The Fredholm properties of Toeplitz operators $T_a$ on Hardy spaces $H^p$ ($1<p<\infty$) with continuous symbols $a$ are well understood. We consider $T_a$ acting on $H^1$, where the operator is bounded provided that $a$ belongs to the class of symbols given by Janson and Stegenga's result on the pointwise multipliers on $H^1$. A necessary and sufficient condition for $T_a$ to be a Fredholm operator is given when $a$ is continuous and satisfies a mild additional condition (much weaker than Hölder continuity). A formula for the index of $T_a$ is also derived. In addition, we study the case of matrix-valued symbols and Toeplitz operators on $\rm{BMO}_A$.