Schur algebras are certain finite-dimensional algebras that completely control the polynomial representation theory of the general linear groups over an infinite field. Infinitesimal Schur algebras are truncated versions of the classical Schur algebras which control the polynomial representation theory of the Frobenius kernels of general linear groups. In this paper we use some elementary results on symmetric powers to classify the semisimple Schur algebras. We then classify the semisimple infinitesimal Schur algebras as well.