Let $X$ be a solenoid, i.e. a compact, finite-dimensional, connected abelian group with normalized Haar measure $\unicode[STIX]{x1D707}$, and let $\unicode[STIX]{x1D6E4}\rightarrow \operatorname{Aff}(X)$ be an action of a countable discrete group $\unicode[STIX]{x1D6E4}$ by continuous affine transformations of $X$. We show that the probability measure preserving action $\unicode[STIX]{x1D6E4}\curvearrowright (X,\unicode[STIX]{x1D707})$ does not have the spectral gap property if and only if there exists a $p_{\text{a}}(\unicode[STIX]{x1D6E4})$-invariant proper subsolenoid $Y$ of $X$ such that the image of $\unicode[STIX]{x1D6E4}$ in $\operatorname{Aff}(X/Y)$ is a virtually solvable group, where $p_{\text{a}}:\operatorname{Aff}(X)\rightarrow \operatorname{Aut}(X)$ is the canonical projection. When $\unicode[STIX]{x1D6E4}$ is finitely generated or when $X$ is the $a$-adic solenoid for an integer $a\geq 1$, the subsolenoid $Y$ can be chosen so that the image $\unicode[STIX]{x1D6E4}$ in $\operatorname{Aff}(X/Y)$ is a virtually abelian group. In particular, an action $\unicode[STIX]{x1D6E4}\curvearrowright (X,\unicode[STIX]{x1D707})$ by affine transformations on a solenoid $X$ has the spectral gap property if and only if $\unicode[STIX]{x1D6E4}\curvearrowright (X,\unicode[STIX]{x1D707})$ is strongly ergodic.