Based on a micropolar continuum of rough granular particles that takes into account the balance equations for the spin (/rotational) velocity and the spin granular temperature, the linear stability characteristics of an unbounded shear flow (${\bm u}\equiv (u_x, u_y, u_z) \,{=}\, (\dot\gamma y, 0, 0)$, where $x$, $y$ and $z$ are the streamwise, transverse and spanwise directions, respectively, and $\dot\gamma$ is the shear rate) are analysed. For pure spanwise perturbations ($k_z\neq 0$, with $k_x\,{=}\,0\,{=}\,k_y$, where $k_i$ is the wavenumber in the $i$th direction), we show that the streamwise translational velocity and the transverse spin velocity modes are subject to linear growths, owing to an inviscid ‘algebraic’ instability (that grows linearly with time). This algebraic instability is shown to be tied to a hidden mechanism of momentum transfer from the translational to the rotational modes, via pure spanwise perturbations to the transverse velocity – in short, we have uncovered an ‘instability-induced rotational-driving’ mechanism. Pure spanwise ($k_z\neq 0$, with $k_x\,{=}\,0\,{=}\,k_y$) and pure transverse ($k_y\neq 0$, with $k_x\,{=}\,0\,{=}\,k_z$) perturbations give rise to ‘exponential’ instabilities (that grow exponentially with time) which are related to similar stationary instabilities in the shear flow of smooth, inelastic particles. Both these instabilities also survive in the limiting case of perfectly elastic but rough particles. The scalings of hydrodynamic modes with wavenumbers have been obtained via the respective long-wave expansion. Perturbations with modulations in all three directions are shown to be stable in the asymptotic time limit, but there could be short-time ‘exponential’ growth of these general perturbations in the long-wave limit for both travelling and stationary waves. The growth rate of all instabilities is maximum at intermediate values of the tangential restitution coefficient ($\beta$), and decreases in both the perfectly smooth ($\beta\to -1$) and rough ($\beta\to 1$) limits; the associated instability length scale is minimum at intermediate $\beta$, and increases in both the perfectly smooth and rough limits. In the perfectly smooth limit, there is a window of particle volume fraction ($\phi$), $\phi_c^s <\phi < \phi_c^t$, over which the flow remains stable to all perturbations. With the inclusion of spin fields, the size of this window decreases and at moderate dissipations with $\beta\,{>}\,0.5$ the flow becomes unstable at all $\phi$.