We present the results of a comprehensive Spitzer survey of 70 radio galaxies across 1 < z < 5.2. Using IRAC (3.6–8.0μm), IRS (16 μm) and MIPS (24–160 μm) imaging, we decompose the rest-frame optical to infrared spectral energy distributions into stellar, AGN, and dust components and determine the contribution of host galaxy stellar emission at rest-frame 1.6 μm (H-band). We find that the fraction of emitted light at rest-frame 1.6 μm from stars is >80% for over half the high redshift radio galaxies. The other radio galaxies have 1.6 μm stellar fractions spanning the range 20–80%. The resultant stellar luminosities imply stellar masses of 1011−12M⊙, independent of redshift, indicating that radio galaxies are amoungst the most massive galaxies observed over this redshift range. Powerful radio galaxies tend to lie in a similar region of mid-IR color-color space as unobscured AGN, despite the inferred stellar contribution to their shorter-wavelength, mid-IR SEDs. The stellar fraction of the rest-frame 1.6 μm luminosity has no correlation with redshift, radio luminosity, or rest-frame mid-IR (5 μm) luminosity. The bolometric energy output of these sources is dominated by the infrared, and the mid-IR luminosities are found to be similar to that of lower redshift (z < 1) radio galaxies. As expected, these exceptionally high mid-IR luminosities are consistent with an obscured, highly-accreting AGN. A weak, but significant, correlation of stellar mass with radio luminosity is found, consistent with earlier results.