The introduction of cover crops as fallow replacement in the traditional cereal-based cropping system of the Northern Great Plains has the potential to decrease soil erosion, increase water infiltration, reduce weed pressure and improve soil health. However, there are concerns this might come at the cost of reduced production in the subsequent wheat crop due to soil water use by the cover crops. To determine this risk, a phased 2-year rotation of 15 different cover crop mixtures and winter wheat/spring wheat was established at the Northern Agricultural Research Center near Havre, MT from 2012 to 2020, or four rotation cycles. Controls included fallow–wheat and barley–wheat sequences. Cover crops and barley were terminated early July by haying, grazing or herbicide application. Yields were significantly decreased in wheat following cover crops in 3 out of 8 years, up to maximum of 1.4 t ha−1 (or 60%) for winter wheat following cool-season cover crop mixtures. However, cover crops also unexpectedly increased following wheat yields in 2018, possibly due in part to residual fertilizer. Within cool-, mid- and warm-season cover crop groups, individual mixtures did not show significant differences impact on following grain yields. Similarly, cover crop termination methods had no impact on spring or winter wheat grain yields in any of the 8 years considered. Wheat grain protein concentration was not affected by cover crop mixtures or termination treatments but was decreased in winter wheat following barley. Differences in soil water content across cover crop groups were only evident at the beginning of the third cycle in one field, but important reductions were observed below 15 cm in the last rotation cycle. In-season rainfall explained 43 and 13% of the variability in winter and spring wheat yields, respectively, compared to 2 and 1% for the previous year cover crop biomass. Further economic analyses are required to determine if the integration of livestock is necessary to mitigate the risks associated with the introduction of cover crops in replacement of fallow in the Northern Great Plains.