Climate change and soil fertility decline are major threats to smallholder farmers' food and nutrition security in southern Africa, and cropping systems that improve soil health are needed to address these challenges. Cropping systems that invest in soil organic matter, such as no-tillage (NT) with crop residue retention, have been proposed as potential solutions. However, a key challenge for assessing the sustainability of NT systems is that soil carbon (C) stocks develop over long timescales, and there is an urgent need to identify trajectory indicators of sustainability and crop productivity. Here we examined the effects of NT as compared with conventional tillage without residue retention on relationships between soil characteristics and maize (Zea mays L.) productivity in long-term on-farm and on-station trials in Zimbabwe. Our results show that relationships between soil characteristics and maize productivity, and the effects of management on these relationships, varied with soil type. Total soil nitrogen (N) and C were strong predictors of maize grain yield and above-ground biomass (i.e., stover) in the clayey soils, but not in the sandy soils, under both managements. This highlights context-specific benefits of management that fosters the accumulation of soil C and N stocks. Despite a strong effect of NT management on soil C and N in sandy soils, this accrual was not sufficient to support increased crop productivity in these soils. We suggest that sandy soils should be the priority target of NT with organic resource inputs interventions in southern Africa, as mineral fertilizer inputs alone will not halt the soil fertility decline. This will require a holistic management approach and input of C in various forms (e.g., biomass from cover crops and tree components, crop residues, in combination with mineral fertilizers). Clayey soils on the other hand have greater buffering capacity against detrimental effects of soil tillage and low C input.