Although black soil in northeast China undergoes severe erosion, the contribution of parent materials, mainly Quaternary loess and non-loess sediments, to soil erodibility remains unclear. Considering the inheritance of ferromagnetic materials by parent materials, changes in magnetic parameters can successfully determine soil erodibility on a regional scale with a close climatic background. Here, we analysed the magnetic indicators of 142 samples from the black soil horizon formed on loess and non-loess sediments, covering areas of severe and slight erosion in the region to determine the effects of parent materials on the erodibility of black soil in northeast China. Both low-frequency magnetic susceptibility and frequency magnetic susceptibility (χfd) were proportional to the decrease in erosion rate due to erosion-induced leaching of ferromagnetic materials, and the change in χfd was narrow for black soil with loess parent materials, corresponding to relatively low soil erodibility. Compared with loess, the addition of soil organic matter could stabilise soils against erosion, thereby inducing a decrease in the erodibility of black soil formed on loess. Additionally, sustainable soil management policies to protect black soil from further erosion are necessary and urgent under the pressure of maintaining high grain yields and preventing erosion in northeast China.