The Rietveld method based on X-ray diffraction (XRD) technology has been commonly applied for the quantitative analysis of crystalline minerals in organic-rich rocks. The presence of organic matter, however, has interference in varying degree to the quantitative result. In this work, the mixed samples of organic matter and crystalline minerals at given proportions were prepared to investigate the effect of organic matter on the Rietveld quantitative analysis of crystalline minerals in coal gangue. The relative standard deviation (RSD) and absolute error (AE) were used to evaluate the precision and accuracy of the quantitative result. The results showed that the presence of organic matter led to a reduction in the precision and accuracy. It was attributed in part to the presence of organic matter, which can produce a background noise in the XRD pattern, especially in the region of 5–30° (2θ). Further results showed that the interference in this region increased with the increase in the organic matter content. The RSD and AE were within the range of 10% and 2%, respectively, when the organic matter content is <30 wt%. Based on this, an approximate method for the Rietveld quantitative analysis of crystalline minerals in coal gangue was established.