Dietary fibre intake has been suggested to reduce blood glucose levels in diabetic patients, particularly when glycosylated Hb (HbA1c) levels are high. In the present study, we used a quantile regression (QR) approach to characterise the possible heterogeneous associations of dietary fibre intake with HbA1c levels in Chinese diabetic patients. A total of 497 diabetic patients participated in the baseline survey in 2006 and in the follow-up survey in 2011, both of which were conducted in Pudong New Area of Shanghai, China. Structured in-person interviews were conducted to collect information on demographic characteristics and lifestyle factors. Dietary intake was assessed using a validated FFQ. Blood samples were collected during the interviews for biochemical assays. QR models were used to examine the heterogeneous associations of dietary factors with HbA1c levels. A significant marginal association of insoluble dietary fibre intake with subsequent HbA1c levels was observed only when the HbA1c level was over 6·8 %. The associations appeared to be greater when the quantile levels of HbA1c were higher. The coefficient estimates were − 0·174 (95 % CI − 0·433, − 0·025) at the quantile of 0·60, − 0·200 (95 % CI − 0·306, − 0·008) at 0·70, − 0·221 (95 % CI − 0·426, − 0·117) at 0·80, and − 0·389 (95 % CI − 0·516, − 0·018) at 0·90. A similar pattern was observed for the associations of dietary glycaemic index (GI) value with HbA1c levels. In conclusion, the present results indicate that the associations of insoluble dietary fibre intake and GI value with subsequent HbA1c levels depend on glycaemic control status in Chinese diabetic patients. More studies are required to confirm our findings.