The contribution of dietary factors to the development and prevention of non-communicable diseases is being increasingly recognized(Reference Amuna and Zotor1). Epidemiological interest currently focuses on examining the association between disease and individual foods, food groups, food patterns, dietary nutrients or healthy eating indices(Reference Willett2–Reference Champagne6).
The measurement of dietary intake remains one of the most challenging tasks in nutritional epidemiology(Reference Drewnowski7). The FFQ is one of the most commonly used methods in epidemiological studies to assess individual long-term dietary intakes of foods and nutrients. Because of its ability to capture usual dietary patterns(Reference Willett and Lenart8), it is crucial to estimate the validity and reliability of an FFQ because, like any other type of dietary assessment, it is affected by error(Reference Black, Goldberg and Jebb9). Information regarding validity and reliability is important and indispensible in interpreting study results to enhance the interpretation of estimated diet–disease associations and to improve the translation of such associations into dietary recommendations(Reference Ogawa, Tsubono and Nishino10, Reference Johansson, Hallmans and Wikman11).
The performance of an FFQ is sensitive to the culture and ethnic background of the study population. Thus the validity and reliability for an FFQ needs to be evaluated for studies conducted in different study populations(Reference Shu, Yang and Jin12).
In recent years, epidemiological studies in Tehran, the capital city of Iran, have shown a high prevalence of metabolic syndrome and CVD in the urban population(Reference Fakhrzadeh, Ebrahimpour and Pourebrahim13, Reference Zabetian, Hadaegh and Azizi14). The Tehran Lipid and Glucose Study (TLGS) was conducted to further investigate dietary relationships, among the other causes of high rates of CVD(Reference Azizi, Rahmani and Emami15). As part of TLGS, we administered a new FFQ (168 items) designed specifically to capture the dietary practices of the study participants. In the present study, we aimed to describe the validity and reliability of this FFQ for assessing nutrient intakes.
Subjects and methods
Subjects
The present study was conducted within the framework of the TLGS, a prospective study conducted in a sample of residents of district-13 of Tehran, to determine the prevalence of risk factors for non-communicable disease and to identify lifestyles to reduce these risk factors(Reference Azizi, Rahmani and Emami15). A random sample of 200 cohort members, aged 20 years and over, were requested to participate in a dietary assessment validation study and 162 subjects agreed. Sample size was determined by considering a confidence interval of 95 %, study power of 80 %, minimum expected correlation coefficient of 0·25 and attrition rate estimation of 50 %. To minimize the effect of under- and over-reporting, we excluded subjects who had left more than seventy items blank on the FFQ and those who reported a total daily energy intake outside the range of 3360–17 640 kJ (800–4200 kcal) on either of the two questionnaires(Reference Willett16); we also excluded those who did not provide a blood or urine sample. A total of 132 subjects (sixty-one males and seventy-one females) remained for the current analysis. The study was approved by the research ethical committee of the Research Institute for Endocrine Sciences of Shahid Beheshti University of Medical Sciences and informed written consent was obtained from each subject.
Assessment of dietary intake
Usual dietary intake was assessed twice using a 168-item semi-quantitative FFQ, one year apart (FFQ1 and FFQ2), all administered by the same trained dietitians for each participant, who had at least 3–5 years of experience in the Nationwide Food Consumption Survey project(Reference Kimiagar, Ghaffarpour and Hoshiar-Rad17) and the TLGS(Reference Mirmiran, Azadbakht and Azizi18), for assessing intra-rater reliability(Reference Cade, Thompson and Warm19). The FFQ consisted of a list of foods with a standard serving size commonly consumed by Iranians. Participants were asked to report their frequency of consumption of a given serving of each food item during the previous year, on a daily (e.g. bread), weekly (e.g. rice, meat) or monthly (e.g. fish) basis. The reported frequency for each food item was converted to a daily intake. Portion sizes of consumed foods were converted to grams using household measures(Reference Ghafarpour, Houshiar-Rad and Kianfar20). Dietary data were also collected by means of 24 h dietary recalls (24hDR), repeated twelve times; 24hDR interviews were performed every month for 12 months by the same trained dietitians according to a standardized protocol and lasted 20 min on average. For all subjects, the recall days included one day of the weekend in Iran (Thursday and Friday); the other days of the week were recalled twice. The first recall was completed one month after FFQ1 administration and the last recall was completed one month before administration of FFQ2. All recall interviews were performed at the subjects’ homes to measure the volume of commonly used household measures. The same interviewer interviewed each subject throughout the study. All 24hDR were reviewed by the investigators and any questions raised were resolved with participants. Because the Iranian food composition table (FCT) is incomplete (limited to only raw materials and a few nutrients), each food and beverage was analysed for energy and nutrient intake using the US Department of Agriculture’s (USDA) FCT. For mixed dishes, nutrients were calculated according to their ingredients. The energy and macronutrients of breads and fruits are almost similar to alternative food items in the USDA FCT, with correlation >0·9. We used the Iranian FCT only for food items like ‘kashk’ which was not listed in the USDA FCT(Reference Azar and Sarkisian21).
Biochemical measurements
Every season, a blood sample was drawn into Vacutainer tubes between 07·00 and 09·00 hours from all study participants in a non-fasting state. Thus we collected four samples (one for each season) for each person. Blood samples were taken in a sitting position according to a standard protocol and centrifuged within 30 to 45 min of collection. Separated plasma was stored at −70°C for up to 17 months until analysed. Serum total cholesterol and TAG concentrations were measured by commercially available enzymatic reagents (Pars Azmoon Inc., Iran) adapted to the Selectra autoanalyaer. Plasma concentrations of retinol, β-carotene and α-tocopherol were measured by the HPLC technique, adapted from Craft et al.(Reference Craft, Brown and Smith22). All samples were analysed when internal quality control met the acceptable criteria. The intra- and inter-assay CV was respectively 2·9 % and 3·2 % for α-tocopherol, 6·8 % and 7·1 % for β-carotene and 5·9 % and 6·8 % for retinol. Also, every season, all participants were asked to provide a 24 h urine collection. Urine was collected in 1-litre plastic bottles containing 5 mg boric acid. On delivery, participants were questioned about the completeness of urine collection; they were then asked to repeat the collection if there was >50 ml loss. Total urinary N was determined by the Kjeldahl technique and urinary K was measured by flame photometry; the reference for urinary K measurement was 25–120 mEq/24 h.
Statistical methods
The SPSS statistical software package version 13 (SPSS Inc., Chicago, IL, USA) was used for all statistical analyses. Normality of the distributions of dietary intake variables was assessed by the Kolmogorov–Smirnov test. When the variables were not normally distributed, we used log-transformed data. All log-transformed variables were normal. All analyses were conducted on the mean of energy and nutrient intake from the twelve 24hDR. Means and standard deviations were calculated for energy and all nutrient intakes from both FFQ and from the twelve 24hDR. The paired t test was conducted to show differences between the two FFQ and between FFQ2 and 24hDR. Pearson correlation coefficients were estimated between energy and nutrient intake variables from FFQ2 and 24hDR. Energy- and age-adjusted nutrient intakes were calculated to remove variation due to energy and age, using the residual method(Reference Rosner and Willett23). Deattenuated correlation coefficients were reported by using Rosner and Willett’s formula to correct within-person variation in the twelve 24hDR(Reference Willett and Lenart8, Reference Rosner and Willett23, Reference Willett24). Crude and energy-adjusted intraclass correlation coefficients were calculated to assess the 1-year reliability(Reference Cade, Thompson and Warm19) of the FFQ. We divided the daily intakes from dietary recalls into thirds and compared them with thirds calculated from FFQ2, expressing the results as agreement, adjacent agreement and complete disagreement percentages.
Sample correlations and estimated validity coefficients between the mean of four measurements of urinary and plasma biomarkers and dietary intakes of comparable nutrients from FFQ2 and the mean of twelve 24hDR were calculated using the method of triads(Reference Ocke and Kaaks25). The correlations between the mean of twelve 24hDR and the mean of four urinary and plasma measurements were corrected for within-person variation(Reference Rosner and Willett23, Reference Willett24). Plasma levels of retinol, β-carotene and α-tocopherol were adjusted for plasma concentration of cholesterol and TAG.
Results
The mean age of the participants was 35·6 (sd 16·8) years, 39·8 (sd 18·8) years for men and 33·4 (15·4) years for women; and their mean BMI was 25·5 (sd 5·2) kg/m2, 24·7 (sd 3·8) kg/m2 for men and 26·0 (sd 5·8) kg/m2 for women. Mean daily intakes from the twelve 24hDR and the two semi-quantitative FFQ are shown in Table 1. The FFQ tended to overestimate intake compared with the 24hDR, especially in women, with the largest discrepancy being seen for β-carotene and vitamin C in men and for vitamin D and β-carotene in women.
Table 1 Daily intake of energy and nutrients estimated by twelve 24 h dietary recalls (24hDR) and two FFQ: Tehran Lipid and Glucose Study
RAE, retinol activity equivalent.
Mean value was significantly different (paired t test) between: a24hDR and FFQ2 (P < 0·01), b24hDR and FFQ2 (P < 0·05), cFFQ1 and FFQ2 (P < 0·01), dFFQ1 and FFQ2 (P < 0·05).
*Percentage difference between intakes calculated with FFQ2 and the average of twelve 24hDR.
†Percentage difference between intakes calculated with FFQ1 and FFQ2.
Crude, energy-adjusted and deattenuated correlation coefficients between mean nutrient intakes of the 24hDR and FFQ2 are shown in Table 2. Overall, adjusted and deattenuated correlation coefficients between the 24hDR and FFQ2 ranged from 0·24 for vitamin A to 0·71 for P in men and from 0·11 for β-carotene to 0·60 for fibre in women. Correlation coefficients were generally higher in men than in women, except for carbohydrate. Mean adjusted and deattenuated correlation coefficients were 0·37 and 0·44 in two age categories, ≤35 and >35 years, respectively.
Table 2 Pearson correlation coefficients of nutrient intake estimated by the average of twelve 24 h dietary recalls (24hDR) and the second FFQ: Tehran Lipid and Glucose Study
*Dietary data were collected by means of twelve 24hDR repeated monthly. The first recall was completed one month after FFQ1 administration and the last recall completed one month before administration of FFQ2.
†Age- and energy-adjusted and deattenuated.
‡Sex- and energy-adjusted and deattenuated.
§Mean of correlation coefficients.
Intraclass correlations between the two FFQ, after adjusting for age and energy intake, are presented in Table 3. Age- and energy-adjusted intraclass correlation coefficients between the two FFQ, administered at a 1 year interval, ranged from 0·41 (monounsaturated fat) to 0·79 (protein) in men and from 0·39 (monounsaturated fat) to 0·74 (saturated fat) in women. Mean adjusted intraclass correlation coefficients between the two FFQ were 0·48 and 0·65 in two age categories, ≤35 and >35 years, respectively.
Table 3 Intraclass correlation for energy and nutrients among the two FFQFootnote *: Tehran Lipid and Glucose Study
* Dietary data were collected by means of twelve 24 h dietary recalls repeated monthly. The first recall was completed one month after FFQ1 administration and the last recall completed one month before administration of FFQ2.
† Mean of correlation coefficients.
Table 4 shows the agreement, adjacent agreement and complete disagreement in nutrient intakes between the 24hDR and FFQ2. The agreement percentages ranged from 39·6 % (vitamin C) to 68·3 % (P) in men and from 39·6 % (K) to 54·1 % (fibre) in women. The complete disagreement ranged from 0 (protein) to 16·3 % (β-carotene) in men and from 1·2 % (thiamin) to 16·2 % (Ca) in women.
Table 4 Percentages of agreement, adjacent agreement and complete disagreement according to tertile classification of daily nutrient intakes based on the average twelve 24 h dietary recalls and the second FFQFootnote *: Tehran Lipid and Glucose Study
* Dietary data were collected by means of 24 h dietary recalls repeated monthly. The first recall was completed one month after FFQ1 administration and the last recall completed one month before administration of FFQ2.
The estimated validity coefficients of protein ranged from 0·38 to 1·16. The ranges of questionnaire validity coefficients, with the sample correlation between the questionnaires and biochemical marker as the lower limit and the estimate obtained by the method of triads as the upper limit, were 0·21–0·56 (protein), 0·37–0·61 (K), 0·38–0·50 (β-carotene), 0·31–0·95 (cholesterol), 0·21–0·55 (retinol) and 0·28–0·38 (α-tocopherol; Fig. 1).
Fig. 1 Tehran Lipid and Glucose Study: sample correlations and estimated validity coefficients between the mean of four measurements of urinary and plasma biomarkers (M) and dietary intakes of comparable nutrients from FFQ2 (Q) and the mean of twelve 24 h dietary recalls (R) using the method of triads; (T) true intake variable. Reference measurements were based on mean values of twelve 24 h dietary recalls (24hDR), completed monthly. Measurements were obtained by a semi-quantitative questionnaire (FFQ) administered twice. The first recall was completed one month after FFQ1 and the last recall was completed one month before FFQ2. Measurements of 24 h urinary N and K and plasma concentrations of β-carotene, cholesterol, retinol and α-tocopherol were taken every season. The correlation between the mean of the 24hDR and the mean of the four urinary and plasma measurements and the other variables were corrected for within-person variation. In addition, plasma levels of retinol, β-carotene and α-tocopherol were adjusted for plasma concentration of cholesterol and TAG. The estimated validity coefficients of protein ranged from 0·38 to 1·16. The ranges of questionnaire validity coefficients, with the sample correlation between the questionnaires and biochemical marker as a lower limit and the estimate obtained by the method of triads as an upper limit, were 0·21–0·56 (protein), 0·37–0·61 (K), 0·38–0·50 (β-carotene), 0·31–0·95 (cholesterol), 0·21–0·55 (retinol) and 0·28–0·38 (α-tocopherol)
Discussion
In the present study we examined the reliability and relative validity of the FFQ developed for the TLGS. We used twelve 24DR to compare nutrient intakes from the FFQ and 24hDR, and to compare serum and urine biomarkers as well. Reliability of the FFQ, as assessed by intraclass correlation coefficients between the results of the two FFQ, was also obtained from the same population. The results showed reasonable relative validity based on true estimated validity coefficients and good reliability of the FFQ over a 1-year period. Cross-classification between these two methods was reasonably acceptable.
The values of correlation coefficients were almost the same between men and women for several nutrients but for some nutrients there were differences between sexes. This may be due to the same portion sizes being used for men and women; in studies in which portion sizes are self-defined, there tend to be differences in portion size between men and women, and furthermore correlations in validity studies tend be highest when subjects are able to describe their own portion sizes(Reference Marks, Hughes and Van der Pols26). The overestimation of the FFQ compared with the mean of the 24hDR may be due to the seasonal availability of food items (like fruits and vegetables when the FFQ was completed), infrequent items with large variation frequency, over-reporting healthy food choices and defining food groups like breads and cereals in great detail; considering that breads and rice are staple foods leads to overestimation of carbohydrates and energy. In addition, relative under-reporting of energy intake was shown by 24hDR compared with doubly labelled water methods(Reference Black, Goldberg and Jebb9). However the problem of our questionnaire estimates of absolute intake should be of less concern when they are applied, because energy-adjusted values are used rather than absolute values.
This is the second validation study in Iran; the first was done in Golestan, a province in the north of Iran, as part of the Golestan cohort study of oesophageal cancer(Reference Malekshah, Kimiagar and Saadatian-Elahi27). In the Golestan study, the correlation coefficients between the dietary recalls and the FFQ ranged from 0·49 to 0·82 and the intraclass correlation between four FFQ ranged from 0·66 to 0·89, but the energy-adjusted correlation coefficients were not calculated. The results of our study have similar ranges of correlation coefficients for the validation of an FFQ as for cohort studies in Japan(Reference Ogawa, Tsubono and Nishino10), northern Sweden(Reference Johansson, Hallmans and Wikman11), Canada(Reference Jain and McLaughlin28) and the Dutch(Reference Ocke, Bueno-de-Mesquita and Pols29) and German(Reference Bohlscheid-Thomas, Hoting and Boeing30) parts of the European Prospective Investigation into Cancer and Nutrition (EPIC).
Correlations comparing nutrients from the dietary recalls with nutrients from FFQ2 might be slightly higher than correlations for nutrients from FFQ1. This difference may reflect some learning bias and change of dietary intake over the years may account for the lower correlations observed with FFQ1(Reference Shu, Yang and Jin12, Reference Rimm, Giovannucci and Stampfer31). However, similar to Rimm et al.’s study(Reference Rimm, Giovannucci and Stampfer31), the FFQ2 used in our study represents the time period during which the 24hDR were collected. Also in our study there was no significant difference between the dietary intake from FFQ1 and FFQ2, except for carbohydrate, folate and riboflavin. Energy-adjusted and deattenuated correlation coefficients were not very different from the crude ones in our study, similar to the findings in a Greek study(Reference Katsouyanni, Rimm and Gnardellis32). Ocke et al.(Reference Ocke, Bueno-de-Mesquita and Pols29) reported higher correlation particularly among men, findings in line with our study. Marks et al. showed that among personal characteristics, sex was most commonly associated with intake estimate errors for food groups(Reference Marks, Hughes and Van der Pols26). We calculated agreement and disagreement percentages to ascertain the usefulness of the FFQ for categorizing individuals based on their levels of consumption. Studies on diet–disease relationships frequently divide nutrient intakes into categories and in epidemiology the primary need is often to place individuals in correct ranking order, rather than to make accurate estimate of absolute intake(Reference Brunner, Stallone and Juneja33). High exact agreement and low complete disagreement percentages were seen in accordance with high and low correlation coefficients, respectively, in our study; hence our FFQ might have the capability to estimate the usual intake at population level. Other validation studies with 24hDR and FFQ showed average exact agreement proportions of 28 %(Reference Paik, Ryu and Choi34) or ranging between 25 % and 58 %(Reference Bohlscheid-Thomas, Hoting and Boeing30).
Using 24hDR as the reference method may be one of the limitations of our study, but we did so because they are less expensive, have high response rate, are easier to use in our population and do not interfere much with the normal dietary habits of subjects. However, their dependence on memory and the inability to incorporate direct measurements are the major limitations of 24hDR. Some previous studies used dietary recalls(Reference Pisani, Faggiano and Krogh35–Reference Kroke, Klipstein-Grobusch and Voss39) and other studies used diet records(Reference Rimm, Giovannucci and Stampfer31, Reference Willett, Sampson and Stampfer40). It seems that characteristics of the study population are important for choosing the reference method, as the diet record is probably not applicable in populations with low or moderate education, or in populations not very experienced in recording food intake, such as the population in the present study. The population in our study was somehow familiar with the dietary recall as it was a subgroup of the TLGS. Regardless of the kind of reference method, under- and overestimation biases and random errors might affect any of the methods normally used as reference in validation studies. Therefore, we used twelve 24hDR, administered monthly for a year, to minimize random errors due to day-to-day variations in food intake and to cover seasonal variations throughout a 1-year period. In addition to repeated dietary recalls, biomarkers from urine and blood samples were used as part of our validation study, as they are not correlated with errors in dietary methods. Using the method of triads, which enables a triangular comparison between questionnaire, reference and biochemical marker measurements, an estimate of the validity coefficient of the FFQ was obtained. In the case of protein, the estimated coefficient of the dietary recall was >1; as a Heywood case that may be acceptable, because a positive correlation between random errors of the FFQ and 24hDR would produce validity coefficients that are overestimated for the FFQ and 24hDR and underestimated for the biochemical marker measurement(Reference Ocke and Kaaks25). The correlation of FFQ measurements of nutrients with biochemical markers as a lower limit and the true estimated coefficient of the FFQ as an upper limit showed that FFQ measurements of K, β-carotene and cholesterol might be reasonably accurate (>0·3) and FFQ measurements of protein, retinol and α-tocopherol appear to be less accurate; however the lower validity correlation of these nutrients may be underestimated(Reference Willett24, Reference Ocke and Kaaks25).
The FFQ administered in the present study was semi-quantitative, such as the FFQ in the Nurses’ Health Study(Reference Willett, Sampson and Stampfer40). However, the portion size was different and we used portion sizes commonly used by Iranians.
To mention other limitations, the methods used for dietary assessment are subject to variations(Reference Ocke, Bueno-de-Mesquita and Pols29, Reference Bingham, Gill and Welch41) and therefore comparison between the two methods may not be precise; however, we evaluated the relative validity(Reference Block42) to partly overcome this problem. Another limitation is that social characteristics, smoking and dietary supplement intake and BMI were not considered in our study. Only a few validity studies(Reference Marks, Hughes and Van der Pols26, Reference Paalanena, Mannisto and Virtanena43) for FFQ have shown results for subgroups other than gender, like BMI groups, and highlight the need to assess FFQ validity in a sample size that is large enough to ascertain differences among subgroups. Using the USDA FCT is another limitation of our study. Not having any complete Iranian FCT with which to compare, we do not know how this affects our results concerning the correlations of dietary intakes of the 24hDR and FFQ with urine and plasma biomarker values.
One of the strengths of the present study is that data were analysed separately in men and women and both men and women were included in the study. Another strength is the reporting of both energy-adjusted and deattenuated correlation coefficients, which reduces the random error due to within-person variation. Clear reasons exist why nutritional epidemiology should focus on energy-adjusted nutrient intakes, which is the nutrient composition of diets in relation to disease occurrence(Reference Willett44).
Over the years investigators have come to recognize that the reported values from FFQ are subject to substantial errors (intake-related bias, person-specific bias and within-person variation) that profoundly affect the interpretation of studies in nutritional epidemiology. It is suggested to structure new models of dietary measurement error for estimation of relative risk based on validation/calibration sub-studies in large epidemiological investigations that include urinary N as a biomarker for protein intake.
In conclusion, the present study shows that using combinations of twelve repeated 24hDR, two FFQ, biochemical markers in serum and urine samples, and true estimated validity coefficients for agreement between two methods, the FFQ used in the TLGS has reasonable relative validity and reliability for nutrient intakes in Tehranian adults and appears to be an acceptable tool for assessing nutrient intakes in this population.
Acknowledgements
The current study was part of the Tehran Lipid and Glucose Study and was supported by grant no. 098 from the Research Institute of Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Islamic Republic of Iran. None of the authors had any personal or financial conflicts of interest. P.M., F.H.E., Y.M. and M.H. designed the study, collected and analysed data, and wrote the manuscript. F.A. supervised the study. We would like to thank the subjects who participated in the study and are grateful to staff of the Nutrition Center for the dietary interviews and laboratory analyses.
