Nutrition has been linked to diseases that cause the highest rates of mortality and morbidity in developed populations. There is convincing evidence of the relationship between dietary factors and the development of chronic diseases(Reference Agudo, Cabrera and Amiano1–Reference Shimazu, Kuriyama and Hozawa6). Several studies have suggested that individuals consuming a traditional Mediterranean diet have a reduced risk of CVD, cancer and neurodegenerative diseases(Reference Estruch, Martinez-Gonzalez and Corella7–Reference Trichopoulou, Bamia and Trichopoulos12).
When we need to explore the nutritional status of a population and associate it to their health status, it is very important to accurately estimate food intake in that population. However, all the available methods for measuring food intake have some limitations that must be taken into account before choosing the one that is most appropriate for the study aims and design. Self-reporting of food intake can lead to a certain degree of measurement error that may dilute and/or bias the relationship between nutrient and/or food intake and the disease of interest. Some suggested reasons for this measurement error are within-subject variability in food intake, difficulties in remembering which and how much food was consumed, and the ability or lack thereof to report food consumption.
FFQ are widely accepted as the most suitable assessment tool in large epidemiological studies which associate diet with chronic diseases because they provide dietary information over a long period of time without altering usual food habits and are relatively inexpensive when self-administered(Reference Willett13).
The FFQ is currently the most frequent method used to assess food intake in large population-based studies. This method, with a single measurement, provides a convenient assessment of the habitual dietary intake of an individual. It is accepted that FFQ allow a reasonably accurate ranking of individuals; however, the ability of FFQ to quantify the absolute intake of foods and specific nutrients(Reference Sullivan, Brown and Williams14, Reference Ishihara, Todoriki and Inoue15) is limited.
Our aim was to assess the reproducibility and the relative validity of a self-administered FFQ designed to measure habitual food and nutrient intake and to be used in the PREDIMED Study(Reference Estruch, Martinez-Gonzalez and Corella7, Reference Zazpe, Sanchez-Tainta and Estruch16), a large multicentre randomised controlled clinical trial for the primary prevention of CVD by means of a traditional Mediterranean diet in a high-risk population.
Materials and methods
The design of the present validation study is shown in Fig. 1. The PREDIMED FFQ was adapted from a previously validated Spanish FFQ(Reference Martin-Moreno, Boyle and Gorgojo17). The original questionnaire had 118 items. After several panel discussions among the researchers of the PREDIMED network, another nineteen new items were added, including food items only recently introduced in Spain (kiwi, all-bran cereals) and fruits, vegetables and whole grains with special interest for a Mediterranean diet trial (for example, onion, garlic, aromatic herbs, mushrooms and some other varieties of fungi). In order to better assess the intake of flavonoids, phytosterols and polyphenols, several additional questions about varieties of olive oil, nuts, cocoa, tea and wines were also added. A complementary question about the frequency of consuming fruit as dessert was also added. Some portion sizes were updated with respect to the previous version to represent the serving size of that item now most frequently consumed currently in Spain. The FFQ was designed to be semi-quantitative and optically readable for data input. The FFQ included 137 food items, and the frequencies of consumption of the food items were reported on an incremental scale with nine levels (never or almost never, 1–3 times per month, once per week, 2–4 times per week, 5–6 times per week, once per d, 2–3 times per d, 4–6 times per d and more than six times per d). A full version of the FFQ can be downloaded (http://www.unav.es/departamento/preventiva/predi_educationals). The reported frequencies of food consumption were converted to number of intakes per d and multiplied by the weight of the portion size indicated.
Fig. 1 Design of the PREDIMED's FFQ validation study. DR, dietary record.
A total of 158 independent individuals (seventy-three men and eighty-five women) aged between 55 and 80 years old and from three different Spanish areas (Tarragona, Pamplona and Valencia) of the PREDIMED Study(Reference Estruch, Martinez-Gonzalez and Corella7) were evaluated. The participants were chosen from users of primary health centres and the compliance rate was 85 %. Exclusion criteria were: dementia, severe terminal disease, enteral or parenteral nutrition, and medically prescribed changes in dietary habits. The participants were asked not to modify their dietary habits during the study's 1-year duration.
The present study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures involving human subjects were approved by the Ethics Committees of the Fundació Jordi Gol, University of Valencia and University of Navarra. Written informed consent was obtained from all subjects. After informed consent was obtained, the sociodemographic characteristics and health habits of the participants were recorded.
The dietary record (DR) was used as a reference to explore validity. The participants were asked to record their food intake for 12 d (3 d for each of the four times).
Food intake was grouped into the food groups (dairy products, eggs, meat or meat products, fish or seafood, vegetables, potatoes, fruits, total nuts, legumes, cereals, olive oil, pastries, cakes or sweets, and alcoholic drinks) that were used in the analysis.
To calculate energy and nutrient intake from both methods (FFQ and DR) we used the Spanish food composition tables(Reference Moreiras, Carbajal and Cabrera18). The nutrient composition of each FFQ item was calculated as the mean of all explicitly mentioned foods.
Nobody reported extremes or implausible energy intakes according to DR (>16 790 kJ/d (>4013 kcal/d) and < 3360 kJ/d ( < 803 kcal/d) for males and >14 690 kJ/d (>3511 kcal/d) and < 2100 kJ/d ( < 502 kcal/d) for females)(Reference Willett13); therefore, nobody was excluded from the analysis of results on this basis.
Statistical methods
The mean daily intake of the twelve (four 3 d) dietary records was used as representative of DR. The days of the week recorded using DR for each individual were randomly selected, and the appropriate weight was given to weekends (3 or 4 d for each participant).
Intakes of food groups, energy and nutrients were transformed (log10) to optimise the normality of the distribution.
We calculated Pearson product–moment correlation coefficients and intraclass correlation coefficients (ICC) of log-transformed food and nutrient intakes and energy-adjusted nutrient intakes using the residual method(Reference Willett13).
To correct for within-individual error in the measurement of the DR that tends to reduce correlation coefficients toward zero, the observed correlation coefficient was multiplied by the de-attenuation factor 
, where 
is the within-individual variance, 
is the between-individual variance, and n represents the number of replicate measurements (here n 12)(Reference Willett13). The within- and between-individual variance components were determined by a random-effects model with the record intake as the dependent and subject identification number as the independent variable(Reference Beaton, Milner and Corey19).
Quintiles of intake were also calculated, and the degree of gross misclassification in FFQ with respect to the DR was evaluated using contingency tables of quintiles, which were constructed using the FFQ2 and the average DR.
To graphically check the agreement between the two methods in terms of absolute intake we used the analysis proposed by Bland & Altman(Reference Bland and Altman20), which plots the difference in intake between the two methods (FFQ2 − DR) against the mean intake of the two measures ((FFQ2+DR)/2). The plots include lines for the mean difference and the so-called ‘limits of agreement’, defined as mean ± 1·96 sd of the mean.
Results
The general characteristics of the 158 participants in the study are shown in Table 1. The majority of the participants were of primary educational level, married, non-smokers and did some physical activity at least once a week (walking, yoga, swimming, gymnastics, bicycle, rowing, gardening, hunting, football, trekking, tennis, aerobics or jogging). Of the participants, 53·8, 37·7 and 12·0 % stated that they had hypertension, dyslipidaemia and type 2 diabetes, respectively.
Table 1 Baseline characteristics of the 158 participants
(Mean values and standard deviations or percentages)
Daily intake of food groups, energy, macro- and micronutrients (as assessed by FFQ1, FFQ2 and DR) are shown in Table 2. To explore relative validity we chose the second FFQ instead of the first one because the former covers the same time period (i.e. previous year) as the four 3 d DR used as the reference. The FFQ2 tended to report higher energy and nutrient intake than the DR, which indicates that the former method may overestimate dietary intake to some degree. On the other hand, this may just be reflecting that the DR underestimates many food groups, with the notable exceptions of eggs, legumes and olive oil.
Table 2 Daily consumption of food groups and nutrients estimated by two FFQ (FFQ1 and FFQ2) and four 3 d dietary records (DR)
(Mean values and standard deviations)
* 1 IU vitamin A = 0·3 μg retinol or 0·6 μg β-carotene.
Table 3 show the reproducibility of the FFQ measurements (FFQ1 v. FFQ2) in terms of the Pearson correlation coefficient and ICC. For food groups the Pearson correlation coefficients ranged from 0·55 to 0·82 and were generally above 0·6 (mean 0·65). ICC ranged from 0·63 to 0·90 and were mostly above 0·75 (mean 0·78). For energy and nutrient intake, the Pearson correlation coefficient ranged from 0·52 to 0·76 (mean 0·64) and from 0·50 to 0·78 (mean 0·63) for energy-unadjusted and -adjusted data, respectively; ICC ranged from 0·68 to 0·87 (mean 0·78) and from 0·67 to 0·88 (mean 0·77) for energy-unadjusted and -adjusted data, respectively.
Table 3 Reproducibility of FFQ: correlations between food groups, energy intake and nutrient intake in FFQ1 and FFQ2*
* Intakes of food groups, energy and nutrients were transformed (log10) to improve normality.
Correlation coefficients that explored the validity of FFQ in relation to DR are shown in Table 4. For food groups, the Pearson correlation coefficients ranged from 0·29 to 0·72 and were mostly above 0·4 (mean 0·53); ICC ranged from 0·40 to 0·84 and were mostly above 0·70 (mean 0·67). For energy and nutrient intake, the Pearson correlation coefficient ranged from 0·24 to 0·49 (mean 0·39) and from 0·23 to 0·68 (mean 0·44) for energy-unadjusted and -adjusted data, respectively; ICC ranged from 0·44 to 0·78 (mean 0·55) and from 0·37 to 0·80 (mean 0·59) for energy-unadjusted and -adjusted data, respectively. De-attenuated coefficients are not shown because their values were very similar, but are available upon request.
Table 4 Validity of FFQ: correlations between food groups, energy intake and nutrient intake in four 3 d dietary records and FFQ2*
* Intakes of food groups, energy and nutrients were transformed (log10) to improve normality.
In Table 5, we present the percentage of gross misclassification (both over- and underestimation by the FFQ) and complete or adjacent agreement as indices of validity of FFQ in categorising individuals. For the food groups, between 68 and 83 % of individuals were in the same or adjacent quintile with both of the two methods (mean 72·3 %) and 55–75 % (mean 67·7 %) were in the same quintile for energy and nutrients. The overall extreme misclassification was always lower than 5 % (with the single exception of potatoes (5·1 %) among food groups, and the single exception of cholesterol (5·7 %) among nutrients).
Table 5 Gross misclassification (%) of food groups, energy and energy-adjusted nutrient distribution: classification in opposite quintiles or in the same/adjacent quintile in the second FFQ (FFQ2) v. classification according to the dietary records mean
The Bland–Altman analysis confirmed a general overestimation – as previously mentioned – of the consumption of food groups and of nutrient intake by the FFQ. Examples of the several patterns of relationship observed between the two methods for the consumption of food groups are plotted in Fig. 2 and for total energy and specific nutrient intakes are shown in Fig. 3. For most food groups, energy and macro- and micronutrients, the mean error in intake assessment by FFQ does not change with increased dietary intake (such as meat and meat products, fish or seafood, potatoes, fruits, total nuts, cereals, alcoholic drinks, energy, protein, carbohydrates, fat, SFA, PUFA, niacin, cobalamin, vitamin D, marine origin n-3 and linolenic acid), despite the wider scatter with increasing mean intake values that is sometimes observed. For other dietary variables a systematic variation in agreement between the two methods was observed with increased dietary intake; for dairy products, vegetables, pastries, cakes or sweets, MUFA, cholesterol, fibre, vitamin C, thiamin, riboflavin, pyridoxine, vitamin A, vitamin E and non-marine n-3, low intakes were underestimated whereas high intakes were overestimated by the FFQ; and for eggs, legumes and olive oil low intakes were overestimated and high intakes were underestimated by the FFQ.
Fig. 2 Bland–Altman plots showing the relationship between difference in the daily intake of (a) vegetables, (b) meat or meat products, (c) potatoes and (d) legumes estimated with the second FFQ (FFQ2) and four 3 d dietary records (DR), and the corresponding mean daily intake estimated by the two methods. Lines are mean difference (—) and lower and upper 95 % limits of agreement (- - -).
Fig. 3 Bland–Altman plots showing the relationship between difference in the daily intake of (a) energy, (b) protein, (c) thiamin and (d) cobalamin estimated with the second FFQ (FFQ2) and four 3 d dietary records (DR), and the corresponding mean daily intake estimated by the two methods. Lines are mean difference (—) and lower and upper 95 % limits of agreement (- - -).
Discussion
In the present study, we assessed the reproducibility and validity of a 137-item semi-quantitative FFQ used in a large multicentre randomised controlled clinical trial for the primary prevention of CVD with a traditional Mediterranean diet in a high-risk population (PREDIMED Trial). The present results show that this FFQ provides a highly reproducible assessment of food and nutrient intake and reasonably good validity in relation to DR for many food groups and nutrients (correlation coefficients>0·60). Agreement between these two methods was also reasonably acceptable and gross misclassification using the FFQ was found to be very unlikely.
Although there are good biomarkers for energy(Reference Black, Coward and Cole21) and some for nutrient intake(Reference Bingham22, Reference Cena, Roggi and Turconi23), there is no available biomarker that can provide a reliable measurement of food habits as a whole. Therefore, a dietary assessment method is frequently chosen as a reference when validating an FFQ(Reference Cade, Thompson and Burley24), even for specific nutrients(Reference Sullivan, Brown and Williams14). Since perfectly error-free reference methods for diet measurement cannot be obtained, it is understood that the reference method provides better data but of limited accuracy (i.e. it is not a true ‘gold standard’). It has been suggested that the measurement errors of FFQ and reference methods may be independent(Reference Kaaks, Riboli and van Staveren25), but there is some disagreement on this point(Reference Kipnis, Subar and Midthune26, Reference Willett27). In any case, because recovery biomarkers are not available for most foods or nutrients, usually there is no other choice but to use self-reported measures as reference methods, despite the bias associated with them. In this context, the DR has been widely used as a reference method in FFQ validation studies. In the present study, the DR was applied for 12 d (in four 3 d periods); although a longer recording period may have better reflected true intake, we chose to record no more than 12 d in order to prevent a possible lack of compliance and the subsequent detriment to the quality of the data obtained.
We found a general overestimation of food group and nutrient intake with the FFQ, though this result could be somewhat artificial given potential underestimations by the DR. FFQ usually reflect higher estimates for most food groups and nutrients than the reference method(Reference Subar, Thompson and Kipnis28), particularly if the FFQ exceeds 100 items(Reference Klipstein-Grobusch, den Breeijen and Goldbohm29), as does ours. A recent quantitative review reported that FFQ with more items are better able to rank individuals according to their intake(Reference Molag, de Vries and Ocké30). This overestimation may be due to measurement errors introduced by differences in conceptualisation of portion sizes, misinterpretation of specific items, and frequency and serving size differences between FFQ and DR. This is especially the case when there are multiple foods in an FFQ item. Moreover, averaging amounts of intake over 1 year, as we have done, is prone to estimation misjudgement that may be different from estimation errors in recording very recent amounts of intake as in DR.
The test–retest interval between the two FFQ is an important factor which influences reproducibility. If the interval is too short, responses to the second FFQ may be influenced by the memory of the first FFQ and reproducibility will be overestimated. However, if the interval is too long, the dietary pattern may have actually changed, which could lead to an underestimation of reproducibility(Reference Tsubono, Nishino and Fukao31). According to a comprehensive review, the time interval in published reports ranges from 2 h to 15 years(Reference Cade, Thompson and Burley24). In the present study, we chose a time interval of 1 year to prevent the types of errors mentioned above, and asked participants not to change their dietary habits during the whole study period. We observed small differences in mean intakes (for food groups, energy and nutrients) between FFQ1 and FFQ2. Studies with the same interval time between the two FFQ have shown similar correlation coefficient ranges(Reference Johansson, Hallmans and Wikman32).
Validity of food group intake was better than that of energy and nutrients.
Correlation coefficients for food groups, energy and nutrient intake were comparable with other similar validation studies(Reference Ahn, Kwon and Shim33–Reference Shu, Yang and Jin37).
The validity study of an FFQ specifically designed to estimate PUFA intake using the weighted food record as its reference method found only slightly better results than those reported here(Reference McNaughton, Hughes and Marks38).
When calculating nutrient intake from the information given in an FFQ, the assumption that each explicitly mentioned food item contributes equally to the intake of the nutrient may be wrong. Perhaps a more realistic approach would be to obtain – in the same population – the relative weight of each mentioned and unmentioned food and use this information to calculate the nutrient intake. Such information could also be used to determine portion size. These limitations may partially explain the relatively low level of disagreement between the two methods compared in the validation results.
We must take into account that our participants voluntarily agreed to take part in the present study, and they may tend to respond more accurately to FFQ and DR or have different dietary habits from non-participants.
The correction for random errors associated to within-individual variation almost always increases correlation values. Nevertheless, in the present results, de-attenuated correlations were similar to non-corrected estimates, perhaps because of a relatively low within-individual variation in relation to the between-individual variation in our participants and/or because of the large number of days – 12 – recorded.
Given that FFQ are often used to categorise individual intake rather than to obtain an exact quantification of nutrient intake, it is appropriate to explore the degree of gross misclassification when quintiles are used to rank individuals according to their relative intake. In this aspect, we found a very low frequency of gross misclassification and a relatively high agreement, especially for the specific food groups and nutrients more strongly related to cardiovascular risk.
In the Bland–Altman plots, no apparent change in the magnitude of between-measurement (FFQ v. DR) differences was observed across mean values of both methods for most foods and nutrients. See, for example, the plot for total energy intake (Fig. 3(a)). The interpretation for these nutrients is that agreement of the FFQ with the DR was similar regardless of their average intake. However, in some other cases, a wider scatter was observed, suggesting worse agreement when mean intakes were higher (for example, meat and meat products, potatoes, protein or cobalamin in Fig. 2 and 3). The interpretation here is that errors were somewhat proportional to mean intake.
In conclusion, this 137-item self-administered semi-quantitative FFQ appears to be a useful instrument for the estimation of food habits and nutrient intake in the epidemiological research of diet as a risk factor for CVD and other chronic diseases and for dietary surveillance in Spain. It was found to have acceptable levels of reproducibility and validity.
Acknowledgements
The authors wish to thank the financial support of RD06/0045/0001 (Alimentación saludable en la prevención primaria de enfermedades crónicas: la Red PREDIMED) of the Instituto de Salud Carlos III, Spain. CIBERobn is an IISCIII initiative. We have no conflict of interest to declare.
J. D. F.-B. participated in design of the study, coordination of the study, analysis of data, interpretation of results and drafting the manuscript.
J. L. P. and M. P.-B. participated in design of the study, acquisition of data, analysis of data and interpretation of results, and revising the article for relevant intellectual content and contribution to drafting of the manuscript.
I. Z., P. C. and E. T. participated in acquisition of data and interpretation of results, and revising the article for relevant intellectual content and contribution to drafting of the manuscript.
D. C. participated in design of the study, interpretation of results and drafting of the manuscript with intellectual and scientific input.
M. A. M.-G. and J. S.-S. participated in design of the study, calculation of nutritional data, interpretation of results and contribution to drafting of the manuscript with intellectual and scientific input.
J. M. M.-M. participated in interpretation of results and revising the article for relevant intellectual content and contribution to drafting of the manuscript.
