The FFQ is a frequently used tool to measure dietary intakes in epidemiological studies on diet and disease. A self-administered semi-quantitative FFQ was used to measure the habitual consumption of foods and nutrients in the Dutch cohorts of the European Prospective Investigation into Cancer and Nutrition (EPIC-NL)( Reference Beulens, Monninkhof and Verschuren 1 ).
In 1991, before the start of the EPIC-NL study, the FFQ was validated against twelve 24-h dietary recalls (24HDR) to study its ability to rank subjects according to several foods( Reference Ocke, Bueno-de-Mesquita and Goddijn 2 ) and nutrients( Reference Ocke, Bueno-de-Mesquita and Pols 3 ), including total fat. However, up to today, this FFQ has not been validated for classes of fatty acids and individual fatty acids, although over time it has become evident that effects of dietary fats on (cardiovascular) health may differ across classes( Reference Skeaff and Miller 4 ), and potentially even across individual fatty acids within these classes( Reference Hu, Stampfer and Manson 5 , Reference Mensink, Zock and Kester 6 ). For the purpose of studying disease risks in relation to individual fatty acids in the EPIC-NL cohort, it is essential to assess the ability of its FFQ to capture their intake.
Several other FFQ were validated against 24HDR or food records for their ability to rank subjects according to several, but not all, individual fatty acids. The majority was focused on individual PUFA( Reference Broadfield, McKeever and Fogarty 7 – Reference Wolk, Ljung and Vessby 16 ) and oleic acid (18 : 1n-9)( Reference Broadfield, McKeever and Fogarty 7 – Reference Kabagambe, Baylin and Allan 9 , Reference McNaughton, Hughes and Marks 11 – Reference Wolk, Ljung and Vessby 16 ), and the validity varied from fair (correlation coefficients (r) between 0·20 and 0·40) up to good (r between 0·60 and 0·80). Concerning individual SFA, studies focused on validating the medium- and long-chained SFA only( Reference Broadfield, McKeever and Fogarty 7 , Reference Kabagambe, Baylin and Allan 9 , Reference McNaughton, Hughes and Marks 11 , Reference Riboli, Elmstahl and Saracci 12 , Reference Wennberg, Vessby and Johansson 15 , Reference Wolk, Ljung and Vessby 16 ), of which only two( Reference Riboli, Elmstahl and Saracci 12 , Reference Wennberg, Vessby and Johansson 15 ) reported on the validity of pentadecylic (15 : 0) and margaric (17 : 0) acid( Reference Wennberg, Vessby and Johansson 15 ), or capric (10 : 0) and lauric (12 : 0) acid( Reference Riboli, Elmstahl and Saracci 12 ). All studies reported moderate to good relative validity( Reference Broadfield, McKeever and Fogarty 7 , Reference McNaughton, Hughes and Marks 11 , Reference Riboli, Elmstahl and Saracci 12 , Reference Wennberg, Vessby and Johansson 15 , Reference Wolk, Ljung and Vessby 16 ), except for one, which observed fair-to-moderate validity( Reference Kabagambe, Baylin and Allan 9 ). The relative validity for trans-fatty acid (TFA) intake was studied less often than the other fatty acid classes( Reference Jaceldo-Siegl, Knutsen and Sabate 8 , Reference McNaughton, Hughes and Marks 11 , Reference Cantwell, Gibney and Cronin 17 ), and ranged from poor( Reference McNaughton, Hughes and Marks 11 ) to good( Reference Cantwell, Gibney and Cronin 17 ).
Other validity studies were carried out in different, non-Dutch, populations with different dietary patterns. As the validity of an FFQ is specific to the study population and FFQ, we cannot translate the validity of other FFQ to the EPIC-NL FFQ. Therefore, in the present study, the reproducibility and relative validity of the FFQ, used in the EPIC-NL study, for measuring fatty acid classes and individual SFA, including short- and medium-chain SFA, TFA, MUFA and PUFA, were investigated.
Methods
Study population and data collection
Description of the study population as well as the collection and processing of the data have been described in detail elsewhere( Reference Ocke, Bueno-de-Mesquita and Goddijn 2 ). In short, the validation study was carried out before the actual enrolment of the EPIC-NL cohort members, and started in 1991. A total of 960 healthy Dutch men and women from two ongoing projects in four towns were invited to participate in the study by postal mail. These subjects were representative of the EPIC-NL cohort members. Of the 240 (25 %) subjects who responded positively, 134 subjects were selected, equally distributed across the four towns, between both sexes, and in 20-year age groups. A total of sixty-three men and fifty-eight women, aged 23–72 years, completed the study. The results presented in this article apply to those 121 subjects. Data were collected over a period of 13 months, starting in October 1991. To assess the reproducibility, the FFQ was administered three times: in months 1, 7 and 13. During the same period, twelve 24HDR were administered once every month in order to assess relative validity.
The questionnaire was self-administered and contained questions on the habitual consumption frequency of seventy-nine main food items during the preceding year. Frequencies could be indicated in times per day, per week, per month or per year. For twenty-one foods, the questionnaire contained photographs of different portion sizes. For other foods, natural or household units were used to indicate portion size. The questionnaire contained additional questions about preparation methods and additions, and provided blank spaces for specification of brand names of margarines and cooking fats. Of the twelve 24HDR, six were administered face-to-face and six by telephone without previous warning. For most subjects, the recall days included one Saturday and one Sunday, and all other weekdays were on average recalled twice. The recalls were performed by trained nutritionists and dietitians, and most subjects were interviewed by the same interviewer throughout the study period.
Data processing and data analyses
For each FFQ and 24HDR assessment, dietary intakes were calculated for each individual subject. The Dutch food composition table 1998 (digital update) was used to calculate the intake of individual fatty acids in grams per day. To correct for under-representation of weekend days, the weighted average of the 24HDR was calculated with a weight of one for weekdays and two for weekend days. The nutrient residual method was used to adjust fatty acid intakes for total energy intake( Reference Willett, Howe and Kushi 18 ). As the majority of fatty acids were not normally distributed (data not shown), intakes were expressed as medians with interquartile ranges. To compare the median intakes of the first FFQ (FFQ1) with FFQ2, FFQ3 and the 24HDR, the Wilcoxon’s signed-rank test was used. Intra-class correlation coefficients (ICC) were calculated with a two-way mixed model to obtain the reproducibility of the FFQ. To investigate the relative validity between FFQ1 and the weighted average of the twelve 24HDR, Spearman’s rank correlation coefficients (r s) were calculated. In addition, weighted κ coefficients (κ w ) were calculated to assess the degree of agreement in fatty acid intake quintiles according to the FFQ1 v. the 24HDR. The ICC, r s and κ w were interpreted according to the following classification: poor (⩽0·20), fair (0·21–0·40), moderate (0·41–0·60), good (0·61–0·80) or excellent (>0·80). All the above-mentioned analyses were performed for both crude and energy-adjusted intakes. To assess absolute agreement between FFQ1 and the 24HDR, we constructed Bland–Altman plots for energy-adjusted fatty acid intakes only. In addition, we investigated whether potential bias was proportional to the levels of energy-adjusted fatty acid intake using linear regression analysis.
Linear regression analysis showed that the relationship between fatty acid intakes as measured by the FFQ and as measured by the 24HDR differed significantly for men and women. Therefore, all analyses were stratified for sex. All the analyses were performed using SPSS version 20.0 (IBM) or SAS 9.2 (SAS Institute).
Results
A detailed description of the baseline characteristics of the study population can be found elsewhere( Reference Ocke, Bueno-de-Mesquita and Goddijn 2 ). In short, the mean age of men and women was 42·6 (sd 11·1) and 49·0 (sd 14·6) years, respectively. The average BMI was 25·5 (sd 2·9) kg/m2 in men and 24·9 (sd 3·5) kg/m2 in women. Furthermore, 28 % of both men and women attained higher vocational education or attended university.
The crude fatty acid intakes as measured by the FFQ and the 24HDR are shown in Tables 1 and 2 for men and women, respectively. Energy-adjusted intakes are presented in the online Supplementary Tables S1 and S2. In both men and women, FFQ1 overestimated the intakes of 16 : 0, TFA and total MUFA as well as individual MUFA and PUFA, as compared with the weighted average of the 24HDR. Similarly, median intakes measured with FFQ1 were significantly higher than those measured with FFQ3, except for PUFA.
TFA, trans-fatty acid.
* P<0·05, ** P<0·005, *** P<0·0001 for significance of difference in median intake compared with FFQ1 as tested with the Wilcoxon’s signed-rank test.
TFA, trans-fatty acid.
* P<0·05, ** P<0·005, *** P<0·0001 for significance of difference in median intake compared with FFQ1 as tested with the Wilcoxon’s signed-rank test.
Table 3 presents the ICC of the three repeated FFQ. ICC for crude fatty acids ranged from 0·56 to 0·75 in men and from 0·57 to 0·82 in women, indicating moderate to good reproducibility. The results were comparable for energy-adjusted fatty acids. The r s for the fatty acids as measured by FFQ1 and the weighted average of the 24HDR are shown in Table 4. In men, the relative validity was moderate to good for crude intakes of total and individual SFA and MUFA, TFA, linoleic acid (LA; 18 : 2n-6) and α-linolenic acid (ALA; 18 : 3n-3), with r s between 0·53 and 0·67. For energy-adjusted intakes of these fatty acids, the coefficients were slightly different but still within the same range, except for stearic acid (18 : 0) (r s=0·47) and ALA (r s=0·45), which were lower. Relative validity was lower for the low-abundant PUFA including arachidonic acid (AA; 20 : 4n-6) (r s=0·42) and the marine n-3 PUFA EPA (20 : 5n-3) and DHA (22 : 6n-3) (r s<0·40). Energy adjustment did not materially change these coefficients.
TFA, trans-fatty acid.
* All fatty acid intakes were adjusted for total energy intake using the residual method( Reference Willett, Howe and Kushi 18 ).
TFA, trans-fatty acid.
* All fatty acid intakes were adjusted for total energy intake using the residual method( Reference Willett, Howe and Kushi 18 ).
In women, the r s between FFQ1 and the 24HDR showed moderate to good relative validity for all SFA (r s from 0·51 to 0·62), except for caprylic acid (8 : 0) (r s=0·35) and lauric acid (r s=0·33), for which validity was fair. Energy adjustment lowered most correlations (r s from 0·30 to 0·50), except for palmitic acid (16 : 0) (r s=0·62) and capric acid (r s=0·66). For MUFA and TFA, the r s were, respectively, 0·63 and 0·56 for crude intakes and 0·58 and 0·49 for energy-adjusted intakes. For individual PUFA, the r s varied from 0·33 to 0·44 for n-6 PUFA and from 0·28 to 0·36 for n-3 PUFA. The correlation coefficients for energy-adjusted PUFA intakes were higher for AA and EPA but lower for total PUFA, ALA and DHA.
Table 5 presents the κ w between FFQ1 and the 24HDR. In men, the agreement between FFQ1 and the 24HDR was fair for crude intake of lauric acid and moderate for the other individual SFA and total SFA (κ w from 0·40 to 0·48). After energy adjustment, the agreement was fair for the short-chain SFA, caprylic acid, lauric acid and margaric acid (κ w from 0·34 to 0·38), and moderate for all other SFA (κ w from 0·44 to 0·52). Moderate agreement was observed for crude intakes of MUFA, TFA and all PUFA, except AA (κ w =0·31) and marine n-3 PUFA (median κ w =0·21), which were considered fair. In general, the κ w were slightly lower for energy-adjusted intakes of MUFA, TFA and PUFA. In women, κ w between FFQ1 and the 24HDR were 0·47, 0·41, 0·50 and 0·43 for crude intakes of, respectively, total SFA, capric acid, palmitic acid and stearic acid (18 : 0). For the other SFA, κ w coefficients were lower, ranging from 0·19 to 0·39. For crude intakes of MUFA and TFA, κ w coefficients were 0·43 and 0·34, respectively. κ w Coefficients for PUFA ranged from 0·17 (EPA) to 0·28 (total PUFA). Energy adjustment in general lowered the κ w coefficients for all fatty acids.
TFA, trans-fatty acid.
* All fatty acid intakes were adjusted for total energy intake using the residual method( Reference Willett, Howe and Kushi 18 ).
Bland–Altman plots showed systematic, non-proportional overestimation by FFQ1 as compared with the weighted average of the 24HDR of intakes of palmitic acid and TFA in both men and women (see online Supplementary Fig. S1–S44). In men, proportional bias was observed for butyric acid (4 : 0), caproic acid (6 : 0) and pentadecylic acid, indicating underestimation at lower intake levels and overestimation at higher intake levels. In addition, for PUFA and LA, the overestimation was positively proportional to the levels of intake. For AA and n-3 PUFA, the proportional bias was negative, demonstrating underestimation by the FFQ at higher levels of intake. In women, a slight overestimation was observed for most SFA, which was positively proportional for capric acid only. Intake of total PUFA was systematically overestimated, showing no proportional bias, whereas the overestimation of LA increased with increased levels of intake, and intakes of AA, EPA and DHA showed negatively proportional bias.
Discussion
The reproducibility of the FFQ, used in EPIC-NL, was moderate to good for all fatty acid classes and individual fatty acids in both men and women. In men, the relative validity of the FFQ was moderate to good for all fatty acids, but fair for the low-abundant long-chain PUFA. In women, moderate to good relative validity was observed for SFA that are highly abundant in the Dutch diet, as well as for TFA and MUFA. The relative validity of low-abundant SFA and PUFA was fair to moderate, with the lowest validity observed for the marine n-3 fatty acids. Compared with the weighted average of the 24HDR, the FFQ generally overestimated fatty acid intake, and showed proportional bias for low-abundant fatty acids, particularly the short-chain SFA and the PUFA.
Strengths of this study include the size of the study population and the equal distribution of subject characteristics such as age and sex. Furthermore, we used data from twelve repeated 24HDR and the FFQ was administered three times at 6-month intervals. A limitation of the study is that the reference method we used, the 24HDR, has correlated errors with the FFQ, such as the reliance on memory, socially desirable answering and use of the same food composition database for calculations of nutrient intakes. Such correlated errors can lead to artificially high correlations between the two methods( Reference Willett 19 ). A reference method that has no correlated errors to the FFQ is the biomarker. Fatty acid levels measured in, for instance, erythrocytes, plasma or adipose tissue can be used as biomarkers for dietary fatty acid intake, but only for the (largely) exogenously derived ones such as EPA, DHA, TFA, pentadecylic acid and margaric acid. Fatty acid biomarkers do not reflect dietary intakes of fatty acids that are largely endogenously derived, such as SFA and MUFA( Reference Arab 20 ). For the present study population, no biomarkers were available. A previous study in a subsample of the total EPIC cohort (which apart from EPIC-NL includes cohorts from nine other countries( Reference Riboli 21 )) compared mean plasma phospholipid fatty acid profiles with mean intakes of food groups as measured by the country-specific FFQ, including the EPIC-NL FFQ( Reference Saadatian-Elahi, Slimani and Chajes 22 ). In that study, exogenously derived fatty acids significantly correlated with those foods that are important contributors to their intake. To illustrate, plasma phospholipid measures of the sum of pentadecylic acid and margaric acid were correlated with dairy product intake as measured by the FFQ. Moreover, 18 : 1n-9t correlated with intakes of dairy foods and margarine, and DHA correlated with fatty fish intake. This indirectly suggests that the EPIC FFQ are well capable of measuring the intakes of these fatty acids. However, we should be careful with directly applying this to the EPIC-NL FFQ as the previous findings are based on combined study populations from different European countries with each having their own FFQ, and it does not compare estimates on the individual fatty acid level. Our study showed that the reproducibility of the FFQ for fatty acid intake assessment in general was good, with ICC ranging from 0·56 to 0·83. These ICC are of the same magnitude as those presented in other studies that assessed the reproducibility of an FFQ for classes of fatty acids( Reference Collins, Boggess and Watson 23 – Reference Laviolle, Froger-Bompas and Guillo 26 ) and a limited number of individual PUFA( Reference Horn-Ross, Lee and Collins 25 , Reference Laviolle, Froger-Bompas and Guillo 26 ). One study reported lower ICC ranging from 0·28 for total PUFA to 0·61 for DHA( Reference Filippi, Amodio and Napoli 27 ).
We observed an overestimation of intake of the majority of fatty acids assessed by the first FFQ as compared with the third FFQ, which is in line with a previous reproducibility study on dietary fatty acid measurements( Reference Filippi, Amodio and Napoli 27 ). The first FFQ also overestimated fatty acid intakes as compared with the 24HDR, which was also observed in several previous validation studies( Reference Jaceldo-Siegl, Knutsen and Sabate 8 – Reference Lora, Lewis and Eskridge 10 , Reference Segovia-Siapco, Singh and Jaceldo-Siegl 13 ), although not in all( Reference Wennberg, Vessby and Johansson 15 ). Overestimation is very common for questionnaires that cover more than 100 food items and pertain to a long time period( Reference Segovia-Siapco, Singh and Jaceldo-Siegl 13 ), such as the FFQ used in our study.
In general, the relative validity for subject ranking in our study was lower among women than among men. This is in line with the lower validity among women in a previous validation study of this FFQ for food groups that largely contribute to fatty acids intake, including cheese, nuts and seeds, and biscuits and pastries( Reference Ocke, Bueno-de-Mesquita and Goddijn 2 ). Previously, it was shown that under-reporting more often applies to foods that are rich in fats( Reference Lafay, Mennen and Basdevant 28 ), and some studies( Reference Johnson, Goran and Poehlman 29 – Reference Ferrari, Slimani and Ciampi 32 ), although not all( Reference Heitmann 33 – Reference Tooze, Subar and Thompson 35 ), showed that under-reporters are more often women, which may explain the lower validity we observed.
Energy adjustment is often used in validation studies to cancel out correlated errors between the two measurement tools( Reference Willett 19 ). In the present study, energy adjustment of fatty acid intake did not improve the validity, and in many cases even lowered the validity. This is in contrast to what is expected based on a study that reported improvement in the validity of three different FFQ after energy adjustment( Reference Subar, Thompson and Kipnis 36 ). It is unclear why energy adjustment caused lower relative validity in our study.
In general, the relative validity of the FFQ in the present study was moderate to good for intakes of individual SFA. Results from previous validation studies on SFA with chain lengths of ten carbon atoms and over that used 24HDR( Reference Kabagambe, Baylin and Allan 9 , Reference Wennberg, Vessby and Johansson 15 ) or (weighed) food records( Reference Broadfield, McKeever and Fogarty 7 , Reference McNaughton, Hughes and Marks 11 , Reference Riboli, Elmstahl and Saracci 12 , Reference Wolk, Ljung and Vessby 16 ) as their reference method were similar to ours. The ability to rank subjects according to intake of SFA that are less abundant in the diet, including short-chain SFA and odd-chain SFA, was less among women. To our knowledge, no previous studies have validated an FFQ against 24HDR or diet records for shorter-chain SFA. It is conceivable that because of the small between-subject and within-subject variation in intake of these SFA, overestimation by the FFQ as compared with the 24HDR will easily lead to changes in subject ranking, and thus to lower validity.
For measurement of individual PUFA, and in particular the marine n-3 PUFA, which are less abundant in the Dutch diet, the relative validity was low in our study, and considered fair. Previous validation studies showed varying results for the measurement of EPA and DHA. Some studies observed fair validity (r<0·40)( Reference Lora, Lewis and Eskridge 10 , Reference Segovia-Siapco, Singh and Jaceldo-Siegl 13 , Reference Wennberg, Vessby and Johansson 15 ), similar to our study, whereas other studies report moderate (0·40≤r<0·60)( Reference Broadfield, McKeever and Fogarty 7 , Reference Jaceldo-Siegl, Knutsen and Sabate 8 , Reference Riboli, Elmstahl and Saracci 12 , Reference Wolk, Ljung and Vessby 16 ) to good validity (r≥0·60)( Reference Sullivan, Brown and Williams 14 ). The lower validity in our study may be caused by the type of reference method used. Studies that showed the lowest validity all used 24HDR( Reference Jaceldo-Siegl, Knutsen and Sabate 8 , Reference Lora, Lewis and Eskridge 10 , Reference Segovia-Siapco, Singh and Jaceldo-Siegl 13 , Reference Wennberg, Vessby and Johansson 15 ), whereas the reference method in the majority of studies that showed higher validity were food records( Reference Broadfield, McKeever and Fogarty 7 , Reference Riboli, Elmstahl and Saracci 12 , Reference Sullivan, Brown and Williams 14 , Reference Wolk, Ljung and Vessby 16 ). Other validation studies used erythrocytes( Reference Ingram, Stonehouse and Russell 37 – Reference Zhang, Wang and Chen 40 ), adipose tissue( Reference Wallin, Di Giuseppe and Burgaz 41 ) or plasma( Reference McNaughton, Hughes and Marks 11 , Reference Sullivan, Williams and Meyer 38 , Reference Swierk, Williams and Wilcox 39 ) as their reference method. Such biomarkers are considered to be a better reference for n-3 PUFA than 24HDR and food records, because of their uncorrelated errors to the FFQ. The observed validity in these biomarker studies ranged from fair( Reference Zhang, Wang and Chen 40 – Reference Lepsch, Vaz and Moreira 42 ) to excellent( Reference Swierk, Williams and Wilcox 39 ). In general, the validity was higher for FFQ that were specifically developed to measure n-3 PUFA intake( Reference Ingram, Stonehouse and Russell 37 – Reference Swierk, Williams and Wilcox 39 ) than for FFQ that were similar to the EPIC-NL FFQ, developed with the aim to measure the total diet( Reference Zhang, Wang and Chen 40 – Reference Lepsch, Vaz and Moreira 42 ). This illustrates another potential explanation for the lower validity in our study. In the EPIC-NL FFQ, intakes of fish products, the main food sources of EPA and DHA, were not asked separately but aggregated into three items, which could have led to an underestimation of intake( Reference Serdula, Byers and Coates 43 ). Correspondingly, a previous validation study of the FFQ used in the present study( Reference Ocke, Bueno-de-Mesquita and Goddijn 2 ) showed similar fair validity for intake of fish (r 0·32 in men, r 0·37 in women).
In contrast to the underestimation of EPA and DHA, LA intake was overestimated by the FFQ as compared with the 24HDR in our study population. This overestimation increased with higher intake levels, and may be caused by the additional and detailed questions about added fats and margarines in the questionnaire, which are an important source of LA in the population.
The validity of an FFQ is specific to the FFQ and to the study population it is administered to. In general, validation studies show obvious differences in validity across FFQ and also across all types of fatty acids. There is no indication that one particular fatty acid is commonly better captured by FFQ as compared with another fatty acid. This implies that we cannot generalise the validity of one FFQ to another, but each FFQ needs to be validated separately for its ability to measure fatty acids.
To conclude, the FFQ used in EPIC-NL showed moderate to good reproducibility for the assessment of intakes of specific fatty acid classes and individual fatty acids. Furthermore, for the fatty acids that are highly abundant in the Dutch diet, this FFQ is an adequate tool to rank people according to their intakes. Relative validity was less for intakes of low-abundant fatty acids including short-chain SFA, AA and marine n-3 PUFA.
Acknowledgements
The EPIC-NL study was funded by ‘Europe against Cancer’ Programme of the European Commission; the Dutch Ministry of Health, Welfare and Sports; The Netherlands Organization for Health Research and Development; and the World Cancer Research Fund.
The authors contributions to this manuscript were as follows: J. W. J. B. and Y. T. v. d. S. designed the study; J. P. and A. P. J. A. conducted the research and analysed the data; J. P., A. P. J. A., I. S., Y. T. v. d. S. and J. W. J. B. interpreted the data; J. P. and A. P. J. A. drafted the paper; C. T. M. v. R., I. S. and Y. T. v. d. S. and J. B. critically revised the paper for intellectual content and provided final approval of the manuscript.
J. P. is financially supported by a restricted research grant from Unilever Research and Development, Vlaardingen, The Netherlands. The other authors report no conflicts of interest.
Supplementary Material
For supplementary material/s referred to in this article, please visit http://dx.doi.org/doi:10.1017/S000711451600132X