Chronic non-communicable diseases are the global health problem of greatest magnitude and represent 58 % of all causes of death. The total burden of these diseases falls mainly on low- and middle-income countries( Reference Alwan, Amstrong and Cowan 1 , Reference Alwan, Maclean and Riley 2 ). In 2008, about 65 % of deaths in Ecuador were attributable to chronic non-communicable diseases (including diabetes, cancer, cardiovascular and respiratory diseases)( Reference Alwan, Amstrong and Cowan 1 ). According to the Ecuadorian National Health and Nutrition Survey carried out in 2012, metabolic syndrome was present in more than 35 % of the adult population aged 20 to 59 years( Reference Freire, Ramírez-Luzuriaga and Belmont 3 ). Evidence shows that at least 40 % of all deaths in individuals affected by these chronic conditions each year are attributed to the consumption of foods with high contents of saturated and trans fats, Na and sugar( Reference Beaglehole, Bonita and Horton 4 ).
Previous epidemiological studies have shown consistent associations between usual dietary intake and the occurrence of these diseases( Reference Millen and Quatromoni 5 , Reference Nelson 6 ). Several methods such as food records, multiple 24 h recalls and FFQ are used to measure dietary intake. Food records and multiple 24 h recalls are accurate methods for assessing food intake but the cost, time, motivation and literacy of participants are important factors that can limit the use of these instruments in large epidemiological studies. Therefore, the FFQ has been the most widely used method because of its practicality, low cost and ability to report usual intake, defined as ‘the long run average of daily intakes of a dietary component by an individual’( Reference Nusser, Carriquirry and Dodd 7 ), which is the main interest in studies on the effects of diet on long-term health( Reference Gibson 8 – Reference Dehghan, Martinez and Zhang 12 ).
As with all dietary assessment methods, the FFQ also has some limitations. An FFQ developed for a given population cannot be readily used in other populations because demographic, socioeconomic, cultural and other differences influence the food intake of each population, so that incorrect estimates of exposure can lead to false associations( Reference Willett 9 , Reference Thompson and Subar 10 , Reference Cade, Thompson and Burley 13 ). Consequently, accuracy and precision in food intake estimates need to be evaluated by studying the validity and reliability of the FFQ in each population. There is no gold standard to develop or validate an FFQ, but multiple 24 h recalls are used as a reference method by 75 % of validation studies( Reference Cade, Thompson and Burley 13 ).
Although several validated questionnaires that have been adapted for cultural variations are currently in use in Latin America( Reference Dehghan, Martinez and Zhang 12 , Reference Chinnock 14 – Reference Dehghan, del Cerro and Zhang 18 ), to our knowledge, no FFQ had previously been developed and validated for the Ecuadorian population. Therefore, the goal of the present study was to assess the reliability and validity of an FFQ designed to evaluate the usual nutrient intake of the adult population of the Metropolitan District of Quito, Ecuador, during the past year.
Methods
The study was conducted in Quito, Ecuador between November 2011 and August 2012 and comprised two phases: (i) the development of a semi-quantitative FFQ; and (ii) assessment of the reliability and validity of the FFQ. Initially, the FFQ food list was compiled from information based on 24 h dietary recalls (24hR). For the reliability and validity study, we administered three 24hR, which is considered to be the standard method, and two FFQ. The study design is shown in Fig. 1.
FFQ development
The FFQ was developed based on 24hR administered to a convenience sample of 250 adults of both sexes of high, medium or low socio-economic status and living in Quito. Of the 250 participants, 134 (53·6 %) were males and 116 (46·4 %) were females, with a mean age of 36·6 (sd 14·3) years. The interviews were performed in different locations in the city of Quito (parks, shopping centres, mass gathering places, markets, etc.). The locations were considered proxies for socio-economic level. In order to administer the 24hR, we developed a specific protocol with an instruction manual and conducted intensive interviewer training on the detailed collection of dietary intake data using a photographic album of food servings and eating utensils( 19 , Reference Zaboto, Viviana and Gil 20 ). The 24hR were administered from Monday to Saturday. The FFQ list included all food items that had appeared with a frequency of 5 % or more in the 24hR and those with the highest percentage of relative contribution for the following nutrients: carbohydrate, protein, total fat, vitamin C, vitamin A, Ca, Fe, K and Na. The relative contribution of each nutrient (e.g. vitamin A) consumed by the population was estimated by the weighted sum of that nutrient in all servings of all foods reported( Reference Block, Dresser and Hartman 21 ). The proportional contribution provided by a particular food i is determined by:
Thus, we selected food items that were responsible for approximately 90 % of the total intake of energy and each selected nutrient.
In addition to the foods recorded in the original 24hR, three traditional items that are commonly consumed in Quito were added (hominy, corn and fried pork), as well as two processed foods that are an increasingly prominent part of the diet (pizza and hamburger), and beef liver because it is rich in Fe, protein and vitamin A. In all, 110 food items were included. In order to determine the usual serving of each food item, we used the average of servings (in grams) reported in the 24hR. After conducting a pilot study (n 35), we noted that it was necessary to add five other foods (beets, string beans, oat soup, coffee and broad beans) and remove four (mango, cabbage, Swiss chard and giblets). Accordingly, the final version of the FFQ included 111 food items divided into eleven groups: (i) dairy foods; (ii) fruit; (iii) greens and vegetables; (iv) rice and tubers including potatoes; (v) legumes and eggs; (vi) meat and seafood; (vii) soups; (viii) cereals and flours; (ix) drinks; (x) sweets and desserts; and (xi) miscellaneous or others. Frequency of consumption was based on the FFQ published by Sichieri and Everhart, with eight response options ranging from ‘never or almost never’ to ‘more than three times/day’( Reference Sichieri and Everhart 22 ). The final version of the FFQ-Quito is presented in the online supplementary material.
Reliability and validity study
Sample
One hundred and twenty adults between 20 and 65 years of age who were employees of a university in Quito were selected and recruited for participation in the reliability and validity study. The sample size calculation was based on the guidelines of Burley et al., which recommend a sample of fifty to 100 people for this type of study( Reference Burley, Cade and Margetts 23 ). Sampling was random and took into account the size ratio of each job category (support staff, technical staff and faculty) as a proxy for socio-economic status. Individuals under medical or nutritional treatment during the previous month and pregnant women were excluded from the study.
FFQ reliability
The reliability of the tool was determined using the test–retest methodology. The study participants responded to the FFQ at baseline (FFQ1) and at the end of the study (FFQ2), with an average interval of 30 to 40 d. For the purpose of reliability, the same interviewer conducted both interviews.
FFQ validity
The average consumption of foods and nutrients reported in three non-consecutive 24hR was used as a reference dietary intake. The dietary surveys were administered by trained interviewers from the university nutrition course, starting with the FFQ and then the 24hR( Reference Cade, Thompson and Burley 13 ). To ensure the quality of the data, we developed an explanatory manual with specific protocols for standardizing the interviews and ensuring the correct completion of the FFQ and the 24hR. In the introduction of the FFQ and for each food group, the interviewer repeated that answers should be based on the previous 12 months. Next, the list of food items was read aloud and respondents were asked how many times the consumption of each item occurred per day, week or month and how many servings were consumed each time.
The 24R were collected over a period of 4 months with an interval of 15 to 20 d between each interview. We used a standard form for administering the 24hR and followed the multiple-pass approach for its completion( Reference Moshfegh, Rhodes and Baer 24 ). In each interview, the participant was encouraged to thoroughly report all foods and beverages consumed in the last 24 h, as well as their respective amounts, sizes, number of servings and preparation methods. A photographic food atlas was used to reduce inaccuracies in reporting the size or volume of food consumed( 19 , Reference Zaboto, Viviana and Gil 20 ). In order to include the weekend food consumption, which in many parts of the world is likely to be different from that during the working week, one of the interviews was conducted on a Monday.
Estimates of nutritional composition
In order to estimate the nutritional composition of the food items included in the FFQ and of those food items reported in the 24hR, we used the Ecuadorian National Health and Nutrition Survey food composition table( Reference Ramírez-Luzuriaga, Silva-Jaramillo and Belmont 25 ), the US Department of Agriculture food composition and nutrient database release 25( 26 ) and information from commercial food labels. The ADSnutri software( 27 ) was used to quantify the nutritional content from the 24hR. In order to determine the nutrients from the FFQ, a syntax was developed in the PASW statistical software package version 18, based on the following calculation: quantity of servings consumed per time×weight (in grams)×frequency of consumption×nutritional composition of the food serving. The coefficients used for the daily frequency equivalents were the following: 3 for >3 times/d; 2 for 2–3 times/d; 1 for 1 time/d; 0·79 for 5–6 times/week; 0·43 for 2–4 times/week; 0·14 for 1 time/week; 0·07 for 1–3 times/month; and 0 for never/almost never.
Statistical analyses
For all nutrients examined, normality was verified and the means and standard deviations were estimated. For those nutrients not showing a normal distribution, logarithmic transformation was performed; in particular, this procedure was necessary for Ca and cholesterol in the reliability analyses. The differences and ratios between the mean values obtained with the FFQ and the 24hR were calculated. Paired t tests were used to determine statistically significant differences between means. Additionally, beanplots of centesimal values of energy and nutrients from the FFQ and 24hR were built. This procedure combines a one-dimensional scatter plot with an estimated density curve( Reference Camey, Nunes and Cruz 28 ).
For reliability analyses, we used the intra-class correlation coefficient (ICC) per point and 95 % CI; the model that was used specifies the random effect for individuals. The validity of the FFQ was assessed by comparing the nutrient intake estimates using the average of the two FFQ and the average for the three days of the 24hR using the Pearson correlation coefficient (r). In order to correct the estimates of nutrients for total energy intake, an adjustment by energy was made using the method proposed by Willett et al.: the correction by energy was made by computing the residuals of the regression model, with energy intake as an independent variable and nutrient intake as a dependent variable( Reference Willett, Howe and Kushi 29 ). Due to the occurrence of within-subject variability in food intake, the correlation coefficients were corrected by the ratio of the within- to between-subject variances in the three 24hR using the following formula:
where r v is the true correlation, r o is the observed correlation between the average of the FFQ and the average of the 24hR, λ is the ratio of within- to between-subject variances in the 24hR and n is the number of replicates; in this case, three recalls( Reference Beaton, Milner and McGuire 30 , Reference Liu, Stamler and Dyer 31 ). The components of the variance within and between subjects were established by conducting ANOVA.
The agreement between each FFQ and the average of the three 24hR was also evaluated by classifying participants according to their distributions into quartiles of energy and other nutrient intakes for each method. The percentages of exact agreement (classification in the same quartile) and disagreement (classification in opposite quartiles) were estimated. For this analysis, the classification into quartiles was compared by using the weighted kappa statistic (κ w).
In order to evaluate the level of discrepancy between energy and nutrient values calculated by both methods (FFQ and 24hR), we built scatter plots with the mean intake values obtained by the two methods on the x-axis and the absolute differences in intake (bias) between the two methods on the y-axis according to the Bland–Altman method( Reference Bland and Altman 32 , Reference Bland and Altman 33 ). For the scatter plots, we calculated the 95 % limits of agreement (LOA), which were obtained as: $\bar{d}\,\pm\,1\!\!\cdot\! 96{\times}SD$ , where $\bar{d}$ is the bias and SD is the standard deviation. These analyses were performed using the R statistical program version 2·15·2 and the PASW statistical software package version 18.
Ethical considerations
This research project was approved by the Ethics Committees at the Universidad San Francisco de Quito (Ecuador) and at the Universidade Federal do Rio Grande do Sul (Brazil). Survey participants were informed about the purposes of the study and gave their consent by signing the informed consent form.
Results
Of the 120 individuals selected, twelve (10·0 %) refused to participate in the study and ten (8·3 %) did not meet the inclusion criteria. Among the individuals who refused to participate in the study, 60 % were male, with a mean age of 42 years, and occupied positions as support staff, technical staff and faculty members. Among the participants (n 98), 97 % completed the validation and reliability study. Thus, the final sample comprised ninety-five adults, with a mean age of 40 (sd 11·2) years, of whom 64·2 % were females. According to job categories, faculty members represented forty-seven participants (twenty males and twenty-seven females) with a mean age of 39 (sd 12·0) years, technical staff comprised thirty-two participants (seven males and twenty-five females) with a mean age 38·8 (sd 11·2) years and sixteen participants were support staff (seven males and nine females) with a mean age 46·3 (sd 10·5) years.
Table 1 shows the mean values and standard deviations for energy and nutrients obtained from FFQ1 and FFQ2. The average intakes of energy and all nutrients analysed were lower in FFQ2 compared with FFQ1. Table 1 also shows the crude and adjusted ICC values for energy and nutrients measured by the FFQ on both occasions. ICC values adjusted for energy ranged from 0·62 (protein) to 0·88 (Ca).
* Log-transformed nutrients.
† Retinol equivalents.
Figure 2 reveals that there were differences between the daily energy and nutrient distribution profiles obtained by the average of both FFQ and the average of the three 24hR. Based on the FFQ, the energy distribution showed a higher mean, greater amplitude, more extreme minimum and maximum consumption values, as well as a concentration of intake between 6276 kJ and 18 828 kJ (1500 kcal and 4500 kcal). On the other hand, energy dispersion was smaller in the 24hR, with a concentration of intake of between 5439 kJ and 12 552 kJ (1300 kcal and 3000 kcal). The same tendency was observed in all nutrients studied, in that there was a higher mean and a greater amplitude of the density and dispersion curve for the FFQ data compared with the 24hR data.
Table 2 shows the values for mean intake of energy and each nutrient estimated by the average of both FFQ and the three 24hR, as well as the variance ratios of the 24hR. Intakes of energy and other nutrients measured by the FFQ were higher than intakes measured by the reference method (24hR). Also, the within-subject variance was greater than the between-subject variance for most nutrients. Table 2 also shows the crude, adjusted and de-attenuated Pearson correlation coefficients (r) for within-subject variance between the two dietary methods (FFQ and 24hR). For energy, de-attenuated r was 0·55 and the values for macronutrients ranged from 0·21 for total fat to 0·42 for carbohydrate. De-attenuated correlations for micronutrients ranged from 0·33 for Fe to 0·65 for Ca. When evaluating FFQ1 and FFQ2 separately, it was observed that the average from the FFQ showed higher correlations for most nutrients as compared with the average from the three 24hR (data not shown).
* Between-subjects/within-subject.
† Retinol equivalents.
‡ Mean values were significantly different from those estimated from 24hR: P<0·05 (paired t-test).
By analysing the percentage of exact agreement between quartiles of energy and nutrient intakes as measured by the FFQ and 24hR, it was possible to observe values between 52·7 % and 27·4 % for Ca and total fat, respectively. The average percentage of disagreement between the methods was 4·1 %, ranging from 0 % for Ca to 7·4 % for fat. The κ w values ranged from 0·25 for total fat to 0·62 for Ca. For the nutrients evaluated, 46 % had a κ w value greater than 0·40 (Table 3).
* Retinol equivalents.
Table 4 shows the mean differences between the methods with the respective upper and lower 95 % LOA. The mean energy difference was 2696·13 kJ (upper/lower LOA=8173·40 kJ, 2781·15 kJ). For carbohydrate the observed difference was approximately 100 g (upper/lower LOA=295·88 g, −94·12 g). Protein and total fat showed a similar mean intake differences between the methods: 29·17 g (upper/lower LOA=35·29 g, 23·05 g) for protein and the same 29·17 g (upper/lower LOA=88·91 g, −30·57 g) for total fat.
* Retinol equivalents.
† All mean differences (biases) between the methods were significant: P<0·05 (paired t test).
Figure 3 provides scatter plots (Bland–Atman) of the differences in intake between the two methods (FFQ and 24hR) v. the mean intakes of the two methods for energy and macronutrients. Comparisons of energy and protein demonstrated greater variability, with a tendency for bias to increase according to the increase in average energy intake measured by 24hR and energy intake measured by FFQ. This indicates that higher energy and protein intake enhanced disagreement between the methods. With respect to carbohydrate, there was a mean difference of 100·88g between methods, which was constant for all levels of intake. In contrast, the scatter plot for total fat showed a continuous bias, although it was smaller and similar for all levels of intake.
Discussion
The present study developed an FFQ for the adult population of Quito, Ecuador, and evaluated its reliability and relative validity. The results show that the tool provides reliable measurements and satisfactory relative validity for most macro- and micronutrients.
It is worth noting that the present study is the first one that proposes a validated FFQ for Ecuador. The findings are based on a rigorous study design with regard to selection of participants, sample size, instrument design, validation and statistical analyses. Standardization of data collection included training and certification of interviewers and the use of food photograph albums to carry out the 24hR and FFQ. Collection of 24hR data included consumption on Sundays because food intake on that day is often significantly higher than on the other days of the week( Reference Beaton, Milner and McGuire 30 ). Some limitations should also be noted. Both FFQ and 24hR rely on the ability of respondents to accurately recall the past, so that some degree of measurement errors due to under- or over-reporting of consumption is inevitable. Nevertheless, the process of data collection was designed to optimize the participation in order to reduce measurement errors and follow-up losses.
With regard to reliability, we used relative comparison techniques such as the ICC, which allows evaluation of the exact agreement between the variables analysed without presuming a linear relationship as in the Pearson correlation coefficient( Reference Molina MDCB and Cardoso L de 34 ). In the present study, the adjusted, uncorrected correlation coefficients for energy and nutrients were always greater than 0·77 and although the adjustment for energy reduced the correlations, the adjusted and de-attenuated values were within acceptable range, with the lowest value of 0·62 for protein and the highest value of 0·88 for Ca. Similar ICC were reported by other reliability studies( Reference Molina MDCB and Cardoso L de 34 , Reference Jackson, Motswagole and Kwape 35 ).
Concerning the validation study, it was observed that for most of the nutrients considered, the within-subject variance was greater than the variance between subjects. There was a predominance of females in the study and it has been reported that generally females have higher ratios of within- to between-subject variance than males( Reference Beaton, Milner and McGuire 30 , Reference Nelson, Black and Morris 36 , Reference Lora, Lewis and Eskridge 37 ). The average intake values for energy and macro- and micronutrients were higher in the FFQ compared with the intake averages established in the 24hR. This finding is consistent with those of other studies, which show overestimation in FFQ when compared with other methods such as recalls and food records( Reference Dehghan, Martinez and Zhang 12 , Reference Hernández-Aila, Romiu and Parra 16 , Reference Dehghan, del Cerro and Zhang 18 , Reference Molina MDCB and Cardoso L de 34 , Reference Jackson, Motswagole and Kwape 35 , Reference Zanolla, Olinto and Henn 38 – Reference Block, Wakimoto and Jensen 40 ). The study also found that the adjustment for energy reduced the correlation values for total fat, carbohydrate, protein, cholesterol and to a lesser degree Na; these results are also similar to those of other studies( Reference Hernández-Aila, Romiu and Parra 16 , Reference Dehghan, del Cerro and Zhang 18 , Reference Zanolla, Olinto and Henn 38 , Reference Block, Wakimoto and Jensen 40 – Reference Barrat, Aubineau and Maillot 43 ). According to Willett( Reference Willett 9 ), the adjustment for total energy increases the correlation coefficient when variability in nutrient intake is related to energy intake, but produces a decrease when the nutrient variability is due to systematic errors in under- or over-reporting of food intake. Given the results of our study, we cannot rule out that these errors are present in the reports of both methods. Over the years, researchers have recognized that intake values measured by FFQ are subject to both systematic and random errors( Reference Liu 44 ).
Analysis of the Bland–Altman plots demonstrates the degree to which the FFQ is inconsistent with the reference method, thus demonstrating a constant bias with a tendency to increase according to the level of intake. The 95 % LOA were very high for Na (upper/lower LOA=1554·29 mg, −1231·45 mg) and vitamin A (upper/lower LOA=1423·88 µg, −229·34 µg). This finding is also consistent with those of other studies( Reference Dehghan, Martinez and Zhang 12 , Reference Sam, Skeaff and Skidmore 45 ), indicating that wide LOA between FFQ and 24hR are common. This conclusion highlights the difficulties in using the FFQ to estimate absolute intakes of some nutrients.
With regard to further use of the tool discussed here, the FFQ-Quito accurately determines nutrients considered as protective or risk markers for chronic non-communicable diseases. In the context of the epidemiological and nutrition transitions, this point is increasingly important because taking new patterns of food consumption into consideration allows for more informed development of strategies and nutritional recommendations aimed at improving the health status of the population. The FFQ-Quito showed reasonable agreement for fibre (r=0·53), Ca (r=0·65), vitamin C (r=0·62) and K (r=0·62); these markers are considered protective for metabolic syndrome, diabetes, CVD and cancer( Reference Aune, Lau and Chan 46 ). Likewise, for non-communicable disease risk markers, we also found a reasonable agreement between the methods for nutrients such as saturated fat (r=0·48) and cholesterol (r=0·45). In nutritional epidemiology, one of the main requirements when using the FFQ to estimate dietary intake is to analyse the diet–disease relationship. Accordingly, more important than correctly estimating food intake is the ability to correctly classify individuals according to intake levels( Reference Willett 9 ). The results of the present study indicate that the FFQ-Quito fills a real need since it has a low average percentage (4·1 %) of disagreement between intake quartiles according to the different methods.
Conclusion
In conclusion, our results suggest that the FFQ developed for Quito, Ecuador, can be used to assess usual nutrient intake since it provides reliable measures for classifying individuals by their nutrient intakes and has reasonable relative validity, especially for protective and risk markers of chronic non-communicable diseases. To our knowledge, this is the first FFQ designed to assess food and nutrient intakes in Quito and will be a useful tool in future research, particularly in studies on the relationship between diet and chronic diseases. This tool also provides a valuable contribution for policy makers since it may be useful in evaluating interventions or policies aimed at promoting healthy eating in this population.
Acknowledgements
Acknowledgements: The authors are grateful to the participants of the study. They also thank the Nutrition Department/Public Health School from the Universidad San Francisco de Quito for its support. Financial support: This study was supported by the Brazilian National Council for Research and Development (CNPq; grant number 190638/2010-9). CNPq had no role in the design, analysis or writing of this article. Conflict of interest: None. Authorship: K.M.S.-J., M.B.N. and M.D. contributed to concept development, conducted the analysis and interpreted the data. K.M.S.-J. was responsible for execution of the study and processing the data, and drafted the manuscript. All authors critically reviewed the manuscript and have responsibility for the final content. Ethics of human subject participation: This research project was approved by the Ethics Committees at the Universidad San Francisco de Quito (Ecuador) and at the Universidade Federal do Rio Grande do Sul (Brazil).
Supplementary material
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1368980014003346