An adequate diet is essential for optimal well-being and development during childhood and for the prevention of disease( 1 , Reference Moreno, Bel-Serrat and Santaliestra-Pasias 2 ). Therefore, accurate assessment of children’s dietary intake has become important for monitoring the nutritional status of the population, as well as for conducting interventional and epidemiological research on the link between diet and health among children.
Dietary assessment in young children is particularly challenging, and it is important to develop suitable methods for this age group. Below the age of 8 years, children are not capable of reporting their own dietary intake due to limited cognitive and literacy levels. Therefore, a proxy reporter, typically the parent, needs to be involved, which poses additional challenges( Reference Goran 3 , Reference Livingstone and Robson 4 ).
The estimated food record (FR) has been recommended for dietary assessment among young children( Reference Andersen, Lioret and Brants 5 ). The use of FR is labour intensive, but enables the use of several proxy persons to provide accurate collection of dietary information, for example parents and day-care staff. In large studies the FFQ is the preferred method because administration is easy and costs are low. However, FFQ should be validated in the target population( Reference Cade, Thompson and Burley 6 ). The validity of FFQ is likely to be lower in younger children but the evidence is scarce( Reference Willett 7 ). Only a handful of studies have examined the relative validity of parent-administered FFQ among children below the age of 8 years( Reference Matos, Prado and Santos 8 – Reference Jarman, Fisk and Ntani 16 ) and very few have examined food group intakes( Reference Andersen, Lande and Trygg 11 , Reference Huybrechts, De Backer and De Bacquer 13 ). Furthermore, few studies include a reproducibility test( Reference Sahashi, Tsuji and Wada 10 , Reference Kobayashi, Kamimura and Imai 12 , Reference Huybrechts, De Backer and De Bacquer 13 , Reference Metcalf, Scragg and Sharpe 17 ). Reproducibility studies are difficult to design but are equally important( Reference Cade, Thompson and Burley 6 ).
As a part of the Danish community intervention study, ‘Health and Local Community’ (SoL-Bornholm), a questionnaire to assess food intake in 3- to 8-year-old children was developed. Currently, Danish FFQ for adults exist but no (published) Danish FFQ has been developed for young children. This is therefore, to our knowledge, the first Danish FFQ to be validated for this age group.
The purpose of the present study was to assess the relative validity and reproducibility of the semi-quantitative FFQ (SFFQ) for its application in the evaluation of the community intervention study, SoL-Bornholm, in Denmark. The assessment gave priority to food groups relevant to the SoL intervention study (listed below).
Methods
Participants and design
In September 2012 the community-based health promotion intervention study, SoL-Bornholm, was initiated. The main purpose of the intervention study was to promote healthy eating and physical activity among children. Important success criteria were increased intakes of vegetables, fruits, fish and whole grains, and decreased intakes of soft drinks and candy. In order to assess the food intake among participating children, a parent-administered FFQ was developed for the purpose and tested during the year 2013.
A convenience sample was recruited through postings on school intranets, in day-care centres in the SoL-Bornholm communities and in neighbourhoods in the Capital Region of Denmark. Recruitment from the Capital was necessary since very few families from the intervention setting were able to be recruited in this way. The majority of the final sample originated from the Capital. The sample was, however, comparable to the actual final users with regard to age and food culture. Children aged 3–9 years were included. Before the start of the study, parents were invited to a group meeting where they were given detailed instruction by a trained nutritionist on how to fill out the FR. The instruction used examples of child meals and how to estimate portion sizes. Throughout the study, the parents received reminders by email or text message, and telephone support if necessary. The study was carried out at the Research Centre for Prevention and Health.
The FR and SFFQ were entered by a nutritionist, following standardized procedures. All records and questionnaires were screened for completeness and parents were contacted to provide clarifications in the case of missing data or misunderstandings.
Semi-quantitative FFQ
A parent-administered SFFQ was developed based on an SFFQ previously used and validated in a nationwide survey among children in Norway( Reference Andersen, Lande and Trygg 11 ). The SFFQ was translated and modified to the Danish language. It was also adapted to Danish food culture as well as to the target age group using data from national Danish food intake surveys among children( Reference Fagt, Christensen and Groth 18 ).
The nineteen-page questionnaire included twenty-two questions covering 183 food items grouped together according to the Danish meal pattern (Western breakfast, open sandwich for lunch, hot meal for dinner). The recall period was the past 4 weeks, and the frequency alternatives varied from ‘never/less than once per month’ to ‘five or more times a day’. A photographic booklet including sixteen series of colour photographs with four different sizes of portions of meals or food items, ranging from small (A) to large (D), was used by the parents when reporting the amounts of food eaten by the child. The photographic booklet originally included six differently sized portions (A–F) and was developed to assist food reporting among children and adults aged 4–75 years( Reference Nielsen, Biltoft-Jensen and Ygil 19 ). The photographs have been validated among 622 adults and 109 children aged 8–12 years( Reference Nielsen, Biltoft-Jensen and Ygil 19 ). When no photograph was available for a food item, household units were used, e.g. slices, pieces and spoons. The questionnaire consisted of seven sections: (i) introductory questions (height, weight, birth date); (ii) breakfast; (iii) bread (including toppings on bread); (iv) fruits; (v) hot meals (including vegetables); (vi) snacks/sweets; and (vii) beverages. Summary questions about entire food groups were added to the questionnaire to prevent misreporting. A summary question was, for example, ‘How often and how many pieces of fruit does your child eat?’
A one-page information letter was mailed to parents together with the questionnaire. This letter contained brief, but detailed information on how to fill out the questionnaire. It also encouraged parents to seek information on the child’s intake of foods and beverages when meals were eaten outside the home.
Approximately one month after completion of the SFFQ, the parents were asked to fill out the SFFQ once again in order to test its reproducibility. This interval was chosen to avoid seasonal variation. For convenience, the first administration of the SFFQ is called SFFQ1 and the second administration is referred to as SFFQ2.
Estimated food record (reference method)
The estimated FR was chosen as the reference method. This type of FR is characterized by being less time-consuming than the weighed FR because participants are asked to use household units or photographs instead of weighing all foods and meals. The FR covered four consecutive days, including three weekdays and one day during the weekend (Wednesday to Saturday). The parents were asked to register in detail all foods and beverages consumed by their child using photographs of portion sizes (the same picture booklet as described in the preceding section) or household units. Besides food amounts, the FR contained sections for registering time and location of meals, the food preparation method, fat percentage and food brand. The FR were filled out by parents shortly after the first administration of the SFFQ.
Nutrient calculations
Daily individual consumption was calculated from the SFFQ and FR. For the SFFQ, monthly frequencies were divided by 30·4375, and weekly frequencies by 7, to calculate the daily intakes in g/person per d. For the FR, daily intakes were calculated as the mean of the recorded days. Intake in grams was calculated by use of standard weights of food items and portion sizes( Reference Nielsen, Biltoft-Jensen and Ygil 19 , Reference Ygil 20 ). Estimation of the individual daily energy, nutrient and food intakes was calculated using the Danish Food Composition Databank version 7( Reference Saxholt, Christensen and Møller 21 ) and the software program FoodCalc version 1·3( Reference Lauritsen 22 ). Total energy intake was calculated according to the Nordic Nutrition Recommendations( 23 ) with fat contributing 37 kJ/g, available carbohydrates and protein contributing 17 kJ/g and dietary fibre contributing 8 kJ/g.
Furthermore, the intakes of specific foods and food groups were calculated. Eight food groups were developed specifically to be used for the evaluation of the SoL-Bornholm intervention study. The grouping of foods into food groups was based on the standards developed by the National Food Institute, Technical University of Denmark( Reference Saxholt, Christensen and Møller 21 ). These food groups were then modified and supplemented to be more comparable with the Danish dietary guidelines( 24 ) and to suit the SoL-Bornholm intervention study. The adaptations were as follows: (i) for the fruit group, removing olives, jams, juices, dried fruit and nuts; (ii) for the vegetable group, removing juices, ketchup, potatoes, soya drinks and mushrooms; (iii) creating the following new food groups relevant for the SoL-Bornholm intervention study: coarse vegetables, canned fish, sweet drinks, candy and whole grains. The vegetable group served as a measure of total vegetable intake, whereas the group named coarse vegetables included only vegetables with high fibre content (above 2 g/100 g) such as cabbage, root vegetables and onions. Potatoes were excluded from the vegetable group because they do not count as a vegetable in the Danish dietary guidelines. Canned fish included all fish sold in cans or similar packaging, and was included because these fish products were a main intervention focus for supermarkets. The whole grains group included rye bread and oats as a marker of whole grain intake. For convenience, this food group is called whole grains in the following sections. Sweet drinks included all non-alcoholic drinks with natural or added sugars (e.g. juice, chocolate milk and lemonade), not including drinks with artificial sweeteners. The candy group included chocolates, liquorice, fruit gums and similar.
A weighting factor was calculated for each of the individual fifty-four children using SFFQ data and the summary questions( Reference Calvert, Cade and Barrett 25 ) for the following food categories: cold meals (open sandwich); fruit; vegetables; and hot meals. The answers from the summary questions were divided by the sum of frequencies or amounts from the FFQ, for each food group, as shown below:
$${{{\rm Value \;from\; summary \;question}} \over {{\rm Sum \;of \;values \;from \;individual \;food \;items \;in \;the \;SFFQ}}}.$$
For example, the weighting factor was 0·92 for one of the children’s fruit intake. The calculation was based on the calculated intake from the summary question (2 portions of fruit, 6–7 times/week), which was 185·7 g, and the sum of the reported fruit intake from question 15 about intake of different types of fruit, which was 201·55 g. When applying the weighting factor the adjusted fruit intake in grams per day was therefore 10 % smaller.
Previous studies have shown that applying weighting factors often leads to better estimates of nutrient and food intakes( Reference Calvert, Cade and Barrett 25 , Reference Nothlings, Hoffmann and Boeing 26 ).
Statistical analyses
All statistical analyses were performed using the SAS statistical software package version 9·4. Descriptive statistics were used to describe baseline characteristics and average intakes of foods, energy and nutrients. Most of the intake data were non-normally distributed both before and after log-transformation, and therefore the mean was presented along with the median and the 5th and 95th percentile.
The Bland and Altman limits of agreement (LOA) method was used to assess agreement (or bias) of the two administrations of the SFFQ and between the two methods( Reference Bland and Altman 27 ). This method was used to visually inspect agreement using plots of the mean difference and the 95 % LOA. Mean difference was calculated as the mean of all individual differences between methods (or repeated measures); that is, mean (SFFQ1 – SFFQ2)/n or mean (SFFQ – FR)/n, where n=2. The 95 % LOA was calculated for each food or nutrient intake as the mean difference ±1·96 sd of differences. This measure represents a range of values within which 95 % of all differences between methods are expected to fall. Bland–Altman plots were drawn using the differences between the two measurements plotted against the means of the measurements( Reference Bland and Altman 27 ). The middle line of the plot represents the mean difference and the two outer lines represent the 95 % LOA.
A cross-classification analysis was undertaken to calculate the percentage of observations classified into the same or opposite quartile of intake (referred to as gross misclassification in the following)( Reference Masson, McNeill and Tomany 28 ). This was done for testing the relative validity as well as reproducibility. The linear association between the two dietary methods and between the first and second administration of the SFFQ was described using Spearman’s rank correlation coefficients( Reference Masson, McNeill and Tomany 28 ). In addition, intra-class correlation coefficients were calculated to assess reproducibility. For the assessment of the relative validity, correlations were corrected for the day-to-day within-person variation using the de-attenuation method( Reference Willett 7 ). The corrected correlation, r c, was calculated as follows:
$$r_{{\mathop{\rm c}\nolimits} } =r_{{\mathop{\rm o}\nolimits} } \sqrt {{{\left[ {1{\plus}\left( {{{S_{{\mathop{\rm w}\nolimits} }^{2} } \mathord{\left/ {\vphantom {{S_{{\mathop{\rm w}\nolimits} }^{2} } {S_{{\mathop{\rm b}\nolimits} }^{2} }}} \right. \kern-\nulldelimiterspace} {S_{{\mathop{\rm b}\nolimits} }^{2} }}} \right)} \right]} \mathord{\left/ {\vphantom {{\left[ {1{\plus}\left( {{{S_{{\mathop{\rm w}\nolimits} }^{2} } \mathord{\left/ {\vphantom {{S_{{\mathop{\rm w}\nolimits} }^{2} } {S_{{\mathop{\rm b}\nolimits} }^{2} }}} \right. \kern-\nulldelimiterspace} {S_{{\mathop{\rm b}\nolimits} }^{2} }}} \right)} \right]} n}} \right. \kern-\nulldelimiterspace} n}} ,$$
where r
o is the observed correlation,
$${{S_{{\mathop{\rm w}\nolimits} }^{2} } \mathord{\left/ {\vphantom {{S_{{\mathop{\rm w}\nolimits} }^{2} } {S_{{\mathop{\rm b}\nolimits} }^{2} }}} \right. \kern-\nulldelimiterspace} {S_{{\mathop{\rm b}\nolimits} }^{2} }}$$
is the ratio of the within- and between-person variances and n is the number of replicates per person for the given variable. Within-person variation and between-person variation were calculated from replicated FR using the Proc Mixed model in SAS. Additionally, Spearman’s rank correlation coefficients were adjusted for energy and person-specific factors. The residual model was used for energy adjustment of correlations between the FR and the first administration of the SFFQ(
Reference Willett and Stampfer
29
). Person-specific factors were defined as the child’s age category (pre-school or school) and sex. These covariates were included in a model, and Spearman’s partial rank-order correlation coefficients were used for analyses of relative validity and reproducibility.
Results
Fifty-four children were enrolled in the study. Fifty-four completed SFFQ1 (100 %), forty-eight completed SFFQ2 (89 %) and fifty-three completed FR (98 %) were available. One participant had only three fully recorded days in the FR, but of satisfactory quality, and was therefore included in the analysis. The rest were of satisfactory quality and none were considered to have recorded implausible intakes. None of the children had energy intakes below or above the thresholds suggested for quality check by Willett and Stampfer( Reference Willett and Stampfer 29 ). The characteristics of the children are presented in Table 1.
Table 1 Descriptive characteristics of the children included in the study (n 54), Capital Region and the Regional Municipality of Bornholm, Denmark, 2013
* Self-reported by parents.
Relative validity of the semi-quantitative FFQ
Food, energy and nutrient intakes from the SFFQ and the FR are shown in Table 2. The mean difference between methods varied among intakes, but in general the intakes were overestimated when compared with the reference (Table 2). Vegetable intakes were particularly overestimated by the SFFQ (151 g for total intake (118 %) and 73 g for coarse vegetables (189 %)). A few intakes were underestimated, particularly the sugar energy percentage. The lowest degree of over- or underestimation was observed for intake of whole grain foods (−3 g (3 %)). When using Bland–Altman plots to examine agreement between methods, it was observed that the mean difference increased with increasing intake for most intakes. For most intakes, the differences were both positive and negative, implying that the participants both under- and over-reported at high intakes. This was, however, not the case for vegetables and energy, where high intakes were mostly related to over-reporting (Fig. 1). When inspecting the 95 % LOA (the outer lines in the plot) we observed that the 95 % LOA generally were wide, indicating that agreement was better at the group level than at the individual level.
Fig. 1 Bland–Altman plot assessing the relative validity of the parent-administered semi-quantitative FFQ (SFFQ) for assessing daily vegetable intake in Danish children aged 3–9 years (n 53), Capital Region and the Regional Municipality of Bornholm, Denmark, 2013. The difference in intake between the first administration of the SFFQ (SFFQ1) and the average of the 4 d estimated food records (FR) is plotted v. the mean intake from the two methods; —— represents the mean difference and – – – – – represent the 95 % limits of agreement corresponding to ±1·96 sd
Table 2 Relative validity of daily intakes of foods, energy and nutrients estimated by the first administration of the parent-administered semi-quantitative FFQ (SFFQ1) and the average of the 4 d estimated food records (FR), and correlation between SFFQ1 and FR. A total of fifty-three Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in all analyses
P5, 5th percentile; P95, 95th percentile; 95 % CL, 95 % confidence limits.
*P<0·05, **P<0·01, ***P<0·001.
† {[Mean (SFFQ−FR)]/[mean (FR)]} ×100.
‡ Variance ratio is the ratio between within- and between-person variances. Spearman correlation coefficient was adjusted for within-person variance using the de-attenuation method.
§ Including rye bread and oats as a marker of whole grain intake.
|| Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.
¶ Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.
The proportion of children correctly classified into the same quartile ranged from 26 % (sweet drinks) to 49 % (coarse vegetables; Table 3). Gross misclassification into the opposite quartile was lowest for foods, on average 8 %. In general, the cross-classification analyses on energy intake and macronutrient energy percentages revealed slightly lower percentages for classification into the same quartile and higher percentages for gross misclassification (Table 3).
Table 3 Percentage of participants classified by the first administration of the parent-administered semi-quantitative FFQ (SFFQ1) into the same or opposite quartile of consumption as measured by the average of the 4 d estimated food records (FR) or the second administration of the SFFQ (SFFQ2). A total of fifty-three and forty-eight Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in analyses of SFFQ1 v. FR and SFFQ1 v. SFFQ2, respectively
† Including rye bread and oats as a marker of whole grain intake.
‡ Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.
§ Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.
All but two correlations were significant (Table 2). The highest degree of linear association was observed for intakes of whole grains and fish products (r=0·58–0·63), there was a less strong association for vegetables and fruit (r=0·40–0·46), and the lowest degree of association was observed for sweet drinks and candy (r=0·29–0·31). There was a tendency towards weaker correlations for energy and macronutrients and the correlation for energy was especially low (r=0·12). The ratio of within- and between-person variance measured from the 4 d FR was between 1·46 and 9·61, and the de-attenuated (corrected) correlation coefficients were therefore substantially higher than the crude correlations (Table 2). Adjustment for energy did not change the correlation coefficients considerably. The most important and largest change was observed for candy (crude value r=0·31, energy-adjusted value r=0·43). When correlations were adjusted for age category or sex, a small change was observed for correlation of energy intake (r=0·06 when adjusted for age category and r=0·08 when adjusted for sex), but otherwise no considerable changes were observed.
Reproducibility of the semi-quantitative FFQ
Food, energy and macronutrient intakes for the repeated SFFQ are shown in Table 4. Overall, intakes from SFFQ1 were slightly higher than those from SFFQ2 (Table 4). The intake of sweet drinks was especially higher for the first SFFQ (35 g (27 %)). Through visual inspection of the Bland–Altman plots, a tendency towards poorer agreement with high intakes was observed. This, however, was not the case for vegetables and whole grains. We also observed that the 95 % LOA (the outer lines in the plot) were wide, indicating that agreement was better at the group level than at the individual level.
Table 4 Reproducibility of daily intakes of foods, energy and nutrients estimated by the first and second administration of the parent-administered semi-quantitative FFQ (SFFQ1 and SFFQ2, respectively), and correlation between SFFQ1 and SFFQ2. A total of forty-eight Danish children aged 3–9 years, from the Capital Region and the Regional Municipality of Bornholm, were included in all analyses
P5, 5th percentile; P95, 95th percentile; 95 % CL, 95 % confidence limits; ICC, intra-class correlation coefficient.
*P<0·05, **P<0·01, ***P<0·001.
† {[Mean (SFFQ1−SFFQ2)]/[mean (SFFQ1)]} ×100.
‡ Including rye bread and oats as a marker of whole grain intake.
§ Including all non-alcoholic drinks with natural or added sugars, not including drinks with artificial sweeteners.
|| Intakes of macronutrients expressed as a percentage of energy: (macronutrient absolute value × energy conversion factor) ×100.
The cross-classification analysis indicated a low rate of gross misclassification into the opposite quartile for most intakes, on average 6 % for foods and 8 % for energy and nutrients (Table 3). The proportion of children correctly classified into the same quartile of intake was highest for intakes of whole grains (69 %) and lowest for candy and vegetables (33 % and 38 %, respectively; Table 3).
Intra-class correlation coefficients between the first and second administration of the SFFQ were on average 0·58 for foods and 0·64 for energy and nutrients, and all were statistically significant (Table 4). Spearman’s correlation coefficients were generally similar to the intra-class correlations (Table 4). The food group candy and chocolate deviated from the rest with a markedly lower correlation (r=0·30). Correlations for vegetable intakes were also somewhat lower than the majority (intra-class correlation coefficient=0·38; Spearman correlation coefficient=0·43). The strongest correlations for food groups were observed for fish and whole grains (intra-class correlation coefficient=0·81–0·82). When Spearman’s correlation coefficients were adjusted for age category or sex, no considerable changes were observed.
Discussion
The relative validity of the SFFQ, when compared with a 4 d estimated FR, varied across the different food groups and nutrients (Tables 2 and 3). Most intakes were overestimated, which is a similar finding to the majority of previous studies. Only two of the similar studies identified found an overall underestimation of intakes( Reference Kobayashi, Kamimura and Imai 12 , Reference Huybrechts, De Backer and De Bacquer 13 ), whereas seven observed an overestimation( Reference Matos, Prado and Santos 8 – Reference Sahashi, Tsuji and Wada 10 , Reference Del Pino and Friedman 14 – Reference Jarman, Fisk and Ntani 16 , Reference Watson, Collins and Sibbritt 30 ). Likewise, the validation study performed on the original Norwegian version of the SFFQ reported overestimation of intakes; however, these overestimations were not as large as those observed in the present study( Reference Andersen, Lande and Trygg 11 ). A review reported that FFQ overestimated total energy intake by 2–59 % when compared with the doubly labelled water method( Reference Burrows, Martin and Collins 31 ). Conversely, dietary assessment using FR is typically related to underestimation of total energy intakes (19–41 %)( Reference Burrows, Martin and Collins 31 ). When testing an FFQ using FR as the reference, it is therefore not surprising that we observed an overestimation of most intakes.
The results of the present validation study point towards an overestimation of the child’s vegetable intake in particular. This can perhaps be explained by the difficulty parents face when attempting to remember the child’s intake of vegetables or when estimating portion sizes. It has previously been observed that there is a low correlation between parental reports of children’s fruit and vegetable intake and corresponding serum micronutrients (r=0·14–0·29)( Reference McPherson, Hoelscher and Alexander 32 ). It is also possible that some parents have reported vegetables that were also a part of mixed dishes, causing double reporting.
When assessing validity with the use of cross-classification into quartiles, the results pointed towards highest agreement among food group intakes (Table 3). Coarse vegetables, fruit, candy, energy, dietary fibre and macronutrient energy percentages were related to a high degree of gross misclassification into the opposite quartile (>10 %), which were higher than or comparable to other studies using quartiles( Reference Huybrechts, De Backer and De Bacquer 13 , Reference Del Pino and Friedman 14 ). Despite this, the percentages for gross misclassification of vegetables, fish, whole grains and sweet drinks were comparable to (or lower than) a similar study among children( Reference Huybrechts, De Backer and De Bacquer 13 ) and other studies performed among adolescents( Reference Truthmann, Mensink and Richter 33 – Reference Overby, Johannesen and Jensen 35 ), all of which used quartile classification.
The use of correlation coefficients to assess validity is under debate but there is a general agreement that correlations above 0·5 are acceptable or good, and that correlations below 0·3–0·4 indicate a low degree of linear association( Reference Cade, Thompson and Burley 6 , Reference Masson, McNeill and Tomany 28 ). In the view of this, the majority of food group intakes can therefore be considered to have an acceptable validity for assessing intakes on a group level and for ranking of children according to food intakes. There are, however, concerns about the measurement of sweet drinks and candy. Few studies have reported results on correlation coefficients of compared food intakes. When compared with the original Norwegian version of the SFFQ, the results of the present study were not considerably different (r=0·26–0·69)( Reference Andersen, Lande and Trygg 11 ). The Norwegian study, however, is not directly comparable since the children were 2 years old. One other study, performed in a similar age group, reported correlation coefficients for foods in the range of 0·23 to 0·62, comparable to the present SFFQ( Reference Huybrechts, De Backer and De Bacquer 13 ). Not surprisingly, the correlation coefficients improved after correction for within-person variance. A few other similar studies have reported de-attenuated correlations( Reference Fumagalli, Pontes Monteiro and Sartorelli 9 , Reference Huybrechts, De Backer and De Bacquer 13 – Reference Sochacka-Tatara and Pac 15 ) and among these only one study examined food intakes( Reference Huybrechts, De Backer and De Bacquer 13 ). The latter study observed a large variation in variance ratios (−16·25 to 9·87), but likewise higher correlations after de-attenuation.
The correlation coefficients for energy intake and macronutrient energy percentages were generally similar to or lower than those observed in other studies( Reference Fumagalli, Pontes Monteiro and Sartorelli 9 , Reference Kobayashi, Kamimura and Imai 12 – Reference Del Pino and Friedman 14 , Reference Jarman, Fisk and Ntani 16 , Reference Metcalf, Scragg and Sharpe 17 , Reference Watson, Collins and Sibbritt 30 ). All correlations increased after de-attenuation, which is comparable to what is observed in other studies( Reference Fumagalli, Pontes Monteiro and Sartorelli 9 , Reference Del Pino and Friedman 14 , Reference Sochacka-Tatara and Pac 15 ). The correlation for energy intake improved to 0·23 when excluding observations from two children who exceeded the 95 % LOA in the Bland–Altman plot, but the association was still low compared with other studies. The low validity of the energy intake can be explained by a number of different factors. First, dietary assessment in young children is notoriously challenging. As mentioned above, a proxy reporter, typically the parent, needs to be involved, which poses additional challenges. The major challenges include: (i) recall bias; (ii) difficulties in estimating the child’s portion size; and (iii) limited supervision of meals when the child is out of the home( Reference Goran 3 , Reference Livingstone and Robson 4 ). Recall bias is a common problem within retrospective methods and even though parents have been recognized as reliable proxy reporters, remembering the child’s diet places high demands on the parent( Reference Livingstone and Robson 4 , Reference Burrows, Martin and Collins 31 ). Recall bias poses a particular problem when the child spends much of his/her time away from home, which is the standard for most Danish children. A study among 3-year-old children found that the linear association between the FFQ and a food diary decreased markedly for energy intake when the number of meals eaten outside the home increased (r=0·38 for no meals outside the home to r=0·31 for four or more meals outside the home)( Reference Jarman, Fisk and Ntani 16 ). The problem is perhaps minimized when children bring their food from home, which is the case in the present study, but limited supervision of meals will remain a challenge. Estimating children’s portion sizes seems to be a problem across dietary assessment methods( Reference Goran 3 , Reference Livingstone and Robson 4 ) and this could also be a possible explanation for the low validity observed for energy intake. A study indicated that the dietary report is more reliable when both parents participate( Reference Livingstone and Robson 4 ). In the present study, 85 % of the reporters were the mother. According to the additional analyses of person-specific factors, the relative validity of energy intake was affected by age and sex. The sample size was too small to analyse the validity of energy intake in subgroups, but informal tests indicated that the validity was better for children from day-care centres and for boys. If the validity is better in day-care children, this supports the hypothesis that limited supervision of meals is a problem when reporting children’s diet. It is likely that schoolchildren consume more meals and drinks out of the home, when visiting friends and family.
Another explanation, unrelated to the reporter, could be the duration of the recording period in the FR. For pragmatic reasons the FR was filled out for 4 d, but optimally a longer recording period would be needed to reflect a reliable intake( Reference Nelson 36 ). A recent study, with a large sample size, did observe a small insignificant difference in energy intake between 3 d and 7 d periods( Reference Whybrow, Horgan and Stubbs 37 ). The study was conducted in adults, so a longer period would probably be needed for children because of the higher within- to between-subject variation ratio. The problem of high day-to-day variability related to FR can be taken into account by using the de-attenuation method. Finally, a low validity of major food group intakes may have influenced the energy intake. As reported, intakes of sugary foods, sweet drinks and candy had a rather low validity. In addition, it was observed that the reported intake of meat and meat products had a low validity (r=0·21; not reported).
The reproducibility of the SFFQ was tested by comparing two administrations of the SFFQ with a 1-month interval. Mean difference was on average 9 %, indicating a small degree of overestimation by the first SFFQ and a good reproducibility at the group level (Table 2). There was a more moderate agreement at the individual level, as reflected in the wide 95 % LOA. The tendency of overestimation is also reported in other studies, and a previous review stated that the first administration tends to result in greater frequencies of consumption than subsequent administrations( Reference McPherson, Hoelscher and Alexander 32 ). This can probably be explained by the learning effect and the increased awareness of eating habits from filling out the FR.
The proportion of children correctly classified into the same quartile was of a similar magnitude to that observed in a reproducibility study in adolescents that also used quartile classification( Reference Overby, Johannesen and Jensen 35 ). In the present study, gross misclassification into the opposite quartile was below 10 % for most intakes, which is even less than in the adolescent study( Reference Overby, Johannesen and Jensen 35 ). All correlation coefficients were significant and comparable to those in other reproducibility studies performed among children( Reference Sahashi, Tsuji and Wada 10 , Reference Kobayashi, Kamimura and Imai 12 , Reference Huybrechts, De Backer and De Bacquer 13 ). Only the intake of candy was associated with poor correlation. Reproducibility of this food group was not reported by the aforementioned studies among children, but was assessed in adolescents as ‘sugar, sweets and snacks’ (r=0·58)( Reference Overby, Johannesen and Jensen 35 ). Furthermore, it is possible that the Christian holiday, Ascension Day, during the first administration of the SFFQ might have influenced the present results. Two children had a much lower intake of candy and chocolate at the second SFFQ (lower than the 95 % LOA), which was possibly caused by increased consumption during the holiday.
It has previously been suggested that the sample size of a validation or reproducibility study should be at least fifty for the LOA method and a minimum of 100 subjects for assessing correlation coefficients( Reference Cade, Thompson and Burley 6 ). The sample size of the present study is therefore a considerable limitation of the study and most likely influences the results causing a higher degree of variation, for example larger sd and 95 % LOA. In the cross-classification analysis, gross misclassification of a few children can make a large difference to the percentages. It can therefore be hypothesized that a larger sample size would have led to a higher level of reproducibility and relative validity. A small sample size can also be related to sampling bias, for example when responders are better than non-responders when answering FFQ. An analysis of sampling bias was not within the scope of the study. The sample size of the present study was comparable to that in other studies testing validity or reproducibility of FFQ. Participation in these kinds of study is challenging since it is related to a high subject burden.
In the current study, the estimated FR was chosen as the reference instrument. Unlike the FFQ, the FR method is prospective, does not depend on memory, is open-ended and has direct estimation of portion sizes using household units. Therefore, the measurement errors of FR and FFQ are fairly independent, and we considered the estimated FR to be the best alternative when choosing between different dietary assessment methods( Reference Cade, Thompson and Burley 6 ). The FR method has other limitations however, for example a likely change of dietary habits( Reference Magarey, Watson and Golley 38 ). Under optimal research conditions, inclusion of an objective measure (e.g. a recovery biomarker) in addition to the self-reported intake data (the dietary assessment method) would be ideal. Unlike self-reported data, biomarkers are not prone to reporting bias and the measurement errors are therefore less correlated with the FFQ. The inclusion of expensive biomarkers was unfortunately not possible in the SoL-Bornholm study.
A limitation of the reproducibility study could be a possible memory effect during the completion of SFFQ2 as parents could possibly still remember what they filled in 4 weeks ago( Reference Cade, Thompson and Burley 6 ). It is, however, a challenge to conduct a reproducibility study on FFQ assessing food intakes covering a month, and an optimal approach is unlikely to exist. Having repeated the SFFQ2 a year after and during the same month would also have been an option, but when conducted in children this may not be the best approach considering the likely increase in energy and nutrient intakes( 23 ). In addition, it would probably have led to lower response rates. Still, reproducibility studies are important and often neglected. A method that lacks reproducibility cannot be valid. When used in the context of a controlled intervention study with repeated measurements, it is of great importance that the study has an acceptable reproducibility in addition to validity( Reference Cade, Thompson and Burley 6 , Reference Bland and Altman 39 ).
The SFFQ tested in the present study had a low level of relative validity for some intakes, which needs to be taken into consideration when analysing results obtained using the SFFQ. The intake of vegetables had a high level of overestimation, intakes of sweet drinks and candy had the lowest correlation among the food groups, and the results for energy intake in particular indicated a low level of validity across statistical methods when compared with other studies. Concerning the reproducibility, an acceptable level was observed for all intakes, but for candy and chocolate. When considering that this food group also obtained a low validity score, the results for this food group must therefore be interpreted with caution when using the questionnaire for testing intervention effects in the SoL-Bornholm study. For all other food group intakes, the study results indicated that the SFFQ is a useful tool when assessing children’s food intakes on a group level in the SoL-Bornholm study. The SoL-Bornholm study was designed to test post-intervention differences between the intervention and control group and therefore reproducibility was more important than the ability to measure absolute intakes. The effectiveness analyses should be adjusted for energy intakes, age and sex to reduce potential variance from these variables.
For future use of the SFFQ, it is recommended that the food lists, frequency options and portion sizes are further developed. It is possible that some of the overestimation could have been avoided if the possibility of double reporting was reduced. If the low level of validity for energy intakes was caused by difficulties related to portion size estimation, it is possible that conversion to a non-quantitative FFQ would have increased the validity. This would, however, require that standard portions were developed for this age group in a representative sample.
In summary, the relative validity of the SFFQ was generally highest for foods, but overall low to moderate for most intakes, but comparable to findings of other studies among children. The reproducibility of the SFFQ was high or moderate for almost all intakes across statistical methods. The SFFQ can be used for ranking of children according to food intakes and the authors found that the SFFQ is a useful tool for evaluating the intervention effects of the SoL-Bornholm community intervention study. The use of the SFFQ to evaluate intakes of candy, energy and some of the macronutrient intakes is problematic and these intakes need to be interpreted with caution.
Acknowledgements
Acknowledgements: The authors would like to thank all the families who participated in the study. The authors would also like to thank the Department of Nutrition, University of Oslo, for sharing information on its FFQ. Financial support: The study was supported by the Nordea-Fonden. The funder had no role in the design, analysis or writing of this article. Conflict of interest: None. Authorship: T.B.-A. contributed to conception and design of the study; collecting data, analysis and interpretation of data; drafting and revision of the manuscript; approval of the final version of the manuscript. F.J.A.P.-C. contributed to conception and design of the study; revision of the manuscript; approval of the final version of the manuscript. U.T. contributed to conception and design of the study; interpretation of data; revision of the manuscript; approval of the final version of the manuscript. Ethics of human subject participation: The study was conducted according to the guidelines laid down in the Declaration of Helsinki and procedures involving human subjects were approved by an Institutional Review Board, the Ethical Committee of the Capital Region (H-3-2013-036). Written informed consent was obtained from all parents on behalf of their child.
