Hostname: page-component-586b7cd67f-dlnhk Total loading time: 0 Render date: 2024-11-26T09:07:25.142Z Has data issue: false hasContentIssue false

Comparative validation of standard, picture-sort and meal-based food-frequency questionnaires adapted for an elderly population of low socio-economic status

Published online by Cambridge University Press:  01 May 2007

Sara A Quandt*
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Mara Z Vitolins
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Shannon L Smith
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Janet A Tooze
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Ronny A Bell
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Cralen C Davis
Affiliation:
Division of Public Health Sciences, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1063, USA
Robert F DeVellis
Affiliation:
Department of Health Behavior and Health Education, University of North Carolina, Chapel Hill, NC, USA
Thomas A Arcury
Affiliation:
Department of Family and Community Medicine, Wake Forest University School of Medicine, Winston-Salem, NC, USA
*
*Corresponding author: Email [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Objective

To compare the validity of a modified Block food-frequency questionnaire (FFQ), a picture-sort administration of the FFQ (PSFFQ) and a meal pattern-based questionnaire (MPQ) in a multi-ethnic population of low socio-economic status (SES).

Design

Participants completed six 24-hour dietary recalls (24HR) over six months; the FFQ, PSFFQ and MPQ were completed in random order in the subsequent month. Instruments were interviewer-administered. The PSFFQ and MPQ were developed in formative research concerning difficulties for older adults in responding to standard food-frequency instruments.

Setting

Rural North Carolina, USA.

Subjects

One hundred and twenty-two African American, Native American and white adults aged ≥ 65 years, with approximately one-third in each ethnic group. Inclusion criteria included education ≤ 12 years and income ≤ 150% of national poverty level or Medicaid recipient.

Results

Comparing median intakes from the average of the 24HR with the three diet assessment instruments, the MPQ tended to overestimate intakes compared with the FFQ and PSFFQ. Correlations among nutrients obtained by the 24HR and the other three instruments were generally statistically significant and positive. Across nutrients, the PSFFQ was most highly correlated with the 24HR for women, while the FFQ was most highly correlated with the 24HR for men.

Conclusions

Dietary assessments using 24HR and FFQ were similar to results reported elsewhere, although correlations between 24HR and FFQ were somewhat lower. Interviewer-administered dietary assessments should be used with caution to evaluate dietary intake among older adults with low SES. Gender differences and the lower correlations should be investigated more thoroughly to assist in choosing dietary assessment instruments for this population.

Type
Research Paper
Copyright
Copyright © The Authors 2006

Attempts to reduce health disparities in diet-related chronic diseases and their risk factors require valid and reliable diet assessment tools that can be used in diverse segments of the population. The most commonly used measures in epidemiological studies are food-frequency questionnaires (FFQs), as they can reflect long-term intake, have relatively low respondent burden, and can be either self- or interviewer-administered.

The FFQ technique poses a mental challenge for some respondents, asking them to compute averages of what may be seasonally or otherwise varied intakeReference Dwyer and Coleman1. This is likely to be more problematic for some groups than others. Validation studies of the most commonly used FFQs have generally been conducted with well-educated groups of white women and menReference Willett2Reference Block, Coyle, Hartman and Scoppa4. Correlation coefficients with estimates of nutrient intake collected from multiple days of diet recalls or records typically range from 0.40 to 0.70. These values have been accepted as indicative of the ability of the FFQ to rank individuals correctly according to categories of nutrient intakeReference Zulkifli and Yu5. However, validation data from other populations (e.g. minority, low education) are scarce. Population subgroups with low socio-economic status (SES) have a higher degree of underreporting of energy intake than high-SES subgroupsReference Stallone, Brunner, Bingham and Marmot6. Two studies found that individuals with less than high school education perform poorly on FFQs, although there have been no studies to examine why this is the caseReference Kristal, Feng, Coates, Overman and George7, Reference Mayer-Davis, Vitolins, Carmichael, Hemphill, Tsaroucha and Rushing8. It is possible that the mental compilation and estimation required is simply too unfamiliar a task for these respondents to formulate answers to the questions and accurately report their intake. It is also possible that the actual food consumption of such persons differs from that of a higher-SES population, being more constrained by economic cycles (e.g. pay periods) than the seasonality typically built into FFQsReference Quandt, Arcury, McDonald, Bell and Vitolins9Reference Wolfe, Frongillo and Valois11.

There have been several attempts to develop instruments that may be better suited for respondents who are elderly or have low educational attainment. These include a picture-sort food-frequency questionnaire developed and validated in the Cardiovascular Health Study, and another developed and validated in an elderly Utah populationReference Kumanyika, Tell, Fried, Martel and Chinchilli12Reference Wengreen, Munger, Wong, West and Cutler14. In both cases the respondents were elderly, but were predominantly white and had high educational attainment. In the Cardiovascular Health Study, scores of 24 or higher on the Mini-Mental State Examination (MMSE) were required for inclusion regardless of education. Another instrument, the Diet History Questionnaire, was developed with modifications of the National Cancer Institute–Block Health Habits and History Questionnaire FFQ based on extensive cognitive interviewing with a wide range of participants, but its validation was conducted with a highly educated group of largely white and non-elderly personsReference Block, Hartman, Dresser, Carroll, Gannon and Gardner3, Reference Subar, Thompson, Kipnis, Midthune, Hurwitz and McNutt15.

Thus, while the need for alternative methods of collecting dietary data is acknowledged, they have not been tested in those populations for whom effective dietary assessment instruments are lacking. The present study was based on ethnographic and survey research begun in 1996 in a low-income, largely minority rural elderly population. That research included both qualitative exploration of food and dietary practices and quantitative documentation of dietary intake and other health behavioursReference Quandt, Arcury, McDonald, Bell and Vitolins9Reference Quandt, Vitolins, DeWalt and Roos16Reference Bell, Quandt, Arcury, McDonald and Vitolins20. In the course of that research a number of issues were identified that were related to difficulties in administering FFQs. The combination of vision problems and low literacy made the use of self-administered instruments difficult. These older adults also found responding to fixed response questionnaires and abstract categories (e.g. Likert-type scales, ordered frequency categories) confusing. These tasks were unfamiliar; they seemed to be particularly challenging for those with little experience in formal education or occupations that required such reasoning. Some had difficulty in describing foods independent of meals. Many had difficulty maintaining attention through a repetitive food-frequency interview. These findings were used to produce two new dietary data collection instruments based on the food frequency approach.

The goal of the present study was to compare the performance of these two new dietary assessment tools and a modified National Cancer Institute–Block FFQ for assessing usual intakes of energy and other nutrients with repeat 24-hour recalls (24HR). The overall objective of the study was to identify the instrument best able to estimate dietary intake in an elderly, low-SES rural population.

Methods

Study population

Study participants were recruited in two rural North Carolina counties. Recruitment strategies included seeking referrals from senior and low-income housing personnel, senior centre and meal site staff, county Social Service agencies, and interviewers who were native to the research counties. Candidates were screened over the telephone, when possible, in their homes or at a local senior health fair. Local housing personnel allowed research staff to conduct screenings in several low-income apartment complexes. The study was described, brochures provided and screening completed. All participants signed informed consent upon enrolment. The study was approved by the Wake Forest University School of Medicine Institutional Review Board.

Participants were selected to equally represent men and women in three ethnic groups: white, African American and Native American. Eligibility criteria were: (1) residence in a study county; (2) age ≥ 65 years; (3) ≤ 12 years of formal education; (4) low income (defined as ≤ 150% of the national poverty level or Medicaid recipient); (5) community-dwelling; and (6) mentally able to answer interview questions without assistanceReference Dwyer and Coleman1Reference Block, Coyle, Hartman and Scoppa4Reference Stallone, Brunner, Bingham and Marmot6. Additional exclusion criteria included (1) blindness or (2) being on kidney dialysis. Potential participants suspected of poor cognitive function were administered the MMSEReference Cockrell and Folstein21. Persons with scores of 23 or lower (for a ninth grade or higher education) or 17 or lower (for an eighth grade or lower education) were excluded based on recommendations from previous researchReference Crum, Anthony, Bassett and Folstein22.

A total of 137 elders enrolled in the study. Of these, 122 completed at least five 24HR and at least two diet assessment questionnaires and are included in these analyses. Of the 15 who did not complete the study, three died during the study period, five withdrew, one was admitted to a nursing home, one suffered significant cognitive decline and scored below the MMSE cut-off score during the course of the study, one moved out of the study counties, and four became too ill to keep interview appointments or were unable to answer questions without assistance. The 15 who did not complete the study were not significantly different from the 122 who did complete the study in age, education, income, receipt of Medicaid, gender or ethnicity (data not shown). Among the 122 persons included, 117 completed all six 24HR and three diet assessment questionnaires, two completed five 24HR and three questionnaires, and three completed six 24HR and two questionnaires.

Diet assessment

Data collection was completed in three phases (approximately 40 participants per phase) during a 16-month period from mid-February 2003 to early May 2004. Respondents were interviewed nine times in their homes over a 7-month period. The first six interviews were 24HR and were conducted at intervals of approximately 1 month. Each interview had a 7-day window for completion. At the final three interviews, the FFQ, a picture-sort administration of the FFQ (PSFFQ) and a meal pattern questionnaire (MPQ) (described below) were administered in a predetermined random order and recorded on paper forms. Each of these questionnaires was administered at least one day after the final 24HR and at least five days after the previous questionnaire. All three questionnaires were to be completed in a 3-week window after the final 24HR concluded. The study team allowed participants a slightly larger window if scheduling difficulties such as illness or holidays arose.

The 24HR data were entered by interviewers into a laptop computer on site, into the Nutrition Data System for Research (NDS-R) software, version 4.05_33 (Nutrition Coordinating Center, University of Minnesota). Interviewers used the ‘multiple-pass’ method to obtain maximum detailReference Johnson23. Of the six diet recalls, at least one and no more than two were collected for weekend days.

The FFQ used in this study is a validated, semi-quantitative, modified version of the National Cancer Institute Health Habits and History Questionnaire (HHHQ)Reference Mayer-Davis, Vitolins, Carmichael, Hemphill, Tsaroucha and Rushing8. This modified version includes a greater number of ethnic foods than the HHHQ. The FFQ required participants to report typical foods and beverages they had consumed over the previous six months. It consisted of 94 food and beverage line items. For foods, nine fixed response categories were provided and ranged from ‘never or less than once per month’ to ‘two or more times per day’. Beverage consumption ranged from ‘never or less than once per month’ to ‘six or more times per day’. Participants were asked to assign a typical portion size (small, medium or large) to each item with reference to other men/women their age.

The PSFFQ contained the same food and beverage line items as the FFQ. However, a colour photograph of each food or beverage category was presented to participants on a 4 in × 6 in card. Beneath each photo, a large print label described the food/beverage category. Participants selected and put aside those eaten ‘never or less than once per month’. The remaining cards were then sorted onto a 15 in × 36 in cardboard tray displaying each of the eight remaining frequency response categories. Interviewers assisted by reading the card labels or response categories to participants with low literacy skills and visual impairments. Interviewers recorded data on paper forms. Participants assigned a portion size (small, medium, large) to food categories (e.g. fruits, vegetables, beverages as used in the HHHQ). After assigning a general portion size, participants were queried on whether there were any portion size differences within a group, and those differences were recorded on the paper form by the interviewer.

The MPQ format was based on the diet history conceptReference Burke24. It consisted of participants reflecting over the past six months and recalling foods in a meal or snack pattern over the course of a typical week. The form consisted of six separate meal categories: breakfast, morning snack, lunch or midday meal, afternoon snack, dinner or evening meal, and evening snack. Within each meal or snack component, participants were asked to recall their usual intake of foods, guided by a series of prompts. The prompts for foods in each component were determined by the line items of the FFQ and from prior knowledge of eating patterns in the study countiesReference Bell, Quandt, Vitolins and Arcury19. The three snack sections were identical to each other; lunch and dinner consisted of identical food item lists. Open-ended questions were used to address types of sandwiches and mixed dishes consumed. Open-ended questions required interviewers to probe for ingredients and preparation methods for the foods recalled. Participants were probed about condiments and seasonings for items on the MPQ.

Quality control

Before the modified food-frequency instruments (PSFFQ, MPQ) were implemented in the field, they were pre-tested. The FFQ was not pre-tested, because it was used by the research team in previous studies and has been shown to be reliable elsewhereReference Mayer-Davis, Vitolins, Carmichael, Hemphill, Tsaroucha and Rushing8, Reference Vitolins, Quandt, Case, Bell, Arcury and McDonald18, Reference Bell, Quandt, Vitolins and Arcury19. The MPQ instrument, PSFFQ photographs and administration process were each pre-tested with four older adults in a county that was not included in the study sample. The PSFFQ pictures were professional photographs selected from stock photography or created for the research team to accurately represent all food categories and beverages queried. Participants were asked for comments, and interviewers noted any problems with the materials and procedures. For the PSFFQ, photograph content, colour and layout were adjusted as needed based on the comments of the pre-test participants.

Interviewers attended a two-day training session conducted by the investigators. Each was required to audiotape and submit multiple practice diet interviews to receive certification to collect data. Audiotapes were reviewed and compared with data collection forms to evaluate accuracy, as well as to assess interviewer style and skill. In order to prevent bias, one interviewer completed all 24HR interviews with a participant; a different interviewer blinded to the 24HR results conducted the final three interviews.

During the course of the study, 5% of diet recalls were tape-recorded and reviewed to ensure that drift did not occur in interviewer technique and accuracy of data entry. Sixty-four per cent of all questionnaire interviews were recorded. A random selection was reviewed, as well as questionnaires where ambiguous answers were recorded. To verify the interview process and assess participant experience with the interviewer, 19% of participants were contacted by phone regarding their diet recalls and 14% of the participants regarding their FFQs.

Additional measures

Demographic data were collected at baseline including self-reported ethnicity, education, date of birth, length of residence in the study counties, household size and income variables. Vitamin and supplement usage and overall health assessment data were also collected. Vitamin and supplement use data are not included in the present analyses.

Data management and statistical analysis

The 1998 Block Dietary Data Systems software was used for data entry of the FFQ, PSFFQ and the MPQ. Data screens for the FFQ and PSFFQ were developed based on the order in which the food item was presented on the form. For the MPQ, data entry screens were developed for three meals (breakfast, lunch, dinner) and three snacks (morning, afternoon, evening) to enable nutrient analyses by meal and snack. In addition, the data were combined in a SAS dataset to generate total daily energy and nutrient intake resembling the FFQ and PSFFQ nutrient datasets.

Descriptive statistics were used to summarise the demographic and health characteristics of participants and median nutrient intake (excluding supplements) for nutrients of interest. For macronutrients, percentage of energy was also included in this analysis. Agreement between the average of the 24HR and each of the dietary assessment instruments was assessed in two ways. First, Wilcoxon signed rank tests were used to determine if the median intake for the dietary assessment instrument (FFQ, MPQ, PSFFQ) differed from the average of the 24HR. Next, we considered the Pearson correlation between the dietary assessment instrument and the average of the 24HR. Due to the skewness of the nutrient data, the nutrient values were transformed to approximate normality using a Box–Cox transformation before the correlations were computedReference Box and Cox25. Both unadjusted correlations and correlations adjusted for energy intake using the residual method were computedReference Willett2. The same analytical procedure was used to perform pairwise correlations between the FFQ and either the MPQ or the PSFFQ. In order to determine whether correlation coefficients differed by gender, education, health status or interviewer assessment, we used Fisher's z-testReference Rosner26. All tests performed were two-sided with an α level of 0.05. SAS software version 8.2 (SAS Institute) was used for all statistical analyses.

Results

The characteristics of the study participants are shown in Table 1. The sample was 52.5% female and comprised African Americans (33.6%), Native Americans (36.1%) and whites (30.3%). Mean age was 75.2 ± 6.6 years. The level of formal education was low, with over half of each gender reporting an eighth grade education or less. Approximately 60% reported annual household income less than $US 10 000. Most had spent the majority of their lives in the study area. The most commonly reported occupations were machine operators, agricultural workers or service industry workers. Participants had high rates of chronic disease and risk factors. Over 75% reported being diagnosed with high blood pressure and over 50% with high cholesterol. For both males and females, the modal category of self-rated health was ‘fair’.

Table 1 Demographic and health characteristics of participants

Data are presented as n (%), except for age (mean ± standard deviation).

Percentages may not add to 100% due to ‘Don't know’ responses.

The median nutrient intakes estimated from the average of the 24HR and from the three summary instruments are shown in Table 2 by gender. Results for percentage of energy from alcohol are not presented because reports of alcohol consumption were extremely rare. For both males and females, the MPQ tended to overestimate intakes of most nutrients relative to the 24HR. For females, the FFQ and PSFFQ accurately estimated energy and intake of fats on average, but protein was significantly underestimated on the FFQ and carbohydrates were significantly underestimated on the PSFFQ. The PSFFQ accurately estimated all vitamins and minerals for females except vitamin C, potassium and sodium, while the FFQ underestimated most of them. For males, the FFQ tended to underestimate energy and nutrient intakes. The FFQ, and to a lesser extent the PSFFQ, underestimated most dietary vitamins and minerals for males. On average, the PSFFQ was the most accurate at estimating nutrient intake for both males and females.

Table 2 Comparison of median nutrient intakes estimated from the average of six 24-hour recalls, the food-frequency questionnaire (FFQ), the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender

%E – percentage of energy; RE – retinol equivalents; TE – tocopherol equivalents.

* Significantly different from 24-hour recalls, P < 0.05 (Wilcoxon signed rank test).

** Significantly different from 24-hour recalls, P < 0.01 (Wilcoxon signed rank test).

Table 3 shows the Pearson correlation coefficients for the associations of the average of the 24HR and each of the FFQ-type instruments. Most are statistically significant and positive; those adjusted for energy intake are lower than those unadjusted. In general, correlations are stronger and more often reach statistical significance for males than females. For females, the MPQ and PSFFQ are more highly correlated with 24HR than is the FFQ. For males, the FFQ and MPQ appear to have a stronger pattern of correlations with the 24HR than the PSFFQ. No comparisons of correlation coefficients for energy and nutrients between the average of the 24HR and each of the FFQ-type instruments by gender, education and health status were statistically significant (data not shown), except for one. The correlation coefficient for protein between the 24HR and PSFFQ was statistically significantly higher for men compared with women.

Table 3 Correlations between the average of multiple 24-hour recalls and the food-frequency questionnaire (FFQ), the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender

%E – percentage of energy; RE – retinol equivalents; TE – tocopherol equivalents.

Adjustment for energy intake done using the residual method (WillettReference Willett2).

* Correlation significant, P < 0.05.

Pearson correlation coefficients of the FFQ with the PSFFQ and MPQ are presented in Table 4. All of the correlation coefficients are statistically significant and positive. For both females and males, correlations for FFQ and PSFFQ are generally stronger than the corresponding correlations between FFQ and MPQ. Most of the correlation coefficients unadjusted for energy intake are stronger than adjusted, with exceptions for protein, vitamin C and fibre.

Table 4 Correlations of the food-frequency questionnaire with the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender

%E – percentage of energy; RE – retinol equivalents; TE – tocopherol equivalents.

Adjustment for energy intake done using the residual method (WillettReference Willett2).

All correlations significant at P < 0.05.

Discussion

The sample for this study was selected to represent an elderly low-SES group typical of that targeted for many community-based prevention and health promotion interventions. Despite their low educational attainment and occupational histories, the results of dietary assessment using standard data collection methods (repeat 24HR and FFQ) were similar to those reported elsewhereReference Willett2, Reference Subar, Thompson, Kipnis, Midthune, Hurwitz and McNutt15, Reference Wright, Wang, Kennedy-Stephenson and Ervin27, Reference Kumanyika, Mauger, Mitchell, Phillips, Smiciklas-Wright and Palmer28. The nutrient medians reported in the 24HR in this study are comparable to those obtained for persons 60 years and older in the 24HR portion of the National Health and Nutrition Examination Survey, 1999–2000Reference Wright, Wang, Kennedy-Stephenson and Ervin27. The deviation between the 24HR and FFQ seen here is assumed to be fairly typicalReference Willett2, Reference Subar, Thompson, Kipnis, Midthune, Hurwitz and McNutt15, although exceptions occurReference Kumanyika, Mauger, Mitchell, Phillips, Smiciklas-Wright and Palmer28.

In general, the correlation coefficients between 24HR and FFQ were slightly below the range observed in other studies of older or minority populationsReference Mayer-Davis, Vitolins, Carmichael, Hemphill, Tsaroucha and Rushing8, Reference Kumanyika, Mauger, Mitchell, Phillips, Smiciklas-Wright and Palmer28, Reference Morris, Tangney, Bienias, Evans and Wilson29. The correlations in the present study were stronger than those in another study on participants with low education levelsReference Mayer-Davis, Vitolins, Carmichael, Hemphill, Tsaroucha and Rushing8. This might be due to the methods of administration of the FFQ, as that study was conducted by telephone rather than with face-to-face interviews. The poorer performance of women than men on the standard FFQ has not been reported in other studies; however, most other studies that included both genders are non-minority, younger or higher-SESReference Subar, Thompson, Kipnis, Midthune, Hurwitz and McNutt15. In particular, women's reports of energy on the FFQ were uncorrelated with the 24HR, as reflected in non-significant correlations for total fat, unsaturated fats and fat-soluble vitamins. This suggests that the FFQ may not have captured particular foods or food preparation techniques in the same way as the 24HR. It should not necessarily be interpreted that men are able to more accurately report their diet. It is possible that men made consistent errors in reporting on the 24HR and FFQ (e.g. being unaware of added fats used in cooking or not knowing food item varieties such as low-fat dairy products served in the home).

Because adjustment for energy intake generally improves correlation coefficientsReference Willett2, it is surprising that this was not observed in the current study. In general, correlation coefficients were weaker after energy adjustment and for density measures (percentage of energy from fat and carbohydrate). It appears that participants did not underreport all macronutrients to the same degree, but were differentially underreporting some macronutrients. For example, for women (Table 2) on the PSFFQ, the median intakes of protein and fat are similar to the median 24HR amounts, but the median report for carbohydrates is higher and represents the largest proportion of total energy. Once adjustment for energy is made, due to the higher carbohydrate intake, percentage of energy from protein and fat are both lower than the percentage of energy reported on the 24HR. A similar pattern appears to occur for the correlation coefficients.

Picture-sort techniques have been tested by two other groupsReference Kumanyika, Tell, Fried, Martel and Chinchilli12, Reference Wengreen, Munger, Wong, West and Cutler14. The correlations with 24HR were somewhat higher in both of those studies. Neither of those studies administered a standard FFQ such as the Block. Thus, our study allows comparison of the picture-sort technique with both the 24HR and FFQ. The picture-sort technique used here produced stronger and more consistent correlations with the 24HR for women than did the FFQ. This suggests that the picture format may help older women, in particular, to more accurately report their usual dietary intake. This format was designed to make the task less abstract, more interactive and to cue memory. These results indicate that women in this sample responded more than men to such cues.

The MPQ technique developed for use in this study is patterned after the diet history approachReference Burke24. The diet history has been used as a semi-quantitative technique for obtaining descriptions of usual diet in clinical settings and, less frequently, for researchReference Bingham30. Most attempts to validate diet histories against 24HR or food records are somewhat dated (e.g. Reference Black, Prentice, Goldberg, Jebb, Bingham and Livingstone31Reference Jain, Howe, Johnson and Miller34; exception is 35), as more recent research has focused on the food-frequency approach. Our findings are consistent with previous studies that showed an overestimation of intakes with the diet history method relative to recalls or recordsReference Hankin, Wilkens, Kolonel and Yoshizawa32, Reference Jain, Howe, Johnson and Miller34, Reference Black, Welch and Bingham35. One study confirmed this result using urinary nitrogen excretion and doubly labelled waterReference Black, Welch and Bingham35. The degree of overestimation in the present study was higher than reported in previous studies. This may be due to the tendency of this elderly population to eat leftovers from the noon meal at the evening mealReference Quandt, Vitolins, DeWalt and Roos16. When responding to the MPQ, they may have reported the amount prepared rather than the amount consumed for each meal.

This study has several limitations. Despite its use as a reference in this study, the 24HR is known to have measurement error associated with itReference Tooze, Subar, Thompson, Troiano, Schatzkin and Kipnis36. Unfortunately, the nature of this error is not known in the present study, as no validation using doubly labelled water or other technique was used. Since all methods of data collection used are subject to errors in memory, the errors in estimates in both the 24HR and the other measures are likely to be correlated. The study focused on older adults from one particular region (the rural southern USA) that is characterised by health and social disparities. We would expect the performance on dietary assessment to be better if we had used a more advantaged population of older adults. Also, sample size precluded examining ethnic differences in validity of the food-frequency instruments or the interaction of gender and ethnic effects.

Despite these limitations, this study provides evidence that older, community-dwelling adults of low SES, many of whom had serious health conditions, can complete an intensive dietary validation study. The use of five or six 24HR within a six-month period is a strength of our study, as it minimises within-person variability in nutrient intake reporting. Although the participants in the present study performed more poorly than more advantaged participants in other studies, our findings indicate that any of the three instruments will provide results that should permit study participants to be ranked in terms of dietary intake. More research is needed to understand the gender differences in performance on the instruments. In the present study, despite the knowledge that error can be attributed to both the data collection instruments and to the respondents, it is impossible to distinguish one type of error from the other.

Most alternative dietary data collection instruments have been developed to accommodate problems posed by low literacy and other characteristics of low-SES and older participants. The present study shows that for different types of instruments administered by an interviewer, the differences among the scores obtained are small. However, participant burden is often a concern when dietary assessment is part of a large battery of data collection instruments. If alternative dietary data collection instruments (e.g. a picture sort) are more enjoyable for study participants, they may assist with participant retention and therefore have value beyond data quality. Such attributes of alternative data collection techniques deserve further study.

The choice of dietary data collection instruments should be driven by the focus of the research, as well as consideration of the validity of the methods. The instruments tested here either underestimated (FFQ, PSFFQ) or overestimated (MPQ) intakes, consistent with other existing diet history or FFQ studies. For measuring absolute intakes, 24HR appear to be necessary. However, for epidemiological studies of diet and disease, the correlation of the questionnaire and the 24HR, not the absolute amount measured, is often more important. For women in particular, the correlation coefficients between the MPQ and 24HR were comparable to and often better than those of the PSFFQ and FFQ with 24HR. For men, the correlations between MPQ and 24HR were weaker than those between FFQ and 24HR, but comparable to or better than those between the PSFFQ and 24HR. These findings suggest that man and women may respond differently to dietary assessment instruments in different formats. Further research to better understand the source of these response differences is necessary before adopting either of these new instruments.

Acknowledgements

Sources of funding: This work was supported by a grant from the National Institute on Aging (AG 13469).

Conflict of interest declaration: None of the authors had a personal or financial interest in any company or organisation that could benefit directly from this research.

Authorship responsibilities: S.A.Q., T.A.A., M.Z.V., R.A.B. and R.F.D. contributed to the design of the study and interpretation of data. S.A.Q., M.Z.V. and S.L.S. were responsible for the acquisition of data. J.A.T. and C.C.D. designed and conducted the statistical analyses. S.A.Q. drafted the manuscript. All authors were responsible for the critical revision of the manuscript and gave final approval for submission.

References

1Dwyer, JT, Coleman, KS. Insights into dietary recall from a longitudinal study: accuracy over four decades. American Journal of Clinical Nutrition 1997; 65(Suppl.): 1153S–8S.Google Scholar
2Willett, W. Nutritional Epidemiology, 2nd ed. New York: Oxford University Press, 1998.Google Scholar
3Block, G, Hartman, AM, Dresser, CM, Carroll, MD, Gannon, J, Gardner, L. A data-based approach to diet questionnaire design and testing. American Journal of Epidemiology 1986; 124: 453–69.Google Scholar
4Block, G, Coyle, LM, Hartman, AM, Scoppa, SM. Revision of dietary analysis software for the Health Habits and History Questionnaire. American Journal of Epidemiology 1994; 139: 1190–6.CrossRefGoogle ScholarPubMed
5Zulkifli, SN, Yu, SM. The food frequency questionnaire for dietary assessment. Journal of the American Dietetic Association 1992; 92: 681–5.CrossRefGoogle Scholar
6Stallone, DD, Brunner, EJ, Bingham, SA, Marmot, MG. Dietary assessment in Whitehall II: the influence of reporting bias on apparent socioeconomic variation in nutrient intakes. European Journal of Clinical Nutrition 1997; 51: 815–25.Google Scholar
7Kristal, AR, Feng, Z, Coates, RJ, Overman, A, George, V. Associations of race/ethnicity, education, and dietary intervention with the validity and reliability of a food frequency questionnaire. The Women's Health Trial Feasibility Study in Minority Populations. American Journal of Epidemiology 1997; 146: 856–69.CrossRefGoogle ScholarPubMed
8Mayer-Davis, EJ, Vitolins, MZ, Carmichael, SL, Hemphill, S, Tsaroucha, G, Rushing, J, et al. . Validity and reproducibility of a food frequency interview in a multi-cultural epidemiologic study. Annals of Epidemiology 1999; 9: 314–24.Google Scholar
9Quandt, SA, Arcury, TA, McDonald, J, Bell, RA, Vitolins, MZ. Meaning and management of food security among rural elders. Journal of Applied Gerontology 2001; 20: 356–76.CrossRefGoogle Scholar
10Kempson, KM, Palmer Keenan, D, Sadani, PS, Ridlen, S, Scotto Rosato, N. Food management practices used by people with limited resources to maintain food sufficiency as reported by nutrition educators. Journal of the American Dietetic Association 2002; 102: 1795–9.CrossRefGoogle ScholarPubMed
11Wolfe, WS, Frongillo, EA, Valois, P. Understanding the experience of food insecurity by elders suggests ways to improve its measurement. Journal of Nutrition 2003; 133: 2762–9.CrossRefGoogle ScholarPubMed
12Kumanyika, S, Tell, GS, Fried, L, Martel, JK, Chinchilli, VM. Picture-sort method for administering a food frequency questionnaire to older adults. Journal of the American Dietetic Association 1996; 96: 137–44.CrossRefGoogle ScholarPubMed
13Kumanyika, SK, Tell, GS, Shemanski, L, Martel, J, Chinchilli, VM. Dietary assessment using a picture-sort approach. American Journal of Clinical Nutrition 1997; 65(Suppl.): 1123S–9S.Google Scholar
14Wengreen, HJ, Munger, RG, Wong, SS, West, NA, Cutler, R. Comparison of a picture-sort food-frequency questionnaire with 24-hour dietary recalls in an elderly Utah population. Public Health Nutrition 2001; 4: 961–70.CrossRefGoogle Scholar
15Subar, AF, Thompson, FE, Kipnis, V, Midthune, D, Hurwitz, P, McNutt, S, et al. . Comparative validation of the Block, Willett and National Cancer Institute food frequency questionnaires. The Eating at America's Table Study. American Journal of Epidemiology 2001; 154: 1089–99.CrossRefGoogle Scholar
16Quandt, SA, Vitolins, MZ, DeWalt, KM, Roos, GM. Meal patterns of older adults in rural communities: life course analysis and implications for undernutrition. Journal of Applied Gerontology 1997; 16: 152–71.CrossRefGoogle Scholar
17Quandt, SA, Arcury, TA, Bell, RA, McDonald, J, Vitolins, MZ. The social and nutritional meaning of food sharing among older rural adults. Journal of Aging Studies 2001; 15: 145–62.CrossRefGoogle Scholar
18Vitolins, MZ, Quandt, SA, Case, LD, Bell, RA, Arcury, TA, McDonald, J. Ethnic and gender variation in the dietary intake of rural elders. Journal of Nutrition for the Elderly 2000; 19: 1529.CrossRefGoogle Scholar
19Bell, RA, Quandt, SA, Vitolins, MZ, Arcury, TA. Dietary patterns of older adults in a rural, tri-ethnic community: a factor analysis approach. Nutrition Research 2003; 23: 1379–90.CrossRefGoogle Scholar
20Bell, RA, Quandt, SA, Arcury, TA, McDonald, J, Vitolins, MZ. Health behaviors of rural white, African-American and Native American elders. American Journal of Health Behavior 2000; 24: 349–60.Google Scholar
21Cockrell, JR, Folstein, MF. Mini-Mental State Examination (MMSE). Psychopharmacology Bulletin 1988; 24: 689–92.Google ScholarPubMed
22Crum, RM, Anthony, JC, Bassett, SS, Folstein, MF. Population-based norms for the Mini-Mental State Examination by age and educational level. Journal of the American Medical Association 1993; 269: 2386–91.Google Scholar
23Johnson, RK. Dietary intake – how do we measure what people are really eating? Obesity Research 2002; 10(Suppl. 1): 63S–8S.CrossRefGoogle ScholarPubMed
24Burke, BS. The dietary history as a tool in research. Journal of the American Dietetic Association 1947; 23: 1041–6.CrossRefGoogle Scholar
25Box, G, Cox, D. An analysis of transformations. Journal of the Royal Statistical Society (B) 1964; 26: 211–52.Google Scholar
26Rosner, B. Fundamentals of Biostatistics, 5th ed. Pacific Grove, CA: Duxbury, 2000; 464–5.Google Scholar
27Wright, JD, Wang, CY, Kennedy-Stephenson, J, Ervin, RB. Dietary intake of ten key nutrients for public health, United States: 1999–2000. Advance Data from Vital and Health Statistics No. 334. Hyattsville, MD: National Center for Health Statistics, 2003.Google Scholar
28Kumanyika, SK, Mauger, D, Mitchell, DC, Phillips, B, Smiciklas-Wright, H, Palmer, JR. Relative validity of food frequency questionnaire nutrient estimates in the Black Women's Health Study. Annals of Epidemiology 2003; 13: 111–8.CrossRefGoogle ScholarPubMed
29Morris, MC, Tangney, CC, Bienias, JL, Evans, DA, Wilson, RS. Validity and reproducibility of a food frequency questionnaire by cognition in an older biracial sample. American Journal of Epidemiology 2003; 158: 1213–7.CrossRefGoogle Scholar
30Bingham, SA. The dietary assessment of individuals: methods, accuracy, new techniques and recommendations. Nutrition Abstracts and Reviews (Series A) 1987; 57: 705–40.Google Scholar
31Black, AE, Prentice, AM, Goldberg, GR, Jebb, SA, Bingham, SA, Livingstone, MB, et al. . Measurements of total energy expenditure provide insights into the validity of dietary measurements of energy intake. Journal of the American Dietetic Association 1993; 93: 572–9.CrossRefGoogle ScholarPubMed
32Hankin, JH, Wilkens, LR, Kolonel, LN, Yoshizawa, CN. Validation of a quantitative diet history method in Hawaii. American Journal of Epidemiology 1991; 133: 616–28.Google Scholar
33Jackson, N, Little, J, Wilson, AD. Comparison of diet history interview and self completed questionnaire in assessment of diet in an elderly population. Journal of Epidemiology and Community Health 1990; 44: 162–9.Google Scholar
34Jain, M, Howe, GR, Johnson, KC, Miller, AB. Evaluation of a diet history questionnaire for epidemiologic studies. American Journal of Epidemiology 1980; 111: 212–9.Google Scholar
35Black, AE, Welch, AA, Bingham, SA. Validation of dietary intakes measured by diet history against 24 h urinary nitrogen excretion and energy expenditure measured by the doubly-labelled water method in middle-aged women. British Journal of Nutrition 2000; 83: 341–54.Google Scholar
36Tooze, JA, Subar, AF, Thompson, FE, Troiano, R, Schatzkin, A, Kipnis, V. Psychosocial predictors of energy underreporting in a large doubly labeled water study. American Journal of Clinical Nutrition 2004; 79: 795804.CrossRefGoogle Scholar
Figure 0

Table 1 Demographic and health characteristics of participants

Figure 1

Table 2 Comparison of median nutrient intakes estimated from the average of six 24-hour recalls, the food-frequency questionnaire (FFQ), the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender

Figure 2

Table 3 Correlations between the average of multiple 24-hour recalls and the food-frequency questionnaire (FFQ), the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender

Figure 3

Table 4 Correlations of the food-frequency questionnaire with the picture-sort food-frequency questionnaire (PSFFQ) and the meal pattern questionnaire (MPQ), by gender