A large body of literature has suggested that dietary patterns rich in fruits and vegetables, the main source of naturally occurring pigments with antioxidant capacity called carotenoids, are important for the prevention of CVD, diabetes and some cancers(1). In large population studies, the majority of data relating diet to individual risk of disease relies on self-report assessment methods(Reference Mayne2) such as the FFQ, which asks about habitual intake over the past 6 to 12 months, or the 24 h dietary recall (24HR), which inquires about intake of foods in the past day. However, self-report methods for assessing diet are prone to measurement error, and misclassification of exposures and covariates can create bias in estimates of diet–disease associations(Reference Bingham, Luben and Welch3–Reference Schatzkin, Kipnis and Carroll6).
Blood carotenoid concentrations are thought to be useful biomarkers of fruit and vegetable intake. Due to a variety of factors that affect concentrations of carotenoids in the serum, such as the characteristics of foods, as well as host factors such as body size, gender, smoking status, cholesterol level and inter-individual variability in absorption, carotenoid concentrations cannot be directly translated to fruit and vegetable intake(Reference Rock7). Nevertheless, correlations with concentrations may provide some degree of face validity for carotenoid intake estimates from dietary assessment instruments.
Given the feasibility of using self-report measures in large studies, ongoing evaluation of how well our self-reported dietary measures compare with blood estimates of carotenoids is important for interpretation of results in studies examining carotenoid–disease associations(Reference Schatzkin and Kipnis8). Past validation studies of this kind have typically evaluated two self-report methods of assessment and the majority of these studies have been performed in relatively small subsamples. Among 470 women and men in the National Cancer Institute's (NCI) Observing Protein and Energy Nutrition (OPEN) Study, we had the opportunity to evaluate the utility of various commonly used dietary instruments, and compared the correlations between self-reported intake of carotenoids estimated from an FFQ, recalls and a novel fruit and vegetable screener and their corresponding serum values. We also evaluated the impact of energy adjustment using doubly labelled water (DLW) v. estimates of energy intake from self-report measures.
Experimental methods
Sample and study design
OPEN was conducted by the NCI from September 1999 to March 2000. A complete description of the study can be found elsewhere(Reference Subar, Kipnis and Troiano9). The original purpose of OPEN was to assess dietary measurement error by comparing measurements obtained via self-reported dietary instruments with unbiased biomarkers of energy (DLW) and protein intake (urinary N) among 484 men and women. The current investigation was restricted to the 470 participants who completed a Diet History Questionnaire (DHQ; baseline visit), at least one of two recalls (baseline and third visits), a fruit and vegetable screener and a blood draw (second visit; see Fig. 1). The fourteen participants excluded from the analysis were slightly older and more overweight than the 470 who were included.
Fig. 1 Study flow and activities, the Observing Protein and Energy Nutrition (OPEN) Study, September 1999–March 2000
Measures
Diet History Questionnaire
In advance of the first visit, participants were mailed and asked to complete a DHQ, an FFQ developed and evaluated at the NCI(10). The DHQ was collected on the first day of the study. Participants reported their usual frequency of intake and portion size of 124 food items over the last 12 months. For forty-four of the foods, participants were asked about seasonal intake, food type (e.g. low fat, lean, diet, caffeine free) and/or fat uses or additions. The DHQ also featured six additional questions about use of low-fat foods, four summary questions, and ten dietary supplement questions, including one on the dose and frequency of β-carotene supplement intake.
Each food item in the DHQ includes values of fruits and vegetables in pyramid servings (PYR) based on US Department of Agriculture (USDA) data(11). One PYR of fruit was equivalent to 1 whole fruit, 
cup of chopped fruit or 
cup of fruit juice. One PYR of vegetables was equivalent to 1 cup of raw, leafy vegetables, 
cup of other vegetables or 
cup of vegetable juice(11). Potatoes were not included in the calculation of vegetable intake. Although fruit and vegetable intake recommendations now use the standard unit of a cup (http://www.choosemyplate.org) instead of a PYR, the rankings of participants’ fruit and vegetable intake are not expected to differ between methods(Reference Nöthlings, Murphy and Sharma12).
To obtain dietary carotenoid values from DHQ fruit and vegetable intake data, we matched food codes from the 1994–96 Continuing Survey of Food Intakes by Individuals (CSFII) to similar foods in the nutrient database of the Nutrition Data Systems for Research (NDS-R) from the University of Minnesota (Nutrition Coordinating Center, Minneapolis, MN, USA)(Reference Dixon, Zimmerman and Kahle13, Reference Subar, Midthune and Kulldorff14).
24 h dietary recalls
Participants completed highly standardized recalls utilizing the five-pass method, developed by the USDA for use in national dietary surveillance(Reference Moshfegh, Raper and Ingwersen15). At the baseline (first day) and third (days 102–105) visits, trained interviewers administered recalls in-person to participants. Participants were asked about their food and supplement intake from midnight to midnight the previous day, as in the National Health and Nutrition Examination Survey (NHANES). Similar to the DHQ, the 24HR probed for specific dose for β-carotene supplements. To calculate carotenoid values, the 24HR data were coded and linked to a nutrient database, the Food Intake Analysis System version 3·99 (Health Science Center, University of Texas, Houston, TX, USA), which obtained its database from updates to CSFII(Reference Tippett and Cypel16).
Fruit and vegetable screener
At the second visit (day 12, 13, 14 or 15), participants completed a Multifactor Screener(Reference Thompson, Midthune and Subar17), a seventeen-item dietary instrument which asked frequency of intake of seven fruit and vegetable food groups in the past month (see Appendix). CSFII data were used to generate sex/age-specific portion sizes of fruit and vegetable intake in PYR and assign dietary carotenoid values in micrograms (μg/PYR). The Multifactorial Screener was developed using national data and its fruit and vegetable component was strongly correlated with estimated true intake in the OPEN study, at about 0·6–0·7 among men and 0·5–0·8 among women(Reference Thompson, Midthune and Subar17).
Serum
Blood specimens were collected for all participants at the second visit (day 12, 13, 14 or 15) and were processed and stored at −80°C until thawed for laboratory analysis. Samples were sent to Craft Technologies (Wilson, NC, USA) for analyses of carotenoids by HPLC. The laboratory provided results for α-carotene, cis- and trans-β-carotene, cis- and trans-β-cryptoxanthin, lutein, zeaxanthin, and cis- and trans-lycopene. Northwest Lipid Metabolism and Diabetes Research Laboratories (Seattle, WA, USA) performed the lipid analyses and reported results for cholesterol and TAG. For both carotenoid and lipid analyses, we inserted 10 % blind quality control samples to monitor performance of the laboratory assays in each batch of study participants’ samples.
The CV for external quality control samples were acceptable for most of the carotenoids and the lipids (CV (%): 4·9 for trans-β-carotene; 8·9 for α-cryptoxanthin; 7·8 for β-cryptoxanthin; 3·0 for lutein; 6·5 for zeaxanthin; 3·4 for trans-lycopene; 4·7 for cis-lycopene; 5·9 for cholesterol; 3·8 for TAG). The CV for α-carotene (13·2 %) was higher than for most other carotenoids, likely due to low concentrations in the blood. Given its quality control results, we chose to not include cis-β-carotene in our analyses (CV = 27·5 %).
Doubly labelled water
Total energy expenditure was measured by DLW. The DLW studies are described in detail elsewhere(Reference Scholler18). Briefly, we used a five-specimen protocol, with total body water measured by the plateau method(Reference Racette, Schoeller and Luke19). At the baseline visit (day 1), DLW was given orally at a dose of approximately 0·12 g of 10 atom% 18O-labelled water and 0·12 g of 99·9 % 2H-labelled water per kilogram of estimated total body water along with a subsequent 50 ml water rinse of the dose bottle(Reference Scholler18). After consuming nothing for 1 h, participants were then allowed to consume 200–400 ml of juice, a liquid replacement meal, or coffee during the next 2 h(Reference Scholler18). Volume of liquids consumed and time of consumption were recorded. Urine specimens were collected at 2, 3 and 4 h after the dose. The 2 h specimen was discarded. Total energy expenditure was calculated according to Racette et al.(Reference Racette, Schoeller and Luke19) and by using the modified Weir equation, assuming a respiratory quotient of 0·86.
Other covariates
Age, race/ethnicity and education level were obtained in advance of the baseline visit from the telephone screening interview. At the baseline visit (day 1), trained staff measured participants’ weight and height while they were wearing light indoor clothing and no shoes. All measurements were performed twice and averaged for a final value. If weight measurements differed by 0·3 kg or height measurements differed by 0·5 cm or more, then a third measurement was taken and used for final weight and height values. BMI was calculated as kg/m2. For the DHQ and fruit and vegetable screener, energy was estimated from the DHQ, while for the 24HR, energy was estimated from the average of two recalls.
Statistical analysis
We chose to present all data stratified by sex, because of past evidence showing under-reporting is more common among women than men(Reference Briefel, Sempos and McDowell20–Reference Price, Paul and Cole26). All analyses were executed using the SAS statistical software package version 9·1·3 (SAS Institute Inc., Cary, NC, USA). Geometric means and 95 % confidence intervals were calculated for log-transformed serum and dietary carotenoid estimates and TAG. Means were reported for cholesterol due to its normal distribution.
Sex-specific energy-adjusted and multivariate-adjusted Spearman correlations between serum carotenoids and dietary carotenoids from the DHQ, average of the recalls and fruit and vegetable screener were performed using the PROC CORR procedure. For the recalls, we also calculated multivariate-adjusted sex-specific Pearson correlations using PROC CORR and deattenuated the estimates by adjusting for intra-individual variation in the recalls(Reference Beaton, Milner and McGuire27). Deattenuated correlations were calculated by multiplying the Pearson correlation by 
, where R 1 and R 2 are the first and second applications of the 24HR, and 
is the mean of R 1 and R 2.
Correlations of diet measures with combined cis- and trans-lycopene, combined lutein and zeaxanthin, and combined α- and β-cryptoxanthin were not reported because individual serum carotenoid measures correlated as well as or better than the aforementioned combined measures and are often of interest in diet–disease associations (e.g. lutein and age-related macular degeneration). For the DHQ and recalls, for β-carotene, we also performed analyses with and without incorporating self-reported β-carotene supplement use.
Covariates considered in the multivariate correlation models were age, energy, serum total cholesterol, serum TAG, BMI, race/ethnicity and education. Given that the large majority of our population was not currently smoking (89 %), and that additional control for smoking did not affect the magnitude of correlations, we did not include smoking in our model building. Potential covariates were assessed via likelihood ratio tests (α = 0·05) and examination of residuals in linear regression models. Our final model included diet-derived energy, BMI, serum total cholesterol and TAG. Substituting energy derived by DLW did not result in substantial changes in magnitude of the correlations.
In an attempt to adjust for potential under- or over-reporting of individual fruits and vegetables on the DHQ, we performed a fruit and vegetable adjust procedure as described by Block with the DHQ data(28). The adjustment did not affect the previous correlations obtained, so the original estimates are presented.
Results
Descriptive statistics of the sample are presented in Table 1. On average, both men and women were 53–54 years of age, non-Hispanic white, overweight, and reported consuming 2–3 servings of fruit and 2–4 servings of vegetables daily depending on the dietary assessment measure. Men had higher mean energy intake than women.
Table 1 Characteristics of the study participants: 470 women and men, the Observing Protein and Energy Nutrition (OPEN) Study, September 1999–March 2000
DHQ, Diet History Questionnaire; PYR, pyramid servings; 24HR, 24 h dietary recall; F&V, fruit and vegetable; DLW, doubly labelled water.
Table 2 shows the geometric means of self-reported dietary carotenoid intakes obtained from the DHQ, recalls and screener, as well as concentrations of serum carotenoids and lipids. Mean dietary intakes of all individual carotenoids were highest from the DHQ and lowest from the screener. Screener estimates were higher than 24HR estimates for cryptoxanthin (men and women) and lycopene (women only).
Table 2 Unadjusted geometric means and 95 % confidence intervals of carotenoids and lipids among 470 women and men, the Observing Protein and Energy Nutrition (OPEN) Study, September 1999–March 2000
DHQ, Diet History Questionnaire; 24HR, 24 h dietary recall; F&V, fruit and vegetable
†Only β-cryptoxanthin is calculated from the F&V screener.
‡Means (not geometric means) are reported for cholesterol.
Serum carotenoids correlated more strongly with DHQ dietary carotenoids than the average of 24HR dietary carotenoids except for zeaxanthin, for which 24HR correlations were stronger (Table 3). Deattenuation of the correlations between recalls and serum carotenoids to account for within-person variability in diet resulted in substantially higher correlations, as expected. Correlations of serum and diet trans-β-carotene improved slightly when specifically measured supplemental intake was taken into account. In most cases, multivariate correlations of serum carotenoids with the screener-derived carotenoid estimates resembled the serum–DHQ correlations. Overall, energy adjustment and multivariate adjustment improved correlations between serum and dietary measures.
Table 3 CorrelationsFootnote † of serum and dietary carotenoids among 470 women and men, the Observing Protein and Energy Nutrition (OPEN) Study, September 1999–March 2000
* P < 0·05; **P < 0·01; ***P < 0·001.
† Models are adjusted for energy, serum total cholesterol, serum TAG and BMI.
Discussion
The present study makes a contribution to the field of dietary assessment in that we compared three commonly used self-report measures of fruit and vegetable intake with a spectrum of serum carotenoids. The study featured high-quality serum carotenoid measures, a larger sample size, and included an evaluation of a fruit and vegetable screener that was later used in national health surveys(Reference Colon-Ramos, Thompson and Yaroch29, Reference Thompson, Midthune and Subar30). Also, unlike past validation work in this area, we reported on the effect of adjusting for within-person variation when examining correlations between the average of multiple recalls and serum carotenoids and had the ability to evaluate the effect of adjustment for energy as assessed by DLW.
The geometric means of serum carotenoids (α-carotene, β-carotene, β-cryptoxanthin, trans-lycopene) and cholesterol observed in our study were similar to those in NHANES III(31) and the nationally representative sample in the Eating at America's Table Study (EATS)(Reference Dixon, Subar and Wideroff32). Our sample's mean serum TAG was slightly lower than the NHANES estimate(31).
Similar to other studies that formally examined validity of both FFQ and 24HR measurements(Reference Dixon, Subar and Wideroff32, Reference Natarajan, Flatt and Sun33), our OPEN data showed modest to strong correlations between FFQ and average 24HR diet and serum measures for provitamin A carotenoids (i.e. α-carotene, β-carotene, α-cryptoxanthin, β-cryptoxanthin) and modest correlations for lutein, and trans- and cis-lycopene. Consistent with our previous investigation in EATS (n 163)(Reference Dixon, Subar and Wideroff32), estimates of mean dietary intakes of all individual carotenoids were higher for the DHQ than for the average of recalls. In other past studies examining at least one FFQ and multiple recalls(Reference Natarajan, Flatt and Sun33–Reference Tangney, Bienias and Evans39), no particular method of dietary assessment produced consistently stronger correlations with individual serum carotenoids.
Unlike the DHQ, which is designed to measure usual intake over an extended period, the 24HR is designed to measure intake on a given day and is expected to be more highly correlated with true intake on that day than with true usual (long-term average) intake. Consistent with past studies which have formally examined components of variation in reported intake(Reference Beaton, Milner and McGuire27, Reference Ferrari, Al-Delaimy and Slimani40), a large part of the variability of intake as measured by 24HR in our study was due to day-to-day within-person variation. In order to better judge the 24HR's performance, we calculated deattenuated (Pearson) correlations that remove the effect of day-to-day variability. Before deattenuation, correlations between the average of two 24HR and the serum biomarker were usually lower than the corresponding correlations for the DHQ, while after deattenuation they were usually substantially higher than those for the DHQ. These results underscore the need to adjust for day-to-day variability when estimating diet–disease relationships in epidemiological studies that use the 24HR to assess diet, and thus the need for at least two recalls in at least a subsample of the individuals in the study(Reference Beaton, Milner and McGuire27).
Our study was the first to evaluate how well a fruit and vegetable screener estimates some dietary carotenoids (α-carotene, β-carotene, cryptoxanthin, lutein/zeaxanthin and lycopene) and how this performance compares with that of the 24HR and the DHQ. For α-carotene, trans-β-carotene, α-cryptoxanthin, β-cryptoxanthin and lutein, correlations with the screener were only slightly lower than with the DHQ, pointing to the potential value of inclusion of this screener in cross-sectional or longitudinal studies when respondent burden is a main concern and there is interest in estimating carotenoids.
Measurement error models are often used to estimate Pearson correlations between reported and true usual intake of dietary components(Reference Kaaks, Ferrari and Ciampi41). Such models require a valid reference instrument such as DLW or urinary N that is unbiased at the individual level. Concentration biomarkers such as serum carotenoids have person-specific biases related to bioavailability, absorption, metabolism and other factors, so are not valid reference instruments and cannot be used to estimate the correlation of true and reported intake(Reference Kaaks, Ferrari and Ciampi41). The correlation between a serum carotenoid and reported intake can, however, be considered a lower bound of true and reported intake(Reference Rosner, Michels and Chen42).
In the absence of blood data in many large studies, we are often reliant on self-report dietary measures of carotenoid or fruit and vegetable intake. Our study provided evidence that the DHQ, deattenuated recalls or (in some cases) a fruit and vegetable screener may be useful measures for estimating carotenoid status in studies without serum measures. Further, dietary energy was shown to be a good surrogate for DLW energy in these analyses. Ongoing research is needed on how to use biomarkers to complement self-report measures in prediction of disease or survival from disease(Reference Freedman, Kipnis and Schatzkin43).
Acknowledgements
This research was supported by the Intramural Research Program of the National Cancer Institute, National Institutes of Health, US Department of Health and Human Services. The authors have no conflicts of interest to disclose. S.M.G. analysed the data. All authors played a role in the writing of the manuscript, and S.M.G., F.E.T. and N.P. had primary responsibility for final content. All authors read and approved the final manuscript.
Appendix
Seven items asked on fruit and vegetable consumption as part of the seventeen-item Multifactorial Screener(Reference Thompson, Midthune and Subar17, 44)
How many times per day, week, or month do you usually eat (or drink):
1. 100 % fruit juice such as orange, grapefruit, apple, and grape juices? Do not count fruit drinks such as Kool-Aid, lemonade cranberry juice cocktail, Hi-C, and Tang.
2. fruit? Count fresh, frozen, or canned fruit. Do not count juices.
3. lettuce or green leafy salad, with or without other vegetables?
4. French fries, home fries, or hash brown potatoes?
5. other white potatoes? Count baked potatoes, boiled potatoes, mashed potatoes, and potato salad. Do not include yams or sweet potatoes.
6. cooked dried beans, such as refried beans, baked beans, bean soup, and pork and beans?
7. other vegetables? Count any form of vegetable: raw, cooked, canned or frozen. Do not count: lettuce salads, white potatoes, cooked dried beans, rice.
Frequency response categories are: never, 1–3 times last month, 1–2 times per week, 3–4 times per week, 5–6 times per week, 1 time per day, 2 times per day, 3 times per day, 4 or more times per day.
