The prevalence of obesity and dietary-related non-communicable diseases is increasing, fuelled by energy-dense but nutrient-poor diets and more sedentary lifestyles( 1 , Reference Ng, Fleming and Robinson 2 ). Given the food environment is a major influence on diet( Reference Vandevijvere and Tseng 3 ), measuring the food or nutrition environment is an important objective for researchers to inform the development of both individual and environmental strategies to increase healthful eating.
Defining the food environment and determining how to measure it is a complicated task( Reference Lytle and Sokol 4 ). The food environment encompasses a number of levels: the community level (e.g. the distribution of food sources and types within a suburb), the social level (e.g. the characteristics of the home, school or workplace) and the consumer level (e.g. aspects that the consumer experiences within an outlet)( Reference Ohri-Vachaspati and Leviton 5 ). The current study focuses on the consumer level, where a number of dimensions reduce or increase opportunities to eat healthfully. These include access, availability, affordability, quality, price, promotion, placement, variety, freshness, information, promotion, media and advertising( Reference Ohri-Vachaspati and Leviton 5 – Reference Glanz, Sallis and Saelens 7 ).
The food environment has been measured in many ways( Reference Lytle and Sokol 4 , Reference Lytle 8 ); at least forty-eight different food environment instruments have been used( Reference Ohri-Vachaspati and Leviton 5 ), eighteen of which focused on measuring the consumer environment (in outlets such as stores, restaurants) wholly or as part of the instrument. Of these, the Nutrition Environment Measures Survey for Stores (NEMS-S) and Nutrition Environment Measures Survey for Restaurants (NEMS-R) are the most commonly used measures of the retail food environment( Reference Glanz, Johnson and Yaroch 9 ). NEMS measures have been used in many countries and various contexts to explore the food environment in rural( Reference Pereira, Sidebottom and Boucher 10 ) and urban areas( Reference Cannuscio, Tappe and Hillier 11 ); the food environment that surrounds disadvantaged populations( Reference Gustafson, Sharkey and Samuel-Hodge 12 ); medical( Reference Lesser, Hunnes and Reyes 13 ), educational( Reference Horacek, Erdman and Byrd-Bredbenner 14 , Reference Horacek, Erdman and Reznar 15 ) and recreational facilities( Reference Olstad, Raine and McCargar 16 ); and indigenous( Reference Canto, Engler-Stringer and Muhajarine 17 ) and cultural groups( Reference Khan, Calloway and Maida 18 ). Both instruments are observational checklists – the NEMS-S measuring aspects of availability, placement, quality and pricing in stores, and the NEMS-R measuring aspects of availability, information, promotion, facilitators, barriers and pricing in restaurants – to understand the level of support for healthful eating.
The NEMS-S and NEMS-R observational instruments can be considered time-consuming to administer, taking between 14 and 42 min per outlet (based on the original research studies)( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ). Acknowledging that time to administer equates to a dollar cost for research, examining short-form measurements represents a worthy research endeavour. Partington et al. ( Reference Partington, Menzies and Colburn 21 ) used data mining techniques on data from 820 stores and 1795 restaurants in the USA to select the items most predictive of the final NEMS scores, to produce short-form instruments. The short-form instrument retained only the most predictive items, weighting these in a regression equation according to their capacity to predict the long-form score. The resulting short-forms were then validated against long-form NEMS instruments in 193 outlets, producing comparable results, and a sizeable reduction in time taken to administer the instrument was noted( Reference Partington, Menzies and Colburn 21 ).
The present paper reports the results of a comparison between scores obtained using the longer versions of NEMS( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ) and the short-form versions( Reference Partington, Menzies and Colburn 21 ) on an Australian sample of food outlets. The aim was to critically examine whether important aspects of the food environment known to influence eating behaviour were captured by each instrument, therefore reflecting on the capacity of each tool to assess the healthfulness of the consumer food environment. A further aim was to determine if the short-forms produced comparable scores to the longer versions in an Australian context, thereby demonstrating whether the brief instruments could deliver comparable assessments in a more resource-economical form.
Methods
Data collection
Data from three Australian studies of the consumer food environment, conducted in 2014( Reference Carins and Rundle-Thiele 22 ), 2016( Reference Storr 23 ) and 2018 (RJ Storr, JE Carins & SR Rundle-Thiele, unpublished results), were used for the current analysis. The studies were conducted in four suburban areas in Brisbane, Australia: one area with high socio-economic status (SES) 2 km from city; one medium-SES area 6 km from city; one low-SES area 13 km from city; and a further low-SES area 50 km from city( 24 ). The first two suburbs were less culturally diverse (70–80 % Australian by birth, with 50–60 % both parents Australian by birth) than the second two suburbs (50–65 % Australian by birth, with 25–40 % both parents Australian by birth). The study designs were based on the original NEMS studies( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 , Reference Frank, Glanz and McCarron 25 ) and used the validated NEMS-R and NEMS-S following the protocols specified in the NEMS training manual (www.med.upenn.edu/nems). Both instruments were modified slightly for the Australian context by changing some foods to common, but equivalent, leading Australian foods and brands. Modifications were in accordance with the recommendations found on the NEMS website, were pilot-tested prior to the first study and did not disturb the categories or scoring structure of either instrument. During modification, the researchers considered the applicability of the instruments to the Australian context by referring to available literature on Australian dietary practices and purchasing patterns( Reference McLennan and Podger 26 , 27 ), finding a high degree of similarity between Australia and the USA. Food outlets within a 1 km radius of the centre of each suburb, along with all supermarkets within a 2 km radius, were identified for assessment (because few supermarkets were found within the 1 km zone). Outlets were sorted into types (stores and restaurants); assessed using the correct NEMS instrument (NEMS-S or NEMS-R) by a team of academics and research students; and from this assessment, scored using the long-form scoring method( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ) and the short-form method( Reference Partington, Menzies and Colburn 21 ).
Data analysis
The original NEMS observational instruments require data for a number of items to be recorded in either yes/no form or count form, or to record detail of an observation (e.g. the price of an item). Although the instruments contain many questions, the score in both long- and short-form versions is derived from fewer findings (e.g. one question may require the price of whole milk to be recorded, and another the price of low-fat milk; whereas points are attributed to the result of whether low-fat milk is priced higher or lower than whole milk). Both the NEMS long-form scoring method( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ) and the short-form method( Reference Partington, Menzies and Colburn 21 ) were examined to identify which items contributed to the score and to appraise the scoring procedure used. This allowed critical consideration of whether each instrument captured important aspects of the food environment known to influence eating behaviour and therefore whether they were robust tools for assessing the healthfulness of the consumer food environment. Long-form scores were calculated using the NEMS protocols found in the training manual and the short-form scores calculated using the method specified( Reference Partington, Menzies and Colburn 21 ) to enable comparison between the long- and short-forms in an Australian context.
Statistical analyses
The statistical software package IBM SPSS Statistics version 23 was used for analysis. The statistical analysis sought to determine whether the scores produced by the short-form were comparable to those produced by the long-form. The Wilcoxon matched-pairs signed-rank test was used to compare the long-form and short-form scores for each outlet for the group of restaurants and the group of stores; and for each outlet type within these groups. The Wilcoxon matched-pairs signed-rank test was used to determine whether the score produced by one form of instrument is more often higher, or lower, than the other method. Paired-samples t tests were also conducted to determine the average size of the difference between the instruments. However, paired-samples t tests will not indicate if the scores produced by one method are more frequently higher or lower compared with the other, due to the potential effect of any scores that may be located far from the mean.
Results
Tables 1 and 2 compare scoring opportunities (as a result of individual items or findings derived from multiple items) for NEMS-R and NEMS-S long-form and short-form versions. This comparison demonstrates which aspects of the consumer food environment are measured by each version of the NEMS (long-form or short-form) when determining the level of support for healthful eating.
Table 1 Nutrition Environment Measures Survey in Restaurants (NEMS-R): long-form and short-form scoring opportunities for aspects of support for healthy eating
LF, long-form; SF, short-form.
* NEMS-R original protocol adds information and promotion item scores to form a sub-score for information.
Table 2 Nutrition Environment Measures Survey in Stores (NEMS-S): long-form and short-form scoring opportunities for aspects of support for healthy eating
LF, long-form; SF, short-form.
* NEMS-S original protocol adds availability and placement item scores to form a sub-score for availability.
NEMS-R long-form captured six consumer-level food environment dimensions, while NEMS-R short-form captured between four and five dimensions, omitting between 17 and 33% of the consumer-level food environment dimensions previously measured. NEMS-S long-form captured four consumer-level food environment dimensions while NEMS-S short-form captured between one and two dimensions, omitting between 50 and 75% of the consumer-level food environment dimensions previously measured. The consumer-level food environment dimensions omitted in NEMS-R and NEMS-S are price, quality, placement and facilitators. Specifically, the comparisons in Tables 1 and 2 show that more than half of the scoring opportunities for availability were retained for both instrument types (stores and restaurants); whereas none of the scoring for price was retained. For restaurants, fewer than half of the scoring opportunities for facilitators, barriers, information and promotion were retained. For stores, fewer than half of the scoring opportunities for quality and placement were retained.
A total of 159 outlets were identified for assessment from maps and street directories; however, twelve (8%) had closed prior to assessment, nine (6%) store operators declined assessment and three (2%) outlets were considered unsuitable for assessment (spice store/wedding cake store/bodybuilding supplement store). The studies were conducted in 2014( Reference Carins and Rundle-Thiele 22 ), 2016( Reference Storr 23 ) and 2018 (RJ Storr, JE Carins & SR Rundle-Thiele, unpublished results). The final data set contained twenty-seven sit-down restaurants, twenty-one fast casual restaurants, thirty-four fast-food outlets and eighteen specialty food stores – all scored with the NEMS-R instrument; and fifteen grocery and twenty convenience stores – all scored with the NEMS-S instrument. Distributions of outlet types within each suburb were similar, except that sit-down restaurants were more common in the first (high-SES) suburb and fast-food outlets were more common in the fourth (low-SES) suburb. Each assessment was complete (no missing data). No variety stores were assessed in these studies. Both long-form and short-form median and mean scores are shown in Tables 3 and 4, along with the results of statistical testing.
Table 3 Scores determined by the NEMS-R( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ) (long-form) and Reduced Item Audit Tool( Reference Partington, Menzies and Colburn 21 ) (short-form) for 100 restaurants representing four suburban areas in Brisbane, Australia (2014, 2016, 2018)
NEMS-R, Nutrition Environment Measures Survey in Restaurants; LF, long-form; SF, short-form; P25, 25th percentile; P75, 75th percentile.
Table 4 Scores determined by the NEMS-S( Reference Glanz, Sallis and Saelens 19 , Reference Saelens, Glanz and Sallis 20 ) (long-form) and Reduced Item Audit Tool( Reference Partington, Menzies and Colburn 21 ) (short-form) for thirty-five stores representing four suburban areas in Brisbane, Australia (2014, 2016, 2018)
NEMS-R, Nutrition Environment Measures Survey in Stores; LF, long-form; SF, short-form; P25, 25th percentile; P75, 75th percentile.
Of the 135 outlets assessed, the short-form score was higher for fifty-two outlets compared with the long-form score; whereas for eighty-one outlets the short-form score was lower than the long-form score. The short-form method produced a statistically significant median increase in score compared with the long-form score for stores (z = 3·441, P = 0·001; Wilcoxon’s test). For restaurants, on the other hand, the short-form method produced a statistically significant median decrease in score (z = 5·051, P<0·001; Wilcoxon’s test). For this data set, the results indicate that, in general, the short-form gives an inflated score of support for healthful eating compared with the long-form instrument for stores, but a deflated score for restaurants. When examining the data by outlet type, these patterns were consistent within store types, with the short-form producing a statistically significant median increase in score for grocery and convenience store categories. However, the short-form produced a significant median decrease in the fast-food and specialty restaurant categories only, with differences in the sit-down and fast casual categories failing to reach significance. Calculations indicate that a minimum sample size of ninety-four provides 80 % power at α=0·05 level to detect a small effect (d=0·3) using the Wilcoxon’s test. The mean effect size observed in this sample of 135 outlets was 1·6 points with an sd of 5·7 (equivalent to d=0·3).
Discussion
Researchers seek valid and reliable tools and techniques to provide robust information for research and programmes while keeping time and costs manageable in resource-constrained situations. Reducing the length of existing (long-form) tools is one pathway towards more cost-efficient measurement. Taking this approach, Partington et al. ( Reference Partington, Menzies and Colburn 21 ) found comparable results when measuring support for healthful eating with short-form versions of NEMS in a US sample of stores and eating outlets.
The current study undertook a critical comparison of long- and short-form measures, drawing on available data (( Reference Carins and Rundle-Thiele 22 , Reference Storr 23 ); RJ Storr, JE Carins & SR Rundle-Thiele, unpublished results) scored using the long-form NEMS scoring protocols and following short-form analysis guidelines reported in Partington et al. ( Reference Partington, Menzies and Colburn 21 ). On an Australian sample of stores and restaurants comparable results were not identified: in general, the short-form delivered an inflated score compared with the long-form version for stores, and a deflated score for restaurants. The raw scores varied by as much as double (for stores) and half (for restaurants). This indicates that the pattern of contribution made by individual items to the overall score was different in this Australian sample compared with the US sample( Reference Partington, Menzies and Colburn 21 ). For stores, the long-form score is derived from thirteen availability items, ten price items, two quality items and one placement item; whereas the short-form score is obtained from availability items and one quality item. This suggests that dimensions of price, quality and placement were stronger and more negative influences on support for healthful eating in these Australian studies. For restaurants, interpretation is not as straightforward. For these outlets, the long-form score is derived from ten availability items, four price items, three items for facilitators and three for barriers, five information items and seven promotion items. The spread of items in the short-form is similar, except that availability items are mostly retained and all price items are removed. A cautious interpretation of the deflated short-form scores for restaurants could be that pricing was a stronger, more positive influence – or that availability was a weaker, more negative influence – on support for healthful eating in the current study. Nevertheless, the current results suggest that more testing of the short-form measure( Reference Partington, Menzies and Colburn 21 ) will be required to obtain a short-form measure that delivers comparable results.
Face validity involves making an assessment about the items used to measure a construct( Reference Rossiter 28 ) and is deemed to occur when ‘a scale’s content logically appears to reflect what was (initially) intended to be measured’( Reference Zikmund, Babin and Carr 29 ). Many features of the consumer food environment influence eating behaviour, which makes it difficult to measure them all within an instrument and difficult to understand what should be included in a food environment construct( Reference Lytle and Sokol 4 ). Taking a critical stance, the present study notes that the long-form NEMS-R captures six consumer-level food dimensions while the long-form NEMS-S captures four dimensions. Both long-form measures omit dimensions such as access, affordability, variety and freshness, all of which enhance healthy eating( Reference Caspi, Sorensen and Subramanian 6 , Reference Lee, Ralston and Truby 30 , Reference Cummins, Petticrew and Sparks 31 ). The data-driven approach taken to derive the short-form measure( Reference Partington, Menzies and Colburn 21 ) omitted many environmental dimensions captured in the original long-form measure, reducing it to primarily a measure of availability. Availability of healthful food products which remained a cornerstone of the short-form measure is essential for healthy eating, but many other factors contribute and need to be considered for inclusion in measurement tools like NEMS( Reference Glanz, Johnson and Yaroch 9 ). This ensures these tools can be used to benchmark support for healthy eating within food environments and to measure the progress of initiatives designed to improve food environments.
Consider that products are placed on shelves deliberately to increase prominence and salience( Reference Dreze, Hoch and Purk 32 , Reference Campo, Gijsbrechts and Nisol 33 ), which increases the likelihood of their purchase. Experimental studies have recently shown the effect of modifying shelf placement to increase sales of healthful dairy products( Reference Adam, Jensen and Sommer 34 ). Research has shown that although unhealthy options dominate shelf space in many stores, this varies across stores( Reference Farley, Rice and Bodor 35 ), making prominence of placement an important aspect to consider in food environment measurement tools. Placement was minimally measured in the long-form version of NEMS and was absent from the short-form version.
Price is considered one of the strongest marketing factors predicting energy intake( Reference Chandon and Wansink 36 ). Studies have shown that price increases for supermarket goods can drive down purchases( Reference Andreyeva, Long and Brownell 37 ) and price discounts for healthy foods can stimulate purchases( Reference Waterlander, Steenhuis and de Boer 38 ), suggesting this dimension of the food environment remains an important feature to measure. A large randomised trial found price discounts for healthier products had a significant positive effect on purchases, which lasted beyond the period of discounting. In the same trial, carefully tailored nutrition education had no effect on food purchases( Reference Mhurchu, Blakely and Jiang 39 ). No pricing elements were included in the short-form NEMS version for either stores or restaurants. While a balance must be struck between the need for information and whether the level of detail is easily obtained, the comparative measures included in the long-form versions of NEMS indicated whether healthier alternatives items were sold at a premium or whether discounting favoured overeating. As price is such a strong influence on behaviour( Reference Chandon and Wansink 36 – Reference Waterlander, Steenhuis and de Boer 38 ), consideration should be given to retaining a measure of price in a shortened version of NEMS.
Facilitators and barriers to healthy eating exist in many eating outlets, especially restaurants and fast-food outlets, and often influence diners without awareness. Many eating decisions occur at a low level of consciousness( Reference Cohen and Farley 40 ), with factors such as portion size, presentation, free refills, supply and proximity of foods all affecting consumption( Reference Wansink and Sobal 41 ). Continued or re-emphasised measurement of facilitators and barriers in tools like NEMS is advisable. Nutrition information supports those who are consciously and deliberately trying to eat better, but can take skill and effort to interpret, and hence often does not have the desired intended effect of reducing energy intake( Reference Swartz, Braxton and Viera 42 ). The demand for foods eaten away from home is sensitive to price change( Reference Andreyeva, Long and Brownell 37 ) and is encouraged by bundling (or bulk discounting) as consumers consider value for money to be one of the most important factors influencing their purchase( Reference Vermeer, Steenhuis and Seidell 43 ). Further to this, promotion, advertising and media are important aspects to measure in the food environment, both in supermarkets and food-service outlets. Media and advertising for food products exceeds $AUD 400 million in Australia( Reference Jolly 44 ) and these influences are widely acknowledged to be pervasive( Reference Chandon and Wansink 36 ).
Taken together, discrepancies on dimensions scored and the absence of some dimensions in the short-form measures suggest further research is warranted before conclusions can be drawn on whether the short-form measures adequately capture aspects of the consumer food environment known to affect healthy eating in this Australian context.
Availability of healthful alternatives is more heavily represented in the short-forms, with inflated scores for these dimensions observed in the current study at the expense or complete absence of other dimensions, namely quality, placement, information, promotion, facilitators, barriers and price. Other efforts to develop short-form measures of outlet healthfulness have also focused only on availability of foods( Reference DeWeese, Todd and Karpyn 45 ), thereby removing the ability to assess other aspects of the food environment known to affect consumer purchase and consumption of foods.
Partington et al. ( Reference Partington, Menzies and Colburn 21 ) acknowledge that shortening the form increases the risk that important information may be lost, and the present paper shares this concern. Future iterations of the NEMS should bear in mind the brevity of the instrument, but also aim to include important factors known to influence consumers so that opportunities for intervention can be identified and mapped in communities. Consideration of face validity in addition to criterion validity is recommended.
Limitations and future research
The current sample was drawn from four suburbs in Australia and therefore does not cover the range or variety of food outlets present nationally, with some outlet types not represented in the sample (variety stores and specialty stores were absent). The sample size is more than two-thirds that used previously to validate the short-form( Reference Partington, Menzies and Colburn 21 ) and findings require further verification with a larger, more diverse, sample size. We estimate our study was sufficiently powered to detect small differences across the entire sample, but only larger differences would have been observed as significant in the smaller subgroups of outlets. Assessment of a larger sample of outlets from other areas containing all outlet types with NEMS short- and long-forms represents an opportunity for future research before definitive conclusions are drawn. Further research is recommended to develop a measure of the consumer-level food environment capturing additional dimensions, including affordability, access, freshness and variety. Continued emphasis on developing short-form NEMS-R and NEMS-S measures is recommended to enable cost-effective measurement. Alternative analytical methods are recommended to develop a short-form measure. For example, Mokken’s( Reference Mokken 46 ) model of monotone homogeneity is recommended to investigate item strength. The model of monotone homogeneity estimates the extent to which pairs of items approximate an ideal, permitting an assessment of whether they can be explained by the same underlying trait( Reference Sijtsma and Molenaar 47 ). That said, the number of dimensions and the nature of measurement used in an assessment of the food environment ultimately depend on the objectives of the research at hand and must balance the need to capture important information while keeping costs manageable. Finally, the current study was centred at the consumer level. critically examining NEMS-R and NEMS-S short- and long-form measures. Additional opportunities exist to extend understanding to social and community levels given NEMS measures could potentially accommodate these levels. For example, the number of stores within a given radius is captured in NEMS measurement, permitting the community-level dimension of store types within a suburb to be determined. Future research could employ a mixed-method approach to develop new or alternative versions of NEMS that tap into emerging aspects of the consumer food environment known to influence eating behaviour. The development and validation of new or revised instruments would provide a greater suite of tools to enable researchers to select the measures most suited to their research objectives.
Conclusions
An initial examination of short-form versions of NEMS in an Australian sample did not produce comparable results with the long-forms. Notably, an inflated score for stores using the NEMS-S short-form, and a deflated score for restaurants using the NEMS-R short-form, were evident. Balance is needed to produce tools that provide robust insight and do not take long to administer. It would be desirable to ensure that future versions of food environment measurement tools like NEMS retain items that measure pricing; cover facilitators of and barriers to healthy eating well; and are extended to include the advertising and media environment.
Acknowledgements
Financial support: This research was conducted under a collaborative research agreement between Griffith University and the Defence Science and Technology Group (DST Group). The authors would like to acknowledge DST Group for providing financial support for this study (via a student scholarship). Conflict of interest: None. Authorship: J.E.C. led the study design and execution; analysis and interpretation; and authoring of the manuscript. S.R.-T. contributed to study design; interpretation; and authorship of the manuscript. R.J.S. contributed to study design and execution; analysis; and critical reviewing of the final manuscript. Ethics of human subject participation: The study protocol was considered by the GU Ethics Committee and assessed as not requiring formal ethical review.
