Hostname: page-component-cd9895bd7-q99xh Total loading time: 0 Render date: 2024-12-23T08:55:25.263Z Has data issue: false hasContentIssue false

Don’t lose sight of the forest for the trees: recognising the benefits as well as the limitations of implementation research

Published online by Cambridge University Press:  05 November 2018

Carly J. Moores
Affiliation:
Nutrition and Dietetics, College of Nursing and Health Sciences, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia email [email protected]
Richard J. Woodman
Affiliation:
Centre for Epidemiology and Biostatistics, College of Medicine and Public Health, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia
Jacqueline Miller
Affiliation:
Nutrition and Dietetics, College of Nursing and Health Sciences, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia email [email protected] Healthy Mothers, Babies and Children, South Australian Health and Medical Research Institute, PO Box 11060, Adelaide, SA 5001, Australia
Helen A. Vidgen
Affiliation:
School of Exercise and Nutrition Sciences, Faculty of Health, Queensland University of Technology, GPO Box 2434, Brisbane, QLD 4001, Australia
Anthea M. Magarey
Affiliation:
Nutrition and Dietetics, College of Nursing and Health Sciences, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia email [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Type
Letter to the Editor
Copyright
© The Authors 2018 

We write to thank Hannon et al. for their interest( Reference Hannon, Thomas and Siu 1 ) in our recent paper that reported pre-post programme changes in anthropometric outcomes for the upscaled community programme Parenting, Eating and Activity for Child Health (PEACH) QLD( Reference Moores, Miller and Daniels 2 ). Their letter identified several weaknesses in the study design including a lack of control group and the possibility of a regression to the mean (RTM) effect arising due to selection bias. Whilst we acknowledge there are indeed limitations to this and other implementation research study designs, we nevertheless feel that our findings are important and here provide further details of our data to strengthen the basis of our conclusions.

Hannon et al. suggested that our changes in BMIz can be exclusively explained by RTM which may arise wherever there are random fluctuations in biological variables and where there is error associated with their measurement( Reference Twisk, Bosman and Hoekstra 3 ). We accordingly took appropriate steps to maximise the accuracy of the pre- and post-programme anthropometric measurements, including protocols to minimise within- and between-subject measurement error( Reference Moores, Miller and Perry 4 ) and we also excluded biologically implausible BMI z-scores( 5 ). This will not however have prevented the inclusion of some overweight children that were assessed towards the high point of their natural weight variation and therefore the potential for to RTM effect, in which more extreme population values have greater potential to regress towards the mean when measured again( Reference Morton and Torgerson 6 ).

In our published paper, our population included only children above a healthy weight pre-programme and did not explore changes in BMIz by level of overweight. We have, however, previously reported changes made during PEACH QLD that included healthy weight children as part of a universal approach to child healthy lifestyle promotion( Reference Croyden, Vidgen and Esdaile 7 ). In the present study, the BMIz change in healthy weight children with paired pre-post programme data (n 58) was –0·02 (from 0·32 to 0·31). This was in comparison to pre-post programme BMIz changes for children that were classified( Reference Cole and Lobstein 8 ) as overweight (–0·13), obese (–0·10), or morbidly obese (–0·11) at pre-programme. This absence of any trend towards greater reductions in BMIz with increasing severity of overweight by definition alone reduces the likelihood of any RTM effect being present. In addition, we also measured improvements in important behavioural variables, namely parenting self-efficacy, and child diet and activity behaviours that we a priori hypothesised would mediate reductions in BMIz. Taken together, we believe these data strengthen the plausibility for a true effect of the intervention rather than one that arose purely as a statistical artefact. We cannot, however, also rule out the possibility that the observed changes may have also arisen from other factors given the lack of a control group (e.g. the Hawthorne effect( Reference McCambridge, Witton and Elbourne 9 ) or confounding from other unmeasured exposures).

The evaluation questions in PEACH QLD were taken from the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework( Reference Glasgow, Vogt and Boles 10 ) for which indicators of programme effectiveness included changes in child anthropometric outcomes, and parent-reported child diet, physical activity, and sedentary behaviours( Reference Moores, Miller and Perry 4 ). Whilst this pre-post design limits the interpretability of our data, we also hope this discourse will benefit others working in translation and implementation science to appreciate the potential limitations in evaluating uncontrolled, large-scale studies of community public health programmes implemented in real-world settings. The PEACH QLD study, however, still remains a valuable example of implementation research and immense value and knowledge are gained from describing pre-post outcomes following large-scale community delivery. Further research is needed to identify effective and scalable solutions for child obesity and it is clear that balancing research attention on both health and implementation outcomes is the key progressing the universal availability of childhood obesity management services for families.

Acknowledgements

PEACH Queensland was funded by the Queensland Government of Australia (2014–2016). The views expressed in this publication do not necessarily reflect those held by the Queensland Health Department or the Queensland Government of Australia. The PEACH Queensland Project was originally funded as a joint Australian, State and Territory Government initiative under the National Partnership Agreement on Preventive Health (2013–2014).

A. M. M. is an inventor of the original PEACH programme and received licence money for the use of PEACH materials in the PEACH QLD Project.

All other authors declare no conflicts of interest.

References

1. Hannon, BA, Thomas, DM, Siu, C, et al. (2018) The claim that effectiveness has been demonstrated in the Parenting, Eating and Activity for Child Health (PEACH) childhood obesity intervention is unsubstantiated by the data. Br J Nutr 120, 958959.Google Scholar
2. Moores, CJ, Miller, J, Daniels, LA, et al. (2018) Pre-post evaluation of a weight management service for families with overweight and obese children, translated from the efficacious lifestyle intervention Parenting, Eating and Activity for Child Health (PEACH). Br J Nutr 119, 14341445.Google Scholar
3. Twisk, J, Bosman, L, Hoekstra, T, et al. (2018) Different ways to estimate treatment effects in randomised controlled trials. Cont Clin Trial Comm 10, 8085.Google Scholar
4. Moores, CJ, Miller, J, Perry, RA, et al. (2017) CONSORT to community: translation of an RCT to a large-scale community intervention and learnings from evaluation of the upscaled program. BMC Public Health 17, 918.Google Scholar
5. Centers for Disease Control and Prevention (2016) A SAS Program for the 2000 CDC Growth Charts (ages 0 to <20 years). https://www.cdc.gov/nccdphp/dnpao/growthcharts/resources/sas.htm#modalIdString_CDCTable_1 (accessed June 2017).Google Scholar
6. Morton, V & Torgerson, DJ (2003) Effect of regression to the mean on decision making in health care. Br Med J 326, 10831084.Google Scholar
7. Croyden, DL, Vidgen, HA, Esdaile, E, et al. (2018) A narrative account of implementation lessons learnt from the dissemination of an up-scaled state-wide child obesity management program in Australia: PEACH (Parenting, Eating and Activity for Child Health) Queensland. BMC Public Health 18, 347.Google Scholar
8. Cole, TJ & Lobstein, T (2012) Extended international (IOTF) body mass index cut-offs for thinness, overweight and obesity. Pediatr Obes 7, 284294.Google Scholar
9. McCambridge, J, Witton, J & Elbourne, DR (2014) Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol 67, 267277.Google Scholar
10. Glasgow, RE, Vogt, TM & Boles, SM (1999) Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 89, 13221327.Google Scholar