Introduction
The promotion of physical activity (PA) has become a prominent theme in public health policy and considered worthy of increased investment (WHO, Reference World Health Organization2018) – Jerry Morris famously calling it the ‘best buy in public health’ (Morris, Reference Morris1994: 807). Despite suggestions of a ‘robust scientific evidence base’ and faith in ‘recommended’ evidence-based actions (WHO, Reference World Health Organization2018: 22), little attention has been paid to the basis and rigour of these centrally generated assumptions and their articulation with local PA policy (Schöppe et al., Reference Schöppe, Bauman and Bull2004). In this context, this article reports on a four-year case study that describes and reflects on the development of PA promotion decision-making processes by the Dumfries and Galloway Physical Activity Alliance (DGPAA, a multi-sector strategic partnership) in the region of Dumfries and Galloway (D&G), Scotland.
The perception of the need for action in this domain arose in response to various local drivers. Primarily, there was a belief that the significance of PA policy had remained under-recognised in the region and investment relatively low (Kelly et al., Reference Kelly, McAdam and Turner2017). Also, despite some funding, levels of PA in the region had remained static and comparatively lower than the rest of Scotland (Scottish Government, 2019). As such, senior regional leaders sought to enhance the status of the PA domain by increasing spend on it and tasking DGPAA to create local principles that would inform this investment. This article describes and reflects on this process. It is structured around the journey taken in relation to five sequential and inter-related stages (spanning instigation to enactment), underpinned by the over-arching research question: ‘what was the nature of the process that devised a set of principles that might inform localised physical activity promotion investment decisions?’. This was supported by the specific exploration of: why these principles were needed; what principles already existed; what actions were taken to arrive at new localised ones; what these were; and how they were ultimately expressed in grounded decision-making.
We start by establishing two broad contexts that are significant to our analysis – first, the way that ‘evidence-based’ and ‘best investment’ processes have evolved generally; and second, the specific trajectory of the emergence of PA as a policy domain and expressions of ‘evidence’ within it. We then describe the methodological foundation of the empirical work – an ‘exploratory sequential design’ using mixed methods. We go on to outline the features of the five elements that make up this case study. Most significantly, as well as an objective technology, a central concern for the work was opening up the possibility of seeing ‘best investments’ as a constructed notion (Majone, Reference Majone1989).
So, in introducing social science resources into a domain traditionally governed by narrow economistic values (Burrows et al., Reference Burrows, Bunton, Muncer and Gillen1995), we place the article in a context initiated by Ken Young and colleagues in Social Policy and Society in 2002, that sought to address ‘confusion about what evidence-based policy making actually means’ (Young et al., Reference Young, Ashby, Boaz and Grayson2002: 215). Within this context, we deploy a set of socially and politically oriented theoretical resources to our empirical data. We conclude by reflecting on the potential significance of these insights to public health investment generally.
Conceptual esources
A context for ‘best investment’ approaches
To appreciate the nature and contemporary significance of ‘best investment’ approaches and the principles that already exist, we first need to understand the trajectory of a series of ‘evidence-based’ methodologies (Wampold and Kuldhir, Reference Wampold and Kuldhir2004). The past forty years have seen a growing desire to promote ‘rigor’ in health policy generally (Baicker and Chandra, Reference Baicker and Chandra2017) and public health specifically (Brownson et al., Reference Brownson, Chriqui and Stamatakis2009). An ‘evidence-based’ approach originated in the 1980’s (Young et al., Reference Young, Ashby, Boaz and Grayson2002) in the context of two influences – a perception of uncontrolled and arbitrary health service expenditure (Propper, Reference Propper2001) and increasing funding restrictions (McKee and Stuckler, Reference McKee and Stuckler2012). Consequently, a decision-making approach emerged that variously sought to: ‘commodify’ interventions (Henderson and Petersen, Reference Henderson and Petersen2002); nurture a healthcare ‘market’ (McKee and Stuckler, Reference McKee and Stuckler2012); and foster the possibility of being able to make ‘value-for-money’ decisions (Burrows et al., Reference Burrows, Bunton, Muncer and Gillen1995). In this context, concepts like ‘cost-effectiveness’, ‘cost-utility’ and ‘cost-benefit’ became arguably one of the dominant features of health policy discourse (Burrows et al., Reference Burrows, Bunton, Muncer and Gillen1995: 243) and were further enhanced by the subsequent emergence of ‘return on investment (RoI)’ (Masters et al., Reference Masters, Anwar, Collins, Cookson and Capewell2017) and ‘best investment’ (Craig, Reference Craig2014) methodologies. At the forefront of this movement was ‘evidence-based medicine’ (Sackett and Rosenberg, Reference Sackett and Rosenberg1995), focussing on one-to-one clinical interventions, underpinned by ‘technologies’ (like the Cochrane database) that sought to codify quantitative RCT-based insights (Black, Reference Black2001).
Despite the ubiquity of these approaches, various critical perspectives have surfaced. Technically, the modelling is seen as a theoretical construct that inevitably simplifies reality (Pokhrel, Reference Pokhrel2015), resulting in inexact assumptions and high variability in costings. Saltelli and Giampietro (Reference Saltelli and Giampietro2017: 66) call this “modelling hubris” that ‘erroneously convey(s) an impression of prediction and control’. This in turn is seen to lead to difficulties in drawing valid generalisations and comparing investment evidence across geographies and projects, Muller-Riemenschneider et al. (Reference Muller-Riemenschneider, Reinhold and Willich2009: 70) concluding: ‘appropriate cost effectiveness analyses are rare… and the generalisability of presented findings is limited’. These technical critiques are complemented by increasing appreciation of the contextual complexities in which such evidence is generated and interpreted (Richardson, Reference Richardson2012). There is a belief that the politically invested nature of policy systems inevitably leads to the re-interpretation of intelligence in light of professional and sectoral values and interests (Trueman and Kwame Anokye, Reference Trueman and Kwame Anokye2012). Some also believe that systems were already ‘overwhelmed’ by cost-effective evidence that does not necessarily influence investment decisions (Pokhrel, Reference Pokhrel2015: 908).
In responding to these critiques, various related notions have emerged – evidence-based ‘practice’ (Rycroft-Malone et al., Reference Rycroft-Malone, Seers, Titchen, Harvey, Kitson and McCormack2004), ‘policy’ (Pawson, Reference Pawson2002) and ‘public health’ (Brownson et al., Reference Brownson, Chriqui and Stamatakis2009). These approaches look to move beyond rudimentary ‘economistic’ analyses (Polanyi, Reference Polanyi, Polanyi and Pearson1977), towards one that accommodates ‘societal’ orientations to evaluation (Jonsson, Reference Jonsson2009). This ground sees evidence collection, collation and interpretation as longitudinal processes (Young et al., Reference Young, Ashby, Boaz and Grayson2002), foregrounded by various forms of complexity. These include: that values can shape assumptions of ‘worth’ (Craig, Reference Craig2014); that a variety of social and environmental ‘non-health outcomes’ can be associated with ‘health’ interventions (NICE, 2011); that ‘return’ timeframes are variable (Richardson, Reference Richardson2012); and that, in a ‘systematic’ context (Carey et al., Reference Carey, Malbon, Carey, Joyce, Crammond and Carey2015), the value of one contributory element of a multifaceted approach cannot be seen in isolation – its success being dependent on other parts of a wider ‘ecology’ (Dauphinee, Reference Dauphinee2015). In this context, Moore et al. (Reference Moore, Cambon, Michie, Arwidson, Ninot, Ferron, Potvin, Kellou, Charlesworth, Alla and Discussion2019: 26) call for a shift in the framing of the evaluative domain from single isolated interventions to a series of inter-related events, accompanied by more ‘realistic’ questions (Pawson and Tilley, Reference Pawson and Tilley1997) beyond ‘does it work?’; namely, the ethics of ‘should we do it’, the pragmatic efficacy of ‘can we do it’, the implementation focus of ‘how do we do it?’ and finally the resource-based question of ‘do we have sufficient capacity to do it?’ (Dobrow, et al., Reference Dobrow, Goel and Upshur2004).
The emergence of physical activity policy and associated expressions of evidence
Many of these issues have played out in the PA domain. From traditionally being a marginal public health issue, the topic gained some prominence in the 1980s (Dunn, Andersen and Jakicic, Reference Dunn, Andersen and Jakicic1998). This early focus tended to be on the physiological basis of PA, particularly the association between vigorous exercise and its capacity to protect against ‘non communicable’ diseases like coronary heart disease, stroke and diabetes (Paluch et al., Reference Paluch, Berryman, Powell, Vuori, Tipton, Blair, Ainsworth and Macera2012). Paffenbarger et al. (Reference Paffenbarger, Blair and Lee2001: 1184) note that, at this time, evidence-based research deployed a classic ‘infectious disease epidemiological’ approach, resulting in a pre-occupation with simply establishing a definitive critical ‘dose’ of PA associated with health gain and setting this as a population norm (Whitelaw, Reference Whitelaw2010). Subsequent programme evaluation then tended to centre simply on assessing the proportion of the population achieving these PA targets (Milton and Bauman, Reference Milton and Bauman2015).
However, such work eventually suggested ‘divergent health-related physical activity recommendations’ (Blair et al., Reference Blair, LaMonte and Nichaman2004: 913S), particularly that health gain could come from moderate intensity PA (Blair and Connelly, Reference Blair and Connelly1996). On this basis, an ‘active living’ policy frame emerged in the late-1990s that fore-grounded ‘everyday’ PA such as walking and gardening (Eyler et al., Reference Eyler, Brownson and Schmid2013). From this foundation, policy then extended into novel domains such as: mental health benefits associated with PA (Asztalos et al., Reference Asztalos, Wijndaele, De Bourdeaudhuij, Philippaerts, Matton, Duvigneaud, Thomis, Duquet, Lefevre and Cardon2009); contributions to environmental sustainability via ‘active transportation’ (Sallis et al., Reference Sallis, Frank, Saelens and Kraft2004); and interventions across the whole lifecycle, particularly the ‘(re)enablement’ of older people (Mjøsund et al., Reference Mjøsund, Moe, Burton and Uhrenfeldt2020). This plurality became rooted in global PA policy (WHO, Reference World Health Organization2018), expressed as a ‘whole systems’ approach (Rutter et al., Reference Rutter, Cavill, Bauman and Bull2019). Of most significance to our work, it was articulated in Scottish PA policy as a need for ‘multiple concurrent approaches… involv(ing)… multiple systems and settings: education, transport, planning, healthcare, communications, communities, and sport’ (Scottish Government, 2018).
This ground displays a curious mix of perspectives on the nature of underpinning evidence. Primarily, a series of definitive statements are apparent, suggesting the existence of a relatively robust evidence base. For example, the Scottish Government (2018: 12) cite a ‘body of international evidence on what works….to get people active’ and this broad position is underpinned by specific guidance, such as GAPA’s (2011) ‘7 Best Investments for Physical Activity’ and the WHO’s ‘20 evidence-based policy actions’ (WHO, Reference World Health Organization2018: 62-94). Of particular interest to this article, a series of affirmative ‘cost-effectiveness’ based economic analyses of PA also exist (Wolfenstetter and Wenig, Reference Wolfenstetter and Wenig2010; Wu et al., Reference Wu, Cohen, Shi, Pearson and Sturm2011; Trueman and Kwame Anokye, Reference Trueman and Kwame Anokye2012; Laine et al., Reference Laine, Kuvaja-Köllner, Pietilä, Koivuneva, Valtonen and Kankaanpää2014; Valero-Elizondo et al., Reference Valero-Elizondo, Salami, Osondu, Ogunmoroti, Arrieta, Spatz, Younus, Rana, Virani, Blankstein, Blaha, Veledar and Nasir2016; Mitchell et al., Reference Mitchell, Hardeman, Pears, Vasconcelos, Prevost, Wilson and Sutton2016; Rütten et al., Reference Rütten, Abu-Omar, Burlacu, Schätzlein and Suhrcke2017; Abu-Omar et al., Reference Abu-Omar, Rütten, Burlacu, Schätzlein, Messing and Suhrcke2017; Cowper et al., Reference Cowper, Peterson, Pieper, Sloane, Hall, McConnell, Bosworth, Ekelund, Pearson and Morey2017;). However, at the same time, a paucity of evidence is also implied. The WHO for example concludes that ‘research….is needed to inform new policy and strengthen practice’ (WHO, Reference World Health Organization2018: 44) and this is seen to be the case particularly in relation to ‘whole systems’ evaluation that Bagnall et al. (Reference Bagnall, Radley, Jones, Gately, Nobles, Van Dijk, Blackshaw, Montel and Sahota2019: 11) feel is ‘still in its infancy’. The significance of these features will become evident later in the paper.
Methodology
The nature of the case study
Moving onto our case study, this was built around a series of linked sub-projects undertaken over a four-year period (2015-2019). The research can best be described as ‘mixed method’, involving a ‘exploratory sequential design’, where ‘data collection and data analysis of one component takes place after the data collection and data analysis of the other component and depends on the outcomes of the other component’ (Schoonenboom and Johnson, Reference Schoonenboom and Johnson2017: 117).
The work broadly adopted a ‘naturalistic’ approach (Frey et al., Reference Frey, Botan and Kreps1999) where the actions and dynamics of various events, workshops and meetings were captured ‘in situ’. The authors (SW and CT) acted as ‘participant observers’ and data were collected via field notes. Some elements also used survey, focus group and semi-structured interview methods (see below).
In terms of ethics, most aspects (particularly ‘in situ’ interactions) were classed within NHS Scotland guidelines as ‘service evaluation’, therefore not requiring formal NHS Research Ethics Committee review. Nevertheless, ethical principles were adhered to – participants were made aware of the purpose of data collection and subsequent use, and they were given opportunities to verify and amend contributions throughout the process. The ‘Local Monitoring Review’ work gained approval from the University of Glasgow’s Ethics committee.
Participants across these elements included: the full DGPAA comprised of senior representatives from NHS D&G, various subdivisions within D&G Council (e.g. leisure and sport, education, transport) and other Third Sector and associated agencies with an interest in PA (such as Forestry Commission Scotland and Sustrans); smaller sub-groups drawn from this base; and a range of practitioners from the D&G PA workforce.
Specific case elements
The empirical work had five elements that matched the trajectory of the process that spanned the initial instigation of a perception for a need for localised ‘best investment’ principles through to how this aspiration was enacted and new localised ones arrived at.
-
1. A stakeholder workshop was undertaken in December 2015 to review regional PA promotion. This was led by DGPAA and deployed a ‘QI’ methodology (The Health Foundation, 2013) that brought together senior representatives from the NHS, Local Government and the Third Sector. In a context supportive of the belief that PA could make significant contributions to health and social outcomes, the event aimed to establish strategic actions that would increase and better target PA investment (see Whitelaw et al., Reference Whitelaw, Topping, McCoy and Turpie2017).
-
2. A foundational ‘best investment’ survey was undertaken in 2016 by the University of Edinburgh’s Physical Activity for Health Research Centre (PAHRC) to ‘synthesise and appraise available evaluation data and make recommendations for what activities provide the best return on investment in D&G’ (Kelly et al., Reference Kelly, McAdam and Turner2017: 4). The survey work captured the range of existing PA projects in the region by type, geographical location, age group and gender and suggested and applied preliminary investment principles.
-
3. A ‘principles review’ workshop was convened in March 2018 as a subgroup of DGPAA with the aims of reflecting on the validity and utility of these preliminary principles and making them focussed and operational. This was based on a recognition that the principles needed to be ‘owned’ locally and that this was best done collaboratively.
-
4. A local monitoring review survey was initiated in July 2018 and undertaken by the University of Glasgow’s Crichton Institute (Crichton Institute, 2019). It arose from an observation in the PAHRC report of the patchy nature of grounded PA promotion intelligence within D&G and its aim was to complement the existent centralised view of existing data. Again, seeking to work closely with practitioners, it comprised: a rapid review of global PA monitoring systems; the identification of potential data domains; an audit of existing local PA project monitoring schemes; and consultative stakeholder interviews and focus groups to identify views on the feasibility and viability of ‘essential’ and ‘desirable’ domains and datasets.
-
5. A confirmatory workshop was convened in April 2019 as sub-group of the DGPAA and was set the tasks of: reviewing the recommendations arising from the work to date; developing a practical data collection framework; and establishing the future structure of DGPAA into which this would be expressed.
These case elements are summarised in Table 1.
Table 1. Summary of case study elements and associated methods
Analysis and associated theoretical resources
Quantitative data from the survey aspects of the work were analysed using descriptive statistical techniques. Qualitative data from the naturalistic ‘in situ’ groups and formal interviews and focus groups were interrogated using ‘thematic’ approaches (Braun and Clarke, Reference Braun and Clarke2006), where data were classified into categories and arranged into manageable forms and patterns. The trustworthiness of the qualitative data contained in ‘in situ’ notes and from focus groups and interviews was verified internally by SW and CW and more broadly within the relevant sub and full groups of DGPAA.
In relation to the theoretical context of these insights, a central feature of our work was to explore the social and political processes involved in creating ‘best investment’ guidelines. As such, qualitative data were interrogated using theoretical resources suggested by Dobrow et al. (Reference Dobrow, Goel and Upshur2004, Reference Dobrow, Goel, Lemieux-Charles and Black2006). Its basis involves a shift from a ‘philosophical-normative’ orientation where ‘evidence has inherent value’ to a ‘practical-operational’ one where ‘contextual variations heavily influence the determination of what constitutes evidence’ (Dobrow et al., Reference Dobrow, Goel and Upshur2004: 208-209). They propose two specific explanatory ‘categories’. First, an external one that is ‘fixed, uncontrollable…(that) cannot be manipulated by decision-makers….account(ing) for the environment in which a decision is applied’ (Dobrow et al., Reference Dobrow, Goel and Upshur2004: 210). This environment is considered to be made up of ‘factors’, such as ‘disease-specific’ pressures that emerge from demographic and epidemiological analyses and ‘extra-jurisdictional’ and ‘political’ forces that recognise social, economic and legal influences on the currency of evidence.
Second, an internal ‘decision-making’ one that theorises fluid variables within the context, including the subjectivity of multiple ‘actors’, varied senses of the core purpose of the evidence-based process and types of ‘incremental’ and ‘mixed-scanning’ decision making (Dobrow et al., Reference Dobrow, Goel and Upshur2004: 209). Here, what might be seen as ‘rational’ data becomes a source of contention with a series of potentially idiosyncratic values at play (Richardson, Reference Richardson2012). For example, Trueman and Kwame Anokye (Reference Trueman and Kwame Anokye2012: 32) suggest that public health investment can be thought of as an ‘opportunity cost’ that ultimately might cost more than traditional ‘treatment’. Similarly, Richardson (Reference Richardson2012: 323) notes the existence of an ‘identifiable victim effect’ where there is an inclination to fund interventions that result in immediate and tangible impacts on individuals with clear ‘needs’. Moynihan (Reference Moynihan2006) also recognises the ambiguity of the ‘rules’ that protagonists work within when deploying such techniques, particularly whether the appraisal is being undertaken within the PA domain (as a form of ‘technical’ efficiency) or between PA and other public health priorities as a form of ‘allocative’ efficiency (Palmer and Torgerson, Reference Palmer and Torgerson1999). In light of these themes, we now describe and reflect on our empirical insights.
Empirical insights
Stakeholder workshop
The practical origins of our case study, and the conception of the perception that local investment principles were needed, lay in the 2015 Stakeholder workshop described above (Whitelaw et al., Reference Whitelaw, Topping, McCoy and Turpie2017). Part of the QI methodology used in the event involved inputs from ‘expert witnesses’ on evidence-based PA interventions. One of these presenters (Foster, Reference Foster2015) posed the question ‘does DGPAA have an awareness of the range of PA interventions being delivered and the per annum spend?’. This question was complemented by a request from the day’s chair (CEO of D&G Council) for stronger collaboration, greater innovation and critically in the context of this paper, the identification of PA projects that showed ‘the best bang for our buck’.
‘Best Investment’ review
These statements shaped DGPAA’s immediate response – the commissioning of a regional survey of PA related activity and a preliminary ‘best investment’ analysis (Kelly et al., Reference Kelly, McAdam and Turner2017). fifty-two projects were identified and allocated to categories based on the aforementioned ‘7 Best Investments for Physical Activity’ (GAPA, 2011). The nature of these projects is described in Table 2.
Table 2. Types of PA Promotion in D&G
The work then sought to identify existing approaches considered to potentially offer a ‘best return’ on investment. It considered the cost-weighted return against (where available) three critical aspects of the: (i) number of unique participants involved; (ii) number of repeat attendances; and (iii) project duration or likelihood of ongoing impact. This generated a ranking within each of the eight project types (Kelly et al., Reference Kelly, McAdam and Turner2017), with thirty-eight of the fifty-two projects provisionally considered as conforming to these principles. This work concluded with seventeen recommendations. Most were subject-specific, supporting the need to invest in the specific areas such as ‘schools’, ‘urban design and infrastructure’ and ‘health and social care’.
However, various conceptual and practical difficulties soon began to emerge. First, it quickly became clear that, in the absence of a region wide reporting system, the projects identified represented only a small fraction of what was actually being delivered in the name of PA promotion in the region. Second, data that were captured was of a relatively poor quality, with very few examples of ‘pre and post’ intervention material that would ‘evidence direct behaviour change impact of interventions’ (Kelly et al., Reference Kelly, McAdam and Turner2017: 13) or offer insights into other possible outcomes (such as mental health) and quality related ‘delivery processes’. Third, a ‘lack of consistency in data collection measures’ meant that any comparative appraisal was considered impracticable (Kelly et al., Reference Kelly, McAdam and Turner2017: 13). Finally, given the wide and inclusive breadth of the founding principles within the ‘7 Best Investments’ framework, the sensitivity of the screening process was considered low – with the majority (thirty-eight) of the fifty-two projects surviving the initial analytical process. Of most significance to further steps, the establishment of a ‘project monitoring system across D&G… developed in collaboration with the end users to ensure fitness for purpose’ (Kelly et al., Reference Kelly, McAdam and Turner2017: 14) was therefore recommended.
‘Principles review’ workshop
The seventeen recommendations were then further developed and made more focussed and operational in a DGPAA sub-group workshop. This involved their cross-referencing with broader conceptual insights, particularly ‘best practice’ academic literature (e.g. Giles-Corti et al., Reference Giles-Corti, Timperio, Bull and Pikora2005), creating themes of: the significance of ‘whole system’ approaches to PA promotion; the importance of conducive environmental contexts for PA; undertaking promotion in preferred settings; targeting interventions at priority lifecycle groups; and utilising appropriate forms of PA. Table 3 sets out these enhanced ‘principles’.
Table 3. Wider ‘best investment’ principles
Workshop discussion also highlighed the perceived need for two further elements. First, additional contextual domains were felt necessary: specifically, an account of project linkages to local/national policy; assessment of the extent to which interventions support deep cultural change; expectations of population ‘reach’; and the expression of an underpinning behaviour change theory. Second, by means of fostering change via disinvestment and reinvestment, the group felt that the principles of ‘legacy potential’ (investing in interventions like infrastructure development that are initially expensive but create value over time) and ‘scalability’ (investment in relatively small-scale interventions with accepted value that have the potential to be disseminated) were significant. These additions were accommodated in a comprehensive ‘outcome focused template’, outlined in Table 4.
Table 4. Best Investments project template
As a trial appraisal, data from the thirty-eight projects were entered onto the template and eleven were judged to conform to the principles and formed the basis of focussed investment that would be monitored longitudinally. A further eleven investment areas not included in the original PAHRC review (commenced post review or not submitted) were subsequently identified as being compatible with these best investment categories. These consisted predominantly of capacity building approaches above project level (for example, PA policy change; embedding principles of community development/physical literacy across focused investment areas; and strengthening evaluation data).
Local monitoring review survey
With this retrospective framework in place, attention moved on to a key recommendation in the PAHRC report – the need for a robust regional PA monitoring system. The next aspect of the case therefore sought to create such a system, one that would be able to prospectively and routinely collect data that would map geographical spread and types of PA interventions and appraise the processes, impacts, outcomes and costs of such work.
Building on an acceptance of the multifaceted nature of PA actions and outcomes and fundamental stances on evidence identified above, this work was founded on a premise that there would be a range of potential scenarios in relation to: the quantity of data collected (from possibly none at all to a lot); the frequency of collection (ad hoc to routinely); concern for delivery ‘process’ and/or end ‘outcomes’; and ultimately varied ‘outcome’ types. Within a desire to attain cross-regional consistency, there was also a belief that consensus was needed across all parties on what a workable regional monitoring system might generally look like and what a common dataset should be.
A range of themes was evident (Crichton Institute, 2019). Those delivering PA on-the-ground confirmed that existing data collection was rudimentary, patchy and when undertaken, mostly limited to participant numbers and basic demographic characteristics. Many cited the practical ‘burden’ of collecting data and that this might act to paradoxically interfere with the actual achievement of PA promotion. Conceptual concerns were also expressed, including clarifying the varied nature of ‘physical activity’ outcomes and identifying and capturing the complexity of processes involved in optimal intervention delivery.
Nevertheless, there was almost universal support for such a framework providing it remained simple, flexible and modest in scope. Consensus was achieved in relation to a series of domains: basic demography; measures of project delivery processes (e.g. enjoyment, engagement); measures of project capacity and environmental adaptation for PA; measures of ‘wellbeing’; and ultimately levels of PA. These perspectives were accommodated in our existing framework in the final part of the case study.
Confirmatory workshop
A DGPAA sub-group workshop was convened as a way of achieving a summative conclusion to the work undertaken since 2015. It had two purposes: to finalise a localised self-reporting tool; and to consider the wider mechanisms of governance that would appraise such data and ultimately make investment and disinvestment decisions. In relation to the first, a DGPAA framework and self-reporting tool was agreed upon. The final framework is set out in Figure 2.
Figure 1. Best Investments by Project Type
Figure 2. An evaluative framework
Given the enthusiasm for PA promotion expressed at the initial 2015 workshop and the substantive data that subsequently emerged, the expectation was that the region could move quickly towards a situation where intelligence was being appraised and collective PA investment decision making enacted. However, by early 2019 there was a view within DGPAA that this was not being progressed at a pace that might have been expected. This work fostered a realisation that, based on a modest model of multi-sector communication, the DGPAA had originally been constituted as a self-contained vehicle for descriptive and retrospective information-sharing of individual projects. This was considered problematic in that it prohibited prospective decision-making as well as comparative appraisal. The need to strengthen the DGPAA was therefore considered a necessity. To this end and drawing on the systems-based themes that informed WHO’s 2018 publication ‘Global Action Plan for Physical Activity’, the sub-group recommended that the DGPAA would concentrate on the creation of ‘Active Systems’ creating conducive contexts (e.g. ‘strengthening policy and leadership’, ‘improving and integrating data systems’, ‘building research and development’, ‘expanding advocacy’ and ‘developing innovative finance mechanisms’). This model is outlined in Table 5.
Table 5. Global action plan for physical activity - four objectives
Furthermore, to enact these principles, it was felt that a separation of responsibilities was required and DGPAA was re-structured into two functions – a strategic overview forum that would set and monitor broad investment rules (PAA Strategic Forum) and an operational group (PAA Working Group) that would practically appraise data arising from regional monitoring and advise the strategic forum on investment and disinvestment assessments (see Figure 3).
Figure 3. DGPAA structure
Discussion
Earlier, we introduced Dobrow et al.’s (Reference Dobrow, Goel and Upshur2004, Reference Dobrow, Goel, Lemieux-Charles and Black2006) notion of evidence-based decision-making being a product of ‘external’ and ‘internal’ dynamics and we now return to these resources as a vehicle to reflect on our empirical experiences. Features related to their fixed ‘external’ context – particularly what they term ‘the epidemiological context’ – were evident. PA has increasingly been seen as simple and profound (and perhaps thus attractive) ‘solution’ to multiple ‘problems’: for example, increases in chronic diseases like CHD and diabetes; high levels of mental health difficulties; low levels of social functioning in older people; and most broadly, accelerating environmental degradation (WHO, Reference World Health Organization2018). These beliefs were expressed in our initial stakeholder workshop and were perhaps reflective of the fact that the workshop was able to attract CEO and significant senior-level representation. At the same time, in calling for the PA sector to identify ‘best buys’ that achieve ‘bang for buck’, the stringency of the political and economic features of this external context also set clear boundaries around any notional favourability. Subsequently, this precise and evocative language had significant currency within our decision-making circle. It broadly framed the issue and initiated actions along very particular lines – foregrounding relatively ‘objectivist’ assumptions and approaches that culminated in the commissioning of PAHRC ‘best investment’ review and subsequent phases.
The evocation of the ‘bang for buck’ idiom can be seen here as an explicit ‘political device’ (Stone, Reference Stone2002), having the potential to be purposeful in suggesting ways forward (Annas, Reference Annas1995). For some, the deployment of this uncomplicated and powerfully articulated desire might seem to provide a motivating dynamic in arriving at optimal investment choices (Allbritton, Reference Allbritton1995). However, the potential to utilise inappropriate idioms is also recognised (Lakoff, Reference Lakoff1995). In this case, Safire (Reference Safire1972) notes that ‘bang for buck’ originated in 1950s US defence policy narrative and related literally to ‘the more efficient use of the U.S. defence budget…the ‘bang’ referred to was a nuclear explosion’ (Powers, Reference Powers2010: 135).
Pokhrel (Reference Pokhrel2015) has highlighted a problematic principle of applying simple concepts uncritically to intricate public health issues and the translation here – from the relative clarity of the (defence) business sector to the complexity of public health – might seem a good example of this critique. Furthermore, the ‘bang for buck’ framing was considered relatively open-ended with little notion of whether the appraisal required would contribute to disinvestment and reinvestment within existing PA spend (recognised earlier as a form of ‘technical efficiency’) or would attract additional resources from outside the domain (that is, benefit from a process of assessing ‘allocative’ efficiency).
Various aspects of Dobrow et al.’s (Reference Dobrow, Goel and Upshur2004) ‘internal’ dynamic were also apparent. In contrast to the relatively uncomplicated and absolutist nature of the external features above, these internal perspectives problematised these assumptions. Our informants (at both strategic and operational levels) quickly highlighted inherent complexity and the invested professional and political values at play. In keeping with Allen et al.’s (Reference Allen, Pullar, Wickramasinghe, Williams, Roberts, Mikkelsen, Varghese and Townsend2018) recognition of the shortcomings of WHO ‘best buy’ approaches and the acknowledged difficulties in simply ‘transferring’ or ‘translating’ policies and practices (James and Lodge, Reference James and Lodge2003), this was expressed particularly in relation to scepticism over the utility of centrally derived guidance in their local circumstances. Rather, informants were conscious of various local contingencies – in relation to the possibility of multiple health and social PA outcomes, the complexity involved in understanding the most effectual processes that underpin outcomes and setting realistic timeframes – in which any impacts might be assessed.
A series of more specific themes were also evident. First, the tendency for informants to often feel swamped by supposedly affirmative ‘evidence’ data for a multitude of intervention types was prominent. Participants in various organisational settings repeatedly voiced the fact that they were well-aware of such generalised ‘best investment’ guidelines but noted that, paradoxically, these appeared all-inclusive and thus lacked meaningful specificity. For example, many felt that the GAPA ‘7 Best Investments for Physical Activity’ contained nebulous guidance that uncritically (and apparently devoid of any visible methodological foundation) covered multiple forms of PA, lifecycle groups, preferred settings and public health approaches. They also included some (such as ‘mass media’) that were considered approaches with a relatively poor evidence base. Some also saw these frameworks as overly directive, which if followed in strict utilitarian terms would exclude locally favoured actions falling outwith its guidance. In keeping with Atkins et al.’s (Reference Atkins, Kelly, Littleford, Leng and Michie2017) recognition of the weight of localised political influence and the existence of well-established and funded PA interventions, such evidence was therefore seen as only one relatively minor and contestable ingredient amongst a range of grounded precedents and principles.
Second, the sheer variety of interventions associated with PA promotion posed difficulties in comparing ‘like-for-like’. Masters et al. (Reference Masters, Anwar, Collins, Cookson and Capewell2017) note the general difficulty in defining what constitutes a ‘public health’ intervention, particularly those focused on structural determinants of health. In our case, PA ‘interventions’ ranged from environmental adaptations (such as cycle path construction) through to sports projects. Whilst changes to the built environment can theoretically both contribute to supportive PA infrastructure and be ‘cost-effective’ in increasing PA (Webber et al., Reference Webber, Chalkidou, Morrow, Ferguson and McPherson2018), the tendency in our pragmatic decision-making circumstances was to see these actions as problematic – either too costly to initially establish and maintain in the longer term or an insufficiently robust vehicle to directly and immediately promote PA. As such, the inclination to favour actions that were seen to immediately and directly benefit individuals was clearly evident (Richardson, Reference Richardson2012), many expressing a view that it was easier to fund projects with identifiable participants (such as sports clubs) rather than wider conducive environmental contexts (such as paths).
Third, by implicitly pursuing a competitive league table (‘best’) form of prioritisation, it was felt that the process had the effect of inhibiting a holistic approach to PA promotion. Despite a growing interest in moving towards an integrative situation (King’s Fund, 2013: 24), the tendency for commissioning to be based on assigning blocks of resource to short term ‘individual items’ was still prominent. In this sense, we were acutely aware that in assessing individual ‘best buys’ within the existing format of the DGPAA, we were acting against the need currently being espoused for a ‘systems-based’ PA promotion formed of a combination of complementary approaches (Rutter et al., Reference Rutter, Cavill, Bauman and Bull2019).
Finally, in line with general views suggested by Hunter et al. (Reference Hunter, Boeri, Tully, Donnelly and Kee2015), some expressed concerns that these crude one-dimensional investment rubrics were leading to a drive to simply increase levels of unqualified ‘participation’, masking the fact that these may be individuals who are already ‘active’. As such, some believed that these circumstances overrode our ‘inequality’ principle (addressing those who are the most inactive) and that approaches to appraisal mechanisms needed to be sensitive to policies that sought to redress PA inequalities (Nordha et al., Reference Nordha, Vistadb, Skårb, Woldb and Bærum2017). These circumstances were felt to be exacerbated by local PA programmes often relying on short term funding based on ‘raw participant’ targets.
Conclusion
This article has described a process by which we have established foundational approaches and frameworks that will form the basis of future PA related investment decision-making in D&G. Principally, we sought to resist simple objectivist orientations to evidence (Klein, Reference Klein1993) and have begun to nurture a more comprehensive process that recognises many of the difficulties consistently identified in the literature on making nuanced investment decisions in complex public health circumstances (Tucker and Roth, Reference Tucker and Roth2008).
Practically, we have placed pluralism and communication at the heart of our approach where consultation has been undertaken and critical debate encouraged across diverse groups and sectors. The combined strands of work have put the region in a position of having a foundation for a ‘whole system’ approach to PA investment decisions. We are admittedly still technically and politically a way off from achieving a mechanism that will fully deal with these complex investment questions. Though, in upholding Haynes et al.’s (Reference Haynes, Devereaux and Guyatt2002: 1350) observation that ‘evidence does not make decisions, people do’, we believe in undertaking and reporting on the complex process we have been through, we have started to address Pokhrel et al.’s (Reference Pokhrel, Owen, Coyle and Coyle2017) identification of the lack of detailed grounded insights into the engagement between intervention appraisal and policy and practice.
We recognise that, for some, such acquiescence may be seen as a surrender to the forces of subjectivity and localism – a retreat from aspirations of achieving objective ‘evidence-based’ practice. However, our contention is that in ‘wicked’ (Rittel and Webber, Reference Rittel and Webber1973) policy circumstances, such rudimentary approaches are fundamentally inappropriate. Furthermore, we suggest that compared to clinging on to a shallow and ultimately limited sense of objectivity in ‘evidence’ and ‘best investments’ movements, in being ‘realistic’ about the complex circumstances that surround PA, the potential to achieve some semblance of critical realism is more likely.
In conclusion, we offer a range of specific insights from our experiences. Primarily, we point to a range of alternative ‘multi-criteria’ evidence-based resources (Angelis and Montibeller, Reference Angelis and Montibeller2016: 76) that have informed our approach and have the potential to be deployed more widely. We also suggest the need to see various efforts to assess, interpret and act on the value of PA related interventions as an on-going and incremental social and political process rather than a one-off technical exercise based on seeking simple ‘answers’. Relatedly, within this process, any summative appraisal of impacts or outcomes can be most profitably seen in short, medium and long-term public health contexts and formal ‘best investment’ data considered as one element of a broader heuristic process.
Additionally, recognising Atkins et al.’s (Reference Atkins, Kelly, Littleford, Leng and Michie2017: 1) notion of ‘reverse or re-engineer the traditional pipeline of guideline development by starting with local need and examples of effective local practice rather than….evidence….from the international scientific literature’, these practices should actively involve as wide a range of local stakeholders and associated perspectives as possible. These actors would naturally have a particular interest in PA promotion but could also contain those whose concerns are for generic health and social sector policy and planning. In this context, we believe it is important to come to a common understanding of the nature of the varied ‘returns’ derived from PA – preferably based on more profound ‘socially valued’ impacts rather than simple short-term cost savings.
Finally, we see it as crucial that such decision-making be done ‘in-the-round’ variously: deploying theoretically and empirically derived best investment principles; allowing values (particularly those related to inequality) to calibrate the appraisal of ‘success’ and shape investment decisions; and applying these principles to a pool of possible investments, rather than single isolated interventions.
Acknowledgements
The authors would like to thank the contributions of those from the Dumfries and Galloway Physical Activity Alliance.
