How this fits in with Primary Healthcare Research and Development
What do we know? Evidence for the effect of quality improvement interventions exists across a range of healthcare settings and implementation levels; quality improvement interventions can be successful in improving care structure, processes and outcomes.
What does this paper add? This paper reviews systematic reviews of quality improvement interventions, specifically in primary care and at the practice level, highlighting those interventions most likely to produce a positive quality improvement effect and the context in which this effect may be enhanced – for example, through recognising baseline performance and assessing improvement potential, planning quality improvement design and intervention selection and being aware of the improvement opportunity in targeting prescribing behaviours. The paper identifies areas for future research, such as the use of theoretical quality improvement frameworks and cost-effectiveness analysis in analysing primary-care quality improvement interventions.
Background
Primary care occupies a significant and increasing role in healthcare in the United Kingdom and internationally. There is available research literature on quality improvement (QI) across a range of primary-care settings and contexts and these include QI at various levels of care: the individual patient level (e.g., patient self-care promotion), practitioner level (e.g., practitioner audit and feedback), practice level (e.g., practice-based patient review and recall) and the wider national and policy level (e.g., standard setting, accreditation development and financial incentives schemes such as the UK quality and outcomes framework)(Ferlie and Shortell, Reference Ferlie and Shortell2001). ‘Quality improvement’ is a broad term with a number of different definitions and interpretations. With roots in the early work of W. Edwards Deming and Walter Shewhart in understanding variation, QI is usually described as a continuous effort to improve processes that include identifying a problem, examining and applying solutions and monitoring for improvement (Shewhart, Reference Shewhart1931; Deming, Reference Deming1982; Urowitz et al., Reference Urowitz, Gladman, Ibanez and Berliner2006; Varkey et al., Reference Varkey, Reller and Resar2007).
This tertiary evidence synthesis (a review of systematic reviews) identifies and appraises primary-care QI interventions relevant to UK primary care and other healthcare systems at the practice level. For the purpose of the review, a ‘practice’ is described as an organisation including a multidisciplinary primary-care team of a single or multiple general practitioner(s), health professionals, and administrative staff, often based within an ambulatory setting or environment and acting as a gatekeeper, central entry and co-ordination point for healthcare delivery (Moore et al., Reference Moore, Niebler, Schlundt and Pichert2007). Given that systematic reviews usually focus on one type of intervention, there is an increasing need to carry out an overview, also termed a ‘review of reviews’, of given topics to ensure all relevant systematic reviews are synthesised and presented in a form that is useful to healthcare practitioners and those delivering QI in routine clinical practice (Grimshaw et al., Reference Grimshaw, McAuley, Bero, Grilli, Oxman, Ramsay, Vale and Zwarenstein2003). The specific objective of this review of reviews was to examine and critically appraise the evidence relating to practice-level primary-care QI initiatives relevant to UK primary care.
Methods
This review was informed by available guidance on conducting systematic reviews of systematic reviews and particularly in the context of source identification, study selection, quality assessment, presentation of results and defining the implications of the research conducted (Smith et al., Reference Smith, Devane, Begley and Clarke2011).
Searches
A literature search of the following databases was performed: ATHENS, MEDLINE, EBSCO HOST, EMBASE, CINAHL, the University of Birmingham E-Library and the Cochrane Database (with date search discrimination of papers from June 1994 to June 2014). Use of specified databases was chosen based on reputation and the requirement to use two or more databases to provide a comprehensive search (Ayevard, Reference Ayevard2008). In searching for appropriate literature, key terms related to the research objective were utilised. Key terms included examples such as ‘quality improvement’, ‘variation’, ‘outcome improvement’, ‘performance improvement’ and ‘practice improvement’. Using these key terms, related terms, Boolean and phrase searches to focus the search strategy, and in using operators in between key words to combine them, enabled acquisition of literature through the use of databases and journal searching programmes. Where available for bespoke adaptation, electronic searches were targeted by specified inclusion criteria, search modes, expanders and limiters – for example, by ensuring returns of systematic reviews after the application of limited publication dates and Boolean/phrase searches. The reference lists and bibliographies of the included reviews were also searched for relevant reviews.
Table 1 details the number of results obtained from each database source after applying the search criteria.
Table 1 Results (number) of papers returned by database after applying the search criteria
Selection of systematic reviews
At least two reviewers independently assessed the retrieved studies for inclusion against the inclusion and exclusion criteria and reached agreement through discussion.
Inclusion criteria
Selected studies had to satisfy the following criteria for inclusion based on examination of the full review text: the literature had to be a systematic review (using the key principles and characteristics of systematic reviews as defined by Hemingway and Brereton, Reference Hemingway and Brereton2009: 1), including all relevant Cochrane Effective Practice and Organisation of Care (EPOC) Group reviews working to defined Cochrane review protocols (Cochrane Effective Practice and Organisation of Care Group, 2014); the literature had to be published after 1994 (representing a 20-year search span); the literature had to be relevant to the research questions and include QI and/or understanding variation as included and prioritised aspects of the study; and the literature had to be from the UK primary-care setting and/or a comparable first-contact primary-care settings elsewhere. These were defined through the three-person author group as follows: The United Kingdom, Republic of Ireland, The Netherlands, Finland, Denmark, Sweden, New Zealand, Norway, Spain, Italy and Portugal. This was based on criteria where (a) patients register with a practice that is their usual provider; (b) strong gatekeeping exists – that is, primary care is the first point of contact and non-emergency access to secondary care is via primary care; (c) primary care is delivered through a team rather than a physician alone; (d) primary care is funded via a largely capitation-based payment system (i.e., not fee for service); and (e) the literature had to be focussed at the primary-care practice level or with QI interventions practically implementable at the practice level.
Further verification of comparable primary-care settings was cross-referenced through the following sources: The Health Systems and Policy Monitor, The Commonwealth Fund International Profiles of Health Care Systems and conference proceedings from the European Forum for Primary Care (European Forum for Primary Care, 2011; The Commonwealth Fund, 2012; European Observatory on Health Systems and Policy, 2014). The qualification of countries representing primary care with relevance to the UK setting is provided, given that within different primary-care settings present in international comparisons, variable factors such as payment method, financial incentives and features of the primary-care system can have an impact on areas as diverse as resource utilisation, compliance, performance target improvement, quality and outcomes (Chaix-Courtier et al., Reference Chaix-Couturier, Durand-Zaleski, Jolly and Durieux2000; Atun, Reference Atun2004; Gosden et al., Reference Gosden, Forland, Kristiansen, Sutton, Leese, Giuffrida, Sergison and Pedersen2011).
Exclusion criteria
Reviews were excluded where the primary intervention or focus of the research was on primary-care funding, financing and regulatory change, due to these being influenced at the system level as opposed to the practice level. Reviews covering topics already examined through the Cochrane EPOC Group were excluded and were often part of, or referenced within, EPOC review updates.
Assessment of quality
The lead author (R.I.) assessed the quality of each of the included systematic reviews using the Critical Appraisal Skills Programme (CASP) tool. The other two authors (T.S., T.M.) independently assessed the quality of assessment of a sample of the included studies. The CASP appraisal checklist for systematic reviews was selected for use in the defined literature review in order to provide an economical and structured approach to systematic review appraisal, where the widely recommended and reputable CASP tool in particular promotes understanding about review validity, meaning and applicability (CASP, 2013; McLean et al., Reference McLean, Sheikh, Cresswell, Nurmatov, Mukherjee, Hemmi and Pagliari2013; Singh, Reference Singh2013). Two systematic reviews were excluded after application of the CASP tool due to lack of clarity around defining a clear research question, which resulted in interventions not easily categorised as QI interventions, and where the systematic review was not specifically focussed enough to be relevant to the UK primary-care setting at the practice level. Table 2 provides a summary of the application of the CASP checklist for systematic reviews included for full review.
Table 2 Summary of CASP application to systematic reviews
CASP=critical appraisal skills programme; IQR=interquartile range; RCTs=randomised control trials; BP=blood pressure.
C. denotes Cochrane EPOC review.
Key: ✔=Yes (also indicates not applicable); X=No; –=Can’t tell
Figure 1 details the flow chart and results of papers included for the review after application of the inclusion and exclusion criteria, quality check and CASP appraisal.
Figure 1 Flow chart of paper selection – inclusion and exclusion process. CASP=critical appraisal skills programme.
Data extraction
Data from the included reviews were extracted by one of the authors to record the author(s), date of publication, review purpose, QI intervention, review outcomes and review conclusions.
Data synthesis
The presented review of reviews is a tertiary-level synthesis of secondary-level data – that is, systematic reviews – which were analysed by the overall effect of the intervention examined within each review. The overall effect was determined in different ways across different systematic reviews, as can be seen in Table 3, by review author(s) adopting various quantitative (e.g., meta-analysis) or qualitative (e.g., meta-synthesis) methods for synthesising primary-level data. The overall examined and reported intervention effect within each systematic review and author conclusions were used as the basis for assessing the intervention effect. Clearly, the evidence of overall QI intervention effect in systematic reviews is influenced by the number and quality of the included primary-level studies. However, although individual analysis of the included studies within systematic reviews was not conducted in this review of reviews, quality assessment and application of the CASP tool for systematic review inclusion assessed the appropriateness and rigour of the included studies within reviews. Randomised control trials were the most common study design present in systematic reviews; however, non-randomised controlled clinical trials (CCT), interrupted time series and before and after studies were also present.
Table 3 Quality improvement interventions and effect
RD=risk difference; IQR=interquartile range; SMD=standardised mean difference; 95% CI=95% confidence interval; GP=general practioner; RCT=randomised control trial; RR=risk ratio; OR=odds ratio; EBM=evidence-based medicine; QI=quality improvement; LDL=low-density lipoprotein.
Heterogeneity was present in the included reviews across areas such as interventions applied, disease area and primary outcome measures and was not comparable in nature; therefore, purely quantitative statistical approaches to present the findings were deemed inappropriate. Furthermore, numerical and statistical values across both reviews and their included studies were not comparable, sample size discrimination was not always conducted, and therefore the computation of standard error was deemed unfeasible. As such, the included reviews were examined for whether the QI intervention applied demonstrated evidence of an effect for the intervention with reported confidence intervals where stated (Table 3). Furthermore, data synthesis occurred through examining the QI intervention effect by outcome category (Table 4) and the level of change using the framework produced by Ferlie and Shortell (Reference Ferlie and Shortell2001), as summarised in Table 5. The Ferlie and Shortell framework was applied to further distinguish whether QI intervention effects were primarily conducted within, and influenced by, a specific level of change – for example, interventions applied at the individual compared with the team level within a practice setting.
Table 4 Quality improvement interventions by outcome category
QI=quality improvement.
Table 5 Four levels of change for improving quality
A number of included systematic reviews made reference to QI strategies or multiple interventions as part of a wider QI programme. These reviews were analysed with single-intervention reviews as above and assessed for general themes, assessing any impact that individual interventions had within these combined QI strategies and approaches.
Results
A total of 31 systematic reviews were included for further assessment after application of the search strategy, application of the inclusion criteria and full paper review. Eight systematic reviews were excluded by the three-person review group; common reasons for exclusion included systematic reviews with a published date before 1994, reviews with a focus on primary-care settings either not comparable with the UK setting (e.g., research based exclusively in the United States) or not relevant to the practice setting and reviews not clearly demonstrating a QI intervention.
Single interventions
The specific QI interventions reviewed that the demonstrated strongest effect (whether on care structure, process or outcomes) included audit and feedback, point-of-care reminders, computerised advice, practice facilitation and interventions for evidence-based guideline adoption, adherence to clear practice structures and processes for patient review, continuing education and educational outreach, as presented in Tables 3 and 4 (Renders et al., Reference Renders, Valk, Griffin, Wagner, Van Eijk and Assendelft2000; Grimshaw et al., Reference Grimshaw, McAuley, Bero, Grilli, Oxman, Ramsay, Vale and Zwarenstein2003; Holden, Reference Holden2003; Fahey et al., Reference Fahey, Schroeder and Ebrahim2005; O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; Smith et al., Reference Smith, AllWright and O’Dowd2007; Reference Smith, Soubhi, Fortin, Hudon and O’Dowd2012; De Belvis et al., Reference de Belvis, Pelone, Biasco, Ricciardi and Volpe2009; Forsetlund et al., Reference Forsetlund, Bjorndal, Rashidian, Jamtvedt, O’Brien, Wolf, Davis, Odgaard-Jensen and Oxman2009; Shojania et al., Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009; Gallagher et al., Reference Gallagher, Lusignan, Harris and Cates2010; Baskerville et al., Reference Baskerville, Liddy and Hogg2012; Gillaizeau et al., Reference Gillaizeau, Chan, Trinquart, Colombet, Walton, Rege-Walther, Burnand and Durieux2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012; Lau et al., Reference Lau, Hu, Majumdar, Storie, Rees and Johnson2012; Tricco et al., Reference Tricco, Ivers, Grimshaw, Moher, Turner, Galipeau, Halperin, Vachon, Ramsay, Manns, Tonelli and Shojania2012). The evidence of effect for these interventions was also replicated in reviews of broader QI approaches comprising several interventions. For example, in their review of interventions for improving vaccination rates, Lau et al. (Reference Lau, Hu, Majumdar, Storie, Rees and Johnson2012) observed many of the described interventions (audit and feedback, point-of-care reminders and practice outreach) as having a particularly large individual and combined effect for improvements in disease-specific vaccination rates as measured through statistical analysis of odds ratios. Audit (whether individual, single practice or multi-practice), feedback and continuing education are cited as having the greatest direct effect on patient outcomes, with other interventions primarily affecting process improvement. Where baseline performance or achievement was low for the measured outcome, the resultant effect size was larger (Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012). For those reviews that did not measure outcomes directly, it must be recognised that care structure and process improvement can also be correlated with improved outcomes (The King’s Fund, 2012).
In categorising evidence of the QI effect against three outcome areas of process improvement/adherence with desired practice, improvement in physiological/biological outcomes and other patient outcomes, the QI intervention effect was more consistently successful in improving process and/or adherence with the desired practice compared with other outcomes. However, it must be stated that categorisation of outcomes in this way is not necessarily sensitive or scientific enough to draw conclusions on QI success across these areas.
Prescribing
A number of reviews sought to improve prescribing or prescribing behaviour as the key outcome. Ivers et al. (Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012: 10) describe prescribing behaviours as ‘important but not complex’. Of those reviews aiming to affect prescribing behaviour, point-of-care reminders, computerised advice and educational outreach visits had the greatest impact (O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; Shojania et al., Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009; Gillaizeau et al., Reference Gillaizeau, Chan, Trinquart, Colombet, Walton, Rege-Walther, Burnand and Durieux2012). There was also evidence that QI interventions produced improvements in prescribing behaviour that were greater than the effects from other measured outcomes, such as patient-level physical and mental health outcomes (Smith et al., Reference Smith, AllWright and O’Dowd2007; Reference Smith, Soubhi, Fortin, Hudon and O’Dowd2012). In these cases, QI interventions that affected prescribing behaviour and practice were more effective than other interventions and particularly when the intention was to improve care for patients with multi-morbidity, where disease management above improving prescribing behaviour may be more complex (Smith et al., Reference Smith, AllWright and O’Dowd2007; Reference Smith, Soubhi, Fortin, Hudon and O’Dowd2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012). The effects on noted QI interventions for prescribing also produced improvement across a range of prescribing-related outcomes, from increased appropriate prescribing volume, improved prescribing compliance, improved prescribing accuracy and dosage and reduced adverse prescribing incidents (O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; Smith et al., Reference Smith, AllWright and O’Dowd2007; Reference Smith, Soubhi, Fortin, Hudon and O’Dowd2012; Shojania et al., Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009; Gillaizeau et al., Reference Gillaizeau, Chan, Trinquart, Colombet, Walton, Rege-Walther, Burnand and Durieux2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012).
Multifaceted interventions
A number of reviews looked at combined or multifaceted interventions (defined as more than one intervention) as part of a QI approach, where multifaceted interventions were sometimes used as comparative arms to single-intervention systematic reviews. The effects on the outcomes of multifaceted interventions was seen across a range of review areas, from condition-specific reviews, such as multifaceted interventions for primary-care management of diabetes, to single-measurement areas, such as reductions in hypertension (Renders et al., Reference Renders, Valk, Griffin, Wagner, Van Eijk and Assendelft2000; Fahey et al., Reference Fahey, Schroeder and Ebrahim2005; O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; De Belvis et al., Reference de Belvis, Pelone, Biasco, Ricciardi and Volpe2009; Baskerville et al., Reference Baskerville, Liddy and Hogg2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012). Multifaceted interventions achieved positive outcomes compared with no intervention or single-intervention in both single practice and multiple-practice QI initiatives (Renders et al., Reference Renders, Valk, Griffin, Wagner, Van Eijk and Assendelft2000; Fahey et al., Reference Fahey, Schroeder and Ebrahim2005; O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; De Belvis et al., Reference de Belvis, Pelone, Biasco, Ricciardi and Volpe2009; Baskerville et al., Reference Baskerville, Liddy and Hogg2012; Ivers et al., Reference Ivers, Jamtvedt, Flottorp, Young, Odgaard-Jensen, French, O’Brien, Johansen, Grimshaw and Oxman2012). Hulscher et al. (Reference Hulscher, Wensing, Grol, Van Der Weijden and Van Weel1999) provide a sensible hypothesis for explaining the multifaceted effect applied to primary and preventive care, stating that the high number of influencing factors involved in this care area are more likely to be affected by a greater number of interventions. Table 6 summarises the impact of multifaceted interventions by the included systematic review. In one review, however, completed by Shojania et al. (Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009), examining point-of-care reminders and other interventions, single interventions realised a higher median improvement compared with multifaceted interventions, and previous authors have also indicated no significant effect of multifaceted approaches to QI through guideline dissemination (Grimshaw and Eccles, Reference Grimshaw and Eccles2004; Squires et al., Reference Squires, Sullivan, Eccles, Worswick and Grimshaw2014). For these reasons, the evidence is not conclusive to suggest that multifaceted approaches are more effective than single interventions.
Table 6 Summary of improvement effects for multifaceted interventions
QI=quality improvement.
C. denotes Cochrane EPOC review.
Organisational level interventions
Reviews by Fahey et al. (Reference Fahey, Schroeder and Ebrahim2005) and Rhydderch et al. (Reference Rhydderch, Edwards, Elwyn, Marshall, Engels, Van Den hombergh and Grol2005) focussed their aims on organisational elements of QI in primary care with some important findings. On QI intervention at the organisational level, Fahey et al. (Reference Fahey, Schroeder and Ebrahim2005) observed that QI programmes addressing primary-care structure, management, organisation and systematic process implementation had a statistically significant effect on blood pressure control and hypertension outcomes. Particularly referenced is the ‘Hypertension detection and follow-up programme’, which, although based on some data from trials over 20-years old, demonstrated that organisation-focussed QI intervention that included hypertension detection and regular review, including addressing medication adherence and blood pressure goal achievement, had a significant impact on hypertension outcomes (Davis and Ford, Reference Davis and Ford2001). Hulscher et al. (Reference Hulscher, Wensing, Grol, Van Der Weijden and Van Weel1999) also cite organisational patient identification and follow-up referral as an effective strategy for included reviews, that focus on screening and referral for breast and cervical cancers and alcohol-related problems. The authors additionally note from an included study that when targeting health promotion behaviour change, extended length of GP appointment by just over 1 min (7.16 min in control versus 8.25 min in intervention arm) meant that GPs were more likely to offer screening and provide or elaborate on health promotion advice that could have a positive impact on disease outcomes.
Ferlie and Shortell (Reference Ferlie and Shortell2001) cited four levels of change for improving quality of healthcare – namely, QI at the individual, group/team, organisation and system/environment levels – reference to this framework in the present review was made when analysing results, although recognising the exclusion of system-level QI interventions for the present review. Comparing the different levels of QI intervention from individual to organisational level, all levels contained a mixture of evidence for an effect and evidence of no effect, whether relating to care structure, process or outcomes. In this respect, it did not become immediately apparent that any single level of intervention realised greater QI outcomes than another.
Discussion
Summary of findings
Audit and feedback, point-of-care reminders, computerised advice, practice facilitation and interventions for evidence-based guideline adoption, adherence to clear practice structures and processes for patient review, continuing education and educational outreach all demonstrated evidence of QI effect. Wider QI strategies, which included these interventions such as clinical practice guideline implementation strategies, also demonstrated evidence of effect (Grimshaw and Eccles, Reference Grimshaw and Eccles2004). The evidence of an effect for QI was higher where baseline performance was low. Prescribing was a healthcare process commonly targeted and improved by QI interventions. Multifaceted interventions achieved positive outcomes compared with no intervention across a range of disease areas in both single-practice and multiple-practice QI initiatives. However, this finding was not present in the review conducted by Shojania et al. (Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009) and has not been replicated in the wider literature reviewed here (Grimshaw and Eccles, Reference Grimshaw and Eccles2004). We also note that a recent review of reviews addressing this question also concluded that there is no compelling evidence that multifaceted interventions are more effective than single-component interventions in affecting healthcare professional behaviours (Squires et al., Reference Squires, Sullivan, Eccles, Worswick and Grimshaw2014). The reason for this may be because multifaceted interventions are usually adopted when there is a need to affect change for an outcome or outcomes that are inherently more difficult to improve (Shojania et al., Reference Shojania, Jennings, Mayhew, Ramsay, Eccles and Grimshaw2009). Equally, in examining the literature around the impact of multifaceted interventions, the primary outcome measure is important. In this review, multifaceted interventions predominantly aimed to affect change through structured QI programmes or QI interventions aimed at whole-care areas and appeared to show a positive effect of multifaceted approaches compared with no intervention or single intervention. This is compared with other available literature using multifaceted approaches to affect individual healthcare professional behaviour change or compliance to practice which appears to demonstrate that multifaceted interventions are no more effective than single interventions (Grimshaw and Eccles, Reference Grimshaw and Eccles2004; Squires et al., Reference Squires, Sullivan, Eccles, Worswick and Grimshaw2014). Recognising this, from the reviews examined, multifaceted approaches delivered measured QIs across a range of research areas and outcomes and could lead to greater compounded gains when implemented successfully. It must be noted that challenges also exist in discriminating multifaceted interventions from single interventions (e.g., where interventions such as educational outreach are inherently multifaceted).
Other areas of the literature review demonstrated considerations around QI programmes or interventions that showed negative findings, little or no effect of QI interventions. Both the examples relating to role substitution (e.g., between doctors and nurses) demonstrated relatively weak evidence for any impact on improved outcomes (Thompson et al., Reference Thompson, Summerbell, Hooper, Higgins, Little, Talbot and Ebrahim2003; Royal et al., Reference Royal, Smeaton, Avery, Hurwitz and Sheikh2006). In examining all reviews, there were also less-clear impacts of QI initiatives on areas such as mortality and morbidity, but this was mostly due to the lack of appropriate follow-up or indeed primary measure of these areas (Thompson et al., Reference Thompson, Summerbell, Hooper, Higgins, Little, Talbot and Ebrahim2003; Royal et al., Reference Royal, Smeaton, Avery, Hurwitz and Sheikh2006). In respect of this, it is important not to discount QI interventions as having an effect on morbidity and mortality outcomes, where the wider literature highlights a relationship here, particularly in showing that QI for effective prescribing and drug treatment has a clear impact on mortality and morbidity (Turnbull, Reference Turnbull2003; Fahey et al., Reference Fahey, Schroeder and Ebrahim2005). Smith et al. (Reference Smith, Soubhi, Fortin, Hudon and O’Dowd2012) also found mixed effects of interventions (most commonly care co-ordination and enhanced multidisciplinary team care approaches) for improving outcomes related to multimorbidity, other than an indication of improved prescribing, medication adherence and patient satisfaction following intervention, there was mixed evidence that interventions had effects on a wider range of patient health outcomes, care utilisation and patient behaviours.
Strengths and limitations of this study
A key strength of this review and the methodology adopted is that it allows the findings of individual systematic reviews on QI interventions in general practice to be compared, contrasted and summarised (Smith et al., Reference Smith, Devane, Begley and Clarke2011). We also sought to ensure relevance by explicitly identifying QI research in primary healthcare settings where first-contact primary care is the norm, such as UK primary care. As a consequence, some potentially relevant systematic reviews that did not meet these criteria were excluded (e.g., reviews exclusively including studies set in the United States). The justification provided for this inclusion criteria is that it may be difficult to draw firm conclusions regarding QI intervention impact from other primary-care settings that could be influenced by significant external factors such as the political, regulatory or other contexts that primary care operates within (Chaix-Courtier et al., Reference Chaix-Couturier, Durand-Zaleski, Jolly and Durieux2000; Atun, Reference Atun2004; Gosden et al., Reference Gosden, Forland, Kristiansen, Sutton, Leese, Giuffrida, Sergison and Pedersen2011). In addition, there are other considerations to note, where the variation in primary-care delivery in the United States, for example, makes differentiation of a first-contact primary-care research setting difficult to identify – for example, where primary care operates within an accountable care system, purely corporate arena, demographically biased insurance market or other contexts (Starfield et al., Reference Starfield, Shi and Macinko2005; Kringos et al., Reference Kringos, Boerma, Hutchinson, Van Der Zee and Groenewegen2010). It is suggested, however, that review findings are still likely to be relevant to those operating in other primary-care settings for two reasons. First, due to research summaries and systematic reviews from other settings – for example, the United States and Australia – identifying and promoting similar improvement effects for the key QI interventions examined, such as audit and feedback, practice facilitation and point-of-care reminders (Garg et al., Reference Garg, Adhikari, McDonald, Rosas-Arellano, Devereaux, Beyene, Sam and Haynes2005; Taylor et al., Reference Taylor, Peikes, Geonnotti, McNellis, Genevro and Meyers2014). Second, because, although reviews exclusively from settings outside the defined inclusion criteria were excluded, many reviews included contained a mix of research settings, including those such as North America and Australasia. One of the key limitations to this study is the possibility of omission of QI research that was not captured through the targeted search strategy; however, a broad search strategy was deployed that generated a high number of systematic reviews in primary care, which were subsequently reviewed for relevance to primary-care QI. A potential limitation also arises where the same individual study or studies could be included in more than one review. Although this is possible given the high total number of overall studies, it was not felt that the impact of this would be significant based on assessing the occurrence of this in reviews of similar topics. The systematic reviews included were analysed as the unit of analysis and therefore lack of primary study applicability within reviews to UK primary care could be seen as a limitation of the review. In terms of the methodological quality of the review of reviews it is noted that we did not rigorously apply dual author study appraisal and data extraction. The fact that a single author led on these steps may have introduced bias, although we attempted to minimise these by ensuring that the other two authors quality assured the appraisal process.
Other considerations
Financial and economic considerations
From an economic and financial analysis perspective, cost-effectiveness was not explored in the present review and this is primarily because, as is common in QI literature, data around cost and cost-effectiveness were lacking and limited to evaluation of specific areas identified under health technology assessment as opposed to QI initiatives (NHS Quality Improvement Scotland, 2009). For the included reviews, although cost-effectiveness analysis was limited, for reviews that did make a reference to cost-effectiveness, the impact was either cost neutral or cost saving compared with usual care. However, it must be stated that comprehensive cost-effectiveness analysis was not completed and was usually based on a small number of studies (Thompson et al., Reference Thompson, Summerbell, Hooper, Higgins, Little, Talbot and Ebrahim2003; O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; Gillaizeau et al., Reference Gillaizeau, Chan, Trinquart, Colombet, Walton, Rege-Walther, Burnand and Durieux2012). Furthermore, some authors comment that cost-effectiveness is highly dependent on the manner in which a QI intervention is delivered and the variation present in this respect could be difficult to examine (Thompson et al., Reference Thompson, Summerbell, Hooper, Higgins, Little, Talbot and Ebrahim2003; O’Brien et al., Reference O’Brien, Rogers, Jamtvedt, Oxman, Odgaard-Jensen, Kristoffersen, Forsetlund, Bainbridge, Freemantle, Haynes and Harvey2007; Gillaizeau et al., Reference Gillaizeau, Chan, Trinquart, Colombet, Walton, Rege-Walther, Burnand and Durieux2012).
The importance of cost-effectiveness, although not examined in this review, is recognised as central to supporting effective decision making for the health economy in application of health research.
Implementation and change science and theory
Other combined areas of importance not specifically explored or indeed present in the included reviews are that of theoretical and scientific perspectives of and on change, implementation science, knowledge translation and addressing barriers to change in QI. Grol et al. (Reference Grol, Bosch, Hulscher, Eccles and Wensing2007) describe the importance of using theoretical and scientific perspectives in planning and studying improvement in patient care. Within their paper, the authors describe the complexity of healthcare and indeed QI in healthcare that is influenced by contextual factors not always considered in healthcare improvement interventions, which typically target individual professionals as opposed to the wider, complex environment of change. The authors argue for an increasing need for improvement research to use theoretical perspectives to plan and study for improvement that helps create understanding of the obstacles, success criteria and incentives of change in the healthcare setting (Grol et al., Reference Grol, Bosch, Hulscher, Eccles and Wensing2007). The authors continue to comprehensively review the range of impact and process theories present in change implementation (summarised in Table 7), stating that, although no single theoretical framework appears superior, evidence exists for their use and application depending on the change intervention, and therefore these frameworks should be applied to improve QI understanding and implementation (Grol et al., Reference Grol, Bosch, Hulscher, Eccles and Wensing2007). Furthermore, there is a large and growing body of research that suggests that implementation of improvement initiatives is as important as the evidence-based QI initiative itself (Cheater et al., Reference Cheater, Baker, Gillies, Hearnshaw, Flottorp, Robertson, Shaw and Oxman2009; May et al., Reference May, Mair, Finch, Macfarlane, Dowrick, Treweek, Rapley, Ballini, Nio Ong, Rogers, Murray, Elwyn, Legare, Gunn and Monori2009; Dixon-Woods et al., Reference Dixon-Woods, Bosk, Aveling, Goeschel and Pronovost2011).
Table 7 Theories of change in healthcare
Adapted from Grol et al. (Reference Grol, Bosch, Hulscher, Eccles and Wensing2007: 105–06).
Sustainability
As a final note on implementation, the topic of sustainability deserves attention. Keller and Aiken (Reference Keller and Aiken2009) note that one of the key problems in QI and change management is that of sustainability and draw attention to the findings by John Kotter in 1995 that change programmes typically only have around a 30% success rate; a rate that was mirrored in examining the change programmes of a large, global management consultancy firm in 2008 (Keller and Aiken, Reference Keller and Aiken2009). Among the reasons for lack of sustainability, areas such as leadership commitment, organisational capacity and capability, reinforcement and motivational alignment are cited (Keller and Aiken, Reference Keller and Aiken2009). Models for improving sustainability exist; however, the application of theoretical frameworks in this context is further underlined to understand the issues surrounding sustainability (Feldstein and Glasgow, Reference Feldstein and Glasgow2008).
Related to this, two areas perhaps the most closely related to organisational QI that are interestingly absent from the literature review are features of leadership and the role of leaders in creating a QI culture. In their paper about improving quality of healthcare in the United Kingdom and in previously referencing and utilising their levels of change for QI, Ferlie and Shortell (Reference Ferlie and Shortell2001) discuss leadership and culture extensively in relationship with QI. Leadership may not necessarily present as an individual, but may take the form of a network or group leadership, including clinical leadership, and is cited as being of central importance in creating and working within a culture that facilitates QI through learning, collaboration and a patient focus (Ferlie and Shortell, Reference Ferlie and Shortell2001). It may perhaps be the conceptual levels of terms such as leadership and culture that create their absence in the literature, which is biased to discussing clinical QI interventions.
Implications for policy and practice
The review of reviews presented has many potential implications for policy and practice. Most notably, it highlights key interventions that may be most suitable for designing QI interventions in primary care at the practice level including audit and feedback, point-of-care reminders, computerised advice, practice facilitation and interventions for evidence-based guideline adoption, adherence to clear practice structures and processes for patient review, continuing education and educational outreach. Development of QI interventions should recognise that a larger improvement opportunity may exist where baseline performance is low. Clinicians and non-clinicians alike involved in QI at the primary-care practice level should also recognise the importance of effective QI implementation in parallel with evidence-based interventions, evaluating any QI approaches through a clear framework to support future learning and development.
Areas for future research
Areas for future research identified within the review process were identified as the following:
-
∙ Understanding the role of group/team development in primary-care QI.
-
∙ Understanding the evidence base around leadership and culture in primary-care development; exploring the types and impact of leadership and culture.
-
∙ Understanding the cost-effectiveness of QI interventions in primary care.
-
∙ Identifying the characteristics of successful QI in primary care and successful practices in terms of quality outcomes.
-
∙ Identifying the common barriers to change in primary-care QI.
-
∙ Evaluating the impact of primary-care QI interventions using theoretical frameworks.
Conclusion
There is evidence of effectiveness for a range of primary-care QI initiatives relevant to UK primary care at the practice level. Particularly effective interventions include audit and feedback, point-of-care reminders (computerised and other), practice facilitation and interventions for evidence-based guideline adoption, adherence to clear practice structures and processes for patient review, continuing education and educational outreach.
Multifaceted interventions and interventions aiming to affect prescribing practice appear particularly successful, and QI implementers can enhance success through focussing on effective QI implementation, addressing barriers to change, while recognising that QI will be more effective when baseline performance is low.
More research is required to determine the use and impact of QI interventions using theoretical frameworks and cost-effectiveness analysis.
Acknowledgement
The first author gratefully receives funding from the National Institute of Health Research for primary care research.
Conflicts of Interest
No competing interests or conflicts of interest declared.
Ethical Standards
Ethics committee approval was not required for this review.
Disclaimer
The first author is funded by the National Institute for Health Research School for Primary Care Research (NIHR SPCR). The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health. The NIHR SPCR is a partnership between the Universities of Birmingham, Bristol, Keele, Manchester, Nottingham, Oxford, Southampton and University College London.
Appendix
Table A1 Quality improvement intervention definitions and descriptions
