The number of older adults residing in long-term care (LTC) facilities is projected to triple by 2031 (Brookmeyer, Gray, & Kawas, Reference Brookmeyer, Gray and Kawas1998). Concurrent with this projection, there likely will be a significant reduction in the availability of formal care providers for LTC facilities (Stone, Reference Stone2001). This situation is compounded by high turnover rates caused by care aides’ dissatisfaction with the workplace and high levels of stress and burnout resulting from the physical and emotional demands of their work (Bowers, Esmond, & Jacobson, Reference Bowers, Esmond and Jacobson2003). These trends signal a need for continued attention to the quality of care and life experienced in LTC facilities.
Over the past three decades, regulations have been introduced across North America related to the quality of care provided by the LTC industry (Institute of Medicine, 1996). These reforms were the product of decades of scandals arising from inadequate care and ineffective regulation (Hawes, Reference Hawes, Katz, Kane and Mezey1991), and have resulted in comprehensive changes in the way LTC facilities are regulated and evaluated (e.g., the US Omnibus Budget Reconciliation Act [Morford, Reference Morford1988]; the British Columbia Community Care and Assisted Living Act, Residential Care Regulation [2013]; and the Ontario Regulation 79/10 under the Long-Term Care Homes Act [2010]). Within a context of diminishing and strained resources, there is greater importance being placed upon the provision of high-quality, evidenced-based care in LTC facilities. These efforts have led to the development of practice change interventions aimed at improving the quality of care and life for LTC residents.
Implementing and sustaining practice change in institutions is not easily accomplished, and changing care practices in LTC facilities is especially challenging (Commonwealth Fund, 2007; Corazzini et al., Reference Corazzini, McConnell, Anderson, Reed, Champagne, Lekan and Zimmerman2010). It is imperative that we learn from the experiences and efforts of people who have previously implemented and studied practice change initiatives in LTC facilities.
Previous reviews and critical appraisals of the literature related to intervention research in LTC settings have been published. These reviews have raised serious concerns regarding weaknesses in the methodological approaches of the research undertaken (e.g., small sample sizes, lack of control groups, high attrition rates) (Aylward, Stolee, Keat, & Johncox, Reference Aylward, Stolee, Keat and Johncox2003; Kuske et al., Reference Kuske, Hanns, Luck, Angermeyer, Behrens and Riedel-Heller2007; Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). A common finding of these reviews is that the exclusive provision of lectures or didactic methods of instruction for staff is unlikely to be successful in producing the desired outcomes (Aylward et al., Reference Aylward, Stolee, Keat and Johncox2003; Kuske et al., Reference Kuske, Hanns, Luck, Angermeyer, Behrens and Riedel-Heller2007; Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). Of these reviews, Aylward et al.’s (Reference Aylward, Stolee, Keat and Johncox2003) work is noteworthy because they applied an intervention classification system based on Green and Kreuter’s (Reference Green and Kreuter2005) PRECEDE-PROCEED Footnote 1 model to examine the role of organizational and system factors that facilitated or hindered the implementation of educational initiatives. The PRECEDE-PROCEED model is an evaluation framework that provides a process of identifying desired outcomes and the strategies needed to achieve those outcomes. The PRECEDE-PROCEED model encourages a detailed exploration of the predisposing (knowledge, attitudes, beliefs), enabling (accessibility of resources, motivational devices), and reinforcing factors (rewards or reinforcers) associated with the successful implementation of change. Aylward et al. (Reference Aylward, Stolee, Keat and Johncox2003) reported that in approximately three quarters of the studies in their sample, new knowledge was provided to staff without any organizational or system support to facilitate the transfer of the new knowledge in the workplace.
Building on the work of Aylward et al. (Reference Aylward, Stolee, Keat and Johncox2003), we adopted a modified realist approach for this review. A realist review offers an alternative systematic review process, based on a realist philosophy of science (i.e., a positive epistemic attitude, which endorses belief in both observable and unobservable aspects of the world described by the sciences) (Pawson, Reference Pawson2006). Our aim was to supplement and extend the findings reported by Aylward et al. (Reference Aylward, Stolee, Keat and Johncox2003) and to determine whether the weaknesses they had observed in the research designs and interventions had improved in the past decade. Thus, we modified the realist review method by selecting the PRECEDE-PROCEED model a priori to guide the identification of intervention characteristics that are critical for successful implementation of interventions. Our rationale for using the modified realist review approach was based on the goals of the review and our conceptualization of practice change interventions. The goals were twofold: (a) to explain what practice change interventions “work”, in what circumstances, and why; and (b) to produce pragmatic guidance that could be used by researchers and administrators to optimize the design of future practice change interventions. We conceptualized practice change interventions as “complex interventions” that consist of multiple, diverse participants (e.g., facilitators, learners) and that interact in a non-linear way to produce outcomes that are extremely context dependent (Wong, Greenhalgh, & Pawson, Reference Wong, Greenhalgh and Pawson2010).
Methods
A realist review seeks to explicate the mechanism(s) of how complex interventions work, or why they fail, in particular contexts. This contextually bound approach to causality is represented as context + mechanism = outcome (Pawson, Greenhalgh, Harvey, & Walshe, Reference Pawson, Greenhalgh, Harvey and Walshe2005). For this review, the relevant context was the complex organizational system found in LTC facilities; the mechanisms were the processes operating within an intervention that were meant to enable participants to implement desired changes in practice; and the outcome was the desired change in care staff members’ attitudes, knowledge, or behaviour that would ultimately lead to changes in LTC residents’ behaviour, health, or quality of life.
The steps in a realist review include (a) clarifying the scope of the review; (b) searching for evidence; (c) appraising the primary studies and extracting their relevant data; (d) synthesizing the evidence and drawing conclusions; and (e) disseminating, and possibly implementing and evaluating, the recommendations (Pawson et al., Reference Pawson, Greenhalgh, Harvey and Walshe2005). The approach is based on the principle that, although human agency and interaction are always involved, in certain contexts, people are likely (though not always or certainly) to engage in semi-predictable recurring patterns of behaviour (Wong et al., Reference Wong, Greenhalgh and Pawson2010). Pawson (Reference Pawson2006) referred to these patterns of behaviour as “demi-regularities”. Realist reviews focus specifically on the demi-regularities in the social world that create preconditions for individual human behaviour and, by so doing, enable reviewers to test underlying theories that explain those behaviours (Pawson, Reference Pawson2006). For this review, we sought to determine whether the PRECEDE-PROCEED model would explain whether a practice change intervention in one or more LTC facilities produced desired, positive outcomes for residents and care staff members. The RAMESES criteria guided the conduct and reporting of the review (Wong, Greenhalgh, Westhorp, Buckingham, & Pawson, Reference Wong, Greenhalgh, Westhorp, Buckingham and Pawson2013).
Precede-Proceed Model as a Candidate Theory
In a realist review, the initial identification of a candidate theory (or theories) is necessarily a speculative and iterative process (Wong et al., Reference Wong, Greenhalgh and Pawson2010). It is an expected and acceptable process for members of the review team to initially have theories they believe to be key in explaining why certain outcomes occur. However, these candidate theories are not considered definitive until they have been tested through the review process. We selected Green and Kreuter’s (Reference Green and Kreuter2005) PRECEDE-PROCEED model to assist in determining what it is about practice change interventions in long-term care that make them work and in what circumstances. The PRECEDE-PROCEED model encourages a detailed exploration of the predisposing, enabling, and reinforcing factors associated with the implementation of change, and has been used in previous research in this area (Aylward et al., Reference Aylward, Stolee, Keat and Johncox2003). The data from each study were extracted and classified to determine whether the PRECEDE-PROCEED model had explanatory power in this context, and whether it offered utility in explaining the various outcomes of the research conducted to date.
The PRECEDE-PROCEED model has several strengths as a theoretical perspective in understanding practice change in LTC settings, including recognition that (a) behaviour has multiple causes that must be assessed and evaluated before the development and application of interventions; (b) members of the target population (i.e., care staff members in LTC facilities) must define their own high-priority problems and goals, and develop and implement solutions; (c) change in knowledge and behaviour depends on voluntary cooperation and active participation by members of the target population in a process that enables personal determination; and (d) environmental factors, such as social inequities, policy and procedural guidelines, and the industry itself are salient factors in determining behaviour – that is, the social climate or culture of the facility must be addressed.
Inclusion Criteria
A broad range of intervention studies were included in this review, with a focus not only on psychosocial interventions designed for the care staff, but also on behavioural and pharmacological interventions designed for residents. We chose studies meeting the following criteria for inclusion:
-
1. The setting was a nursing home, LTC facility, or special care unit. Each of these settings is defined as a residential care facility within which 24-hour nursing services are provided.
-
2. The population of interest was LTC care staff members or residents.
-
3. A primary focus of the intervention was practice change related to quality of care, quality of life, or quality of work life.
-
4. There was evident evaluation of the intervention with empirical data provided (i.e., the article manuscript described a primary study as opposed to a review and presented full evaluations, not preliminary results).
-
5. The description of the intervention was detailed enough to enable the reviewers to categorize it based on Green and Kreuter’s (Reference Green and Kreuter2005) characterization of intervention factors.
-
6. The study was published in an English-language journal. Footnote 2
-
7. The publication date was between 1985 and 2014. Footnote 3
Identification of the Primary Studies
The strategy to identify relevant primary studies included (1) searching several electronic databases – Academic Search Complete, Ageline, CINAHL, Medline, PsycINFO, and Social Science – and (2) undertaking manual searches of journals and the reference lists of the retrieved articles. Keywords for the search included long-term care, nursing home, staff training, outcomes, quality of care, evaluation, quality of life, continuing education, and intervention. We conducted several searches using strategic groupings of the keywords in an attempt to more successfully navigate the number of articles that each search yielded. For example, Medline was first searched with the keywords nursing home, staff training, and quality of care, limited to English-language publications dated between 1985 and 2014. The search yielded 115 articles. After removing duplicates, the titles and abstracts were screened to determine if the article met the inclusion criteria. Of the 115 abstracts, five articles met all the criteria and received a full review. In addition, we conducted a subsequent search of each retained article’s reference list. This process was repeated for each database and different strategic groupings of keywords until 94 articles were found that met the inclusion criteria. All keywords were ultimately included in searches in every database. See Figure 1 for further details.
Figure 1: Document flow diagram
Data Management, Analysis, and Synthesis
Phase One – Appraisal and Data Extraction
The first two authors (SC and HC) independently extracted the relevant data from each study with a structured extraction form organized with the following five categories: (a) intervention focus; (b) study goal/intended outcomes; (c) sample size and description; (d) details of the intervention; and (e) intervention factors and findings (including whether follow-up occurred, and if so, over what period) (see Figure 2 for the structured extraction form). Footnote 4 The two coders then independently evaluated the methodological quality of each study by assessing the quality of the evidence and the risk of bias to determine an effectiveness rating (ratings are further described in subsequent paragraphs). Following completion of the data extraction phase, the reviewers compared their completed forms; any discrepancies were discussed and resolved by discussion and consensus.
Figure 2: Sample of the extraction form used to collect data for the review
The intended study outcomes of the selected studies were stratified by the target population: residents or staff. Resident outcomes were classified with three categories:
-
1. Behaviour (e.g., decreased agitation)
-
2. Health (e.g., nutrition, fall rates)
-
3. Quality of Life (e.g., outcomes based on residents’ perceived sense of emotional, social, or spiritual well-being, including safety, physical comfort, enjoyment, meaningful activity, relationships, or dignity)
Staff outcomes were classified with five categories:
-
1. Attitudes (e.g., beliefs regarding person-centred care practices)
-
2. Knowledge (e.g., improved knowledge of delirium)
-
3. Quality of Care: change in actual care practices related to physical and cognitive health and well-being of residents (e.g., improved oral care)
-
4. Quality of Life: changes in actual care practices related to the social, emotional, and spiritual health or well-being of residents (e.g., providing choices)
-
5. Quality of Work Life: changes in quality of work life (e.g., team building, recognition)
The format of the program or intervention (e.g., lecture, videos, on-site consultation, written materials) was noted to better understand the formats used in practice change interventions in LTC facilities. We categorized the interventions on the basis of Green and Kreuter’s (Reference Green and Kreuter2005) characterization of three intervention factors:
-
1. Predisposing factors entail the creation of a shared vision that provides the foundation for change in practice. They involve effectively communicating and disseminating information designed to modify staff members’ knowledge, beliefs, skills, or attitudes (e.g., lectures, written information, group work, didactic training, experiential learning, video presentations, role-playing, or computerized learning).
-
2. Enabling factors are conditions and resources within the environment developed to allow or enable staff members to implement new skills or practices (e.g., modified work schedules, practice opportunities, changes to policy or treatment guidelines, development of new treatment protocols, or access to appropriate resources).
-
3. Reinforcing factors are mechanisms that reinforce the implementation of new skills or practices and motivate their continued use (e.g., providing cues or reminders, improved peer support, timely and appropriate feedback, timely and consistent follow-up, and rewards and recognition for success).
The intervention factors were categorized as (a) predisposing factors only (Type I), (b) predisposing and enabling factors (Type II), (c) predisposing and reinforcing factors (Type III), or (d) predisposing, enabling, and reinforcing factors (Type IV) (Davis et al., Reference Davis, O’Brien, Freemantle, Wolf, Mazmanian and Taylor-Vaisey1999). This classification scheme has been used extensively to design and evaluate interventions intended to influence human behaviour (Glanz, Rimer, & Viswanath, Reference Glanz, Rimer and Viswanath2008).
The quality of the published evidence was categorized as the level of evidence generated from (a) properly randomized controlled trials (RCTs), (b) quasi-experimental (QE) designs (i.e., well-designed controlled trials without randomization), or (c) descriptive case studies (DCSs) or case reports.
Follow-up evaluation of the intervention (i.e., whether the researchers evaluated the sustainability of practice change over time) was noted to better understand the typical length of follow-up used in intervention studies in LTC settings to assist in the evaluation of the intervention effectiveness and to determine whether it was sustained.
The Cochrane Collaboration’s tool for assessing the risk of bias (Higgins et al., Reference Higgins, Altman, Gøtzsche, Jüni, Moher, Oxman and Sterne2011) was used to evaluate risks related to five key domains for each of the randomized studies (i.e., selection, performance, detection, attrition, and reporting bias). We used the Risk of Bias Assessment for Non-Randomized Studies (RoBANS) (Kim et al., Reference Kim, Park, Lee, Seo, Sheen, Hahn and Son2013) to evaluate risks in the non-randomized studies related to five key domains: (a) selection, (b) confounding variables, (c) performance, (d) inadequate blinding, and (e) incomplete outcome data. We classified the risk of bias as 1 = low risk of bias in all key domains with plausible bias unlikely to seriously alter the results; 2 = unclear risk of bias for one or more key domains with plausible bias that raised some doubt about the results; and 3 = high risk of bias for one or more key domains with plausible bias that seriously weakened confidence in the results.
The effectiveness of each study was evaluated independently by the first and second authors. The effectiveness rating was based on a combined overview of the sample size, effect size, outcomes and impact, level of evidence, follow-up, and risk of bias. Following the format utilized by Aylward et al. (Reference Aylward, Stolee, Keat and Johncox2003), the effectiveness was rated as A = good evidence to support a recommendation of effectiveness, B = fair evidence to support a recommendation of effectiveness, C = insufficient evidence to recommend for or against effectiveness, D = fair evidence to support a recommendation of ineffectiveness, and E = good evidence to support a recommendation of ineffectiveness.
Phase Two – Synthesis and Interpretation
Upon completion of the data extraction, we computed descriptive statistics for each of the factors. Next, we identified prominent demi-regularities that might help to explain the divergent outcomes of the practice change interventions undertaken. In phase two, we focussed on the ability of the PRECEDE-PROCEED model to explain the findings reported in the primary studies. Specifically, we attempted to identify recurrent demi-regularities that might have acted as barriers to, or enablers of, the success of the practice change interventions, and tested the explanatory ability of our candidate theory against these. Throughout this process, we sought and highlighted disconfirming data.
Results
Search Results and Study Characteristics
Ninety-four studies met the selection criteria. Twenty-two (23%) of the studies were published between 1985 and 1995; 42 (45%) were published between 1996 and 2005; and 30 (32%) were published between 2006 and 2014. Studies from 10 countries were included in this review with the majority having been conducted in the United States (see Table 1). The most prevalent intervention focus was on some form of putative best-practice in person-centred dementia care (see Table 2). It should be noted that although workplace empowerment and quality of work life were the primary focus of three studies, aspects of these outcomes were included in several studies.
Table 1: Number and relative frequency of studies by country
Table 2: Intervention focus of studies
All of the studies included an educational component in the intervention, and the majority used multiple teaching methods for staff members. The most common method was lecture in combination with role play, group work, experiential learning, or written materials offered as supplementary resources. One study reported having specifically focussed on staff members’ learning preferences and styles, emphasizing, for example, the importance of creative, experiential learning for care aides, who may lack confidence with traditional, passive approaches to learning, such as classroom style teaching (Kemeny, Boetther, DeShon, & Stevens, Reference Kemeny, Boettcher, DeShon and Stevens2006). Smith et al. (Reference Smith, Buckwalter, Garand, Mitchell, Albanese and Kreiter1994) made a similar argument when they suggested that how an educational initiative is delivered is as, if not more, important than the content.
The duration of the interventions varied widely, ranging from one 40-minute session to a 60-hour training initiative. The reported ongoing support for the staff, following the initial training intervention (classified as a reinforcing factor), ranged from a three-week hands-on phase to 24 months of follow-up support. The duration and depth of support following the intervention seemed to be predictive of effectiveness. This observation is examined in more detail within the discussion that follows on the intervention factors.
The majority of the studies (n = 72; 77%) had multiple goals and intended outcomes related to the interventions. Of the studies reviewed, 31 (33%) had goals related to changes in residents’ behaviour, 13 (42%) of which reported significant improvement following the intervention. The most common goal related to residents’ behaviour was reduced agitation or aggression. Of the studies that reported follow-up evaluations, the majority (n = 51; 79%) reported some aspect of the outcome being sustained over time. Nineteen (29%) of these studies reported sustained changes in the residents’ behaviour.
Improvement in the care staff’s provision of care was the most frequently cited goal of the studies reviewed (n = 43; 46%). Of these studies, 25 (58%) reported improvements in the quality of care, 14 of which were sustained for one month or more following the intervention. The second most common goal included changes in the care staff members’ knowledge (n = 36; 38%). Twenty-six (72%) of these studies reported improvements as outcomes. However, less than one half of the studies reporting improvements in knowledge (n = 10) found these improvements to be sustained for three months or more. The most frequent method used to measure change in the care staff members’ knowledge was a simple pre- and post-intervention written test.
Thirty-two (34%) of the studies indicated that improvement in the quality of work life of the care staff was a goal. Of these, only eight (25%) of the studies found that improvements resulted. Quality of work life was most often measured in relation to absenteeism, intention to leave the employer, and employee turnover rates. In addition, some studies included measures related to care staff members’ experiences of physical and verbal assault while on the job. Others measured the quality of work life with questionnaires about job satisfaction. It is important to note that this outcome was measured in a variety of ways, some highly objective while others were relatively subjective in nature.
Our review found that many intervention studies conducted in LTC facilities have relied on the same person or persons to lead the intervention and to collect the data. When this was not the case, the studies relied on the often under-trained and overworked staff to collect the data, many of whom participated in the intervention. This leads to serious concerns regarding the risk of bias. For example, Lekan-Rutledge, Palmer, and Belyea (Reference Lekan-Rutledge, Palmer and Belyea1998) reported perceptions of improvement in residents’ behaviour; fewer skin problems; and improved family satisfaction, all based on staff members’ self-reports. However, none of these claims was verified with objective evaluations of the residents’ records, by direct observation, or direct feedback from family members.
Of the studies reviewed, only 12 (13%) ensured that the data collectors were blind to group assignment. The lack of blinding of outcome assessors was a major source of potential bias in the studies we reviewed. The two other sources of potential bias were related to incomplete outcome data or selective outcome reporting. Of the studies we reviewed, 11 (12%) appeared to selectively report their findings, excluding findings for variables initially indicated as outcome goals of the intervention.
Testing the Precede–Proceed Model
The data extraction process demonstrated that including enabling and reinforcing factors in an intervention was clearly an important feature that helped explain the outcomes of the interventions. We found that care staff members were more likely to engage in practice changes that produced positive resident and staff outcomes if the intervention included more than predisposing factors.
Predisposing Factors
Predisposing factors focus on the communication and dissemination of information designed to modify staff members’ knowledge, beliefs, skills, or attitudes. All (100%) of the studies included predisposing factors as part of their interventions. A significant proportion of the studies (n = 40; 43%) were classified as Type I; they evaluated predisposing intervention factors alone. Each of these studies relied solely on information sharing as their strategy to produce change. It is widely recognized that providing education alone is rarely effective in producing actual change in practice (Aylward et al., Reference Aylward, Stolee, Keat and Johncox2003; Kuske et al., Reference Kuske, Hanns, Luck, Angermeyer, Behrens and Riedel-Heller2007; Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). Seven of the studies demonstrated that improvements in staff members’ knowledge or attitudes did not lead to improvements in care practices (Campbell, Knight, Benson, & Colling, Reference Campbell, Knight, Benson and Colling1991; Cohen-Mansfield, Libin, & Marx, Reference Cohen-Mansfield, Libin and Marx2007; Lintern, Woods, & Phair, Reference Lintern, Woods and Phair2000; Moniz-Cook et al., Reference Moniz-Cook, Agar, Silver, Woods, Wang, Elston and Win1998; Parker, Leggett-Frazier, Vincent, & Swanson, Reference Parker, Leggett-Frazier, Vincent and Swanson1995; Smyer & Brannon, Reference Smyer and Brannon1992; Visser et al., Reference Visser, McCabe, Hudgson, Buchanan, Davison and George2008).
Only three of the 40 Type I studies received an effectiveness rating of A (Gozalo, Prakash, Qato, Sloane, & Mor, Reference Gozalo, Prakash, Qato, Sloane and Mor2014; Isaksson, Paulsson, Fridlund, & Nederfors, Reference Isaksson, Paulsson, Fridlund and Nederfors2000; Ray et al., Reference Ray, Taylor, Meador, Lichtenstein, Griffin, Fought and Blazer1993), and eight (20%) received a rating of B (Avorn & Soumerai, Reference Avorn and Soumerai1992; Bryan, Axelrod, Maxim, Bell, & Jordan, Reference Bryan, Axelrod, Maxim, Bell and Jordan2002; Daly, Smith, Rusnak, Jones, & Giuliano, Reference Daly, Smith, Rusnak, Jones and Giuliano1992; Fossey et al., Reference Fossey, Ballard, Juszczak, James, Alder, Jacoby and Howard2006; Linn, Linn, Stein, & Stein, Reference Linn, Linn, Stein and Stein1989; Pillemer et al., Reference Pillemer, Suitor, Henderson, Meador, Schultz, Robison and Hegeman2003; Rosen et al., Reference Rosen, Mulsant, Kollar, Kastango, Mazumdar and Fox2002; Smith et al., Reference Smith, Buckwalter, Garand, Mitchell, Albanese and Kreiter1994). Thus, the results of this review lend further support to the assertion that interventions classified as Type I, which primarily use information sharing, are largely ineffective in producing change in LTC care practices.
According to Green and Kreuter (Reference Green and Kreuter2005), the creation of a shared vision is a predisposing factor. Only two of the studies indicated that they incorporated vision setting as part of their initiative (Boumans, Berkhout, Vijgen, Nijhuis, & Vasse, Reference Boumans, Berkhout, Vijgen, Nijhuis and Vasse2008; Hegeman, Hoskinson, Munro, Maiden, & Pillemer, Reference Hegeman, Hoskinson, Munro, Maiden and Pillemer2007). In addition, only eight indicated that they sought input from staff members and administration about their training needs and the appropriate focus and content of the intervention (Castle & Bost, Reference Castle and Bost2009; Hegeman et al., Reference Hegeman, Hoskinson, Munro, Maiden and Pillemer2007; Kemeny et al., Reference Kemeny, Boettcher, DeShon and Stevens2006; McGilton et al., Reference McGilton, O’Brien-Pallas, Darlington, Evans, Wynn and Pringle2003; Morgan & Konrad, Reference Morgan and Konrad2008; Palmer & Withee, Reference Palmer and Withee1996; Stevens-Roseman & Leung, Reference Stevens-Roseman and Leung2004; Wilkinson, Reference Wilkinson1999).
Enabling Factors
Enabling factors include conditions and resources within the environment that are developed to allow, or enable, staff members to implement their newly acquired skills and information. Eleven (12%) of the studies included enabling factors and were classified as Type II (included both predisposing and enabling factors). Of these, only one received an effectiveness rating of A (Sloane et al., Reference Sloane, Hoeffer, Mitchell, McKenzie, Barrick, Rader and Koch2004), and two received an effectiveness rating of B (Matteson, Linton, Cleary, Barnes, & Lichtenstein, Reference Matteson, Linton, Cleary, Barnes and Lichtenstein1997; Rokstad et al., Reference Rokstad, Røsvik, Kirkevold, Selbaek, Saltyte Benth and Engedal2013). The majority of the studies (n = 56; 59%) did not include any enabling factors within their interventions.
The most common enabling factor identified was the development and implementation of care plans for residents (n = 11; 29%). It should be noted that individualized care plans are now considered a basic standard of practice in most LTC facilities; however, the creation of care plans does not necessarily ensure that they will be used or implemented. This may be why so many of the studies that used this approach as the primary enabling factor did not produce change in behaviour or outcomes (see Figure 3). Thus, unless other enabling and reinforcing factors are in place, the development of care plans is not sufficient in administrators’ efforts to create sustained change.
Figure 3: Effectiveness rating by intervention type
Administrative commitment to assist and enable the translation of learning into practice is potentially an important enabling factor. However, of the studies reviewed, only five discussed the ways in which they addressed, ensured, and measured administrative support for their initiatives (Hegeman et al., Reference Hegeman, Hoskinson, Munro, Maiden and Pillemer2007; Lintern et al., Reference Lintern, Woods and Phair2000; McGilton et al., Reference McGilton, O’Brien-Pallas, Darlington, Evans, Wynn and Pringle2003; van Weert, van Dulmen, Spreeuwenberg, Ribbe, & Bensing, Reference van Weert, van Dulmen, Spreeuwenberg, Ribbe and Bensing2005; Wilkinson, Reference Wilkinson1999). The literature consistently notes that lack of administrative support likely explains the failure of education, training, and practice change initiatives (Aylward et al., Reference Aylward, Stolee, Keat and Johncox2003; Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). Only one study provided a two-day training course that specifically targeted leadership skills and support from management and senior staff and which subsequently enabled staff to create a development plan to address enabling factors before providing education to the direct care staff (Lintern et al., Reference Lintern, Woods and Phair2000).
Reinforcing Factors
Reinforcing factors are the mechanisms that reinforce and encourage the sustained implementation of new skills and practices. Forty-three (46%) of the studies included reinforcing factors. Of these, 16 were classified as Type III (predisposing and reinforcing only), and 27 were classified as Type IV (predisposing, enabling, and reinforcing factors). Of the 38 studies that received an effectiveness rating of A (i.e., good evidence to support a recommendation of effectiveness) or B (i.e., fair evidence to support a recommendation of effectiveness), 24 (63%) included reinforcing factors as part of the intervention.
A chi-square test for independence revealed a significant association between the intervention type and the rated effectiveness of the intervention; χ2 (6, n = 94) = 24.26, p < .001, Cramer’s V = .36, p < .001. Interventions classified as Type IV (i.e., included predisposing, enabling, and reinforcing factors) were most likely to be rated as effective (see Figure 3). The reinforcing factors included such features as (a) on-the-job coaching, (b) hands-on practice, (c) supportive mentoring, (d) increased supervision, and (e) team meetings. These descriptions all indicate that resources in the form of direct human involvement and interaction were provided to the care staff as they worked to implement their newly learned skills and care practices. Given the influence that reinforcing factors exert on the success of practice interventions, the importance of these types of supports for care staff should not be underestimated.
On-site consultation and hands-on supervision and support were the most common forms of reinforcement (n = 17). Staff motivation systems through the provision of rewards or recognition for intervention implementation were used in six studies, five of which received an effectiveness rating of B (Bourgeois et al., Reference Bourgeois, Camp, Rose, White, Malone, Carr and Rovine2003; Burgio et al., Reference Burgio, Stevens, Burgio, Roth, Paul and Gerstle2002; Resnick et al., Reference Resnick, Gruber-Baldini, Galik, Pretzer-Aboff, Russ, Hebel and Zimmerman2009; Roth, Stevens, Burgio, & Burgio, Reference Roth, Stevens, Burgio and Burgio2002; Stevens et al., Reference Stevens, Burgio, Bailey, Burgio, Paul, Capilouto and Hale1998). This finding is noteworthy because motivational factors such as recognition and a sense of achievement have been found to be important to care aides’ perceived empowerment and quality of work life (Caspar & O’Rourke, Reference Caspar and O’Rourke2008).
Discussion
This review elucidated many of the intervention factors and methodological approaches associated with the effectiveness of interventions designed to produce change in care practices in LTC settings. The continued extensive use of education alone (predisposing factors) as an intervention to change practice is troubling. Our review shows that this approach is often unsuccessful, and Aylward et al.’s (Reference Aylward, Stolee, Keat and Johncox2003) conclusion is as relevant today as it was 12 years ago: “The critical issue in training is effecting change in practice. Successful implementation of training must include organizational and system changes” (p. 269).
The ineffectiveness of enabling factors used in the primary studies also merits further exploration. The use of individualized care plans was the most frequently evaluated enabling factor. It is noteworthy that care staff members’ access to the care plans was not addressed in any of these studies. Rather, the researchers seemed to assume that the information contained within the residents’ care plans would be accessed and applied in the day-to-day practices of the health care staff. This presupposition, and the consequent lack of attention to how residents’ individualized care information is accessed and exchanged among care staff, is pervasive in the literature. We recommend that researchers attempting to change care practices in LTC facilities stop relying on this understandable yet erroneous presupposition. This recommendation aligns with the findings of Caspar (Reference Caspar2014) who demonstrated that care aides lacked practical access to residents’ care plans in LTC facilities. Caspar (Reference Caspar2014) found that residents’ assessments and care plans, which were developed specifically to organize and prescribe their care, exerted little, if any, influence on care aides’ daily care practices. This finding is consistent with the literature that has suggested that care plans do not guide the daily care provided in LTC facilities (Adams-Wendling, Piamjariyakul, Bott, & Taunton, Reference Adams-Wendling, Piamjariyakul, Bott and Taunton2008; Daly, Buckwalter, & Maas, Reference Daly, Buckwalter and Maas2002; Dellefield, Reference Dellefield2006).
The presence of reinforcing factors was found to be especially influential in successful practice change interventions. This is perhaps one of the most noteworthy findings of this review because it indicates that, with the presence of reinforcing factors, interventions related to aspects of care that are challenging and complex to change (e.g., communication techniques; McGilton et al., Reference McGilton, O’Brien-Pallas, Darlington, Evans, Wynn and Pringle2003) can be as successful and effective as interventions related to more concrete and basic care practices, such as improved oral care (Isaksson et al., Reference Isaksson, Paulsson, Fridlund and Nederfors2000) and infection control (Daly et al., Reference Daly, Smith, Rusnak, Jones and Giuliano1992).
The literature indicates that reinforcing factors in the form of rewards and recognition are particularly important to care aides and are significantly lacking in their day-to-day work experiences. For example, Pennington, Scott, and Magilvy (Reference Pennington, Scott and Magilvy2003) found that basic motivational factors, such as recognition and a sense of achievement, are most important for care aides. They concluded that, to achieve positive outcomes for both care aides and residents, management needs to become more creative in finding ways to recognize care aides and to enhance their sense of achievement. Yet Caspar and O’Rourke (Reference Caspar and O’Rourke2008) found that a majority of care aides indicated that they received few if any rewards or recognition for a job well done. Further, Scalzi, Evans, Barstow, and Hostvedt (Reference Scalzi, Evans, Barstow and Hostvedt2006), who reviewed barriers and enablers to changing organizational culture in nursing homes, found that few, if any, incentives and rewards were linked to the implementation of quality of life or person-centred care practices. Rather, incentives in LTC facilities were often linked to competing or conflicting goals, such as performance-based outcomes that placed more emphasis on regulatory compliance than on residents’ individualized needs. Unless reinforcing factors such as motivation, recognition, and incentives are part of an intervention, the day-to-day care practices and routines may be in direct conflict with the successful implementation of an intervention.
As for the methodological approaches used in the primary studies, our review has shown that minimizing the risk of bias is an area in need of significant improvement in intervention research conducted in LTC facilities. Without this, the ability to confidently claim the effectiveness of an intervention is seriously weakened. Specifically, researchers who relied upon the care staff to obtain data for their studies should have taken additional measures to ensure that the practice did not confound the outcomes.
This review shed light on the many methodological challenges that researchers encounter when attempting to conduct intervention studies in LTC facilities. Some of the challenges include, but are not limited to, high attrition rates due to high turnover of care staff members and the death of residents, limited numbers of available participants due to the complexity of obtaining informed consent, lack of active support from management for the initiatives, and the inability to take the time needed to create and sustain trusting relationships between the research team and the care staff (Caspar et al., Reference Caspar, Cooke, Dahlke, Moody, Phinney, Puurveen and Stern2012). Consequently, we recommend that researchers embarking on the extremely important yet very challenging task of conducting research in LTC facilities establish ways to adequately address these challenges.
Researchers conducting studies in LTC settings must be sensitive to the limited resources available for the education and training of care staff. The vast majority of the studies we reviewed did not specifically provide cost-benefit analyses of the interventions. Cost-benefit analyses, however, must be conducted if researchers want to make claims related to the viability or sustainability of the interventions they are designing and evaluating.
Stakeholder engagement was rarely employed in the development of the reviewed interventions. It is essential to involve staff members as fully as possible in the design, content, delivery, and evaluation of education and training initiatives and to create a consensus among them for the need for education along with the desired goals and outcomes (Burgio et al., Reference Burgio, Stevens, Burgio, Roth, Paul and Gerstle2002; Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). The practice of including staff members in the development stage of an intervention is said to improve their sense of ownership and to help to establish the appropriateness and relevance of the intended programme (Nolan et al., Reference Nolan, Davies, Brown, Wilkinson, Warnes, McKee and Stasi2008). Our review found that this practice is not the norm in intervention studies conducted in LTC facilities.
It is widely recognized that change causes stress. It also is widely recognized that LTC care staff often experience high levels of stress related to their job tasks (Kane, Reference Kane2001). Only three of the studies included in our review specifically addressed stress management and time management for staff members (Hegeman et al., Reference Hegeman, Hoskinson, Munro, Maiden and Pillemer2007; Schonfeld et al., Reference Schonfeld, Cairl, Cohen, Neal, Watson and Westerhof1999; Wilkinson, Reference Wilkinson1999). We recommend that any intervention research conducted in LTC settings should include at least some aspect of stress and time management for care staff asked to embark on a change initiative that would likely produce additional stress, at least in the short term, in their work environment.
Conclusions
This review found that, with appropriate intervention factors (i.e., the inclusion of more than predisposing factors) and stronger study designs (i.e., appropriately addressing the risk of bias), changes in care practices are indeed possible and measurable. Three key factors need to be addressed for changes to occur in care practices related to the quality of life of, and quality of care provided for, residents. First, information designed to modify care staff members’ knowledge, skills, beliefs, or attitudes must be effectively communicated and disseminated (i.e., predisposing factors). Second, conditions and resources must be developed within LTC facilities to enable staff members to implement their new skills (i.e., enabling factors). Third, mechanisms must be in place to support the sustained implementation of new skills or practices (i.e., reinforcing factors). The majority of researchers attempting to change practice have not ensured that all of these factors are addressed. This is an important consideration given the amount of research published regarding the lack of change in care practices during the course of the past two or three decades, despite concerted efforts (Commonwealth Fund, 2007; Corazzini et al., Reference Corazzini, McConnell, Anderson, Reed, Champagne, Lekan and Zimmerman2010).
