Definition of SOS

We define support for our supervisors (SOS) as an evidence-based, systematic, organizational process which ensures that supervisors receive the necessary leadership, support and development to fulfil their role effectively and with job satisfaction. It differs from informal, traditional, or organizationally flawed arrangements, such as ‘peer supervision’. The basic functions of our SOS model are to embed supervision within a normative, formative and restorative infrastructure. Respectively, these are achieved by evidence-based methods such as drawing on competencies frameworks, undertaking ‘gold standard’ training in supervision, and providing supervision-of-supervision (or consultancy). The effectiveness of these various organizational interventions could be measured by instruments assessing such variables as the development of competence in supervisors (e.g. Milne & Reiser, Reference Milne, Reiser, Watkins and Milne2014), their job satisfaction or occupational burnout.

This model adopts a research and development (R&D) implementation strategy, which consists of two nested problem-solving cycles (Department of Health, 1994). In Figure 1, we have translated this original R&D framework into supervision-related terminology, and have proposed a supervision-specific extension of the model by incorporating the steps in a general problem-solving strategy (Hayes, Reference Hayes1989).

Fig. 1. The SOS model, a systematic way of thinking about supporting our supervisors.

The SOS model set out in Figure 1 has the supervisee at its heart, and depicts the experiential learning cycle, a formative emphasis on learning and professional development. The supervision cycle is nested in the middle of Figure 1, describing how supervision operates through a series of repeated needs assessments, goal-setting and evaluation cycles borrowing from Kolb's (Reference Kolb1984) experiential learning cycle. These cycles have already been described in some detail elsewhere (Milne, Reference Milne2009). The novel aspect of Figure 1 is the outer ring, the ‘support our supervisors’ (SOS) cycle. This organizationally focused cycle adds a more explicit, evidence-based, CBT consistent, and coherent dimension to existing systemic supervision models (e.g. Hawkins & Shohet, Reference Hawkins and Shohet2000; Holloway, Reference Holloway, Watkins and Milne2014; Hoge et al. Reference Hoge, Migdole, Cannata and Powell2014). To conceive this SOS cycle, we adopted the basic tasks of R&D (‘conceptualization’, etc.), assimilating where helpful the organizational model and evidence-based approach to supervision outlined by Milne (Reference Milne2009). In addition to a primary formative function (i.e. developing supervisors’ expertise), Figure 1 also emphasizes the normative function of supervisor support, recognizing how organizations influence supervision through such means as R&D and effective leadership. Note in particular that, by means of the cited examples, we have added an emphasis on the organizations’ supportive function (e.g. drawing on reviews of the key research findings to best train supervisors in facilitating peer support groups). An organization that fulfils its restorative function helps supervisors ‘feel that they are supported, accepted, nurtured, acknowledged and validated’ (Milne, Reference Milne2009, p. 185). That is, just as in most models of supervision, in the SOS model we envisage the need for normative, formative and restorative functions to be performed by supervisors’ organizations. Although for space reasons, and to simplify the Figure, all three functions are not specified within Figure 1, we believe that only when all three functions are fulfilled can we expect supervisors to be properly developed, guided and supported. With respect to supervision, these influences were first described by Kadushin (Reference Kadushin1976) and popularized by Proctor (Reference Proctor, Marken and Payne1988), and they remain key constructs for identifying modes of effective supervision (e.g. White & Winstanley, Reference White, Winstanley, Watkins and Milne2014). In Figure 1 we extend these three functions to the support of supervisors, believing them to be equally necessary conditions for a systematic support infrastructure.

We next describe and illustrate each of the seven SOS cycle tasks with some representative research and development activity or relevant theory.

Conceptualization: How should we construe a supervision infrastructure?

Operationalization: What are the key elements to measure within the supervision infrastructure?

Our selected conceptualization should indicate which variables to measure, such as factors within the organizational climate. For example, one study of supervision contrasted two different milieus for the care of dementia sufferers, a collective living (CL) unit and a traditional nursing home (Kihlgren & Hansebo, Reference Kihlgren, Hansebo, Watkins and Milne2014). The CL unit staff received 1 month of training on dementia care before the unit opened, followed by supervision, support and monthly feedback during the 22-month study period. These interventions were designed to improve their care practices. Specifically, the style of caring in the CL unit engaged the residents in decisions about their unit, encouraged autonomy, and promoted dignity. These organizational changes were associated with a significantly smaller deterioration in intellectual functioning for the CL unit patients. Therefore, these CL unit changes seem to be key elements and should be measured, in order for supervision to be as effective as possible. Several general instruments are available for measuring such organizational factors, including ‘readiness for change’ tools (e.g. Helfrich et al. Reference Helfrish, Li, Sharp and Sales2009; Shea et al. Reference Shea, Jacobs, Esserman, Bruce and Weiner2014). More specific to supervision are instruments like The Clinical Learning Environment, Supervision and Nurse Teacher Evaluation Scale (CLES+T; Saarikoski, Reference Saarikoski, Watkins and Milne2014), and the Manchester Clinical Supervision Scale (MCSS; White & Winstanley, Reference White, Winstanley, Watkins and Milne2014). These tools measure the ‘cultural environment’, the ‘leadership style’ of the unit manager, and the ‘resource’ aspect (i.e. importance and availability of supervision in the workplace). At the micro-environmental level of the supervision group there are also instruments that tap the organizational system (e.g. Ogren & Sundin, Reference Ogren and Sundin2009), but overall there are few supervision-related measurement tools, and even fewer which are psychometrically sound (Wheeler & Barkham, Reference Wheeler, Barkham, Watkins and Milne2014).

But there are welcome exceptions. Godley et al. (Reference Godley, Garner, Smith, Meyers and Godley2011) present a comprehensive implementation model that includes systematic measurement of training and supervisory effectiveness using standardized measures. To their credit, they implemented a fully manualized CBT treatment that was supported by initial therapist training workshops, follow-up supervisor training (including review of taped supervision sessions), certification of competent supervisors, bi-weekly telephone-based coaching and support for therapists, and ongoing fidelity monitoring of both therapists and supervisors.

Experimentation: What kinds of research will best guide us?

In previous reviews of research on supervision (Milne, Reference Milne2009), we have drawn upon the fidelity framework (Borrelli et al. Reference Borrelli, Sepinwall, Ernst, Bellg, Czajkowski, Greger, DeFrancesco, Levesque, Sharp, Ogedegbe, Resnick and Orwig2005) to provide an organizing principle to capture the complexity entailed in evaluating supervision systemically. This framework captures the successive stages of the design, training, delivery, receipt and enactment of supervision (see Reiser & Milne, Reference Reiser and Milne2014, for a comprehensive description of the framework). This framework is designed to address the following types of questions about supervision: ‘What is the right thing to do?’ (Design); ‘Has the right thing been done?’ (Training); ‘Has it been done right? (Delivery); ‘Did it result in the right outcome?’ (Receipt); and, ‘Did it result in the right impact?’ (Enactment). The fidelity framework appears to be valuable in pinpointing problems in providing support for supervisors. This is similar to the ‘organizational behaviour management’ approach to diagnosing organizational functioning. But such wide-ranging evaluations are rare. In our review of studies evaluating the outcomes of supervision, we determined that these fidelity framework variables had not been measured adequately in about 50% of the studies that we reviewed (Reiser & Milne, Reference Reiser and Milne2014). Therefore, a first step in improving implementation research on supervision will be to fully address the successive stages within the fidelity framework, as they apply to developing and implementing an organizational approach to supporting supervisors.

In order to address fidelity and other research questions, it is likely that the use of large group, randomized controlled designs (RCTs) will be cumbersome, resource intensive and premature. In addition, RCTs have become associated with failed top-down implementation efforts (e.g. clinical guidelines; Greene, Reference Greene2014). Instead, we recommend ‘upstream’, qualitative, quasi-experimental studies concerning pragmatic issues, combined with rigorous small sample research designs. These might include the effectiveness of activities like supervisor training, surveys to establish priorities and progress, together with n = 1 designs to study supervision processes intensively (e.g. Milne et al. Reference Milne, Reiser and Cliffe2013). These types of designs are also consistent with underlying premises of the CBT model, involving the use of collaborative empiricism to develop specific, well-tailored interventions.

Interpretation: What conclusions can we draw, to guide the supervision infrastructure?

Such ‘small n’ designs offer high internal validity and seem well-suited to our present knowledge base, being sufficient for drawing some important inferences about the effects of various kinds of institutional support on supervision. For example, Milne et al. (Reference Milne, Reiser and Cliffe2013) compared standard CBT supervision with evidence-based clinical supervision, and were able to draw helpful conclusions about the effectiveness of consultation on the supervisor's performance. Other organizational factors may supplement or replace arrangements like consultancy. In a best-evidence synthesis of 24 supervision studies, Milne et al. (Reference Milne, Sheikh, Pattison and Wilkinson2011) noted that the contextual variable ‘administrative support’ was consistently identified as an important implementation variable. Beidas & Kendall (Reference Beidas and Kendall2010) also identified organizational support as a key factor in maintaining the ongoing transfer of learning for therapists, and others have noted how an organizations policy and culture greatly shape the implementation of EBP (e.g. Proctor et al. Reference Proctor, Landsverk, Aarons, Chambers, Glisson and Mittman2009).

It is clear that there is still a dearth of research focused on the impacts of organizational and contextual factors affecting the development of an enriched supervision culture. Based on the current undeveloped state of research in this domain, it is difficult to draw substantive, general conclusions. One can only assert that there is an ongoing need for consideration of systems-contextual factors in future supervision research. Local research and service evaluation efforts, grounded in the best-available research and theory, may therefore be the best short-term guide to developing such a culture. However, the point of this step in the SOS cycle is to ensure that any appropriate implications are duly drawn out, for the benefit of research and practice.

Dissemination: How should we train and develop supervisors?

Supervisor training is a vital mechanism within a systematic, evidence-based strengthening of supervision, and it is the aspect of support that has been most studied. Yet surprisingly little is known about how we might best train supervisors. Indeed, it appears that the majority of supervisors do not even receive training in supervision, leading to the oft-quoted lament that ‘something does not compute’ (Watkins, Reference Watkins and Watkins1997, p. 604). In a recent ‘informal’ survey across 12 countries, Watkins & Wang (2014, p. 193) determined that to date only Australia had a mandatory system for approving supervisors, with experience as a clinician continuing to be the only requirement to be met in becoming a supervisor in the other 11 countries. Despite the advent of competence frameworks (Falender et al. Reference Falender, Cornish, Goodyear, Hatcher, Kaslow, Leventhal, Shafranske, Sigmon, Stoltenberg and Grus2004; Roth & Pilling, Reference Roth and Pilling2007; Olds & Hawkins, Reference Olds and Hawkins2014), the content of training in these countries varied considerably (Fleming, Reference Fleming, Fleming and Steen2012). By contrast, within an SOS infrastructure, relevant (needs-led) competencies would be combined with supervisor training methods, ones that are based on expert consensus (Kaslow et al. Reference Kaslow, Borden, Grus, Collins, Campbell Forrest, Illfelder-Kaye and Nelson2004; Milne et al. Reference Milne, Scaife and Cliffe2009) and research findings (e.g. Godley et al. Reference Godley, Garner, Smith, Meyers and Godley2011). It is surely time to consider replacing supervisor training as something of a local art form with such evidence-based approaches.

Although there are now a few promising studies concerning the training of supervisors (Watkins & Wang, Reference Watkins, Wang, Watkins and Milne2014), little is known about the kind of complementary support arrangements that should be in place to nurture supervisors and to foster the transfer of their training. In a systematic review of 24 successful supervision studies, Milne et al. (Reference Milne, Aylott, Fitzpatrick and Ellis2008) noted that five types of contextual variables were thought to have moderated the effectiveness of supervision. These were the ‘general organizational context’ (e.g. administrative support and staff turnover), ‘intervention factors’ (e.g. the acceptability of supervision and the use of incentives); ‘research factors’ (e.g. reactivity to observation and unreliable observation tools); ‘learning factors’ (e.g. training was based on educational needs); and ‘participant factors’ (e.g. clinical outcomes, supervisee anxiety). As this list indicates, some of these variables were judged by the study authors to be boosters to supervision, while other variables were deemed barriers. There are logical and ethical requirements to make such ‘boosting’ support arrangements, as reflected in the UK Department of Health policy to support supervisors with ‘multiple knowledge and learning sources, technical and other resources’ (e.g. Department of Health, 2001, p. ix). But according to surveys, such infrastructure tends to be weak, undermining the provision of supervision (Harmse, Reference Harmse2001). In turn, this presents a barrier to accessing CBT (Stallard et al. Reference Stallard, Utwin, Goddard and Hibbert2007). It also represents a failure to systematize support for supervisors, an unacceptable strategic and moral oversight, leading to burnout and to dissatisfaction over arrangements in 82% of surveyed supervisors within one study (Gabbay et al. Reference Gabbay, Kiemle and Maguire1999).

Utilization: How can we support and guide the transfer of supervision training?

Thankfully, there are also some highly informative accounts of implementing training (and related developments), serving as useful guides to the trials and tribulations of innovation. For example, Lynch & Happell (2008) described an audit of the implementation of supervision in rural Australia. They highlighted the need for a strategic plan (incorporating the vision of the organization and place of the clinical supervision innovation), guided by the innovation literature (e.g. establishing a system for training, supported by implementation ‘champions’).

Two aspects of the Lynch & Happell (2008) training innovation intended to increase utilization were consultancy and supervision of supervision. In addition to the traditional focus on addressing organizational barriers and boosters, consultants are now being viewed as key agents within more active and prospective implementation strategies (Nadeem et al. Reference Nadeem, Gleacher and Beidas RS2013). Drawing on their experiences as consultants and informed by a review of the literature, Nadeem et al. (Reference Nadeem, Gleacher and Beidas RS2013) defined some core functions of implementation consultancy (e.g. accessing additional resources, encouraging engagement, facilitating the supervisory infrastructure, and fine-tuning). Supervision-of-supervision overlaps with consultancy in both form and function, though there is little by way of research. From a review, Milne (Reference Milne2009) concluded that ‘the supervision of supervisors is the most deficient area in the whole enterprise of clinical supervision’ (p. 186). But according to a survey by Wheeler & King (Reference Wheeler and King2000), the majority of supervisors reported receiving supervision of their supervision. In a rare study, Milne et al. (Reference Milne, Reiser and Cliffe2013) utilized a supervision manual, incorporating supervision guidelines, to develop a supervisor's fidelity to CBT supervision. An account of the supervision-of-supervision procedure was provided in Milne & Reiser (Reference Milne, Reiser, Watkins and Milne2014).

Evaluation: Which data will provide the optimal corrective feedback?

Any healthy system requires information on progress to adapt, in a form which empowers corrective action. In terms of supervision there have been particular challenges in developing specific feedback instruments (Wheeler & Barkham, Reference Wheeler, Barkham, Watkins and Milne2014), such as the diverse definitions of supervision (Milne, Reference Milne2007), and the divergence over defining supervisory success. Even when a measurement system is in place, questions may remain concerning the most valuable feedback data. Regarding supervisor training, researchers have tended to apply educational evaluation criteria, such as acceptability, learning, transfer and clinical benefit (Kirkpatrick, Reference Kirkpatrick, Craig and Bittel1967). Within clinical service evaluations, practical solutions to measurement problems have included the clinical outcome monitoring approach within IAPT (Richards, Reference Richards, Watkins and Milne2014). A broader approach to the evaluation of a system such as SOS is the approach proposed by Proctor et al. (Reference Proctor, Silmere, Raghavan, Hovmand, Aarons, Bunger, Griffey and Hensley2011). They helpfully delineated this taxonomy of implementation outcomes:

• • acceptability (e.g. satisfaction with the simplicity and credibility of an innovation),

• • adoption (uptake and utilization),

• • appropriateness (perceived fit or compatibility),

• • feasibility (actual fit or suitability),

• • fidelity (adherence and service quality),

• • costs (cost-effectiveness),

• • innovation penetration (e.g. improved service and clinical outcomes),

• • innovation sustainability (durable, incorporated, routinized).

We believe that to be optimal, feedback should encompass such outcome dimensions, possibly best studied in a stepwise manner, as per the fidelity framework mentioned earlier.