The purpose of this paper is to identify the important gaps in research coverage, particularly in areas key to the National Service Framework for Mental Health (NSF-MH) (Department of Health, 1999) and the NHS Plan (Department of Health, 2000), and to translate these gaps into researchable questions, with a view to developing a potential research agenda for consideration by research funders.

Three sets of source material were subject to a series of expert assessments by the review panel: the Thematic Review conducted by our team and completed in October 2000 (Reference Wright, Bindman and ThornicroftWright et al, 2000), the Scoping Review of the Effectiveness of Mental Health Services, produced by the Centre for Reviews and Dissemination at the University of York in September 2000 (Reference Jepson, Di Blasi and WrightJepson et al, 2000) and the Report of the Mental Health Topic Working Group, which reported to the Clarke Research and Development Review Committee (Mental Health Topic Working Group, 1999). The focus is on adults of working age with mental health problems, and upon mental health services for adults, including their interfaces with services for substance misuse, older adults, children and adolescents, and people with learning disability. We have produced 11 recommendations designed to strengthen the research infrastructure.

Creating the research infrastructure to answer the researchable questions

The recently published Medical Research Council (MRC) framework (Campbell et al, 2001; Fig. 1) offers a structure to the chain of events from initial idea to dissemination of proven intervention, and can be seen as a parallel to the five phases recognised in the development of pharmaceutical interventions. In terms of the research that has been conducted over the past decade in particular, several issues are notable in relation to the five stages of this framework.

Fig. 1. A structural framework for development and evaluation of randomised controlled trials (RCTs) for complex interventions to improve health. (Sources: Reference Campbell, Fitzpatrick and HainesCampbell et al, 2000, BMJ, 321, 694-696.) Printed with permission from the BMJ Publishing Group.

Recommendation 9.

Where appropriate, research should address the mechanisms whereby interventions are effective.

Pre-clinical phase

The large majority of studies reviewed for this report are largely or completely atheoretical, the exceptions being mostly those that concern psychological interventions. The MRC framework encourages the development of a theoretical understanding of the processes at work in effective interventions, and we concur.

Phase I: manualisation

This phase allows the components of the intervention to be modelled, together with the underlying mechanisms by which they influence outcomes to provide evidence that their relationship and interaction with each other can be predicted. In this phase, the intervention may be manualised.

Phase II: exploratory trials

These may be carried out to explore the feasibility of a new procedure before full randomisation is undertaken; to gain a rough idea of the effectiveness of a new intervention; or because a full RCT is not feasible at the time of the study. An example of the first would be Lam et al's (Reference Lam, Bright and Jones2000) exploratory study of the feasibility of CBT, of the second is Sutherby & Szmukler's (Reference Sutherby and Szmukler1998) investigation of crisis cards and self-help initiatives in patients with chronic psychoses, and of the last would be the adoption by Brooker and his colleagues (Reference Brooker, Falloon and Butterworth1994) of a ‘before and after’ design in gauging the effectiveness of training community psychiatric nurses in family interventions.

Phase III: definitive trials

We have discussed above the lack of sufficiently powered trials in most areas relevant to the NSF-MH and NHS Plan, and the trials that have been completed are reviewed in the York Scoping Review (Reference Jepson, Di Blasi and WrightJepson et al, 2000). We therefore refer to our recommendation 3.

Phase IV: dissemination

This is arguably the area that needs most research input. Although there is a fairly large number of studies that include a training component, there are no studies at all that randomise a sufficiently large sample of staff to an experimental training condition or a no-training control. In order to test whether an intervention (which has previously been proven to be effective) can be effectively ‘trained’ in the workforce, trials need to be designed wherein the unit of analysis is the member of staff. There is no existing literature, so power calculations need to be based on theoretical estimates of differences in outcome between trained and non-trained staff. This calculation may be further complicated by the likelihood that there will be a considerable variation in skill uptake between different members of the training cohort, leading to a variation in the way that the intervention is delivered to the patient.

The effectiveness of the training needs to be measured against two important sets of variables. The first set of variables relates to the trainee and to the increase in his/her skill and knowledge. The second set relates to the change in patient outcomes, and these should include economic as well as clinical and social measures. These are two important areas where training needs to be tested.

Studies have already been conducted to examine effectiveness, but there have been no controlled studies at all to examine the current training models (such as the Thorn Programme, the Cope Initiative from the University of Manchester and the Sainsbury Centre for Mental Health training courses. These consume a large proportion of the mental health component of consortium budgets).

Fidelity and implementation

What naturally follows from the issue of disseminating research findings into the workplace and, hence, into the wider population of those with mental health problems, is research into the long-term maintenance of skills. Furthermore, we need to examine the implementation issues concerning the way that services may, or may not, adapt to the presence of a newly-trained worker. We have evidence from the family intervention area (Kavannagh et al, 1992) that once trained, staff may not continue to use their skills. Furthermore, although there is widespread anecdotal evidence that there are barriers to implementing new models of working, there is absolutely no robust research evidence in this regard. This area of research raises important methodological questions, notably the place for qualitative research. It seems clear that quantitative findings from trials of training need to be complemented by qualitative studies that tease out some of the important implementation issues, such as attitudes of co-workers and organisational and management issues.

Declaration of interest

None.