Acquiring skills in evidence-based medicine

To provide such an approach, researchers at McMaster University in the USA proposed the concepts of evidence-based medicine (EBM) as a way of ‘systematically finding, appraising and using contemporaneous research’ (Reference Rosenberg and DonaldRosenberg 1995). These can be summarised in four steps (Reference GilbodyGilbody 1996), expanded on in Box 1:

1. 1 formulate the question

2. 2 search the literature

3. 3 evaluate the evidence

4. 4 implement the findings.

EBM became seen as the best way of ensuring that a clinician's opinion, potentially limited by knowledge gaps or biases, is supported by evidence from the scientific literature so that best practice can be determined in the context of the patient's views and values. As EBM evolved, journal clubs aspired to the more specific goal of helping clinicians, and in particular trainees, to acquire the skills necessary to adopt this approach. EBM clubs covering all four steps outlined by Gilbody and integrated with clinical work may be seen as the gold standard of journal clubs. In an EBM journal club, the presenter describes a clinical scenario, formulates a question emerging from it, demonstrates the search strategy to find relevant, reliable research, appraises a relevant paper and considers how applicable it is to their initial scenario.

Journal clubs traditionally last 1 hour, but covering all the steps set out by Gilbody is difficult in this brief time. Some EBM clubs are therefore run in cycles over 2–4 weeks, with different combinations of question generation, search, appraisal and presentation of a critically appraised topic. For example, in a 2-week cycle the first meeting might involve 45 minutes spent critically appraising a paper, followed by 15 minutes focusing on formulating a specific question that emerges from a clinical scenario. The new presenter then takes the question, searches the literature (with support from a facilitator) and brings back a paper to appraise the following week.

Reference Phillips and GlasziouPhillips & Glasziou (2004) have suggested that the weakest aspect of most clubs is the part focused on searching the literature, so it may be useful to have longer cycles of 3 or 4 weeks, where the literature search is conducted live during one session (Fig. 1). The challenge with running a journal club over several weeks is that the audience may vary depending on leave, rest periods and so on, which can make it difficult to maintain momentum.

FIG 1 An EBM journal club run over a 3-week cycle.

Passing the critical review component of the MRCPsych

Running an EBM journal club relies on having facilitators with appropriate skills to formulate questions and teach search strategies (Box 2). Jobbing clinicians do not always feel confident in these areas (Reference Warner and KingWarner 1997), particularly if they are required to demonstrate these skills in front of an audience of peers and trainees.

Moreover, assessing the ability to formulate questions and conduct literature searches does not lend itself easily to a formal examination process. From the spring of 1999, the Membership examination of the Royal College of Psychiatrists (the MRCPsych) has included a critical review paper, the aspiration being that trainees studying for this paper would subsequently have a greater commitment to EBM throughout their career. A new paper, MRCPsych Paper B, was launched in April 2015, replacing the old Papers 2 and 3, and the critical review component makes up one-third of Paper B. The syllabus for the critical review topic (Royal College of Psychiatrists 2011) has not changed: theoretically, it covers all four steps of EBM, but in practice the emphasis in the exam tends to be on critical appraisal.

A key aim for many psychiatric journal clubs has become to support psychiatric trainees in passing the critical review component of their exams. Although a national survey (Reference Taylor, Reeves and EwingsTaylor 2004) suggested that most clubs are run on evidence-based lines, respondents were specifically asked whether their trainees received teaching in formulating questions and searching the literature. Receiving teaching is not the same thing as putting these skills regularly into practice. It would be an interesting research project to explore the extent to which journal clubs in psychiatry cover the whole range of EBM. My impression is that such clubs are rare, both because of the practical difficulties described above, and because of a lack of confidence among facilitators. In contrast, clubs that focus on the critical appraisal of a specific study are relatively easy to run: there is an agreed process and trainees are often encouraged to work their way systematically through a series of questions (known as critical appraisal tools) such as those suggested by the JAMA users’ guide to the medical literature (Reference Guyatt and RennieGuyatt 1993), the Critical Appraisal Skills Programme (CASP: www.caspuk.net) or the Centre for Evidence-Based Medicine (CEBM: www.cebm.net/critical-appraisal). The risk with these tools is that it is possible to take a ‘painting-by-numbers’ approach, commenting on study design, bias and so on without feeling confident to give an overall judgement on the quality of the paper and its applicability to clinical practice.

An alternative approach and one that often provokes lively debate is the reverse critical appraisal journal club, where a clinical question is posed and the audience is asked to design a study that would best answer it (Reference Stallings, Borja-Hart and FassStallings 2011). The ensuing discussion then informs the subsequent critical appraisal of the chosen article. This approach helps the audience recognise the challenges faced by the researchers: however, it is more demanding of the facilitator's abilities in critical appraisal.

Learning theory (Box 3) reminds us that the majority of students will focus their learning on the demands of the examinations facing them. Perhaps it is not surprising that the other steps in EBM, namely formulating questions from clinical scenarios, undertaking literature searches and thinking about application to clinical practice, have been relatively ignored in many psychiatric journal clubs as, although these steps mirror best clinical practice, they are not formally examined in the same way as critical appraisal.

Another difficulty with having examination success as a primary aim for a psychiatric journal club is that it may not engage potential audience members. Adult learning theory stresses the importance of real-life scenarios, establishing a need to know and a task-centred approach. Senior clinicians are likely to be less interested in attending journal clubs if the format does not relate to the dilemmas they encounter in routine practice, and educational meetings where senior staff are absent tend to be less positively rated by trainees (Reference Heilligman and WollitzerHeilligman 1987). Many trainees based in psychiatry are in their foundation years or are working in psychiatry as part of general practice training: education targeting a specific psychiatric exam may appear less relevant to them too, although critical appraisal is a formal part of general practitioner (GP) training.

Other aims: professional development and bonding

Critical appraisal clubs focus on assessing the validity of specific studies. Other formats may emphasise different components of EBM more relevant to particular audiences.

Reference Cave and ClandinnCave & Clandinn (2007) described a journal club that focused on books rather than research papers. The books chosen were written by physicians, ‘stories of their lives, the lives of their patients and the clinical problems with which they live’. They give the example of a discussion on The Anatomy of Hope by Jerome Reference GroopmanGroopman (2005). Groopman, Professor of Medicine at Harvard, writes of his personal and professional struggle to balance the needs of nurturing hope with those of being truthful and realistic in discussing prognosis with patients and friends facing serious illness. This journal club was attended by doctors from trainees to highly experienced consultants and it aimed to foster links between participants and help doctors develop different ways of understanding relationships between colleagues and between doctors and patients. Discussing such works may seem unusual in the context of a journal club, but the authors quote Reference SackettSackett (1997) as including in his principles of EBM, ‘individual clinical expertise that incorporates “thoughtful identification and compassionate use of individual patients’ predicaments, rights and preferences in making clinical decisions”’. Such a journal club format can therefore be considered to relate to the fourth step of EBM, namely how to understand and apply the research evidence in the context of the individual patient.

Cave & Clandinn were interested in helping doctors deepen their relationship with medical colleagues at different stages in their careers. Other authors have focused on increasing cohesion and mutual support in other professional groups or in multidisciplinary teams (Reference Dobrzanska and CromackDobrzanska 2005; Reference Mukherjee, Owen and HollinsMukherjee 2006; Reference Bilodeau, Pepin and St-LouisBilodeau 2012). There is increasing awareness of the importance of a strong bond between colleagues in contributing to a culture of compassion and intelligent kindness in healthcare (Mid Staffordshire NHS Foundation Trust Public Inquiry 2013). Community-based specialties such as psychiatry face a particular challenge in fostering camaraderie in a workforce that is geographically dispersed, and there are interesting parallels with groups such as school nurses, where an online journal club presentation and moderated discussion gave participants a sense of connecting with each other and decreased isolation (Reference SortedahlSortedahl 2012).

In the nursing literature, Reference Campbell-Fleming, Catania and CourtneyCampbell-Fleming et al (2009) suggest that busy in-patient and community nurses are less interested in ‘the luxury to critique literature’. In their US study, the facilitator's role thus became to select high-quality, relevant studies where the results could be accepted and the discussion could concentrate on the applicability of the findings to their daily work. The same lesson emerged from Reference Dobrzanska and CromackDobrzanska & Cromack's UK study (2005) where the nurses likened an academic focus on critical appraisal to ‘a college lecture’. The club was more successful when the emphasis was on a presentation of a clinical topic based on high-quality studies identified by a research facilitator. Dobrzanska & Cromack remind us that: ‘the reality of enhancing critical appraisal skills among staff who may not have studied for a considerable length of time is difficult to achieve’.

Where participants bring differing levels of expertise and skills in critical appraisal, clubs need to find ways of ensuring that the format allows all present to engage. One option suggested by Reference Mukherjee, Owen and HollinsMukherjee et al (2006) in a group including doctors, psychologists, and clinical and social science researchers is to alternate qualitative and quantitative papers. Some professional groups, such as clinical psychologists and social science researchers, often have greater expertise in assessing qualitative papers, in contrast to the experience of doctors and clinical researchers in quantitative approaches, so this facilitates learning from each other.

Thus, journal clubs need to have clear, agreed aims that meet the needs of their target audience. The next step is that they should meet regularly and consistently – what helps them do this?