We were concerned about our journal club. Traditionally, based as it was in the department of psychiatry in a large teaching hospital, the weekly meetings had been successful in terms of longevity and attendance. In contrast, over the previous 6 months we had experienced last minute cancellations, presentations that were poorly prepared and low levels of attendance. All this despite the fact that we fulfilled many of the criteria recommended by Sidorov (Reference Sidorov1995): a small training programme, available food, mandatory attendance and an active research programme. At a focus group held to identify the causes of the recent decline in enthusiasm, trainees attributed the problem to the introduction of the new clinical appraisal paper in the Membership exam. The traditional journal club format, where an article was presented and then discussed, was seen as no longer relevant to trainees' needs. When an attempt was made to introduce a more evidence-based format as described by Gilbody (Reference Gilbody1996) pre-Membership trainees did not feel confident in presenting. Moreover, more senior trainees who had taken the Membership exam before the introduction of the new paper also lacked confidence in chairing the meetings. Taking into account these concerns about the journal club, the focus group generated several suggestions to develop the journal club. We have now implemented these suggestions and the aim of this paper is to evaluate the impact of the new format on attendance and satisfaction levels.

Method

Assessment of impact of new format

We studied attendance and satisfaction in the last 3 months of the traditional journal club and the first 3 months of the new format using the following measures:

1. (a) the number of meetings cancelled

2. (b) the average number of people attending the meeting

3. (c) a 4-point Likert scale (1=no interest/satisfaction to 4=highly interesting/very satisfied) to examine clinical interest, academic interest, enjoyment and overall satisfaction.

T-tests were used to look for significant differences between the mean scores in each of the categories before and after the new format was introduced. Finally, participants were asked for any additional comments.

Results

Prior to the intervention, the core group attending the journal club consisted of 10 doctors: three consultants, three post-Membership trainees involved in teaching and research and four pre-Membership trainees. Two members of the psychology department had been invited to the old format journal club but after one meeting had decided that it was not relevant to their work. In a 3-month period, only four of 14 meetings were actually held, the rest being cancelled owing to trainees not being prepared or failing to attend. Table 1 shows the mean satisfaction levels.

Table 1. Mean satisfaction levels before and after introduction of a new format for journal clubs

					95% Cl
	Before or after intervention	Mean	d.f.	Significance (2 tailed)	Lower	Upper	n
Clinical interest	Before	2.6	21	0.142	-1.22	-0.19	9
	After	3.1	15				14
Academic interest	Before	2.4	22	0.011	-1.9	-0.3	10
	After	3.5	12				14
Enjoyable	Before	2.4	22	0.037	-1.6	-6.11	10
	After	3.3	11				14
Usefulness	Before	2.4	22	0.005	-2.2	-0.5	10
	After	3.8	11				14
Overall satisfaction	Before	2.2	22	0.008	-2.7	-0.4	10
	After	3.5	12				14

Three months after the intervention, the number of people attending the meetings had increased to 15. One new lecturer started working in the department. The two psychologists now attend on a regular basis. Two pre-Membership trainees working in another hospital heard about our new format and felt that the additional statistical input and training in critical appraisal techniques was sufficiently helpful for their exam preparation to travel to the department. All 14 of the scheduled meetings were held. As can be seen in Table 1, satisfaction levels improved in all areas, with statistically significant increases in academic interest, enjoyment, usefulness and overall satisfaction. Comments offered by participants included acknowledgement of a greater sense of ownership of the journal club by trainees. Several people noted that where the meeting would have been cancelled in the past if no consultants had been able to attend, now there was enough confidence to carry on regardless. Trainees also said that they found it easier to ask questions of someone who was outside the medical hierarchy, particularly when these questions might reveal their lack of knowledge in statistics. Finally, there were several comments about the benefits of regular scheduled contact with a statistician, when informal advice could be sought regarding on-going research projects.

Discussion

This study examines the impact on our journal club of a new format that placed increased emphasis on critical appraisal and used adult learning principles. We found a significant increase in satisfaction levels and improved attendance rates.

Some limitations to this study are obvious. The length of follow-up is short, merely 3 months. It is likely that any change that is perceived as emanating from the group will be associated with a sense of dynamism and enthusiasm, which is then reflected in higher morale. It is interesting that in the literature to date, although many different formats for journal clubs have been studied, every intervention has been reported as having positive effects on attendance and satisfaction.

A second limitation is that we have no objective measurements of improvements in statistical knowledge or ability to critically appraise a piece of research. Previous studies in this field have found inconsistent evidence of objective improvement in critical appraisal skills even where trainees have reported satisfaction (Reference SeeligSeelig et al, 1991; Reference Sandifer, Lo and CromptonSandifer et al, 1996; Reference Norman and ShannonNorman & Shannon, 1998; Reference Fu, Hodges and RegehrFu et al, 1999). Most of these studies were based in the US or Canada where, as far as we are aware, trainees are not formally assessed in critical appraisal as part of their qualification as a specialist. It is possible that trainees in the UK and Ireland may be more realistic in their evaluation of their own abilities given that they will be examined in critical appraisal skills. Therefore, their expression of satisfaction, and particularly the attendance at the journal club of trainees who are not obliged to attend, may be given more weight. The advantages of our new format are several. First, identifying specific learning objectives (namely statistical knowledge and critical appraisal ability) that are seen as relevant to participants' needs has created an improved sense of ownership. Both Sidorov (Reference Sidorov1995) and Linzer et al (Reference Linzer, Delong and Hupart1987) suggested that a balance needed to be struck between allowing trainees to feel a sense of self-determination and involving faculty in teaching. In a similar vein, Rowlands and Geddes (Reference Rowlands and Geddes1993) pointed out the importance of active involvement in a journal club to the morale of junior doctors. We feel our format encourages trainees to take responsibility for the journal club while ensuring that expertise is readily available.

Second, dividing participants into small groups and requiring them to devise answers and to debate these answers with other participants leads to active learner involvement. Like Elnicki et al (Reference Elnicki, Halperin and Shockor1999), we were concerned about the risk of embarrassing individuals who might feel self-conscious about their lack of knowledge in an area. In general we find that this is not an issue once there are at least three participants in each group. Moreover, the involvement of all those present reduces the workload for the presenter: in general, participants have found that the preparation time is less than for the traditional format.

Finally, the presence of statisticians at each meeting has created an environment where consultants, psychologists and trainees feel freer to question each other in a scientific manner. Knowing that a statistician is there to answer and elaborate on difficult points encourages all those present to focus on the statistical strengths and weaknesses of a study.

We are not aware of other journal clubs that have sought regular statistical support. Other groups seeking outside expertise have mainly looked to librarians for assistance in conducting adequate literature searches (Reference ScherrerScherrer, 1999). Clearly the determining factor here is the learning objectives set for the meetings, but we feel that other groups might, like ourselves, identify statistical analysis as a particularly challenging area.

Conclusion

Reviews of journal clubs (Reference Jouriles, Cordell and MartinJouriles et al, 1996; Alguire, 1998) suggest that there is no one ideal format: rather, different groups need to tailor a journal club according to their needs. In our department, trainees and researchers expressed an interest in meetings that focused primarily on critical appraisal. Combining expert statistical advice with active learning has produced a format associated with improved participant satisfaction. To maintain enthusiasm we hope to develop this format further, allowing participants to move from pure critical appraisal to a broader evidence-based medicine approach, formulating a clinical question and searching for the most relevant literature.