Origins of case-based discussion

Case-based discussion is one of a family of workplace-based assessment methods that have developed from an instrument in Canada and the USA called chart-stimulated recall (CSR). In CSR an assessor, having reviewed a selection of patients’ charts (clinical records), discusses with the practitioner data-gathering, diagnosis, problem-solving, problem-management, use of resources and record-keeping to validate information gathered from the records.

This method was shown to have good face and content validity (Reference Jennett, Scott and AtkinsonJennett 1995). In addition, it has been demonstrated that (with sufficient sampling) good levels of reliability (Reference Norman, Davis and LambNorman 1993) and validity with assessor training (Reference Solomon, Reinhardt and BridghamSolomon 1990) can be achieved. Furthermore, in emergency medicine Reference Maatsch, Huang and DowningMaatsch etal(1984) demonstrated concurrent validity in the relationship between CSR scores and results from the American Board of Emergency Medicine examinations.

Reference Jennett and AffleckJennett & Affleck (1998) produced a major literature review on case-based discussion. It includes examples in the literature of chart audit and case (note) review skills being considered as a dimension of clinical competence extending back to the 1970s, for example by Reference Goetz, Peters and FolseGoetz etal(1979).

Work in the USA using CSR as part of the recertification of practising doctors showed scores that were highly correlated with performance in a clinical examination using standardised patients. It also showed that CSR scores distinguished between doctors who had been ‘referred’ because of concerns and those about whom there were no concerns (Reference Goulet, Jacques and GagnonGoulet 2002).

Adoption and development in the UK

Case-based assessment was adapted from original work in Canada and the USA by the General Medical Council (GMC) in its performance assessment procedures. Currently in the UK, case-based discussion is used extensively by the GMC and the National Clinical Assessment Service (NCAS) in their assessments of performance, as well as forming a key component of the assessment framework for foundation and specialty training.

The use of case-based discussion in the foundation programme has been reported by Davies et al in a summary of the workplace-based assessments in postgraduate training of physicians in the UK (Reference Davies, Archer and SouthgateDavies 2009). Their data suggest good levels of reliability for case-based discussion using four, but preferably eight, cases. The authors also established that scores for case-based discussion increased between the first and second half of the year, indicating validity. Early findings from a pilot study by the Royal College of Physicians (Reference Booth, Johnson and WadeBooth 2009) supported the notion of case-based discussion as a valid and reliable instrument in the assessment of doctors’ performance. Further results are now emerging from the Royal College of Physicians following the full introduction of the curriculum and assessment reforms in 2007 (details available from N.B. on request).

The Royal College of Psychiatrists’ curricula for specialty training in psychiatry currently use case-based discussion. In addition, there are a number of variants on the theme used in encounters with simulated or standardised patients (including incognito standardised patients, so-called ‘mystery customers’) in clinical settings in many other specialties and in many different countries. Other variants on case-based discussion can be used away from the workplace in formal examination such as objective structured clinical examinations (OSCEs) and the Clinical Assessment of Skills and Competencies (CASC).

Assessment programmes

Before proceeding with further consideration of case-based discussion it is useful to place it within the context of contemporary thought on assessment in medical education. Postgraduate medical education is a growing field with a language, set of terms and, some would say, fashions of its own. It is important to note that contemporary best practice in assessment in medical education is moving in a direction that is significantly different from the traditional model. This issue is discussed in detail elsewhere, particularly in the seminal paper by Reference Van der VleutenVan der Vleuten (1996) and later papers by Reference Van der Vleuten and SchuwirthVan der Vleuten & Schuwirth (2005) and Reference Schuwirth and Van der VleutenSchuwirth & Van der Vleuten (2006a,Reference Schuwirth and Van der Vleutenb). It has also been discussed and developed in a broader context by Reference Holsgrove, Davies, Jackson, Jamieson and KhanHolsgrove & Davies (2007, Reference Holsgrove, Davies, Carter and Jackson2008).

However, put briefly, there are two main issues. First is the point raised in Van der Vleuten's earlier article (1996) that assessment is not a measurement issue intended, for example, to reduce clinical competence into its supposed component parts and express the levels of attainment in numerical terms. Instead, it should be seen as a matter of educational design aimed to assess clinical competence as a global construct. Second, assessments should have a variety of purposes (Reference Southgate and GrantSouthgate 2004; Postgraduate Medical Education and Training Board 2010b). These purposes include feedback – not just feedback to the person being assessed, but also to teachers, assessors and other interested parties (Reference Holsgrove, Davies, Carter and JacksonHolsgrove 2008). This philosophy leads us to the concept of a utility model in which assessment programmes are designed, with context-dependent compromises where necessary, to suit a particular set of requirements and circumstances. This model recognises that assessment characteristics are weighted depending on the nature and purpose of the assessment (Reference Van der Vleuten and SchuwirthVan der Vleuten 2005). Reliability depends not on structuring or standardisation but on sampling. Key issues concerning validity are authenticity and integration of competencies; thus, alignment with the curriculum and its aims is essential. In sum, this model suggests that fundamentals of a successful assessment programme are adequate sampling across judges, instruments and contexts that can then ensure both validity and reliability.

The assessment programme designed to support the College curricula includes both formal (MRCPsych) examinations and workplace-based assessments. Case-based discussion is among the workplace-based assessments in that programme and, along with the observed assessment (assessment of clinical expertise – ACE – or mini-assessed clinical encounter – mini-ACE), case-based discussion assesses the very heart of daily clinical performance by a doctor.

Postgraduate medical education – what does this actually mean? What do trainers and trainees need to know?

It is worth considering once more the context in which case-based discussion has appeared. As with many assessments in contemporary medical education, it is important to remember that case-based discussion is attempting to serve both formative (‘How am I doing?’) and summative (‘Have I passed or failed?’) assessment purposes. It is therefore an instrument that can be used to provide assessments both for and of learning (Postgraduate Medical Education and Training Board 2010a). The content and application of learning and the outcomes of assessment are defined in the curriculum and so the selection and use of learning and assessment instruments must always be undertaken with the curriculum in mind. This, although apparently obvious, is in sharp contrast to learning and assessment in the past. Consideration of the place of case-based discussion is therefore an exemplar of the need for trainers (and trainees) to be fully aware of the curriculum in setting learning plans with intended learning outcomes that match curriculum outcomes and are subject to continuing assessment that is carefully planned and not simply a spontaneous occurrence.

Planning – first stage

The fundamental nature of case-based discussion or assessment is that the doctor's own patients (cases) are used as the starting point for a conversation or discussion that looks into that doctor's applied knowledge, reasoning and decision-making. Case-based discussion is an assessment instrument that probes the doctor's clinical reasoning; it may be described as a structured interview designed to explore professional judgement exercised in clinical cases. Professional judgement may be considered as the ability to make holistic, balanced and justifiable decisions in situations of complexity and uncertainty. Case-based discussion can explore a full range of these issues, such as the ability to recognise dilemmas, see a range of options, weight these options, decide on a course of action, explain the course of action and assess its results. It draws on practical aspects of a doctor's routine clinical activity either in the form of documented observed practice (an ACE or mini-ACE would make suitable materials) or an examination of entries made in the case notes and subsequent discussion of the case. Based on such records of a patient recently seen by the trainee, and to whose care they have made a significant contribution, case-based discussion is conducted as a structured discussion.

A few days ahead of the scheduled case-based discussion, either the trainee gives the assessor two sets of suitable notes or they agree on the discussion stemming from an observed interview. The assessor reads them and selects one for the case-based discussion. The potential curriculum domains and specific competencies should be mapped at the outset. In their initial meetings trainers and trainees will find it invaluable to draw up a blueprint (either prepared themselves or from their postgraduate school) on which to plot their learning plan, including specific learning objectives and assessments. This will ensure full and proper curriculum coverage. Case selection should be kept simple but must enable good curriculum coverage. A grid may be constructed that plots cases, questions from the case-based discussion and curriculum objectives (Fig. 1).

FIG 1 Case-based discussion – example planning grid (the list of objectives is not exhaustive). After National Clinical Assessment Service 2010.

Planning – second stage

The next stage of planning is for the assessor to write out the questions that they intend to ask. This may initially appear over-prescriptive and somewhat obsessional. However, it is critical that the discussion retains its focus and avoids false trails, for example, into discussing the patient's problems or reviewing the notes themselves or turning the case-based discussion into a factual viva. Good practice is to prepare around three questions for each case, preferably covering different curriculum or performance objectives (e.g. as marked with a tick in Fig. 1). The use of three questions will facilitate coverage of more than one competency with the same case. Some thoughts and suggestions on construction of questions are offered in Box 1.

The discussion

It is important to ensure that doctors being assessed have time to review the records and refresh their memory before the case-based discussion starts and that the records are present throughout the discussion for reference. Questioning should commence with a reminder of the case and, ideally, the competency and/or curriculum domain that is being covered and should begin in an open fashion. The subsequent discussion is then anchored on this particular case and is clear in its aim of assessment against a curriculum domain or intended learning outcome. This is facilitated by the assessor, who might use prompts such as those in Box 1.

It can clearly be seen that the major contribution that case-based discussion makes to the overall assessment programme is that it allows the assessor to explore clinical reasoning and professional judgement. However, it is important again to distinguish case-based discussion from a traditional viva because a case-based discussion must not be a viva-type interaction exploring the trainee's knowledge of the clinical problem. It must focus on what is in the notes and the trainee's thinking in relation to the diagnosis and management of the case.

Feedback

It is axiomatic that a well-conducted assessment must be accompanied by timely and effective feedback, as has previously been discussed in this journal (Reference Brown and CookeBrown 2009).

Finally

Case-based discussion needs preparation by both trainer and trainee. It is important to select cases carefully and to look at the case before the meeting. Assessors must not attempt to make it up as they go along. The discussion can be quite challenging and a trainer may find an early difficulty in suppressing the urge to say ‘What if … ?’ (the discussion is an assessment of what the doctor did with a particular case, not what they might have done). So, doctors can be asked what they did and why, what evidence they have for that action and even ‘What is your next step?’ However, assessors should not go down the line of hypothetical exploration. Colleagues in all specialties have discovered that case-based discussion needs considerable practice to develop new skills and the discipline not to slip into tutorial or viva modes of questioning. Case-based discussion tests what the doctor actually did rather than what they think they might do (which is what a viva or OSCE might test). Thus, case-based discussion assesses at a higher level on Miller's pyramid (Reference MillerMiller 1990) than most other assessments because it tests what the doctor or trainee did as opposed to what they show or know they can do (Fig. 2).

FIG 2 The hierarchy of skills and related assessment instruments. CASC, Clinical Assessment of Skills and Competencies; CBD, case-based discussion; CRQ, constructed response question; EMQ, extended matching question; OSCE, objective structured clinical examination; WPBA, workplace-based assessment. After Reference MillerMiller 1990.

The discussion and feedback should take around 25–30 minutes, of which about 10 minutes should be reserved for feedback.

Back to the educational (including assessment) programme

Case-based discussion can, and indeed should, be used in conjunction with other assessment instruments, in particular those assessing observed clinical practice (ACE or mini-ACE), written clinical material such as out-patient letters (Reference Crossley, Howe and NewbleCrossley 2001) and the case presentation assessment used in the College curriculum (Reference SearleSearle 2008). Moreover, as well as being a useful assessment instrument, case-based discussion can be an effective learning method using, for example, multimedia adaptation and presentation of cases (Reference Bridgemohan, Levy and VeluzBridgemohan 2005) or interdisciplinary discussion with a personal development plan (PDP) peer group.