thirteen - Non-experimental quantitative methods
Published online by Cambridge University Press: 09 September 2022
Summary
Introduction
As Chapter Three noted, the use of the randomised controlled trial(RCT) has become widespread, and in many senses obligatory, in manyareas of healthcare. However, there are legitimate grounds forconcern about uncritical adoption of RCT methods, even inhealthcare. As the chapters in this book indicate, the extent towhich the principles of controlled experimentation have penetratedother areas of public sector activity has been much more limited.Even in education services, which share many of the characteristicsof healthcare, the RCT methodology has not been embraced withenthusiasm (see Chapter Four), in other services – such as welfare(Chapter Seven) – its use clearly poses substantial problems.
There are numerous reasons why RCT methods might not be adopted in aparticular setting. For example:
• it might be difficult to recruit participants willing toenter a trial;
• professionals might be reluctant or unable to administer atrial;
• a trial might for a number of reasons be consideredunethical;
• the costs of undertaking a prospective trial might beconsidered unacceptably high;
• the length of time before results emerge from a prospectivetrial might be considered unacceptably long;
• it might be considered impractical to design ascientifically acceptable RCT for the intervention inquestion;
• the intervention under investigation may have population aswell as individual effects, bringing into question therelevance of the traditional RCT;
• any trial would have to be undertaken in an artificialsetting, which might compromise the general relevance of theresults;
• outcomes and side-effects of an intervention may bedifficult to capture in any meaningful way;
• the impact of the intervention may be widely heterogeneousmaking the assessment of aggregate effects inappropriate(see Chapter Twelve).
Underlying several of these concerns is the fact that participants ina behavioural trial (either professionals or subjects) are consciousof their own role in the experiment, and are therefore capable ofsubverting the outcome. More generally, there is often a concernthat the very act of studying a social phenomenon changes itsbehaviour, frustrating the intended purpose of the examination (theso-called ‘Hawthorne effect’). For these and other reasons (seeChapter Twelve), there appears to have been a marked reluctance onthe part of UK researchers to pursue experimental methods in publicservices, especially when compared to their US counterparts, whohave developed a distinctive ‘evaluation’ tradition.
- Type
- Chapter
- Information
- What Works?Evidence-Based Policy and Practice in Public Services, pp. 277 - 290Publisher: Bristol University PressPrint publication year: 2000