Introduction: social science as understanding, measurement and comparison

Evaluating the effectiveness of probation has always been a knotty problem for social science. Like most (perhaps all) social science, it depends on achieving the right combination of different sources of knowledge. This chapter reviews key stages in the development of evaluation research on probation services, with a major but not exclusive focus on England and Wales, and ends with some suggestions about the future. Research on the skills used by probation staff marks a particular and important step in this development.

Social science, as used in evaluative research, is a three-legged enterprise supported by three sources of knowledge or forms of investigation: understanding, measurement and comparison. (Three is usually the minimum number of legs required to support a stable structure.) We need understanding, usually acquired by qualitative research methods, to bring into focus the aims of social actors, their beliefs about the processes they are involved in, and the meanings they attach to what they do and to what happens to them. The criminologist David Matza called this ‘appreciation’ (Matza, 1969), meaning the attempt to understand social situations from the point of view of those involved. Human action is socially constructed and our social environments are structured by our actions and by those of others (particularly those more powerful than ourselves). Understanding also requires awareness of our own assumptions and our ways of interpreting and shaping experience, because what we learn will be the product of an interaction between our own perceptions and those of our research subjects. This is why qualitative researchers have to try hard to be guided by what they actually find rather than what they expect or hope or prefer to find. However, it is not clear how social science can be social without an attempt to understand the meanings of social experience for the people involved. Some notable recent examples of probation research have relied on qualitative methods: for example, research on the occupational culture of probation staff (Mawby and Worrall, 2013) and on their beliefs about the quality of probation work and the nature of good practice (Robinson et al, 2014). However, evaluation research in probation needs to go beyond practitioners’ beliefs to develop more independent and objective ways to measure the impact of probation practice: what does it change? What difference does it make?