Impact theory

The generally agreed main purpose of formal evaluation in the public sector is to improve services. In order to achieve that broad objective, governments and public sector and voluntary sector bodies produce strategies that culminate in policies which, in turn, inform the design and operation of projects and programmes. In earlier chapters we have made a distinction between outcome and impact. Those who have carried out and written on the subject of evaluation are agreed that the term ‘impact’ refers to the cause–effect relationship between planned activities and the intended social benefit. Rossi et al (1999) use the phrase impact theory to sum up the ‘beliefs, assumptions, and expectations’ (p 78) about this relationship. Here again, we are inclined to discount a definition of a theory that relies on anything less substantial than a firmly evidence-based explanation.

On this point, and in relation to the issue of attempting to identify a cause–effect link between inputs–outcomes–impact, Pawson (2006) offers a rather radical realist perspective. His claim is that ‘interventions are theories’ (p 26). He notes that the conventional interpretation of interventions is to regard them as tangible items such as resources, equipment and personnel whereas, from the realist perspective, interventions are always based on a hypothesis that postulates:

In referring to the supposed ‘theory’ as a hypothesis, Pawson supports our assertion about the misuse of ‘theory’ in setting out to explain the process of outcome and impact assessments. However, what Pawson regards as interventions are more accurately known as ‘inputs’ – those resources – including sets of values – which are used to generate interventions.