When we say that a policy is working well (or badly) or that a service is getting better (or worse) what do we mean? Do we mean that some or all members of the public think that it is so? Or that managers or politicians think so? Or that ‘performance indicators’ suggest that it is so? The words ‘well’, ‘badly’, ‘better’ and ‘worse’ are, to say the least, ambiguous.

For evaluations to be useful we need to reduce such ambiguities as far as possible and to be explicit about what we mean. This is where the notion of a criterion comes in. A criterion is a principle or standard by which something may be judged or decided and derives from the Greek for a ‘means for judging’ (New Oxford Dictionary of English). In evaluating a policy or service it is important to be explicit about the criterion or criteria being used. Otherwise a claim that, for example, a service is ‘good’ or ‘getting better’ would be rather meaningless. Would it be getting better in relation to efficiency, effectiveness, equity, reducing costs, or what?

There are a number of criteria which can be used in making such judgements. The criteria are not mutually exclusive, and sometimes the boundaries between them might be blurred; sometimes the criteria might overlap; for example, the ‘acceptability’ of a service might depend on its ‘responsiveness’ to users’ needs and wishes. We shall examine a range of criteria commonly used in evaluations (in no particular order) and set out what each is usually taken to mean. We also critically comment on the limitations of each criterion, demonstrating that in real-world evaluations judgements will always be problematic and contestable.

It is important to recognise that criteria are not always set or ‘pre-ordained’. Sometimes they ‘emerge from the specific social contexts of various stakeholders’ (Abma, 1997, p 35). For example, the ‘acceptability’ of a spending programme, as far as the electorate is concerned, is clearly a largely political question.