Under the broad aim of examining the development of evaluation and its impact on public policy, we set out in the Introduction to this book a set of key objectives, the first of which was to trace the emergence of evaluation in the political and academic worlds. As noted in Chapter One, evaluation in the UK and the USA emerged in the mid-20th century as a means of improving the performance of public sector organisations against a background of increasing fiscal pressures. In the case of the UK, much of the early work was crystallised in the early 1980s in the form of the Audit Commission, to which the government gave the role of evaluating performance in relation to effectiveness, efficiency and economy – the elements of what was seen at that time as ‘value for money’.

Our second objective was to explore the concept of evaluation and its various interpretations. We have seen, largely in Chapter Two, that the range of interpretations is very wide, encompassing formal mechanisms such as performance indicators (PIs), audits, inspections and accreditations as well as a number of models and conceptions such as goal-oriented and goal-free evaluations, formative and summative evaluations, and evaluations that can be described as naturalistic, realistic, theory-driven and utilisation-focused. There can be no simple and comprehensive fix to the task of evaluation. Evaluators need to be explicit as to the ‘version’ of evaluation being used in any particular context.

The third objective was to identify the potential opportunities and problems in a range of data collection methods and in the analysis of data. We have seen that there are problems to be resolved when using any method of data collection – including interviews, observation, the scrutiny of documents and the use of questionnaires. Those planning and undertaking the collection of data for evaluations need to make many decisions about which methods to use and about exactly how to use the selected methods. The problems are deeply philosophical, in the case of ontology and epistemology, and operational as in the case, for example, of designing the questions to be asked in interviews. Evaluations which are not based on a careful consideration of these sets of issues need to be treated with considerable scepticism as they are likely to fail tests of validity and/or reliability.