Introduction

This chapter considers the evaluation of service systems, with particular reference to non-health outcomes. The nature of the mental health service system is considered and system-level analysis is subdivided into idiographic, normative and legislative types. Examples of measures developed for this kind of evaluation are given. The mental health service system interacts with many other social systems, and illustrations are given from the interfaces with work, criminal justice, housing and health care systems, together with some suggested measures.

The evaluation of system-level outcomes involves the assessment of what Rossi & Freeman (1993) call ‘fully saturated’ services which are intended to provide for the whole population. In this situation there are limited opportunities to undertake evaluative comparisons, because there is no other provider with which to compare results. We shall return to the question of what constitutes a mental health service system below, but if we assume for a moment that the system provides services to the whole population but in a predetermined geographical location, then we do have the possibility of comparing different service systems. The geographical location, or unit of analysis, can be a sector within a district (Thornicroft, Knudsen, this volume), whole districts (Huxley, 1993), or even whole countries. Cultural issues arise in connection with the last form of analysis, but recent developments in standardised assessments have made meaningful interstate and international comparisons possible (see for example Warner & Huxley, 1993). Finally, and this will also be discussed in more detail later, it is possible to examine changes within the same service system over time; this is known as the reflexive control technique, and requires regular repeated measures over a long time period in order to produce credible results (Rossi & Freeman, 1993). The major comparative study of service systems in the United States, the Robert Wood Johnson initiative, has recently begun to produce findings based upon results from the major study sites, and the comparison service in Colorado Springs (H. Goldman, personal communication). An adapted version of the key informant approach to the evaluation of service systems has been used in North Wales, permitting an attempt at cross-cultural comparisons using similar methodologies (W. Barr, personal communication).