Analytical perspective

For the purposes of mental health economics research, it is desirable to measure service use and costs comprehensively, since the broad personal and social impacts of schizophrenia typically result in a need for contact with many different service agencies, including health services, social services, housing and criminal justice services (Reference Weisbrod, Test and SteinWeisbrod et al, 1980; Reference Clark, Teague and RickettsClark et al, 1994; Reference Knapp, Almond, Percudani, Maj and SartoriusKnapp et al, 1999). This comprehensive perspective is particularly important for multinational studies, since different countries have established different boundaries between health and other services, and these boundaries have been known to shift over time as a result of changes in government policy or other forces. Also, the balance of responsibilities between the public sector (state) and other agencies may similarly vary from country to country. Data collection should therefore range beyond the immediately observable health service inputs to include other service supports, contacts with other agencies (such as housing and criminal justice) and non-service implications of mental ill health (particularly the costs of lost employment and productivity, and the economic burden falling on family caregivers Footnote ¹ ). Moreover, data should be obtained on the frequency and intensity of any service contacts, in order to examine service patterns and to estimate costs accurately.

International research on service utilisation patterns, costs and other economic dimensions of mental health care is complicated by the need to reflect the contexts within which people live and receive their care. Arguably this is true of any research tool, but the problems of economic research which crosses international boundaries are especially acute when the objects of comparison are themselves heavily influenced by social, economic, political, historical and cultural structures and forces peculiar to those contries. To a greater degree than for the other instruments developed or tested in this study, therefore, we needed to ensure that the instruments assessing resource use captured the core features of each of five health care and other systems covered by the study, as well as being sufficiently standardised to permit meaningful international comparison. A core feature was thus to balance local relevance with international generalisability.

Sources of resource utilisation and socio-economic data

Selecting the most appropriate method of data capture for economic studies depends on a number of factors, including the primary purpose of the study, the availability of funding and the data collection methods to be used for other evaluative information in the broader study. In this paper we do not discuss what methods to employ in each possible circumstance or in response to each type of eventuality (for helpful discussion of these issues, see Reference Drummond, O'Brien, Stoddart and TorranceDrummond et al, 1997; Reference Hargreaves, Shumway, Hu and CuffelHargreaves et al, 1998; Reference Johnston, Buxton and JonesJohnston et al, 1998). Instead we focus on the type of study within which we would be examining service use and associated patterns. The main features of that type of study are in fact quite common in mental health services research :

1. (a) a representative sample of people with mental health problems (in this case schizophrenia) treated by ‘ordinary’ services;

2. (b) the aim of making comparisons between samples or sites (in this case, comparisons between countries);

3. (c) a cross-sectional design, with the possibility of repeating the observations later on sampled individuals;

4. (d) a limited research budget, making it necessary either to rely on extant information sets and/or to collect interview-based data concurrently with clinical and associated data.

The two broad options for data collection that presented themselves were to use existing information held by service-providing or funding agencies, or to rely on individual informants. One of the study sites (Verona) has a psychiatric case register which contains health service utilisation data, and other sites had some electronic data (for example, secondary health care information systems in London and Copenhagen). However, none of the local ‘ routine information’ systems was sufficiently compatible with the others to provide the basis for comparative research. Even if there had been some compatibility, there would be the question of data breadth: do extant systems keep data on all relevant services ? We know from previous research that schizophrenia sufferers use many different services (Reference Knapp, Almond, Percudani, Maj and SartoriusKnapp et al, 1999); so even if each site had computerised data, would these cover all relevant services ? In the (unlikely) event that they did, there would then be the considerable challenge of merging data-sets designed for different health care systems, to meet different local management needs, and using different software. Data capture via electronic information systems was therefore rejected at an early stage. Information on services used by individual people will usually be held by service providers, and some professionals (such as general practitioners) should have a reasonably broad view of service utilisation. The disadvantage of relying on service providers to produce these data is that records or professionals' knowledge will often be incomplete, partial (generally covering only the agency's own responsibilities) and difficult to access or expensive in researcher time. These latter factors were important in the present study, and we concluded that we could not rely on agency information holdings in the five sites to provide the range and quality of data needed.

Another way of collecting these data is to ask individuals, either through interview or a self-completion questionnaire. Postal or self-completion methods (including diary cards) have been used in some previous studies (Reference Mauskopf, Schulman, Bell and AmaddeoMauskopf et al, 1996; Reference Gosden, Black and MeadGosden et al, 1997) but were ruled out here, partly because we feared a low response rate, but mainly for the pragmatic reason that other study objectives already required face-to-face interviews, and there is a long track record of collecting service utilisation and associated data alongside clinical data (Reference Beecham and KnappBeecham, 1995).

Three potential groups of respondents could be interviewed: case managers (keyworkers or similar), family members, or patients themselves. If there is a case (or care) manager or keyworker to coordinate services for the patient, they might be a good respondent, although this would depend on the breadth of their responsibilities and knowledge (Reference Widlak, Greenley and McKeeWidlak et al, 1992). Case management has been implemented in some guise in all the study sites, but because the particular modes of operation and service structures differ in these different sites, standardised reporting of service uptake by case managers would be problematic. Where a patient lives with their family, another data source could be other family members (who may act as informal case managers, of course). In some sites it transpired that a majority of the sample lived with at least one relative (72% in Santander, 50% in Verona), but elsewhere this was much less common (20% in Amsterdam and London, and only 4% in Copenhagen). Family members were interviewed for another part of the research study (to complete the Involvement Evaluation Questionnaire (IEQ); Reference van Wijngaarden, Schene and Koetervan Wijngaarden et al, 2000, this supplement), but it was felt that this data source was not sufficiently widely available for the purposes of the description of service use and cost calculations.

The patient is the only person who should have all or most of the information on which particular services have been accessed, how often and for how long. A potential concern, however, is that the patient may not report service utilisation accurately, either because of their clinical condition, or because they exhibit the common human failing of poor recall. We did in fact choose to ask patients for these data, but we needed to take especial care with instrument design to improve the likely accuracy of the information provided (for example, by providing clearly defined and identifiable categories of service or state benefits).

The comparative merits of retrospective and prospective data collection have been discussed elsewhere (Reference Johnston, Buxton and JonesJohnston et al, 1998). Prospective data collection essentially requires maintaining a diary of all service contacts, whereas retrospective collection involves occasional completion of an interview, reflecting back on services used in the previous few weeks or months. (Data collection should not be confused with design: prospective trials can of course use retrospective methods for collecting service use data.) In this study, we adopted a 3-month retrospective period, which is sufficiently long to pick up the wide range of services that individuals might take up but without stretching the respondent's powers of recall (there is evidence to suggest that interviewees significantly under-report frequent events when asked to report retrospectively over a 6-month period; Reference Jobe, White and KelleyJobe et al, 1990).

Instrument development

It is possible to distinguish a number of stages in the development of an instrument for collecting service use and related (Table 1). The first task was to identify criteria for selection or development of an instrument. Four requirements were identified :

Table 1

Stages of instrument development

1.	Identify the main desirable characteristics of an instrument in this area
2.	Examine existing instruments
3.	Select an existing instrument for adaptation or develop a new instrument from first principles, aiming for face validity
4.	Translate into other languages
5.	Organise and conduct focus group discussions to refine the content and translation
6.	Revise both the original instrument (in the light of content recommendations) and the translations (in the light of terminological or language recommendations)
7.	Employ the instrument in its various languages in an ‘exploratory’ cross-national sample
8.	Re-examine the performance of the instrument in terms of, for example, response rates to individual questions, ease of completion, preferred aggregation of service categories
9.	Make further revisions to the instrument so as to make it ready for more widespread use

1. (a) It should span the domains of accommodation and living circumstances, employment and income, and service utilisation, so as to allow description of the economic and related circumstances of individual people and the service or care ‘packages’ that support them.

2. (b) It should record the frequency and intensity of service use, so that service costs can be calculated as accurately as possible.

3. (c) After translation, it should be able to be used alongside the other instruments chosen for the EPSILON Study, and it should also be suitable, after only modest adaptation, for use in other European countries.

4. (d) It should be understandable by respondents (people with schizophrenia) and manageable for use in interviews conducted by trained researchers.

The second stage was to establish whether there existed an instrument that would meet these requirements. If so, we could move on to its employment in the empirical part of the study, examining its performance in use with samples of people with schizophrenia. If no existing instrument satisfied our requirements, as was the case here, the next task was to develop or adapt one. Rather than start from very first principles, we chose to build on an existing instrument, the Client Service Receipt Inventory (CSRI). The CSRI has been widely employed and has a multitude of forms, having been used in over 100 studies since it was first developed in England in the mid-1980s (Reference Beecham, Knapp, Thornicroft, Brewin and WingBeecham & Knapp, 1992, introduce this instrument in the context of a wider discussion of cost research methods in mental health). Although it has been used outside the UK, the CSRI has not previously been subjected to the degree of developmental work or scrutiny that we employed in this study. In particular, close attention was paid to the categorisation of housing, employment and service use items of the inventory in order to generate an instrument capable of international use (see below).

A set of baseline questions was generated which covered the topics of interest (initially in English, subsequently translated into the other four languages). This baseline schedule was then subjected to a process of cross-cultural validation and refinement, based on the discussions in focus groups at each site. The final stage in the development of this resource utilisation measure was to administer it to a sample of individuals at each site, and further revise the instrument in the light of any difficulties or misinterpretation.

The form of the instrument

The baseline version of the Client Socio-Demographic and Service Receipt Inventory - European Version (CSSRI-EU) was constructed around five main sections (Table 2). A manual was prepared which contained explanatory notes for particular questions or items in the schedule that required additional information, definition or guidance.

Table 2

Key domains and variables of the Client Socio-Demographic and Service Receipt Inventory - European Version (CSSRI-EU)

Section	Key variables
Socio-demographics	Age, gender, marital status, ethnicity, mother tongue, years of schooling, educational level
Usual living situation	Living situation (alone, with relatives, etc.), type of accommodation, household composition
Employment and income	Employment status, occupational category, days of work lost, state benefits, source/level of income
Service receipt	Hospital in-patient days, out-patient/day care attendances, community-based service contacts (mental health, social services and primary care), criminal justice service contacts
Medication profile	Name/type of drug, dosage level and frequency

Translations and focus groups

Once the baseline version of the instrument had been developed, the next steps were translation into the other four European languages (Danish, Dutch, Italian, Spanish), by either professional translators or local researchers, followed by cross-cultural validation of the translated instrument. Since the CSSRI-EU is an inventory of socio-economic indicators and service variables rather than a multi-item rating scale of a particular outcome domain, the focus in this study has been on achieving face validity and semantic equivalence within and between individual participating sites, rather than formal exploration of the reliability of the measure between raters, sites or time points (Reference Schene, Koeter and van WijngaardenSchene et al, 2000, this supplement). This took place through both informal dialogue and discussion with principal investigators and other interested parties, and more formally through the conduct of focus groups. Focus groups consisted of between six and ten individuals (psychiatrists, other health professionals, social care workers, informal carers and service users), and were intended to address two aspects of the instrument: its content and its language (Reference Knudsen, Vázquez-Barquero and BeckerKnudsen et al, 2000, this supplement).

The CSSRI-EU focus groups generated a number of system-level comments that revolved around the perceived incompatibility of national health, social and welfare structures with the attempted European-wide structures or categories given in the initial version of the CSSRI-RU. These comments related to two sections of the instrument: usual living situation, and employment and income. In particular, focus groups suggested reordering the categories of employment, benefit entitlements and accommodation so as to reflect their own national systems better. These suggestions were incorporated as far as possible, without losing the core requirement of inter-site comparability. For example, four international categories of state benefits were developed (unemployment/income support; sickness/disability; housing; pension), within which sites could specify local variants of these broader categories. Residential care was a further area that required reordering, owing to the heterogeneity of service arrangements in different sites. This problem was overcome by describing the final categories in neutral, broad terms (overnight facility, 24-hour staffed; overnight facility, staffed (not 24 hours); overnight facility, unstaffed).

A second set of comments revolved around country-specific suggestions for enhancing the understanding, definition or measurement of individual items or components included in the service receipt section of the inventory. A particular area of discussion concerned the appropriate classification and definition of day care and support facilities and community-based mental health services. For example, the Dutch system of community mental health centres needed to be correctly classified under the appropriate item in the inventory.

The instrument was then revised, both in its original English form (in the light of focus group recommendations as to content) and in each of four translations (in the light of recommendations about terminology or language).

Exploratory use of the instrument

The tool was now ready for use in empirical research. In this study the sample was selected in each of the five sites by employing the same diagnostic and administrative criteria (adults aged 18-65 inclusive with an ICD-10 diagnosis of schizophrenia: code F20). All patients had been in contact with mental health services during the 3-month period preceding the start of the study (Reference Becker, Knapp and KnudsenBecker et al, 1999). Fieldwork was conducted over 15 months in the five sites. The main findings regarding service utilisation and costs for this sample are to be reported elsewhere, but a number of methodological issues which emerged in the course of data collection and in the following initial examination of the final dataset can be summarised here.

First, the response rate in terms of completion of the CSSRI-EU was 100%; no refusals to be interviewed were encountered across the sites. This unusually high response rate may be attributable in part to the integration of service utilisation questions with key socio-demographic characteristics (the latter being of fundamental importance to analyses, over and above economic evaluation). Response rates to individual items within the CSSRI-EU were also high (where applicable), indicating few difficulties of interpretation. For items or services not available in, or applicable to, a site (for example, long-stay psychiatric wards in certain sites), zero was entered for all cases.

Structured response categories for a range of socio-demographic and socio-economic indicators (such as living situation, level of educational attainment and state benefit entitlements) performed well. The only clear source of inter-site differences in interpretation related to employment status: specifically, the classification of patients not in open or sheltered employment; in certain sites, such patients were categorised as ‘unemployed’, whereas in other sites a large proportion were classified as ‘retired’, in the sense that they were in receipt of a disability allowance or pension and were not expected or likely to be employed in the future.

As expected, and allowed for in the baseline version of the instrument, a number of specific services had been used over and above those specified. These service contacts were recorded as an ‘other service’ under the appropriate service category, together with a brief text description of the service, and subsequently recoded or described in their own right. For one site (Verona), where the same professionals provide both hospital and community care (continuity of care model), there was evidence of ‘double counting’ of service provision, which subsequently required checking of CSSRI-EU data with that centre's psychiatric case register. A final area of difficulty experienced in the service receipt section of the instrument was coding of the dosage frequency for prescribed medications, because sites adopted different approaches for recording frequency rates for depot v. other drugs.

On the basis of these analyses, a final set of revisions was made to the instrument (and accompanying manual) in order for it to be usable in other research studies. For example, specific services included under ‘other’ for a particular service category were incorporated as necessary, and guidance was modified for coding employment status and frequency of medication use.