1. Introduction
There is growing interest in moving from evidence-based planning to evidence-informed policy, which takes into account information on the local context and other factors influencing decision-making [1–Reference Bate, Robert, Fulop, Øvretviet and Dixon-Woods3]. Context refers to the totality of circumstances that comprise the milieu of a given phenomenon [Reference Shogren, Luckasson and Schalock4] and therefore encompasses information on the physical environment, the social and demographic determinants of health and the range of services available in the local system and their costs [Reference Raine, Fitzpatrick, Barratt, Bevan, Black and Boaden5, Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6]. The eventual aim is to allow such information to be incorporated into real world decision support systems to guide planning and resource allocation [Reference García-Alonso, Almeda, Salinas-Pérez, Gutiérrez-Colosía and Salvador-Carulla7] and facilitate interpretation of research results.
Context analysis, including service provision, is part of “healthcare ecosystem research” [Reference Lee, Wong, Bartsch, Yilmaz, Avery and Brown8, Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9], an emerging discipline that analyses the complexity of care systems and interventions in a defined environment, using methods developed in environmental sciences for ecosystem services research [Reference Laihonen10].
PRISMA flowchart of study.
There is wide variability in the terminology referring to services and programs delivered even in the same geographical area, and listings of services by their names alone should be analysed with caution, as the service names do not always reflect activity. Meaningful international comparisons need a common consensus-based terminology to improve health, strengthen health systems and provide essential healthcare for all [Reference World Health Organization11, Reference World Health Organization12]. A common coding system, using a standardised method of assessment, is important to overcome these challenges and enable better comparisons of data to inform policy and practice [Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6].
Comparative description of national and international health systems is important for identification of gaps in care, particularly in mental health. The World Health Organization has raised international awareness of the gap between health care needs and available resources following the launch of the mental health Gap Action Programme in 2008 [Reference World Health Organization13].
Recent guidelines on implementation research (StaRI -Standards for Reporting Implementation Studies) [Reference Pinnock, Barwick, Carpenter, Eldridge, Grandes and Griffiths14] call for transparent and accurate descriptions of the environment in which implementation studies took place [Reference Rycroft‐Malone and Burton15]. The guidelines specify a detailed description of both the general context in which the intervention is implemented, and the service provision structure at personnel, health resources and sites level in which the implementation takes place.
The European Service Mapping Schedule (ESMS) was developed to facilitate the classification of mental health services and the standardised description of the care system for adults experiencing mental health problems by the EPCAT group (European Psychiatric Care Assessment Team) between 1994 and 1997, and it was published in 2000 [Reference Johnson, Kuhlmann and EPCAT group16, Reference Munizza, Tibaldi, Cesano, Dazzi and Fantini17]. This system evaluation toolkit also included the International Classification of Mental Health Care (ICMHC) [Reference De Jong18], which evaluated the different modalities of care available in a service, and the European Socio-demographic Schedule (ESDS) [Reference Beecham, Johnson and EPCAT Group19] for the standardised description of the sociodemographic characteristics of local catchment areas.
The ESMS was expanded and adapted for the assessment of other target groups such as children and adolescents, people with drug and alcohol problems or disabilities [Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20] and ageing populations [Reference Salvador Carulla21]. This expanded version, called “Description and Evaluation of Services and DirectoriEs” (DESDE), was adapted for the evaluation of chronic or long-term care (DESDE-LTC) [Reference Salvador-Carulla, Alvarez-Gálvez, Romero, Gutiérrez-Colosía and Weber22] and for the evaluation of social services (DESDE-AND) including an automated coding system [Reference Almenara Abellán, Gutiérrez-Colosía, Salinas Pérez, Arlette, García Alonso and Salvador-Carulla23]. Hence, the original ESMS instrument for adult mental health care, comprising 36 codes, has been expanded to an international classification of the care sector with 106 codes and over six instruments (see Fig. 1). This classification is here referred as the “ESMS/DESDE” system for health care evaluation. Since the original work of the EPCAT group (1994–2000) several European Union funded research reference groups have continued the seminal work of EPCAT developing and implementing this system (Mental Health Economics Network-MHEEN, Description and Evaluation of Services and DirectoriEs for Long Term Care-DESDE-LTC research Group and REsearch on FINancing systems' Effect on the quality of MENTal health care-REFINEMENT).
The classification and its related instruments describe care provision in catchment areas, comparing the structure, distribution and typology of services across health districts. The ESMS and the instruments derived from it, use a tree diagram to describe health services over four main domains:
A Definition of catchments, target populations and units of analysis for services. Services are analysed as “Care Teams” or “Basic Stable Inputs of Care”. A BSIC is a combined and coordinated set of inputs (including structure, staff and organisation) that delivers care at a micro-organisation level, and has temporal and organisational stability. In summary, a BSIC is characterized by a stable group of professionals who on a routine basis provide coordinated care to the same group of patients or consumers. Catchments, target populations and services can be aggregated to provide higher-level analysis of health systems.
B Availability of care: activities performed by the Care Teams. Each team or program is coded according to the Main Types of Care (MTCs) it provides. The MTC codes describe the principal activities of the service according to the ESMS/DESDE hierarchical tree taxonomy. There are six main MTC mapping branches (Residential, Day, and Outpatient Care, Self-help support, Information and assessment, and Accessibility), as well as optional qualifiers that can be used to develop a more granular description of services as required.
C Resource use: The ESMS/DESDE system provides instructions for collection of standardised counts of service use. As with other sections, various levels of granularity can be obtained as required by the specific evaluation project.
D Service characteristics checklist: A more detailed, standardised analysis of local care organisations and functional teams, including information about governance, funding sources, characteristics of the services and staffing.
This system is intended to be widely used and is open access to favour its use by non-for-profit organisations. Its tree structure has facilitated the incorporation of new codes as new target groups or sectors were coded.
Even though the ESMS/DESDE has been extensively applied in health care assessment in many countries, there has not been a comprehensive review on its “diffusion” in a range of different sectors and target groups across the world. Diffusion refers to the spreading of the innovation tool more widely in a range of different contexts [Reference Greenhalgh, Robert, Macfarlane, Bate and Kyriakidou24].
This study aims to identify, describe and analyze the use and the international diffusion of the ESMS/DESDE system for health service evaluation and systems research and its impact in health policy and decision-making.
2. Method
We conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [Reference Moher, Liberati, Tetzlaff and Altman25]. The protocol of the systematic review was published in PROSPERO, a database of prospectively registered systematic reviews in health and social care (CRD42018104864).
2.1 Search strategy
The search was carried out until December 31, 2018. There was no limitation of the search strategy based on language or year of publication. We searched for scientific publications in the following electronic databases: Web of Science, Scopus, Proquest, (Agricultural & environmental science database, Health and medical collection, Nursing & allied health database, Psycarticles, Psychology database, PsycInfo), Pubmed, Google Scholar and OVID.
We used a similar search strategy for every database search. The key words included different nomenclature applied to the ESMS/DESDE system and the instruments derived from it from its initial development to the present. For example in the case of Pubmed database, the search strategy was as follows: (("EUROPEAN SERVICE* MAPPING SCHEDULE"[All Fields] OR eDESDE[All Fields]) OR DESDE-LTC[All Fields]) OR REMAST[All Fields].
2.2 Eligibility criteria
The adapted PICO method included: Participant/Population (all type of services for people with mental disorders, disability or long-term care), Intervention (assessment of services using ESMS/DESDE), and Outcome (application/diffusion of the ESMS/DESDE and policy impact).
Inclusion criteria were: Journal articles reporting studies about services or health systems evaluation that have used ESMS/DESDE, studies about services aimed at people with mental health, disability or long term care needs and introduction of the instruments of ESMS/DESDE system. No restrictions were set about country of use, year of publication or language.
Studies that did not incorporate a care service or systems evaluation or that only provide a reference to other ESMS/DESDE studies were excluded from the final selection list. The studies that only refer to ESMS/DESDE as part of a conceptual framework in evaluation of services have been counted separately but not included in the systematic review. Grey literature has not been included in this review.
2.3 Study selection
Two authors (CR and MR) carried out an independent screening and eligibility analysis. In the first phase, CR and MR checked the paper’s title and abstract to assess whether it potentially met criteria for inclusion. In the eligibility phase, they reviewed full text articles. Where there was disagreement between the two reviewers, a third researcher (JAS) was consulted to reach consensus on eligibility. A direct content analysis was made of papers published in English, Spanish, French, Portuguese and Italian. External support was required for context analysis of papers published in German, Polish, Farsi and Mandarin.
Titles and abstracts of all citations were obtained for phase 1 of the study selection. Citation indices and reference lists of retrieved articles were checked for additional studies not identified in the original database search. The full text of the screened records was searched.
Fig. 1 provides a visual representation of the current review’s methodological process, according to the PRISMA framework [Reference Moher, Liberati, Tetzlaff and Altman25].
When necessary, the reviewers contacted the leading authors to get further clarification and information on the use of the ESMS/DESDE in other countries.
2.4 Data abstraction
The data abstraction form included: 1) bibliographic information (first author, year of publication); 2) uses of ESMS/DESDE system; 3) specific tools derived from the ESMS; 4) psychometric properties of the tools; 5) language of the version used; 6) country of application; 7) target population; 8) number of evaluated areas. Whenever possible, the catchment areas described in the study were classified in accordance with the territorialisation levels described in the DESDE-LTC instrument (H1-national level, H2-regional level, H3-hospital catchment area and H4-mental health community center catchment areas) [Reference Salvador-Carulla, Dimitrov, Weber, McDaid, Venner and Sprah26]. Twenty-two papers used other territorial jurisdictions such as municipalities, urban districts or research sites and the geographical level was not specified; four studies did not describe areas. In addition, we included a description of the following characteristics: 9) number of services evaluated in functional teams or BSICS; 10) research group; (11) impact on policy for plan and health decision use; (12) funding source.
2.5 Study characteristics
The study characteristics are shown in Table 2. This description of the selected papers includes: the specific ESMS/DESDE tool used in the study, the specific tool and its metric properties (feasibility, reliability, validity), framework of service research included in the study, description of the reference areas and its social and demographic characteristics (context analysis) and demographic context (standard description or basic data), the evaluation of the service provision in the system, the agents using and/or providing care in the system (patient, family, professional), resource utilization, the main aim of the study (e.g. scale development, costs, description of the service delivery, supply and demand, health interventions or decision support system model), the use of visualization tools (geographic information system, spatial analysis and atlas), type of analysis (data analytic or decision support system), and use for decision making.
This study has followed the PRISMA quality criteria for systematic reviews [Reference Moher, Liberati, Tetzlaff and Altman25]. The quality was assessed using a checklist based on the EPCAT model for services evaluation [Reference Salvador, Atienza, Romero, Guimón and Sartorius27] and the domains suggested by Votruba and colleagues for health system and policy research [Reference Votruba, Ziemann, Grant and Thornicroft28]. The quality checklist included the following ten domains listed above. Studies were rated as high quality when they fulfilled at least six criteria in this checklist.
3. Results
3.1 Literature search
The search performed in our review retrieved 1.114 references. One additional paper was included after checking the list with the experts of the group. Fig. 1 shows the flowchart of the selection process. After removing duplicates, 830 abstracts were reviewed by two independent researchers. A total of 603 records were excluded because they were not focused on assessment of health systems and services. The full text of the remaining 227 articles was text assessed for eligibility. Of these, 72 papers were excluded, of which 61 did not include a reference to the ESMS/DESDE system, six were conference abstracts and five were not available for the full text. Another 84 texts referred to the ESMS/DESDE system in the introduction or discussion but did not actually use the ESMS/DESDE classification or its related instruments.7
3.2 General characteristics
Since 1997, 155 papers have mentioned the ESMS/DESDE system. Out of them, 71 articles have actually used the ESMS/DESDE for service research. It should be noted that three papers authored by members of the core EPCAT team were published before the ESMS was officially released in 2000 [29–Reference Haro, Salvador-Carulla, Cabasés, Madoz and Vázquez-Barquero31].
The data abstraction form is included in Annex 1 with information about key features of the papers reviewed. The main characteristics of the reviewed articles are shown in Table 2.
Eleven tools for health services and system research have derived from the original ESMS [Reference Johnson, Kuhlmann and EPCAT group16] (Fig. 2). Six are versions of the ESMS and DESDE instruments aimed at different target groups or levels of research [Reference Prot-Klinger and Pawłowska32, Reference Ala-Nikkola, Pirkola, Kontio, Joffe and Pankakoski33, 20–Reference Almenara Abellán, Gutiérrez-Colosía, Salinas Pérez, Arlette, García Alonso and Salvador-Carulla23]. In addition three instruments have been derived from the ESMS by independent groups for a) cost analysis in schizophrenia (Service Utilization Sheet –SUS-, [Reference Salize, McCabe, Bullenkamp, Hansson and Lauber34], b) analysis of community mental health services in South Africa (framework for CMHS, [Reference Lund and Flisher35], and c) for evaluating transition services from child to adult mental health care in Finland (European CAMHS Mapping-ECM-Q Questionnaire) [Reference Tuomainen, Schulze, Warwick, Paul and Dieleman36]. The REFINEMENT Decision Support Toolkit includes two tools derived from DESDE: the Mental Health Service Inventory–MHSI- [Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6] that summarises information from DESDE-LTC; and the REFINEMENT Glossary of terms for mental health system research [Reference Montagni, Salvador-Carulla, Mcdaid, Straßmayr and Endel37] that extended the vocabulary developed in the ESMS/DESDE system [Reference Salvador-Carulla, Dimitrov, Weber, McDaid, Venner and Sprah26]. Other instruments such as World Health Organization Assessment Instrument for Mental Health Systems (WHO-AIMS) [Reference World Health Organization38] have included terms from ESMS. We have only included instruments directly derived from ESMS in Fig. 2.
The ESMS/DESDE system has been translated from English into eight languages: Italian, Finnish, German, Norwegian, Polish, Russian, Slovenian and Spanish. Contact with the authors allowed the identification of two unpublished versions of ESMS in France and Brazil, that have not been included in this review. Six papers provide a description of versions of the ESMS/DESDE system in different countries: [Reference Johnson, Kuhlmann and EPCAT group16, Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6, Reference Montagni, Salvador-Carulla, Mcdaid, Straßmayr and Endel37, Reference Salvador Carulla, Torres Gonzalez, Johnson, Kuhlman and Varquez-Barquero29, Reference Baková, Raboch and Stárková39, Reference Adamowski and Trypka40].
From the selected papers, 21.1% fulfilled at least six criteria (high quality) of the 10 quality criteria included in this systematic review. Fifteen papers fulfilled three or fewer criteria and were considered low quality. For further details, see Table 2.
More than 64% of the papers provided a detailed description of the specific instrument used. The inclusion of a context analysis (social and demographics and other characteristics of the area) in the study is an important quality indicator of 29 articles analyzed, 18 of them using standard methodology.
3.3 Psychometric properties of ESMS/DESDE system
The psychometric properties of the different ESMS/DESDE tools have been described in 11 papers (Table 2). The feasibility of ESMS/DESDE was considered adequate in all of them, although the coding system requires intensive training. The instruments ESMS, DESDE and DESDE-LTC have shown optimal levels of consistency, descriptive validity and inter-rater reliability in studies conducted by the core group [Reference Munizza, Tibaldi, Cesano, Dazzi and Fantini17, Reference Salvador‐Carulla, Romero and Martinez41, Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20], [Reference Salvador-Carulla, Alvarez-Gálvez, Romero, Gutiérrez-Colosía and Weber22]. The ontological properties, content analysis and hierarchical structure of the DESDE-LTC classification has also been published [Reference Salvador-Carulla, Alvarez-Gálvez, Romero, Gutiérrez-Colosía and Weber22, Reference Salvador-Carulla, Romero, Weber, Dimitrov and Sprah42].
In addition, four psychometric studies have been carried out by other independent research groups [43–Reference Skiba and Prot46]. Becker and colleagues indicated a low usability of ESMS, but this evaluation did not follow the training recommended by the EPCAT core group. Two articles using ESMS/DESDE data for decision support system (DSS) include metric properties of the derived decision support tools such as agreement, predictive validity, feasibility and technology readiness level (TRL) [Reference Salvador-Carulla, García-Alonso, Gonzalez-Caballero and Garrido-Cumbrera47, Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9].
Pilot and demonstration studies have been carried out in Spain [Reference Salvador‐Carulla, Romero and Martinez41, Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20], Italy [Reference Munizza, Tibaldi, Cesano, Dazzi and Fantini17] and Australia [Reference Fernandez, Gillespie, Smith-Merry, Feng and Astell-Burt48].
3.4 Use of ESMS/DESDE in service research
3.4.1 International diffusion of ESMS/DESDE system
ESMS/DESDE system has been used in 34 countries comprising four WHO world regions. The distribution of the countries is shown in Table 1. There are five European countries with 10 or more publications related to the application of ESMS/DESDE system.
Out of the 71 papers selected for this review, 40 have used a version of ESMS. Two have used the ESMS-b, four the ESMS-R (actually the expanded version of DESDE-LTC). Sixteen papers have used a version of DESDE (Table 2). The remaining four papers have used another tool derived from the ESMS/DESDE system (Fig. 2).
The utilization of the ESMS/DESDE system fell into six domains categories: Methodology aspects of the system (11 papers), description of Health Interventions and Services (13 papers), Context analysis (29 papers), articles describing the relationship between Supply and Demand (8 papers), use in Health Economics (6 papers) and use in Decision Support Systems (4 papers) (Annex 1).
3.4.2 Use by different research reference networks (Annex 1)
Eleven national and international research reference networks have used ESMS/DESDE system in their studies. The ESMS/DESDE system core group (EPCAT, PSICOST, DESDE-LTC and REFINEMENT) has been involved in 31 papers related to the development and use of ESMS/DESDE system. Other research networks that have applied the ESMS/DESDE in their studies include (full names available in annex 1): the EDEN study, the EUNOMIA project, EPSILON Study, LIDO Study, EuroSC project, and MILESTONE project, in Europe. In Latin America, the ESMS/DESDE has been used by the Maristan Network.
3.4.3 Use in different care sectors and target groups
Of the articles that applied ESMS/DESDE system, 48.4% carried out a cross-sectoral evaluation, in some cases focused on specific target groups. The health sector was the focus of 18.7% of the papers; 26.7% related to specific health care: one paper described primary care services for patients with depression [Reference Chisholm, Amir, Fleck, Herrman and Lomachenkov51], three papers evaluated mental health and substance abuse services [Reference Ala-Nikkola, Pirkola, Kontio, Joffe and Pankakoski33, Reference Dahl, Rezvyy, Bogdanov and Øiesvold88, Reference Ala-Nikkola, Pirkola, Kaila, Joffe and Kontio90], one paper evaluated transition services from child and adolescent to adult mental health care [Reference Tuomainen, Schulze, Warwick, Paul and Dieleman36] and others assessed other specific mental health services. One article focused on the evaluation of vocational services for people with schizophrenia [Reference Marwaha, Johnson, Bebbington, Angermeyer and Brugha70] and nine papers described services used for specific target population like people with schizophrenia [Reference Salvador-Carulla, Haro, Cabases, Madoz and Sacristan30, Reference McCrone, Leese, Thornicroft, Schene and Knudsen52, Reference Knapp, Chisholm, Leese, Amaddeo and Tansella53, Reference Bebbington, Angermeyer, Azorin, Brugha and Kilian57, Reference Roick, Heider, Toumi and Angermeyer61, Reference Marwaha, Johnson, Be bbington, Stafford and Angermeyer62, Reference Marwaha, Johnson, Bebbington, Angermeyer and Brugha70, Reference Salize, McCabe, Bullenkamp, Hansson and Lauber34] and intellectual disabilities [Reference Salvador-Carulla, Martinez-Leal, Poole, Salinas-Perez and Tamarit78].
3.4.4 Use in healthcare ecosystem research (context analysis)
Twenty-six of the selected papers highlighted the relevance of a standard model and method for service research for evaluating health systems. Specific references to ecosystem research were mentioned in two papers [Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9, Reference Sadeniemi, Almeda, Salinas-Pérez, Gutiérrez-Colosía and García-Alonso91].
In spite of its wide use for describing catchment areas, only 29 studies (40.8%) provided a full description of the areas following an ecological approach. Most of these studies (18) used the European Socio-Demographic Schedule [Reference Beecham, Johnson and EPCAT Group19], another instrument of the EPCAT Toolkit, or derived instruments from ESDS including more contextual indicators. The remaining documents did not describe socio-demographic characteristics of evaluated areas or presented a poor description (Table 1). These 29 studies provide a standard description of 585 catchment areas. These geographical areas include a wide array of urban and rural districts and different jurisdictions at meso and macro levels that have been described using the health area classification provided in the DESDE-LTC manual (H codes) [Reference Salvador-Carulla, Dimitrov, Weber, McDaid, Venner and Sprah26]. One study described a whole country (Level H1) [Reference Tuomainen, Schulze, Warwick, Paul and Dieleman36]. Fourteen studies have provided descriptions at regional/state level or in health districts (H2). The H3 level (hospital catchment area) has been used in 10 studies, another 16 studies describe catchment areas of community mental health centers (mesolevel-H4), and three studies provide descriptions of a combination of both H3 and H4 areas. These ecological studies should be differentiated from other studies describing purpose areas (e.g. research sites in 18 studies). Finally, three studies describe jurisdiction boundaries that do not facilitate international comparability such as municipalities or urban districts (Table 1).
These studies have allowed standard description of over 6.279 different services (Basic Stable Inputs of Care - BSICS) following the ESMS/DESDE system methodology. In some cases, the paper did not specify the number of BSICs evaluated in the study (see Annex 1)
Thirty-two papers provided comparative analysis of the context of care in nine different countries at national level: seventeen studies in Spain, five in Finland, four in Germany, three in Poland, two in Australia, Chile, Italy and Slovenia; and one in South Africa and Canada.
We also found 24 papers comparing mental health areas or health systems at international level. Six studies compared regions or districts across two countries: Russia and Norway [Reference Rezvyy, Oiesvold, Parniakov, Ponomarev and Lazurko64, Reference Dahl, Rezvyy, Bogdanov and Øiesvold88], and Spain versus a) Bulgaria [Reference Salvador-Carulla, Romero, Weber, Dimitrov and Sprah42], b) Italy [Reference Salvador-Carulla, Tibaldi, Johnson, Scala and Romero59], c) Chile [Reference Salvador-Carulla, Saldivia, Martinez-Leal, Vicente and Garcia-Alonso69] and d) Finland [Reference Sadeniemi, Almeda, Salinas-Pérez, Gutiérrez-Colosía and García-Alonso91]. Other international papers described service provision across three to nine countries including the EDEN study, LIDO Study, EPSILON Study, EuroSC project, eDESDE-LTC project and REFINEMENT project. The EUNOMIA study included comparisons across 12 countries in Europe: Germany, Bulgaria, Czech Republic, Greece, Israel, Italy, Lithuania, Poland, Slovak Republic, Spain, Sweden and UK [Reference Mastrogianni, Georgiadou, Iosifidis, Kotrotsios and Karastergiou56, Reference Kallert, Glöckner, Onchev, Raboch and Karastergiou58, Reference Raboch, Kalisova, Nawka, Kitzlerova and Onchev73, Reference Kalisova, Raboch, Nawka, Sampogna and Cihal80].
A significant percentage (66.2%) of the articles described the provision of services of a specific area. Twenty-eight papers used ESMS/DESDE either to provide context to local outcomes or to analyse the relationship between the service delivery system and outcomes such as family burden (e.g. [Reference Roick, Heider, Toumi and Angermeyer61]), needs (e.g [Reference Ala-Nikkola, Pirkola, Kaila, Saarni and Joffe84].) and costs (e.g [Reference Haro, Salvador-Carulla, Cabasés, Madoz and Vázquez-Barquero31].).
Eleven papers used visualization tools for representing data including basic geographical information (4), spatial analysis (1) or advanced geographical information system incorporated to atlases of care (4) (even though the maps did not appear in the paper) [Reference Beperet, Nadal, Martínez, Salvador and Romero49, Reference Fernández, Salinas-Perez, Gutiérrez-Colosía, Prat-Pubill and Serrano-Blanco81, Reference Iruin-Sanz, Pereira-Rodríguez and Nuño-Solinís82, Reference Rodero-Cosano, Salinas-Pérez, González-Caballero, García-Alonso and Lagares-Franco86]. One paper used ESMS/DESDE to inform machine learning using Self-Organising Maps (SOMNET) for health planning [Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9].
3.4.5 Use in longitudinal studies of the evolution of care systems
ESMS/DESDE has been used to monitor the evolution of the mental health care provision in several countries and regions. The mental health improvement and its relationship to the regional plan was analysed in Catalonia (Spain) in 2002, 2006 and 2010 [Reference Fernández, Salinas-Perez, Gutiérrez-Colosía, Prat-Pubill and Serrano-Blanco81]. Two health districts in Central Chile were evaluated in 2004/05 [Reference Salvador-Carulla, Saldivia, Martinez-Leal, Vicente and Garcia-Alonso69], 2008/09 [Reference Romero, Salinas, Saldivia, Grandón, Poole and Salvador-Carulla95] and 2012 [Reference Salinas-Perez, Salvador-Carulla, Saldivia, Grandon and Minoletti94]. Three hospital districts in Finland were assessed in 2004 [Reference Pirkola, Sund, Sailas and Wahlbeck71], 2011/12 [Reference Ala-Nikkola, Pirkola, Kontio, Joffe and Pankakoski33] and 2012/14 [Reference Ala-Nikkola, Pirkola, Kaila, Saarni and Joffe84].The area of Verona (Italy) has been analysed in 2002 and 2010 [Reference Gutiérrez-Colosía, Salvador-Carulla, Salinas-Pérez, García-Alonso and Cid89]; and the areas in Northwest Russia and Northern Norway in 2004/05 and 2011/12 [Reference Rezvyy, Oiesvold, Parniakov, Ponomarev and Lazurko64, Reference Dahl, Rezvyy, Bogdanov and Øiesvold88].
Tools for service and health systems research derived from the European Service Mapping Schedule and related to the ESMS system for the international classification of care provision.
1ESMS (European Service Mapping Schedule) [Reference Johnson, Kuhlmann and EPCAT group16], the brief version (ESMS-b) [Reference Prot-Klinger and Pawłowska32] and the extended Revised version (ESMS-R) [Reference Ala-Nikkola, Pirkola, Kontio, Joffe and Pankakoski33] for the assessment of adult mental health care.
2DESDE (Description and Evaluation of Services and DirectoriEs). Apart from mental health it has been used for the classification of services for Disabilities (DESDE) [Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20], ageing (DESDAE) [Reference Salvador Carulla21], Long Term care (DESDE-LTC)[Reference Salvador-Carulla, Alvarez-Gálvez, Romero, Gutiérrez-Colosía and Weber22], automated evaluation of social services in Andalucia (Spain) (DESDE-AND) [Reference Almenara Abellán, Gutiérrez-Colosía, Salinas Pérez, Arlette, García Alonso and Salvador-Carulla23]. It has also been used for mapping Drug and Alcohol Services, Child and Adolescent Mental Health, Chronic care and care for homeless.
3SUS (Service Utilization Sheet) [Reference Salize, McCabe, Bullenkamp, Hansson and Lauber34].
4CMHS (Community Mental Health Services provision) [Reference Lund and Flisher35].
5ECM-Q (European Child and Adolescent Mental Health Mapping Questionnaire) [Reference Tuomainen, Schulze, Warwick, Paul and Dieleman36].
6MHSI (Mental Health Service Inventory) [Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6].
7Glossary of terms REFINEMENT EU project [Reference Montagni, Salvador-Carulla, Mcdaid, Straßmayr and Endel37].
3.4.6 Use in health economics and health financing analysis
Seven papers have used the ESMS/DESDE system for health economics including the analysis of the context of health expenditure [Reference Chisholm, Amir, Fleck, Herrman and Lomachenkov51], the development of units of cost analysis and service utilization [Reference Moreno, Sanchez and Salvador-Carulla68, Reference Salize, McCabe, Bullenkamp, Hansson and Lauber34], cost of illness [Reference Haro, Salvador-Carulla, Cabasés, Madoz and Vázquez-Barquero31, Reference Knapp, Chisholm, Leese, Amaddeo and Tansella53] and efficiency analysis [Reference Torres-Jiménez, García-Alonso, Salvador-Carulla and Fernández-Rodríguez83, Reference Almeda, García-Alonso, Salinas-Pérez, Gutiérrez-Colosía and Salvador-Carulla87].
3.5 Impact of the use of ESMS/DESDE system on decision making
A substantial number of the selected papers have been funded by public agencies or international bodies such as the European Union. Twenty-two articles (31%) received funding from national, regional or local governmental agencies. Two papers mentioned the use of the ESMS/DESDE terminology and coding structure in the development of other major international classification and service assessment tools such as the International System of Health Accounts (SHA 2.0) [96, Reference Salvador-Carulla, Romero, Weber, Dimitrov and Sprah42]; and the WHO Assessment Instrument for Mental Health Systems (WHO-AIMS) [Reference World Health Organization38, Reference Salinas-Perez, Salvador-Carulla, Saldivia, Grandon and Minoletti94].
The ESMS/DESDE system has been adopted for health and social policy planning by public agencies in several countries. ESMS and ESMS-R have been extensively used for health planning in Finland [Reference Ala-Nikkola, Pirkola, Kaila, Saarni and Joffe84]. The DESDE coding has been adopted for the classification system for disability services in three Autonomous Communities in Spain (Navarra, Castilla la Mancha and Andalucia) [Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20]. Data from the Dernovšek & Šprah study was used to inform the National Mental Health Plan in Slovenia [Reference Dernovšek and Šprah65].
Four papers focused on models to improve decision support in healthcare systems using ESMS/DESDE system [Reference Gibert, García-Alonso and Salvador-Carulla72, Reference Torres-Jiménez, García-Alonso, Salvador-Carulla and Fernández-Rodríguez83, Reference Almeda, García-Alonso, Salinas-Pérez, Gutiérrez-Colosía and Salvador-Carulla87, Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9].
4. Discussion
To fill the existing gap between the burden of diseases and the resources available to treat them, it is imperative to obtain basic information about care provision at local, regional and national levels [Reference World Health Organization97]. The WPA-Lancet Psychiatry Commission on the Future of Psychiatry [Reference Bhugra, Tasman, Pathare, Priebe, Smith and Torous98] has also underscored the need to reform the “traditional structure of services”. However, the evaluation of a system´s reform requires a detailed knowledge of the existing structure of services and how these services change over time. The ESMS/DESDE set of instruments provides key information for monitoring health systems. It incorporates a common terminology, an international taxonomy and coding of health services, a standard procedure for data collection and meaningful comparisons across and within countries.
ESMS/DESDE addresses three key problems in health service evaluation: commensurability, terminological variability and the lack of contextualisation. First, the incommensurability bias is due to the existence of different units of analysis in health care. “Service” and ‘interventions” are vague terms that could refer to very dissimilar units of analysis impeding comparisons like-with-like. The ESMS/DESDE has introduced an operational unit called “Basic Stable Inputs of Care” (BSIC) that allows comparison across jurisdictions [Reference Salvador-Carulla, Amaddeo, Gutiérrez-Colosía, Salazzari and Gonzalez-Caballero6].
Second, the terminological variability is a major source of ambiguity in healthcare research. The name of the service does not always reflect the activity it performs and this causes major problems when services classified by their names such as “nursing homes” or “day hospitals” are aggregated for care gap analysis, financing or planning. To overcome this problem ESMS/DESDE has developed a fully operational taxonomy for coding BSICS based on their Main Type of Care (MTCs). Its metric properties have been extensively analysed by several independent groups and the usability of the system has been demonstrated around the world, including a number of key international studies on mental health service research. Papers have been appearing more frequently recently, with 29.5% published in the last five years due to growing interest in health agencies supporting this type of study.
Third, services should be understood in the context of the local system that they belong. Context analysis is now considered a key component of healthcare ecosystem research [Reference Chung, Salvador-Carulla, Salinas-Pérez, Uriarte-Uriarte and Iruin-Sanz9, Reference Furst, Gandré and Romero-Lopez-Alberca99]. This approach was advanced 20 years ago by the EPCAT group when the assessment of services provided by ESMS was accompanied by the standard analysis of the social and demographic context using ESDS [Reference Beecham, Johnson and EPCAT Group19], and by the description of the main modalities of care using ICMHC [Reference De Jong18]. Whilst WHO-AIMS [Reference World Health Organization38] and the Mental Health Country Profile (MHCP) [Reference Jenkins, Gulbinat, Manderscheid, Baingana, Whiteford and Khandelwal100] have been used for describing national mental health systems worldwide, ESMS/DESDE is the only system that provides local, bottom-up information that can be used across different sectors (health, social, education, employment, housing and justice) [Reference Furst, Gandré and Romero-Lopez-Alberca99] and for coding services for different target groups such as mental disorders [Reference Gutiérrez-Colosía, Salvador-Carulla, Salinas-Pérez, García-Alonso and Cid89, Reference Chisholm, Amir, Fleck, Herrman and Lomachenkov51] intellectual disabilities [Reference Salvador-Carulla, Martinez-Leal, Poole, Salinas-Perez and Tamarit78], substance abuse [Reference Ala-Nikkola, Pirkola, Kontio, Joffe and Pankakoski33, Reference Dahl, Rezvyy, Bogdanov and Øiesvold88, Reference Ala-Nikkola, Pirkola, Kaila, Joffe and Kontio90], general disabilities [Reference Salvador-Carulla, Poole, Gonzalez-Caballero, Romero and Salinas20], aging and long term care [Reference Salvador Carulla21, Reference Salvador-Carulla, Alvarez-Gálvez, Romero, Gutiérrez-Colosía and Weber22]. The usability of ESMS/DESDE for the analysis of local change and improvement has been tested in Catalonia (Spain) where the evolution of the mental health care system was analysed before and after the implementation of the 2006 regional mental health plan (2002–2010) [Reference Fernández, Salinas-Perez, Gutiérrez-Colosía, Prat-Pubill and Serrano-Blanco81]. A 15 years on analysis is currently under way in this region.
This coding system has provided the basic information for producing local atlases of mental health care in America, Australia and Europe. These atlases are one of the visual tools that are being used to analyse mental health systems and support better decision making [Reference Jenkins, Gulbinat, Manderscheid, Baingana, Whiteford and Khandelwal100]. This has contributed to the development of regional and national action plans in mental health [Reference Fernández, Salinas-Perez, Gutiérrez-Colosía, Prat-Pubill and Serrano-Blanco81], intellectual disability [Reference Salvador-Carulla, Martinez-Leal, Poole, Salinas-Perez and Tamarit78] and social care [Reference Almenara Abellán, Gutiérrez-Colosía, Salinas Pérez, Arlette, García Alonso and Salvador-Carulla23]. In addition, the use of the ESMS/DESDE system provides relevant information for the assessment of equity to universally accessible services, an essential component of the Sustainable Development Goals and the new global health agenda on Universal Health Care [Reference World Health Organization101]. Furthermore, the realisation of integrated people-centred health services depends on health system inputs, which require reliable and standardised information on service provision [Reference World Health Organization102].
The use of ESMS/DESDE system in health economics and health financing identified in this review is particularly relevant for health policy and its incorporation into real world decision support systems routinely used by public agencies to inform their planning strategies [Reference Iruin-Sanz, Pereira-Rodríguez and Nuño-Solinís82, Reference García-Alonso, Almeda, Salinas-Pérez, Gutiérrez-Colosía and Salvador-Carulla103].
4.1 Limitations
However, the use of this system has several limitations. ESMS/DESDE requires extensive training and the interpretation of the results by decision makers should involve additional support from experts. To overcome these difficulties the core group developed open source online training material, made the system fully accessible to non-for-profit organizations and produced a brief and user-friendly version (ESMS b). However, these initial problems persist in the current versions as the problem may not rely on the difficulty of the instrument but on the inherent complexity of care services. In addition, its easy access has led to certain nomenclature confusion in the use of the different versions of the tools, and to the development of instruments not supervised by the original authors. The participation of two members of the original core group in this study (SJ and LSC) may have skewed the review towards positive results. However, these two authors have not participated in the selection process nor the individual analysis of the documents and negative results have been reported. This review does not include grey literature or technical reports by public agencies even though these sources of information are key for its use in policy and practice. A complementary review of the grey literature of the ESMS/DESDE system is currently under way.
1. Territorialisation levels for mental health planning and policy [Reference Moher, Liberati, Tetzlaff and Altman25]. H: Health areas as defined in DESDE-LTC. H2 macro level (regional), H3 meso-level (e.g health district, catchment area), H4: micro level MH: Mental Health.
* Implicit information. Type of analysis: Data analytic (statistical analysis of the data to draw conclusions); Decision support systems (tools supporting decision making processes).
5. Conclusion
The ESMS/DESDE system provides a common terminology, an ontology based classification of care services, a set of instruments covering different aims in healthcare research, a standard method for data collection of service provision in health and social care, and facilitates comparisons across and within countries. This system has been extensively used to provide context information at every level of the health system (local, regional, national), for care gap analysis, health economics, and for modelling healthcare ecosystems. It has been used across different care sectors and has been effectively incorporated into decision support systems to guide evidence-informed planning.
Availability of data and materials
Supplementary materials (Annex 1) are available in the http://psicost.org/ website, and upon request.
Declaration of Competing Interest
The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing their work. Authors declare no conflicts of interest.
Acknowledgments
This systematic review is part of the PECUNIA project. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 779292
We want to acknowledge Nasser Bagheri, Rose Marie Fischer, Instituto Internacional de Idiomas and Mingming Zhou for the translation of Iranian, Polish, German and Mandarin texts.
References marked with an asterisk indicate studies included in the systematic review.
Appendix A Supplementary data
Supplementary material related to this article can be found, in the online version, at doi: https://doi.org/10.1016/j.eurpsy.2019.07.003.
Comments
No Comments have been published for this article.