Evaluating treatment of Axis I mental health disorders in Aceh, Indonesia

if any, resulted from participation in the programme. Results The system is functional in 3 of 4 subdistricts, and 47 of 53 subdistrict clinics ( puskesmas ) have trained mental healthcare nurses. Both patients and caregivers reported statistically signiﬁcant differences when ranking patient well-being and were able to qualitatively describe speciﬁc changes in patient symptomatology and social functioning. Clinical implications Results indicate that (a) the creation of a decentralised system with outreach at multiple levels, (b) emphasis on staff capacity building within a wider household-to-hospital continuum of care, and (c) incorporation of community volunteers working with trained medical personnel led to effective treatment options for people with Axis I disorders in a resource-poor setting. Declaration of interest M.P. is employed by John Snow Inc.

A significant portion of the global burden of disease has been attributed to mental illness, which accounted for 13% of all the disability-adjusted life-years lost in 2003. Additionally, approximately 33% of all years lived with disability are attributed to neuropsychiatric conditions. [1][2][3][4][5] This burden of mental illness is expected to rise, and yet despite increasing documentation of its tremendous negative impact worldwide, treatment has largely been Aims and method To share evaluation findings of a new decentralised mental healthcare system addressing Axis I disorders, developed in four subdistricts of Aceh Besar in Indonesia following the 2004 Asian tsunami. Two complementary methodologies were employed: an adequacy survey that assessed whether agreed programme implementation tasks were completed, and an outcome study that utilised patient, caregiver and staff assessment of the programme to determine what changes, if any, resulted from participation in the programme.

Results
The system is functional in 3 of 4 subdistricts, and 47 of 53 subdistrict clinics (puskesmas) have trained mental healthcare nurses. Both patients and caregivers reported statistically significant differences when ranking patient wellbeing and were able to qualitatively describe specific changes in patient symptomatology and social functioning.
Clinical implications Results indicate that (a) the creation of a decentralised system with outreach at multiple levels, (b) emphasis on staff capacity building within a wider household-to-hospital continuum of care, and (c) incorporation of community volunteers working with trained medical personnel led to effective treatment options for people with Axis I disorders in a resource-poor setting.
Declaration of interest M.P. is employed by John Snow Inc.

Evaluating treatment of Axis I mental health disorders in Aceh, Indonesia
Neil Boothby, 1 Maggie Veatch, 1 Matina Pentes 2 neglected. 6 With resources scarce and mental healthcare costs high, there has been a push towards innovative solutions to alleviate the suffering of people with mental disorders and their caregivers. In spite of these efforts and information, the international community has been slow to respond. 7,8 The aftermath of the 2004 Asian tsunami provided an opportunity to raise the status of mental healthcare in Aceh, Indonesia. Advocates in Indonesia used resources allocated for mental health not only to alleviate people's anguish but also to develop a decentralised mental healthcare system providing a household-to-hospital continuum of care. Before the tsunami, Aceh's two psychiatrists and sole mental health treatment facility were located in the provincial capital of Banda Aceh, where the 220-bed treatment facility was at 150% capacity. Both mental health expertise and medications were absent at the district and subdistrict levels. The hospital used electroconvulsive therapy without anaesthesia to treat patients, and high levels of prescribed medication resulted in polypharmacy. 9 Medications were available only in Banda Aceh, and community support for individuals with Axis I disorders and their families was lacking.
In this article we describe the results of an evaluation of a new decentralised mental healthcare system in the district of Aceh Besar. The mental healthcare system addressed the clinical needs of people with Axis I disorders, defined within the DSM-IV-R multi-axial system as 'Clinical disorders [and] other conditions that may be a focus of clinical attention', and including all disorders and conditions except personality disorders and mental retardation. 10 (Major disorders listed under Axis I include schizophrenia, anxiety disorders, sleep disorders, substance-related disorders and mood disorders.) It also took into account stigma resulting in part from cultural beliefs that view mental illness as a supernatural occurrence rather than as a treatable medical condition. The decentralised system of mental healthcare is within Aceh's normative primary healthcare system, consisting of district, subdistrict and village levels. Where the system is functional, the evaluation team found significant improvement in the lives of people with Axis I disorders. Although programme improvements are needed, the evaluation found the system to be viable and replicable.

Method
This evaluation was part of a broader USAID evaluation of JSI's contracted work in Indonesia. Two complementary evaluation methodologies were employed: an adequacy survey and an outcome study.

Adequacy survey
The adequacy survey examined programme management and implementation as measured against the initial output indicators agreed with USAID. The survey examined key activities, training and services planned for the four districts in which JSI operated: Banda Aceh, Aceh Besar, Aceh Barat and Aceh Jaya.
At the most basic level, the adequacy survey sought to verify the extent to which the implementing partners met their input and output commitments. Beyond verifying that tasks were (or were not) completed, the adequacy survey also enabled a more realistic examination of the programme design. If most aspects of the programme were implemented, negative or positive outcomes were reflective of the programme design. Conversely, if key tasks were not implemented, evaluation of the programme's design would not have been feasible.

Outcome study
The outcome study focused on what changes, if any, resulted from people with Axis I disorders being served by the new decentralised mental healthcare system. It focused exclusively on Aceh Besar, the first district in which JSI's mental healthcare programme was implemented and where the programme was most developed. This gave the evaluation team the opportunity to look beyond the successes and challenges associated with programme start-up and identify how the programme was functioning following its inception.
A portion of the outcome study focused on health system development and sought to answer the key question: 'Is the mental healthcare system established by the programme at village, subdistrict and district levels effective, sustainable and replicable?' The study asked patients and their families (primary caregivers) whether the patient's mental health and social functioning had changed since their involvement in the new mental healthcare system (online Appendix DS1). Structured interviews with community mental health nurses (online Appendix DS2) and subdistrict level general practitioners explored these health professionals' perceptions of functionality at various levels of the mental healthcare system. Field observations of community mental health nurses working in puskesmas (subdistrict health clinics that are part of Aceh's normative healthcare system) and of volunteer village workers (mental health cadres) further elucidated mental healthcare provision at different levels of the system.

Establishing a baseline
Patient improvement following participation in an intervention is best measured when baseline data are available on patients' pre-programme status; such data were not available for patients in this evaluation. To capture the status of intended beneficiaries as accurately as possible, it was necessary to create a retrospective baseline.
Establishing such a baseline is most effectively done when clear timeframes are established and questions requiring recall are straightforward.
In Aceh Besar, a clear timeframe (before and after patients' first contact with any level of the decentralised mental healthcare system) was established, along with simple procedures for gauging perceptions of change between these time periods. Patients and their families (primary caregivers) were asked to describe the patient's mental health and social functioning status before and after entering the system. A 1-5 scale, representing no improvement to high improvement, was adopted following the recommendation of local staff, because of its familiarity throughout Indonesia, including Aceh Besar.
In addition to the retrospective baseline, one of the four subdistricts of Aceh Besar inadvertently served as a pre-tsunami baseline. Although it is often unethical to establish comparison groups in emergencies if it means intentionally withholding assistance to people in dire need, there may be naturally occurring opportunities for comparisons. 11 This was the case in the fourth subdistrict of Lhoong, where the programme did not fully materialise. Lhoong is ethnically and linguistically unique and traditionally has been beyond the reach of the Ministry of Health due to its isolation. Analysis of outcomes in this subdistrict was not possible owing to lack of validity in the data collected from the Lhoong puskesmas, which, upon fact-checking, was found to have fewer than half of the patients reported as receiving visits actually having received them.
National staff and community members suggested several reasons for the programme's failure to function, including a historically remote location further exacerbated by the road being washed away by the tsunami; frequent staff absence; high turnover at the puskesmas; and a saturation of non-governmental organisations and related externally funded activities following the tsunami, which may have undermined collective self-reliance and volunteerism. Open conflict between people with Axis I disorders and other community members was also reported. This subdistrict was therefore used by the evaluation team as a proxy for conditions existing before the implementation of the decentralised mental healthcare system.

Patients and caregivers
To determine whether people with mental health disorders and their families were well-served by the newly decentralised mental healthcare system, the study interviewed patients who were reported by the District Health Office to have received clinical services and home visits as of 31 March 2007. When possible, both the patients and their family members who served as primary caregivers were interviewed, to triangulate findings and better identify gaps and challenges. Patient selection was purposeful, based on Ministry of Health records. Participating patients were all taken from the Ministry of Health lists and may be representative not of the broader population of people with Axis I mental health disorders in Aceh Besar but only of those people within the health system. The prevalence of Axis I disorders in Aceh, Indonesia, is not known.
The 1-5 ranking exercise sought to document both the patient's and the caregiver's perceptions of the patient's mental health and social functioning status before and after entering the newly decentralised mental healthcare system. Patients were asked to rank the severity of their conditions before their first contact with any level of the decentralised system, with 1 being the worst problem that they had ever experienced, and 5 being no problem at all. The caregiver was asked to rank their perception of the patient's condition, using the same scale. A total of 36 households, with 29 patients and 30 caregivers, were interviewed. The ranking scale was accompanied by a qualitative question asking what changes, if any, the caregiver and patient had observed since receiving services (online Appendix DS1).

Structured interviews with community mental health nurses and mental health cadres
Questions for the community mental health nurses were designed to capture their insights into the new mental healthcare system and their roles within it. Nurses were asked about the quality and level of training they received, the strengths of the system, the challenges they faced in their new positions, and their working relationships with the general practitioner and village-level mental health cadres. A total of six nurses from four puskesmas were interviewed. Structured group interviews with 82 mental health cadres in 13 villages explored their roles, daily activities (including awareness-raising and support to families) and working relationships with the community mental health nurses (online Appendix DS3).

Key government and agency staff interviews
Clarification interviews took place with key staff members in government bodies, non-governmental organisations, academic institutions and donor agencies to seek their insights and follow-up on outstanding issues. These interviews included senior staff of the:

Data collection and analysis
Household-and community-level data were collected by a team of trained national staff using structured interviews in the local language. All questionnaires were piloted with representative samples and then adapted before being deployed. In one conflict-affected subdistrict, villagers were reluctant to speak to the interview team because they were viewed as outsiders. In this village, on four occasions, a community mental health nurse introduced the interviewers to the household and waited elsewhere in the community while the interviews were being conducted.
Data were entered into a Microsoft Access 2003 database. In addition, an expatriate member of the evaluation team debriefed field interviewers each day to ensure qualitative and observational information was recorded. Data were then imported and analysed using SPSS version 16.0 for Windows.

Decentralised system of care
As the adequacy and outcome studies are complementary, the results of these different methodologies are presented together. Figure 1 describes the levels of care and staffing of the new decentralised mental healthcare system in Aceh Besar.

District level
At the district level, JSI built and equipped a ten-bed acute care psychiatric unit located within Jantho District Hospital (Aceh Besar). It has interactive workstations, tranquil spaces for patients and secure areas, including contained rooms available as needed during care crises. The unit is designed to provide a safe environment for people experiencing high levels of distress in their mental illness and meets international standards. There are no bars and no dormitory sleeping; instead, patients have bathrooms connected to their rooms, and fixtures and fittings are selected and designed to minimise opportunities for harming oneself or others.
This unit opened on 2 April 2008. Staffing includes six nurses who have completed the advanced training programme and one psychiatrist, who will initially be supported by the Asian Development Bank. In addition to working at the acute care unit, these seven members of staff provide regular support to puskesmas staff. However, coordination is still lacking between the district hospital and puskesmas. For example, when patients are discharged from the hospital, they are provided with documentation pertaining to the nature of their stay and recommended medication. Neither this documentation nor a discharge notification is transferred to the puskesmas level, which would facilitate follow-up by the community mental health nurses.

Subdistrict level
The programme calls for two community mental health nurses per 30 000 residents and the four puskesmas in Aceh Besar are staffed accordingly. Community mental health nurses at the puskesmas attend a structured training programme of beginning, intermediate and advanced courses. The competency-based curriculum was reviewed by WHO and senior psychiatrists working in Indonesia; training methodologies include classroom learning, fieldwork and 2-6 months of supervised case management, depending on the level of training. 7 The role of community mental health nurses includes case management, community outreach, home visitations and referral procedures, and standards have been established against which they are expected to perform. Each puskesmas serves a wide geographical area, presenting a challenge to individual patient follow-up. The community mental health nurses report receiving referrals from village-based volunteers (mental health cadres), midwives and village leaders. They also report that home visits are considered 'extras' by their supervisors, with some community mental health nurses required to conduct home visits outside of their normal working hours; the community mental health nurses do so using their own transportation and funds to perform the visits. Consequently, community mental health nurses as a whole report working significantly less than the amount of time specified by the programme plan (Fig. 2).

Village level
At the village level, the programme called for the establishment of volunteer mental health cadres to collaborate with the community mental health nurses. The mental health cadres were trained to work with the nurses to identify villagers in need of mental healthcare and treatment, raise awareness and reduce stigma about mental illness, provide referrals to mental health services, and conduct home visitation and outreach to families that require extra support for family members with mental disorders. Several non-governmental organisations provided additional support to mental health cadres for activities designed to promote psychosocial well-being, primarily through women's and children's groups.
The village level of the programme was found to be functioning in three of the four subdistricts, with 70% of village mental health cadres in these areas engaged in routine home visits and awareness-raising, according to patient, caregiver and community mental health nurse reports. In Lhoong, the subdistrict where the JSI programme was not functioning, the community mental health nurse spoke of difficulties with cadre availability. This corresponded with patients' reports of a lack of contact with mental health cadres.

Work of the community mental health nurses and mental health cadres
Both community mental health nurses and mental health cadres are an integral part of the decentralised system's ability to address community concerns about mental illness and open discussions around commonly held beliefs. Community mental health nurses and mental health cadres introduced or increased awareness of the scientific model of understanding disease and treatment, including medication, while remaining sensitive to other explanations. Patients also likely benefited from community mental health nurses and mental health cadres working together to mutually reinforce programme messages and services. The mental health cadres identified are trusted community members who serve as a bridge between the subdistrict and village levels of care. They not only help raise awareness but also allow community mental health nurses access to communities that might otherwise consider the nurses outsiders. In turn, the community mental health nurses add credibility to the work of the cadres.
The community outreach and home visitation aspects of the programme are vital in achieving improved mental health outcomes for patients. The importance of outreach beyond the acute care level was reinforced with findings from the non-functioning subdistrict of Lhoong. There, medication was available at the puskesmas, as in other subdistricts, but patients did not receive mental health interventions at this level because of a lack of coordination between the subdistrict and village levels. Failure to challenge stigma and discuss beliefs related to mental health also contributed to the lack of positive outcomes for patients in this subdistrict.
Overall, community mental health nurses need more support to continue to be effective. Remediable issues such as lack of transportation undermine the community outreach and home visitation duties carried out by community mental health nurses, especially in more distant and remote villages. The aforementioned effectiveness of the outreach and visitation system is thus currently based on individuals' extraordinary efforts rather than the system's success. Motorcycles and fuel allowances are needed to ensure key community mental health nurse functions are possible, as is the education of general practitioners regarding the importance of including home visits as part of community mental health nurses' regular activities.

Analysis of patient and caregiver reports
In three of four subdistricts of Aceh Besar, the decentralised mental healthcare system is achieving positive results. Patients and caregivers in 34 of 36 households surveyed in these districts reported improvements in patients' mental health following the commencement of the decentralised mental healthcare system. Using a paired t-test, patients' Differences between patients' and caregivers' reports of patient improvement were not statistically significant (Figs 3 and 4). Where there were differences, patients almost always reported side-effects from the medication, whereas caregivers reported improved social functioning without mention of the medication's side-effects. Patients in the majority (29 of 36) of all households surveyed reported experiencing mental health problems before the tsunami, indicating that the tsunami was neither a trigger nor a cause of their mental disorder. None of the patients or caregivers in the three functioning subdistricts reported a worsening of mental health or social functioning. One patient, who temporarily stopped seeing the community mental health nurse after a change of residence, reported significant improvement while receiving home visits, but returned to a 1, or 'worst problem ever', rank after losing contact with the nurse. He was recorded as having experienced 'no improvement' and was reconnected with the nurse after the evaluation team's visit.
Qualitative feedback received from both community mental health nurses and patients shed further light on programme-related changes that they experienced. One nurse described the following interaction: 'About five months ago, this particular patient was hallucinating a lot. He walked around the village naked and would eat faeces. After regular visits and medication, the patient improved significantly. He started dressing properly, taking a bath and doing other things like that. I think that the reason for this was that the patient was able to take his medications regularly and had full support from his family during his recovery. I also think that it helped that we told him how to deal with his hallucinations and reintegrate into the community.' Another community mental health nurse described a patient who was repeatedly admitted and discharged from the provincial hospital, and claimed that this patient's health status improved only following the commencement of nurse home visits. Several community mental health nurses described recognising the symptoms of mental illness in ostracised community members during their training and returning to their communities to seek out and treat these patients following the training.
Patients and caregivers largely corroborated the community mental health nurses' reports when asked an open-ended question about what changes they had experienced since receiving care from various levels of the new programme. Changes that the patients described ranged from simply being less disruptive, to significant and positive alterations in the quality of life for both patients and their caregivers (Fig. 3). The most frequently reported improved symptoms were individual functioning (regular bathing, being able to be left alone, a willingness to leave the house); decreased aggression (assaulting people, breaking plates); family functioning (communicating more, participating in household activities, contributing to the household income); and mental state (decrease in hallucinations, no longer talking to oneself, sleeping better). Of the 36 patient households surveyed, 6 reported that the programme enabled patients who had never been employed to gain employment after their treatment and support plan came into effect.
In open-ended questions regarding symptomatology, differences existed between patients who described themselves as 'high improvers' (those who improved by three or four points on the 1-5 scale) and 'low improvers' (those who improved by zero, one or two points on the 1-5 scale). A w 2 analysis was used to show that patients whose score differences put them in the range of 'low improvers' were significantly less likely to report an improvement in their mental state, with 19 of 25 'low improvers' reporting no change in mental state, compared with 5 of 12 'high improvers' (P50.041).
Further, caregivers whose scores placed patients into the 'high improver' range were more likely to describe an improvement in family function than caregivers whose scores led to patients being categorised as 'low improvers', with 9 of 11 caregivers who had ranked their patients as 'high improvers' describing an improvement in family functioning compared with 7 of 26 caregivers of 'low improvers' (P50.002) (Fig. 4). Caregivers whose rankings identified 'high improver' patients were also more likely (8 of 11) to report an improvement in mental functioning than caregivers whose reports led to patients being identified as 'low improvers' (P50.04).
Patients who reported the most significant improvements (meaning a difference of four to five points in their pre-and post-rankings) often received more sophisticated medications at the Banda Aceh hospital. Currently, there is advocacy for a wider range of medications to be available at the puskesmas, which would further decentralise care and increase accessibility at subdistrict levels. One community mental health nurse suggested that securing a psychiatrist  to consult on medication a few times per year at the puskesmas would increase patient attendance and improve medication regimes.
Comparison with non-functioning subdistrict As noted, in Lhoong the components of the mental healthcare system were not functioning, allowing this subdistrict to serve as a naturally occurring baseline against which to measure the achievement of the other three programme sites. Residents of Lhoong reported experiencing delusions and voice commands, and patients' households reported frequent admissions to Banda Aceh Psychiatric Hospital and security situations requiring police intervention. These situations were not reported by any of the households in the three subdistricts in which the programme was functioning.
Major deficits identified in the existing mental healthcare programme in Lhoong subdistrict included the absence of active outreach or surveillance by mental healthcare staff, poor diagnosis and treatment by mental healthcare professionals, and medications and treatment accessible at the district rather than the subdistrict level (i.e. at Banda Aceh psychiatric hospital rather than the puskesmas). Further, there was a lack of support to families and caregivers; use of restraining devices, such as locking/ chaining of patients to homes or beds; and widespread stigma regarding mental health disorders, possibly related to superstitious beliefs regarding causation of mental disorders.

Discussion
This article documents measurable outcomes achieved in a relatively short period of time through the development of a decentralised mental healthcare system in Aceh Besar, Indonesia. All three levels of the programme -district, subdistrict and village care -were found to be critical to establishing a household-to-hospital continuum of care. Although programme achievements in patient outreach and the establishment of a multi-level system incorporating both healthcare providers and non-professionals are to be commended, continued decentralisation is necessary to further improve patient outcomes.

Limitations
The evaluation's findings are limited by the retrospective nature of the baseline, which was established at the time of the evaluation owing to a lack of pre-existing data. To account for the decreased ability to establish temporality, clear timelines for the respondent recall period were sought and a subdistrict where the programme had not materialised was used as a proxy for the conditions existing before the programme's inception. Results were further limited by the small sample size on which statistical analysis was applied, and the lack of a comparison group large enough to be statistically significant. At this point in the programme's implementation, it is not possible to evaluate programme impacts; rather, an evaluation focusing on programme outputs and outcomes was undertaken.

Programme improvements
Additional support for the puskesmas and village-level components of this continuum of care is required, including engagement of those within the Ministry of Health who have shown resistance towards decentralisation in mental healthcare provision and the involvement of village mental health cadres. Additional resources are needed to maintain the quality of advanced-level community mental health nurse training and the community mental health nurse staffing ratios necessary for the population served. Retention of the services of the district-level psychiatrist (beyond the initial 6 months) and continued decentralisation of medication provision are further priorities.
At the subdistrict and village levels, the symbiotic relationship between community mental health nurses and mental health cadres augments the benefits of their respective work. Yet the functioning of the decentralised system's three levels of care would benefit from stricter adherence to the two-way referral system. By actively maintaining a system to notify staff at the puskesmas when patients are discharged from the district-level hospital, continuity of care could be greatly improved. Better overall coordination between district and subdistrict level health services, 7 and the continued decentralisation of medications to district and subdistrict levels, are key next steps in improving mental health outcomes.

Replication
Mental health services in many parts of Indonesia have remained at a low level of organisation and development. The Aceh Besar programme is a model for what is possible in other provinces throughout the country. The decentralised mental healthcare system's infrastructure, personnel, standards and materials have been developed, and there is documented evidence of programmatic achievements in improving patient outcomes. The establishment of an acute care in-patient facility at the district level, with a design that respects the dignity of patients both individually and culturally, is an innovation for Indonesia as a nation and represents a major shift in paradigm from custodial care to a human rights-based model. What is needed is the will to ensure such services are maintained and replicated at all levels.

Advocacy and funding
The decentralised mental healthcare system is an example of the opportunity to save lives, to assist and protect vulnerable groups, and to strengthen mental health systems after an emergency. The tsunami's aftermath presented both an opportunity and a space -in terms of funds and interest among different emergency responders (e.g. national and international agencies, donors) -to make systematic changes to healthcare provision, exceeding what was available before the crisis. As the international mental health community becomes aware of Aceh's example, it is vital that key health and political leaders within Indonesia are also aware of what is taking place. 12,13 Following the end of the emergency phase in the tsunami response, funding for this programme and others like it has waned.
Despite the ongoing availability and interest of the University of Indonesia's Faculty of Nursing to continue providing technical assistance, as well as a positive evaluation of the programme's work conducted by WHO, the Asian Development Bank and USAID, none of the decentralised mental healthcare system's initial donors plan to dedicate development-oriented aid to the programme's continuation. Additionally, mental health is not a priority within the Indonesian Ministry of Health, exacerbating a situation in which this newly established system of mental healthcare is unlikely to receive adequate financial support without the launch of a strong national advocacy campaign.
Although it is too early to make definitive statements regarding the programme's impact, senior policy makers and Ministry of Health officials must be made aware of the documented benefits provided by developing a multilevel decentralised mental healthcare system in Aceh Besar, as well as opportunities for the programme's replication in other parts of Indonesia. For the system to continue, concerned international agencies such as USAID, WHO and the Asian Development Bank must also continue to support the programme and coordinate and prioritise awarenessraising activities. In addition, advocacy should include a focus on establishing national legislation on mental healthcare that codifies support for the household-tohospital continuum of care in addition to a psychiatric hospital component.

Conclusion
Some progress has been made towards the development of a household-to-hospital continuum of mental healthcare in Aceh Besar. Where the system is functioning, it establishes three clear levels of mental health and psychosocial care (district, subdistrict and village), which effectively decentralise the provision of mental healthcare services and contribute to community awareness of mental health disorders. This programme serves as an example of how primary healthcare providers and non-specialists can work together to provide mental healthcare and treatment to people with Axis I disorders.