Designing superior evaluations of complex interventions

Randomised trials of health interventions appear to yield mostly negative results (<70%), and this trend worsens for mental health interventions (80%).Reference Crawford, Barnicot, Patterson and Gold¹ Explanations for these findings include poorer development of interventions usually on insufficient funds, reliance on small and poorly designed exploratory trials and pilot studies that overestimate effects that sanction progression to phase III trials that then fail. Another reason for negative trial results is most interventions are complex, multicomponent and dependent on local health, social and societal systems for implementation; measuring and taking account of the impact of these contexts is not easily achieved.Reference Bonell, Fletcher, Morton, Lorenc and Moore² If context-dependent intervention components are scaled up, it may be impossible to replicate the original critical contexts – from pilot studies – in different parts of the world or in different areas in any one country with contrasting urban and social milieu. Furthermore, for public health preventive campaigns, trials may not alone be sufficient to estimate the optimal duration and dosage of interventions.Reference Colditz and Taylor³ Trials to address equity also need tailoring and appropriate trial design.Reference Mbuagbaw, Aves, Shea, Jull, Welch and Taljaard⁴ The Medical Research Council guidelines for the development of complex interventions have revolutionised trial design,Reference Anderson⁵ but variation in exposures to local health and social systems and naturalistic assets such as health campaigns, community actions, parks and non-governmental organisations, if not fully considered, may differentially undermine the detection of any differences between the new intervention and comparison arms of a trial.Reference Porter, McConnell and Reid⁶ Specific examples of such context dependence include trials of arts interventions in the community, through arts and cultural industries. There are arts-specific evaluation frameworks;^7–Reference Fancourt and Aesop⁹ these are suited to arts interventions and may be very appropriate for complex mental health interventions. Another approach to evaluation is not to assume the discreteness of an intervention nor that is universally applicable, and move to explore processes and mechanisms of what works in the hope that these can be generalised and take account of contexts. Although trials and realist approaches may seem to emerge from different philosophical and theoretical positions, realist approaches to evaluation may help by formulating a programme theory and then testing and modifying it during the delivery of a trial, and also take account of the contexts in understanding how interventions work as well as whether they work (see Duncan et al. pp. 451–453).

Critical evaluation questions

There are many methodological and design questions that need a creative response and better exploration in mental health settings. Psychological therapy for psychosis among admitted patients appear to not be consistently studied or reported, with variable outcomes and models of delivery raising some doubt about effectiveness (Jacobsen et al pp. 490–497). Given that most individuals admitted to in-patient wards are severely ill and in crisis, and often detained under the powers of the Mental Health Act, undertaking research with in-patients is challenging. Methodologists, ethics committees and research teams raise questions about capacity to consent to participate in research. Spencer et al (pp. 484–489) elegantly show that capacity to consent to treatment does not map well onto capacity to consent to research, and that the two should not be conflated. These findings should spark a fresh commitment to improving research that benefits patients admitted and often confined to in-patient settings.

Attrition in research cohorts is a major concern; interventions to retain patients in services and in research are needed. First-episode psychosis cohorts offered new hope that we might better treat patient early, and reduce poor experiences or chronicity. Therefore, it is worrisome that in Solmi et al’s (pp. 477–483) study of retention, over half (54.3%) of the patients in a first-episode cohort were discharged before receiving the intended minimum of 3 years of care; 11.7% of participants were discharged because of disengagement. Disengagement was associated with negative symptoms, severe hallucinations, diagnostic uncertainty, polysubstance use and employment, suggesting that patients with these coexisting complicating factors will also need more targeted research and adapted treatments.

Therapeutics for comorbid depression

Depression is a common and disabling condition and one for which complex and effective interventions are critical to reduce disability and improve function and quality of life.Reference Bhui¹⁰ Studies of antidepressants for treating depression need clearer inclusion and exclusion criteria, consistent staging and phenotypes for comparisons across studies, and even then judgement and interpretation of findings can suggest conflicting evidence for practice. Parker (pp. 454–455) entertainingly exposes sensational reporting of studies that suggest antidepressants are placebos and should be prescribed for their powerful placebo properties, revealing why public and professional confidence in effective antidepressant therapy can be undermined. The treatment of depression in people with schizophrenia poses particular predicaments. Fond et al (pp. 464–470) show that more intensive and refined interventions are needed, especially for people with schizophrenia who experience paranoia and alcohol use. Although comorbidities may undermine effective care, reassuringly, Camacho et al (pp. 456–463) show collaborative care improves depressive symptoms in patients with diabetes and coronary heart disease. Depression is reported to be an early signal of later-life dementia, and the mechanisms may involve inflammation that reflects a shared aetiology or that depression itself raises the risk of dementia.Reference da Silva, Goncalves-Pereira, Xavier and Mukaetova-Ladinska^11, Reference Saczynski, Beiser, Seshadri, Auerbach, Wolf and Au¹² Despite concerns about depression comorbidity leading to poorer outcomes, depression is reported by Lewis et al (pp. 471–476) to not increase mortality of in-patients with dementia.