Psychotherapy is effective and efficient for treating patients with a variety of mental disorders. Reference Shadish, Matt, Navarro and Phillips1–Reference Lambert, Ogles and Lambert3 If psychotherapeutic treatment (alone or in combination with psychopharmacotherapy) is indicated, the primary task of the psychiatrist or psychotherapist at the initiation of treatment is to offer the appropriate treatment option and to decide which therapeutic approach would be the most beneficial. Reference Tillett4 This prognostic assessment of the expected outcome of particular treatment methods is part of the daily clinical routine and is an important issue of mental healthcare. The question of ‘what works for whom?’ and of an adequate and evidence-based treatment selection was identified early as a vital issue of psychotherapy research. Reference Paul5 It remains of continuous interest for both practitioners and the scientific community. Reference Roth and Fonagy6,Reference Beutler and Harwood7 Treatment selection is salient within healthcare policy, where the current controversial (and sometimes ideologically guided) discussion focuses mainly on diagnostic groups/mental health problems as the only criterion for treatment selection. Reference Chambless and Ollendick8
For decisions on treatment selection, cognitive–behavioural therapy (CBT) and psychodynamic therapy (PDT) take a key position owing to their widespread application. Reference Barghaan, Schulz, Koch and Watzke9 There are numerous studies demonstrating the efficacy and effectiveness of both CBT (e.g. Emmelkamp Reference Emmelkamp and Lambert10 and Hollon & Beck Reference Hollon, Beck and Lambert11 ) and, to a lesser extent, PDT. Reference Leichsenring, Rabung and Leibing12 Psychodynamic therapy and CBT are clearly distinct from one another, not only with regard to their basic concepts, training modalities and treatment rationales, but also concerning their actual therapeutic implementation (i.e. therapeutic interventions, attitude, style and content). Reference Goldfried and Castonguay13,Reference Watzke, Rueddel, Koch, Rudolph and Schulz14 Despite all efforts to develop integrative and adaptive treatment models integrating or combining both methods (e.g. Keller et al Reference Keller, McCullough, Klein, Arnow, Rush and Nemeroff15–Reference Grawe17 ) these are rarely implemented in routine mental healthcare. Reference Barghaan, Schulz, Koch and Watzke9
Despite the wide application of both CBT and PDT and their distinct procedural differences, to date no empirical research has addressed the question of whether treatment outcomes can be optimised by employing the method of systematic treatment selection Reference Beutler and Harwood7,Reference Harwood, Williams, Beutler and Groth-Marnat18 to either treatment method by increasing the match between the individual patient and the applied treatment. Some approaches try to improve this match by adapting the applied treatment to the patient's characteristics within one therapeutic concept. However, at least for CBT, results of a randomised controlled trial (RCT) show that individualisation of treatment allowing therapists to use interventions based on their own inclinations did not produce superior effects compared with standardised interventions. Reference Schulte and Kunzel19
We present results from an RCT under clinically representative conditions comparing systematic treatment selection and random treatment selection for both CBT and PDT in terms of long-term treatment response. Besides the general effectiveness of systematic treatment selection, we also investigated its differential effectiveness for CBT and PDT (i.e. which therapeutic approach benefited the most from systematic treatment selection). We hypothesised that systematic treatment selection is generally superior to random treatment selection because of the expected optimisation of fit between the individual and the selected treatment approach. Concerning differential effectiveness, we hypothesised that PDT benefits more from systematic treatment selection than CBT, as patients receiving PDT are considered to require more abilities (e.g. sufficient degree of psychological mindedness, good quality of object relations Reference Valbak20 ) than those receiving CBT.
We conducted the RCT under clinically representative conditions in order to provide for generalisability in routine practice. We chose an in-patient setting to ensure better monitoring of assignment and treatment.
Method
Between October 2002 and May 2003, we studied a consecutive sample of in-patients recruited from a psychotherapeutic in-patient unit in Germany. All patients fulfilled the criteria for at least one mental disorder according to ICD–10 (Chapter V). 21 Patients without a complete diagnostic procedure (i.e. with less than 1 week of in-patient stay) and patients admitted a second time to this specific clinic for treatment continuation (i.e. their treatment was already established prior to admission) were excluded from the study. Additionally, patients meeting specifically defined criteria (extreme under- or overweight with a body mass index <12 or >50 kg/m2, decompensated tinnitus with the patient's focus being symptomatic treatment, and psychosis-like states) were also excluded from the study, because they would always need to receive systematic treatment selection (‘compelled STS’) according to the clinic's standards and therefore could not undergo randomisation for clinical and ethical reasons. Because of the naturalistic context of the study, no further exclusion criteria were set.
The unit in which the study took place is part of routine mental healthcare in Germany, which yields larger in-patient psychotherapeutic capacities than healthcare systems of most European countries. There are about 300 000–400 000 in-patient treatments per year in Germany compared with approximately 280 000 out-patient treatments. Reference Schulz, Barghaan, Harfst and Koch22 In contrast to other countries, in-patient psychotherapeutic treatment is very common and an important part of routine care in Germany. The in-patient system focused on in this study serves individuals with a broad spectrum of F-diagnoses based on ICD–10 (main diagnostic groups: depressive, adjustment and anxiety disorders). Hence, the in-patient system in Germany is not directly comparable to in-patient treatment in other countries. However, with regard to the clinical characteristics of the patients (diagnoses, symptom severity), the treatment dose, and the treatment methods delivered, it is rather comparable to the out-patient sector in other countries.
Study design
The study was approved by the human ethics committee of the responsible medical association. After a complete description of the study to the patients, written informed consent was obtained. Sociodemographic and clinical characteristics were assessed using clinical interviews and self-ratings as well as psychometric testing. This diagnostic procedure was carried out by experienced clinicians. Participants were randomly assigned to either the experimental group (systematic treatment selection, STS) or the control group (random treatment selection, RTS) in a 1:2 ratio. In the experimental group, patients were assigned to either CBT or PDT in a 3:2 ratio using systematic treatment selection (see below). Patients in the control group were randomly assigned to either CBT or PDT in a 3:2 ratio using random treatment selection. The ratio was determined by the capacities of the PDT and CBT treatment facilities in the unit.
We implemented a single-blind randomised trial with two levels of randomisation. The randomisation to systematic and random treatment selection (first randomisation) and the randomisation to CBT or PDT for patients in the control group (second randomisation) were conducted at the University Medical Centre Hamburg-Eppendorf using an external computerised randomisation schedule. According to this procedure, first the study group (STS v. RTS) was set by randomisation. Then the treatment method (CBT or PDT) was determined for each patient. In the experimental group (STS), the treatment method was selected according to the decision of the therapeutic team. In the control group (RTS), treatment was selected according to the second randomisation. This information on treatment method (but not the information on belonging to the experimental or the control group) was given to the person in the clinic entering data in the patient file. The patient file included no information on the selection process itself and the assigning therapeutic team was not involved in the further treatment of a patient. Thus, neither therapists nor patients knew which patients belonged to the experimental group and which patients belonged to the control group.
In order to assess the long-term outcome, patients filled in self-rated questionnaires at the beginning of treatment and at follow-up 6 months after termination of treatment. (An additional assessment took place at the termination of treatment; analyses regarding this assessment are not presented here.) The treatment success for systematically and randomly assigned patients at follow-up was compared across both therapeutic methods (general effectiveness of systematic treatment selection) and differentially for CBT and PDT (differential effectiveness).
Systematic treatment selection
Evidence-based medicine refers to the integration of the best available evidence from systematic research with the individual clinical expertise of the therapist and the preferences of the patient in order to make a decision in medical care. Reference Sackett and Rosenberg23 Applying these principles of evidence-based medicine, systematic treatment selection for either CBT or PDT comprised an expert consensus-based procedure owing to the lack of systematic empirical knowledge. Reference Beutler and Harwood7 Systematic treatment selection was determined during the admission phase within a multistage diagnostic procedure integrating clinical interviews and psychometric tests. It was carried out by a multiprofessional team (psychiatrists and psychologists from both treatment approaches) and was based on a consensus decision. The implemented systematic selection was the result of a long-term development within the unit (since the 1990s). The main criteria for STS related to the patient's therapeutic goals (for 86% of all decisions) and their diagnoses (74%), with 68% of all decisions being based on the combination of both criteria. In accordance with, for example, Ambuehl et al Reference Ambühl and Orlinsky24 those therapeutic goals focusing on ‘the integration of unconscious aspects of experience’ and ‘the ability to fully allow the experience of emotions’ supported a decision for PDT, whereas goals emphasising ‘learning to cope with problematic situations effectively’ and ‘modifying or changing problematic behaviour’ facilitated a decision for CBT. In combination with this, patients with an anxiety disorder, eating disorder or a post-traumatic stress disorder, as well as patients lacking a sufficient level of cognitive reflection, received more CBT. Figure 1 displays all criteria and their relevance for systematic treatment selection.
Treatment and treatment integrity
Both PDT (focal approach) Reference Heigl-Evers and Ott25 and CBT Reference Fiedler and Tschuschke26 comprised brief group therapy held in separate departments of the clinic, and were of similar dose and duration (three to four sessions per week with average treatment duration of 6 weeks). An additional individual short session per week served as an option for the patient to discuss more general therapeutic issues regarding the management of therapy. Patients in both treatment groups (i.e. CBT and PDT) received these sessions.
Criteria for systematic treatment selection (STS) to cognitive–behavioural therapy (CBT) or psychodynamic therapy (PDT) and their relevance for STS (percentage of decisions for which the criteria were referred to).
Because treatment was carried out in routine care, the treatments were not manualised. However, the degree of sufficient treatment differentiation (i.e. adherence to treatment integrity) Reference Waltz, Addis, Koerner and Jacobsen27 between the two therapies was assessed at the unit using video-based expert ratings as well as patient and therapist ratings (patient and therapist checklists were filled out after each session). This assessment of treatment integrity was implemented some months before this trial of STS (i.e. not the same sample of patients but the same therapists as in the present study were investigated). Results confirmed distinct profiles for PDT and CBT according to their therapeutic models: CBT therapists used more cognitive, behavioural and psychoeducational strategies, fostered self-efficacy to a larger extent and were more supporting and empathic. On the contrary, PDT therapists used more interpretive and confrontational interventions concerning intra-psychic issues and focused on interactional aspects as well as on transference and on the past more intensively. Reference Watzke, Rueddel, Koch, Rudolph and Schulz14
In addition to psychotherapy, about a quarter of the patients took psychotropic medication on a daily basis.
Therapists
All therapists in the PDT and CBT departments had completed or were of advanced standing in a multiyear postgraduate professional training programme within their specific therapeutic orientation. Psychotherapeutic training in Germany consists of a combination of theoretical education and practical work (600 h of theory; 1800 h of practical work under supervision). After their university degree in psychology or medicine, therapists are trained in either psychodynamic (focal analytic or psychoanalytic) or cognitive–behavioural treatment at specialised, licensed institutions. They then finish their training with an ‘Approbation’ (licence to practise psychotherapy) for either PDT or CBT. In the unit, all therapists receive supervision from a psychotherapist of the respective therapeutic orientation at least every 2 weeks. Every therapist delivered only one form of treatment.
Outcome measures
The primary outcome was symptom severity at 6-month follow-up (T 1) adjusted for symptom severity at baseline (T 0). Symptom severity was assessed using the General Severity Index (GSI) of the Symptom Checklist–14 (SCL–14), a short version of the Symptom Checklist–90–Revised (SCL–90–R), Reference Derogatis28 which comprises nine subscales assessing the current status of symptoms and a summary scale of overall symptom severity (GSI). It is an internationally well-established instrument with proven psychometric properties. Reference Derogatis28 Its internal consistency (Cronbach's alpha) coefficients range from 0.77 to 0.90 and test–retest coefficients from 0.78 to 0.90. Several studies have demonstrated its concurrent and construct validity as a global measure of symptom severity. The SCL–14 was empirically constructed and refers to three major subscales (depression, anxiety and somatisation) using the original item wording and response format of the SCL–90–R. The GSI of the SCL–14 correlates to r = 0.96 with the GSI of the SCL–90–R. Reference Harfst, Koch, Kurtz von Aschoff, Nutzinger, Rüddel and Schulz29 Scores vary from 0 (‘not at all’) to 4 (‘extremely’), with higher scores reflecting higher symptom severity.
The secondary outcome was the mental component summary of the Short Form–8 (SF–8, a short version of the SF–36) Reference Ware30 assessing health-related quality of life (HRQoL), namely, functional health. Health-related quality of life refers to the degree to which medical problems and disorders, as well as their treatment, impair somatic, emotional and social well-being. This is a superior concept incorporating a variety of dimensions, for example somatic problems, emotional problems, functional impairment, impairments in activity and participation with others. The SF–8 comprises a psychometrically based physical and mental health component summary (PCS and MCS respectively) measure. After standardised norm-based scoring (including recoding and weighting), the resulting summary scores vary from 0 (‘low HRQoL’/‘high impairment in HRQoL’) to 100 (‘high HRQoL’/‘low impairment in HRQoL’), with all scores above and below 50 being above and below the average respectively. In our analyses, we linearly transformed the original scale into a scale ranging from 0 (‘no impairment in HRQoL’) to 4 (‘high impairment’), with a score of 2 indicating the average. The instrument is widely used internationally and shows good psychometric properties, with internal consistency from 0.70 to 0.88 and a convergent and discriminant validity shown in several studies. Reference Ware30
Statistical analysis
For the baseline comparison of the experimental (STS) and control (RTS) groups as well as for the comparison of the PDT and CBT patients (in both groups), we analysed demographic and clinical characteristics at T 0 using t-tests for continuous variables and χ2-tests for categorical variables.
To test our hypotheses, both the experimental and control groups were compared using analyses of covariance (ANCOVAs) of T 1 for the factors assignment (STS–RTS) and treatment method (PDT–CBT) with T 0 as covariate in order to control for baseline differences. The general effectiveness of systematic selection was assessed with the statistical main effect of the factor assignment. The differential effectiveness of systematic selection for CBT and PDT was assessed with the statistical interaction of assignment × treatment method.
To examine the degree of difference between systematic and random treatment selection, we calculated effect sizes Reference Cohen31 using partial η 2 for ANCOVAs. Owing to the lack of existing data for the classification of effect sizes, we used Cohen's Reference Cohen31 interpretation for small (partial η 2 >0.0099), medium (partial η 2 >0.0588) and large (partial η 2 >0.1379) effect sizes. Partial η 2 can be transformed into other effect sizes, for example into Cohen's d (partial η 2 = 0.0099 corresponds to d = 0.20; partial η 2 = 0.0588 corresponds to d = 0.50; partial η 2 = 0.1379 corresponds to d = 0.80).
Our primary analyses refer to all patients with complete data-sets, i.e. with available assessments at T 0 and T 1 (completer analyses). Additionally, we conducted intention-to-treat (ITT) analyses as sensitivity analyses. For ITT analyses, missing values were imputed by carrying forward the baseline value (assuming that individuals who dropped out did not respond to the treatment at all). Therefore we could include all patients with a measurement at baseline in these analyses.
We also conducted drop-out analyses comparing the demographic and clinical baseline characteristics of completers (i.e. primary analyses sample) with non-completers (t- and χ2-tests).
In order to estimate the possible risk of a selection bias with regard to the recruitment process, analyses of representativeness were additionally carried out on the demographic and clinical baseline characteristics of completers and all other patients being treated in the clinic during the study period and not meeting the exclusion criteria (i.e. patients refusing to participate in the study and non-completers). For the comparison of these two groups we used t- and χ2-tests.
Sample size
The sample size was determined to provide the study with a statistical power of 80% for the detection of a small effect between random and systematic treatment selection using ANCOVA, with treatment method as the additional factor and baseline severity as the covariate. Since this is the first study investigating this research question, it seemed appropriate to apply a small effect size. We defined a small effect as η2 = 0.02 (corresponding to f = 0.14 according to Cohen). We assumed that the error variance decreases to an extent of 30% owing to inclusion of baseline severity as covariate (while being non-correlated to condition assignment owing to randomisation and being moderately correlated to treatment method owing to systematic selection in one of the groups), which led to an adjusted effect size of η2 = 0.03 (f = 0.17). Setting the level of significance at α = 0.05, a total sample size of 261 patients was needed. However, because of higher drop-out rates than expected, only 226 patients could be included in the primary analysis (see Results). Thus, the actual power of our analyses was about 74%. In the ITT analyses, the higher number of patients is expected to be compensated by a potential dilution of treatment effects, so the statistical power should be about the same.
Results
Participants
Patient flow through the study is shown in the CONSORT diagram in Fig. 2. Reference Begg, Cho, Eastwood, Horton, Moher and Olkin32 Of the 614 patients admitted to the clinic during the study period, 44 were excluded because they were admitted for treatment continuation (n = 20) or stayed less than 1 week (n = 24). Additionally, n = 65 patients with ‘compelled STS’ were excluded from the study because random allocation was not possible for clinical and ethical reasons (as described in the Method). Of the remaining 505 patients, 291 patients (58%) agreed to participate in the study. The main reasons for study refusal were that patients did not want to be randomised, or were not willing to fill in forms at baseline and at follow-up. A further common reason was that they did not want to take part in research projects in general. These reasons have to be put into perspective – the RCT took place in a clinically representative setting where the conducting of research and the implementation of trials is not very common.
A high response rate at 6-month follow-up (77.7%) was reached by contacting the patients up to four times (one reminder letter and up to three reminders by telephone). Drop-out rates did not vary substantially between the four study conditions (RTS–PDT: 76.3% completers; RTS–CBT: 78.8%; STS–PDT: 79.6%; STS–CBT: 75.9%; χ2 = 0.36, P = 0.95). We included 226 patients with complete data-sets in the completer analyses referring to the primary outcome, i.e. the GSI (for the MCS: n = 233).
Baseline clinical and demographic characteristics of the experimental (STS) and control (RTS) groups for the completer and ITT analyses are displayed in Table 1. Despite randomisation, the statistical comparison between the two groups showed a small but systematic difference in GSI and MCS scores, with slightly higher values for patients in the STS groups at baseline (completer analyses: GSI t(224) = 2.0, P = 0.045 and MCS t(220) = 1.9, P = 0.061; ITT: GSI t(275) = 2.2, P = 0.028 and MCS t(265) = 2.5; P = 0.014). For all other variables there were no systematic differences between study conditions.
Baseline characteristics of the study populationa
| Completer analyses | Intention-to-treat analyses | |||||
|---|---|---|---|---|---|---|
| Characteristics | STS (n = 79) | RTS (n = 147) | Total (n = 226) | STS (n = 97) | RTS (n = 180) | Total (n = 277) |
| Female, % | 73.4 | 74.8 | 74.3 | 72.2 | 73.3 | 72.9 |
| Age, years: mean (s.d.) | 44.5 (9.3) | 43.4 (10.8) | 43.8 (10.3) | 43.4 (10.2) | 42.8 (110.9) | 43.0 (10.7) |
| Professional degree, % | ||||||
| University degree | 7.1 | 8.8 | 8.2 | 5.7 | 8.5 | 7.5 |
| Non-university degree | 71.4 | 74.5 | 73.4 | 70.5 | 71.5 | 71.1 |
| Without professional degree | 10.0 | 8.0 | 8.7 | 11.3 | 9.7 | 10.3 |
| Other | 11.4 | 8.8 | 9.7 | 12.5 | 10.3 | 11.1 |
| Work incapacity before treatment, months: % | ||||||
| <3 | 45.9 | 49.1 | 48.0 | 45.5 | 49.3 | 47.9 |
| 3–6 | 18.0 | 16.7 | 17.1 | 16.9 | 16.4 | 16.6 |
| >6 | 19.7 | 15.8 | 17.1 | 19.5 | 14.2 | 16.1 |
| Not employed | 16.4 | 18.4 | 17.7 | 18.2 | 20.1 | 19.4 |
| Number of diagnoses (ICD–10), mean (s.d.) | 3.4 (1.4) | 3.4 (1.4) | 3.4 (1.4) | 3.4 (1.4) | 3.4 (1.4) | 3.4 (1.4) |
| Duration of treatment, days: mean (s.d.) | 42.7 (10.3) | 42.2 (11.5) | 42.4 (11.1) | 41.6 (12.0) | 41.4 (13.3) | 41.5 (12.8) |
| Diagnoses (ICD–10),b % | ||||||
| Depression (F32-34) | 40.8 | 42.2 | 41.7 | 42.6 | 41.7 | 42.0 |
| Anxiety disorder (F40, F41) | 17.1 | 12.2 | 13.9 | 16.0 | 11.1 | 12.7 |
| Adjustment disorder (F43.2) | 19.7 | 25.2 | 23.3 | 18.1 | 24.4 | 22.3 |
| PTSD (43.1) | 2.6 | 3.4 | 3.1 | 2.1 | 3.3 | 2.9 |
| Somatoform disorder (F45) | 6.6 | 4.8 | 5.4 | 7.4 | 6.1 | 6.6 |
| Eating disorder (F50) | 9.2 | 4.8 | 6.3 | 8.5 | 5.6 | 6.5 |
| Personality disorder (F60-61) | 0.0 | 0.7 | 0.4 | 2.1 | 1.1 | 1.5 |
| Other mental disorder | 2.6 | 5.4 | 4.5 | 2.1 | 5.6 | 4.4 |
| Somatic diagnosis | 1.3 | 1.4 | 1.3 | 1.1 | 1.1 | 1.1 |
| SF–8 MCS score, mean (s.d.) | 2.52 (0.77) | 2.30 (0.84) | 2.37 (0.82) | 2.56 (0.77) | 2.30 (0.84) | 2.39 (0.82) |
| SF–8 PCS score, mean (s.d.) | 2.18 (0.92) | 2.22 (0.89) | 2.20 (0.90) | 2.20 (0.89) | 2.22 (0.87) | 2.21 (0.88) |
| GSI of the SCL–14 score, mean (s.d.) | 1.39 (0.84) | 1.17 (0.76) | 1.25 (0.79) | 1.40 (0.80) | 1.18 (0.77) | 1.26 (0.79) |
Baseline clinical and demographic characteristics of CBT and PDT patients in both groups are displayed in Table 2. As expected, PDT and CBT patients did not differ significantly in the control group at T 0. In the experimental group, systematic differences between PDT and CBT were expected as such differences are supposed to be an inherent part of the intervention under investigation (i.e. the systematic treatment selection) owing to the association of certain baseline characteristics with certain factors guiding the treatment selection. In fact, PDT and CBT patients showed clinically relevant differences: PDT patients had a higher professional degree (with 20% more academics receiving PDT than CBT) and less impairment in HRQoL concerning physical aspects (initial SF–8 PCS) than CBT patients allocated to systematic selection. Additionally, there were more women receiving PDT in the STS group (80% v. 68% receiving CBT).
Baseline characteristics of patients assigned to cognitive–behavioural therapy (CBT) and psychodynamic therapy (PDT) in the experimental and control groups (completer analyses)a
| Experimental group (STS) | Control group (RTS) | |||||
|---|---|---|---|---|---|---|
| Characteristics | CBT (n = 44) | PDT (n = 35) | t-test/χ2 | CBT (n = 89) | PDT (n = 58) | t-test/χ2 |
| Age, years: mean (s.d.) | 43.8 (8.6) | 45.3 (10.1) | t(77) = 0.72, P = 0.47 | 42.0 (10.9) | 45.5 (10.2) | t(144) = 1.92, P = 0.056 |
| Female, % | 68.2 | 80.0 | χ2(1, n = 79) = 1.39, P = 0.23 | 74.2 | 75.9 | χ2(1, n = 147) = 0.54, P = 0.81 |
| Professional degree, % | ||||||
| University degree | 2.5 | 13.3 | χ2(3, n = 101) = 7.73, P = 0.052 | 10.0 | 7.0 | χ2(3, n = 137) = 4.37, P = 0.22 |
| Non-university degree | 80.0 | 60.0 | 68.8 | 82.5 | ||
| Without professional degree | 12.5 | 6.7 | 8.8 | 7.0 | ||
| Other | 5.0 | 20.0 | 12.5 | 3.5 | ||
| Work incapacity before treatment, months:b % | ||||||
| <3 | 39.4 | 53.6 | χ2(3, n = 61) = 1.29, P = 0.73 | 49.3 | 48.9 | χ2(3, n = 114) = 2.77, P = 0.42 |
| 3–6 | 21.2 | 14.3 | 18.8 | 13.3 | ||
| >6 | 21.2 | 17.9 | 11.6 | 22.2 | ||
| Not employed | 18.2 | 14.3 | 20.3 | 15.6 | ||
| Number of diagnoses (ICD-10), mean (s.d.) | 3.5 (1.4) | 3.1 (1.3) | t(74) = –1.34, P = 0.18 | 3.4 (1.4) | 3.4 (1.2) | t(134) = 0.22, P = 0.82 |
| Duration of treatment, days: mean (s.d.) | 44.5 (10.7) | 40.2 (9.3) | t(77) = –1.84, P = 0.06 | 43.2 (12.7) | 40.5 (9.1) | t(143) = – 1.50, P = 0.13 |
| SF–8 MCS score, mean (s.d.) | 2.46 (0.78) | 2.59 (0.75) | t(75) = 0.73, P = 0.46 | 2.32 (0.87) | 2.25 (0.79) | t(143) = – 0.45, P = 0.65 |
| SF–8 PCS score, mean (s.d.) | 2.32 (0.91) | 2.00 (0.90) | t(76) = –1.54, P = 0.12 | 2.17 (0.93) | 2.28 (0.81) | t(143) = 0.70, P = 0.48 |
| SCL–14 GSI score, mean (s.d.) | 1.40 (0.91) | 1.37 (0.75) | t(77) = –0.15, P = 0.87 | 1.17 (0.80) | 1.17 (0.69) | t(145) = 0.23, P = 0.98 |
CONSORT flow of patients through the trial.
RTS, random treatment selection; STS, systematic treatment selection; PDT, psychodynamic therapy; CBT, cognitive–behavioural therapy. ITT, intention-to-treat analyses. Patients with ‘compelled STS’, patients with defined criteria who had to receive STS for clinical and ethical reasons (see Method). a. Referring to the analyses of the primary outcome (Global Severity Index of the 14-item Symptom Checklist).
Psychotropic medication did not differ between CBT and PDT patients in either group (either at T 0 or at T 1).
The drop-out analyses (i.e. comparison of completers with non-completers) of baseline characteristics revealed a systematic difference regarding age (non-completers were on average 4 years younger; t(288) = 2.1, P = 0.039) and professional degree (non-completers more often had no professional degree or had not finished it yet; χ2 = 7.81, P = 0.05). Women were slightly overrepresented among completers (completers, 74% v. non-completers, 63%). For all other variables being examined (marital status, work incapacity, diagnoses, comorbidity, chronicity, initial scores of SCL–14 and SF–8), no systematic differences between the two groups were found.
The analyses of representativeness (i.e. comparison of completers and all other patients being treated in the clinic during the study period and not meeting the exclusion criteria) also revealed a small systematic difference regarding age (completers were on average 2 years older; t(502) = 2.4, P = 0.015). For all other variables being examined (gender, marital status, professional degree, work incapacity, diagnoses, comorbidity, chronicity, initial scores of SF–8 and SCL–14), no systematic differences between the two groups were found.
Effectiveness of systematic treatment selection
Primary analyses (completer analyses)
General effectiveness. The results of the ANCOVAs are displayed in Table 3 and Fig. 3. There was no significant main effect for the factor assignment either for the GSI (STS: marginal mean (m.m.)T1_STS =0.98, s.e.=0.06; RTS: m.m.T1_RTS = 1.00, s.e. = 0.07) or for the MCS (STS: m.m.T1_STS = 1.81, s.e. = 0.11; RTS: m.m.T1_RTS = 1.77, s.e. = 0.08). Thus, systematically assigned patients did not benefit more from treatment than patients randomly assigned (GSI: F(1, 226) = 0.13, P = 0.721, partial η 2 = 0.001; MCS: F(1, 233) = 0.12, P = 0.732, partial η 2 = 0.001). This means that, irrespective of treatment method and adjustment for baseline differences, we found no evidence for the general effectiveness of systematic treatment selection.
Comparison of systematic (STS) and random treatment selection (RTS) at follow-up (completer analyses)
| ANCOVAb | ||||||||
|---|---|---|---|---|---|---|---|---|
| STSa | RTSa | Assignment (main effect) | Assignment × method | |||||
| CBT, marginal mean (s.e.) (n = 44) | PDT, marginal mean (s.e.) (n = 35) | CBT, marginal mean (s.e.) (n = 89) | PDT, marginal mean (s.e.) (n = 58) | P | η2 | P | η2 | |
| SCL-14 GSI (n = 226) | 1.03 (0.10) | 0.98 (0.11) | 0.80 (0.07) | 1.15 (0.09) | 0.721 | 0.001 | 0.031 | 0.021 |
| SF–8 MCS (n = 233) | 1.81 (0.14) | 1.81 (0.17) | 1.57 (0.10) | 1.96 (0.12) | 0.732 | 0.001 | 0.153 | 0.009 |
Differential effectiveness. In assessing the effectiveness of systematic treatment differentially for both CBT and PDT, the ANCOVAs revealed a significant assignment × treatment method interaction for the primary outcome (GSI: F(1, 226) = 4.72, P = 0.031, partial η 2 = 0.021). Regarding symptom severity, systematically assigned PDT patients benefited more from the treatment than patients randomly assigned to PDT, although this expected difference was not observed for CBT (Table 3 and Fig. 3). However, we did not find this pattern of differential effectiveness for the secondary outcome, HRQoL. In the ANCOVA, there was no significant assignment × treatment method interaction for this outcome parameter (MCS: F(1, 233) = 2.06, P = 0.153, partial η 2 = 0.009).
Sensitivity analyses (ITT analyses)
General effectiveness. The results of the ITT analyses (Table 4) confirmed the results of the completer analyses. We found no significant main effect for the factor assignment either for the GSI (STS: m.m.T1_STS = 1.06, s.e. = 0.06; RTS: m.m.T1_RTS = 1.04, s.e. = 0.05) or for the MCS (STS: m.m.T1_STS = 1.93, s.e. = 0.10; RTS: m.m.T1_RTS = 1.88, s.e. = 0.07). As in the completer sample, systematically assigned patients did not benefit more from treatment than patients randomly assigned (GSI: F(1, 277) = 0.12, P = 0.734, partial η 2 <0.001; MCS: F(1, 278) = 0.17, P = 0.679, partial η 2 = 0.001).
Comparison of systematic (STS) and random treatment selection (RTS) at follow-up (intention-to-treat analyses)
| ANCOVAb | ||||||||
|---|---|---|---|---|---|---|---|---|
| STSa | RTSa | Assignment (main effect) | Assignment × method | |||||
| CBT, marginal mean (s.e.) (n = 550) | PDT, marginal mean (s.e.) (n = 42) | CBT, marginal mean (s.e.) (n = 109) | PDT, marginal mean (s.e.) (n = 71) | P | η2 | P | η2 | |
| SCL–14 GSI (n = 277) | 1.08 (0.08) | 1.05 (0.09) | 0.89 (0.06) | 1.18 (0.07) | 0.734 | <0.001 | 0.044 | 0.015 |
| SF–8 MCS (n = 278) | 1.93 (0.13) | 1.93 (0.15) | 1.72 (0.09) | 2.03 (0.11) | 0.679 | 0.001 | 0.206 | 0.0096 |
Differential effectiveness. As in the primary analyses, the ANCOVAs for the sample revealed a significant assignment × treatment method interaction for the primary outcome (GSI: F(1, 226) = 4.72, P = 0.031, partial η 2 = 0.021; Table 4) indicating the differential effectiveness of systematic treatment selection regarding symptom severity. For HRQoL, our secondary outcome, the assignment × treatment method interaction reached no significance, replicating the result of the primary analyses (MCS: F(1, 233) = 2.06, P = 0.153, partial η 2 = 0.009).
Discussion
We did not find evidence for the general effectiveness of systematic treatment selection in an in-patient sample receiving psychodynamic or cognitive–behavioural therapy. However, the results support the assertion that systematic treatment selection is effective for psychodynamic treatment, but not for the cognitive–behavioural approach. Systematically assigned PDT patients showed a significantly larger reduction in symptom severity 6 months after treatment compared with patients randomly assigned to PDT. For patients treated with CBT, this hypothesised effect was not found.
This result is in line with theoretical considerations about therapy concepts implying that psychodynamic and psychoanalytically oriented therapies require patients with certain abilities (e.g. the ability to reflect or psychological mindedness), Reference Valbak20,Reference Taylor33,Reference Grand34 whereas such explicit considerations do not exist for CBT. Therefore, it might be easier within the cognitive–behavioural model to respond to a broader unselected population presenting for routine care. Problems for patients receiving PDT who do not meet the requirements for this treatment approach may arise at two levels within the therapy process: not only may these patients not respond adequately to PDT interventions, but also difficulties may occur during the actual implementation and delivery of the interventions. As fewer such problems should arise with CBT (because of its wider applicability), possibly resulting differences in treatment delivery could be an alternative interpretation of our study results.
Improvement from baseline (T0) to follow-up (T1) for the study groups (systematic treatment selection, STS v. random treatment selection, RTS) for cognitive–behavioural therapy (CBT) and psychodynamic therapy (PDT) patients in terms of (a) symptom severity (n = 226) and (b) HRQoL (n = 233). For both variables, higher scores (mean values) reflect higher degrees of distress or impairment.
SCL–14 GSI, Global Severity Index of the 14-item Symptom Checklist; SF–8 MCS, short version of the SF–36 Health Survey, mental component summary; T, assessment at the beginning of treatment (baseline); T, follow-up assessment 6 months after the termination of treatment.
The results of a previous study on prognostic assessment of therapy outcome are also consistent with our findings with respect to PDT: Spinhoven et al Reference Spinhoven and Giesen-Bloo35 found that assessors' ratings at baseline significantly predicted which patients with borderline personality disorder failed to change reliably or had not recovered after 3 years of transference-focused psychotherapy, although they could not predict similar outcomes in patients treated with CBT.
According to our data, the effectiveness of systematic treatment selection for the cognitive–behavioural approaches can neither be confirmed nor ruled out. The treatment improvement seen in randomly assigned CBT patients at least equals that of systematically assigned patients. Considering this data pattern, along with the result that systematically assigned CBT patients seem to have more disadvantageous prognostic factors for improvement (lower professional degree indicating lower education, higher impairment in physical HRQoL) than patients in the other study groups, it can be assumed that instead of explicit CBT indications, PDT contraindications were more likely to lead to a CBT assignment in our study. This would mean that in the STS group, patients who have a poorer prognosis for psychotherapeutic treatment in general were assigned to CBT, whereas in the RTS group, patients were evenly distributed among both treatment methods. This might be also a reason for the inability to detect a general effect of systematic treatment selection (v. random treatment selection) in our study.
One further interesting result refers to the finding that differential effectiveness showed a significant effect for symptom severity, but not for our secondary outcome (HRQoL). However, the resulting slopes for HRQoL showed at least a similar pattern for the four study groups as those for symptom severity.
The sensitivity analyses (ITT analyses), including all patients with a baseline assessment, revealed a significant effect for the differential effectiveness of systematic treatment selection, but not for its general effectiveness. This replicates and confirms the results of the completer analyses, supporting that there was no substantial attrition bias due to drop-out at the follow-up assessment.
The effect of the treatment method itself (CBT/PDT) refers to a further interesting and very relevant research question (regarding the long tradition of outcome research comparing different treatment approaches). However, this is beyond the scope of this paper.
One limitation of the study is the participation rate (58%), which was lower than expected. However, it must be considered that we recruited a consecutive sample of in-patients (with only a few exclusion criteria) in routine mental healthcare. Additionally, the representativeness analyses, which showed no systematic differences between participants and non-participants, except that of age, contribute to the generalisability of the findings.
A further shortcoming might be seen in the exclusion of the group of patients who had to be systematically randomised according to the clinic's standards (compelled STS). Although for these patients systematic treatment selection seemed to be especially important, the effects could not be evaluated because randomisation to either systematic or random selection was not possible for clinical and ethical reasons.
As we were unable to carry out the relatively expensive procedure of a structured diagnostic interview (e.g. Structured Clinical Interview for DSM Disorders) Reference First, Spitzer, Gibbon and Williams36 at the clinic, the diagnoses of our patient sample are based solely on the clinical judgement of the therapists. Although these were experienced clinical psychotherapists with extensive 3- or 5-year training, including comprehensive modules of diagnostic assessment, there is a certain risk of unreliable diagnoses. Reference Shear, Greeno, Kang, Ludewig, Frank and Swartz37 Therefore, although the aim of our study was not to focus on diagnostic subgroups, an additional standardised diagnostic procedure would have been desirable but was not feasible because this study was carried out in routine care.
For the same reason, treatments were not manualised but represent PDT and CBT as conducted in routine care. This implicates certain strengths with regard to external validity, but also leads to uncertainties regarding treatment integrity and treatment differentiation. Our naturalistic study could not reach the gold standard of trials in experimental settings, which refer to manualised treatments and therapists with specialised training. However, the results of our additional trial on treatment integrity (being conducted with the same therapists but not the same patients) Reference Watzke, Rueddel, Koch, Rudolph and Schulz14 at least support a sufficient degree of treatment integrity in the clinic under investigation.
A further potential shortcoming is the slightly different baseline scores of the experimental (STS) and the control (RTS) group in symptom severity and HRQoL. These small differences might have been relevant if there had been significant effects regarding systematic treatment selection in general (for both CBT and PDT), but they are not a plausible alternative explanation for the differential effects (i.e. an effect for PDT, but not for CBT) found in the study. Nevertheless, despite controlling statistically for the differences (ANCOVA) it cannot be absolutely ruled out that the slightly lower initial scores in the control group have led to a bias that overlies the hypothesised superiority of systematic treatment selection. However, the similar gradients of improvement in three of the four study conditions (RTS–CBT; STS–CBT and STS–PDT) do not support this explanation.
Another limitation might be seen in the applied criteria for systematic treatment selection. As the empirical knowledge for criteria and their operationalisations is marginal, we referred to the ‘best available evidence’ in the sense of evidence-based medicine, Reference Sackett and Rosenberg23 a procedure based on expert consensus. However, we do not know whether this procedure is directly reproducible across other diagnostic and treatment settings.
The study implemented an experimental double-blind design in routine in-patient health services by masking both therapists and patients to the information on study group (STS/RTS). This enabled us to assess the long-term effects of a systematic treatment selection model for psychotherapeutic treatments on symptom severity and HRQoL while ruling out expectancy effects that might play an important role in psychotherapeutic treatment. According to the results, systematic treatment selection seems to be able to optimise treatment outcome, at least for the psychodynamic approaches in routine care. Therefore, pursuing strategies for systematic assignment procedures in mental healthcare settings seems worthwhile. In order to be able to succeed in this and to optimise routine care by means of evidence-based procedures, it is necessary to strengthen research on specific prognostic markers for particular types of treatment. With this empirical knowledge it might be possible to develop feasible criteria for systematic treatment selection which can be reliably and validly assessed with standard procedures in clinical practice.
Acknowledgements
The authors thank Julia Deubel and Martina Kraemer for assisting with the study.
eLetters
No eLetters have been published for this article.