Summary of the evidence

In this NMA, behavioural therapy, CBT, short-term psychodynamic therapy and cognitive therapy were superior to treatment as usual in the treatment of the acute phase of panic disorder with or without agoraphobia. At the same time, some of the most efficacious therapies were the lowest in terms of acceptability. For example, behavioural therapy had the best efficacy point estimate but proved to be poorly accepted, performing better than cognitive therapy only. In terms of efficacy, the psychotherapies that outperformed treatment as usual had medium-to-large effect sizes. Nonetheless, the CINeMA assessment showed very low-to-moderate confidence in the evidence, with no psychotherapy proving to have high quality of evidence. No relevant differences emerged when psychotherapies were compared head to head, except for behavioural therapy and CBT, which performed particularly well against physiological therapies and third-wave CBT. It should be acknowledged that although psychodynamic therapy has its own roots and tradition,^{Reference Freud169} the distinction between behavioural therapy, cognitive therapy and CBT is not clear-cut. For example, Clark's cognitive therapy for panic disorder emphasises the cognitive aspects, marginalising the role of fear extinction through habituation, but incorporates ‘behavioural experiments’.^{Reference Clark163,Reference Clark and Salkovskis170} By contrast, Beck's cognitive therapy also makes extensive use of behavioural skills.^{Reference Beck and Emery171} Finally, CBT is a psychotherapy that combines cognitive and behavioural elements.^{Reference Barlow172} It is therefore expected that CBT and its behavioural and cognitive components taken separately would have fairly similar effect sizes in terms of efficacy. Furthermore, our results on the efficacy estimates of psychotherapies for panic disorder are similar to those found for generalised anxiety disorder^{Reference Chen, Huang, Hsu, Ouyang and Lin173} and social anxiety disorder,^{Reference Mayo-Wilson, Dias, Mavranezouli, Kew, Clark and Ades174} suggesting that common therapy factors might play a greater role than specific factors in the treatment of different anxiety disorders.^{Reference Cuijpers, Reijnders and Huibers175}

For acceptability, most psychotherapies had similar effect sizes, but behavioural therapy and CBT were less acceptable than most of the other interventions. Furthermore, the evidence for cognitive therapy, behavioural therapy and physiological therapies was rated as high confidence, strengthening the link between efficacy and poor acceptability for behavioural and cognitive therapy. By principle, a high drop-out rate in psychotherapy trials does not necessarily mean that the intervention is poorly acceptable. In contrast with what happens in psychopharmacology trials, participants may drop out of psychotherapy not because of side-effects but because they get better and do not feel the necessity to be treated anymore. This could explain why some psychotherapies had large effect sizes but also high attrition rates. Nonetheless, among those studies reporting reasons for drop out such a possibility was never mentioned. Instead, the most frequently reported reasons for drop out were lost contact, personal or transportation difficulties and time demands. Regardless, only CBT and short-term psychodynamic therapy performed better than treatment as usual in terms of efficacy, being similar to the same reference comparison in terms of acceptability (although supported by only low-to-moderate confidence of evidence in both outcomes). In general, these results were confirmed by the sensitivity analyses, with the interesting finding that after removing high risk of bias RCTs the overall effect of psychotherapies deflated, and only CBT and behavioural therapy remained significantly more efficacious than treatment as usual. We acknowledge that sensitivity analyses showed a higher degree of incoherence in comparison with primary analyses. Such a finding may be due to the increase in the number of single-study comparisons present in the sensitivity analyses.^{Reference Veroniki, Vasiliadis, Higgins and Salanti176}

The findings of the present systematic review and NMA are consistent with those from the randomised trials comparing psychotherapies head to head, and are also generally aligned with the results of previous pairwise meta-analyses. For example, Mitte et al^{Reference Mitte177} found no differences between CBT and behavioural therapy in terms of anxiety reduction, and Sánchez-Meca et al^{Reference Sánchez-Meca, Rosa-Alcázar, Marín-Martínez and Gómez-Conesa178} showed a general efficacy of psychological therapies for different clusters of symptoms, with the most consistent results in favour of the combination of exposure strategies with relaxation training or breathing retraining techniques, or both. There is also one Cochrane NMA on this topic, which was not able to provide clear-cut suggestions for clinical practice.^{Reference Pompoli, Furukawa, Imai, Tajika, Efthimiou and Salanti5} In the present review we almost doubled the number of included RCTs, sharpening the precision of the meta-analytic estimates especially for behavioural therapy and short-term psychodynamic therapy. For example, the Cochrane NMA called for new studies comparing CBT with short-term psychodynamic therapy,^{Reference Pompoli, Furukawa, Imai, Tajika, Efthimiou and Salanti5} and soon after its publication a relatively large RCT comparing short-term psychodynamic therapy with CBT was published.^{Reference Milrod, Chambless, Gallop, Busch, Schwalberg and McCarthy113} The results of this individual study are consistent with those of the present review, pointing out the slight superiority of CBT over panic-focused psychodynamic psychotherapy in terms of symptom reduction. We further confirmed these findings with the pairwise meta-analysis confronting CBT and short-term psychodynamic therapies head to head.

Strengths and limitations of this research

To the best of our knowledge, this is the largest systematic review on the efficacy and acceptability of psychotherapies for a specific anxiety disorder. It compared psychotherapies for panic disorder using an NMA methodology that included all available psychotherapies, administered in any delivery format, while keeping in the network the contribution of studies that compared psychotherapies with pharmacotherapy to optimise the use of existing evidence. With some negligible exceptions, we were adherent to a protocol that we published in advance.^{Reference Papola, Ostuzzi, Gastaldon, Purgato, Del Giovane and Pompoli8} We selected one outcome measure for each study using a pre-planned hierarchy of rating scales, giving priority to panic-specific scales, aiming to enhance the clinical applicability of study findings. The inclusion of any type of delivery format is another strength, as focusing on one delivery format only would have excluded a relevant proportion of studies.

Despite these strengths, several limitations should be considered when interpreting the results. First, the included RCTs were published over a long time span and this has inevitably introduced heterogeneity in terms of design, diagnostic criteria, follow-up periods and outcomes. To investigate this possibility we conducted meta-regression analyses to measure the potential impact of numerous study characteristics on the results, but we did not find significant associations. Heterogeneity could not be explained by the pre-planned sensitivity analyses either. We acknowledge this as the main limitation of the present study. The overall interpretation of the findings should be cautious owing to the presence of statistical heterogeneity in the efficacy analysis. When we removed the studies in which participants were diagnosed by means of outdated diagnostic manuals (i.e. DSM-III and DMS-III-TR), heterogeneity remained significant but short-term psychodynamic therapy lost its significance over treatment as usual. In light of the inconclusive finding of a meta-regression that tested the potential impact of the year of publication on treatment effect, such a result may be due to the loss of statistical power after removing one^{Reference Wiborg and Dahl155} of the five RCTs on short-term psychodynamic therapy.^{Reference Beutel, Scheurich, Knebel, Michal, Wiltink and Graf-Morgenstern39,Reference Martini, Rosso, Chiodelli, de Cori and Maina105,Reference Milrod, Chambless, Gallop, Busch, Schwalberg and McCarthy113,Reference Milrod, Leon, Busch, Rudden, Schwalberg and Clarkin114,Reference Wiborg and Dahl155} Despite that, the transitivity assumption appeared to be well preserved. In line with this point, we think the all-encompassing consideration of all the available delivery formats might explain at least part of the statistical heterogeneity detected in the efficacy analysis. Alhough there is evidence that different treatment delivery formats of CBT might have different impacts on depressive symptoms,^{Reference Cuijpers, Noma, Karyotaki, Cipriani and Furukawa179} no investigation has been conducted on the same matter for panic disorder so far. We highlighted that only 56% of the included RCTs used the individual face-to-face format, while the remaining 44% tested the validity of psychotherapies that used other delivery formats. A meta-regression analysis showed no impact of delivering the psychotherapy in person or remotely on the efficacy of psychotherapies. At any rate, in the second part of the overarching project described in the protocol,^{Reference Papola, Ostuzzi, Gastaldon, Purgato, Del Giovane and Pompoli8} we plan to deepen the topic by performing an NMA specifically focused on treatment delivery modalities for CBT. Second, more than half of the studies were judged to be at high risk of bias. We reason that this finding should be viewed also in light of meticulous requirements of the second version of the Cochrane risk of bias tool (RoB 2). Some of the key domains needed to grant a low risk of bias status are seldom satisfied in psychotherapy trials, especially those published before 2010. For example, the frequent failure to report details of allocation concealment, and the low rates of studies that analysed data in agreement with a pre-specified protocol, have negatively affected the overall risk of bias rating much more than would have happened applying the first version of the risk of bias tool. To counterweight the heavy impact of the risk of bias evaluation on the CINeMA evaluation we decided to downgrade by half a point for ‘some concerns’ and by one point for ‘major concerns’. This allowed us to produce a clinically informative open range of judgements instead of a less helpful series of very-low-confidence ratings, flattened down by the hypertrophic influence of the risk of bias evaluation. However, a sensitivity analysis showed that outcomes did not change indicatively after removing high risk of bias studies. Third, the imbalance in terms of number of participants between CBT and the other psychotherapies might have affected the reliability of our findings owing to random errors brought into the networks by the nodes with fewer participants (cognitive therapy, behavioural therapy, psychodynamic therapy, physiological therapy) and especially by those with fewer than 100 participants (interpersonal therapy, EMDR, third-wave CBT, psychoeducation, supportive therapy, benzodiazepines). Fourth, only three direct comparisons included ten or more studies, so the risk of publication bias could not be checked for the great majority of the head-to-head comparisons. The only comparison for which a small-study effect was suspected was CBT versus treatment as usual, which was one of the key comparisons in the efficacy analysis. Although there is the possibility that the SUCRA ranking of CBT could be partly explained by a small-study effect, such a suspicion arose from the analysis of 12 RCTs only. This number is just above the threshold suggested for analysing publication bias.^{Reference Sterne, Sutton, Ioannidis, Terrin, Jones and Lau20} Thus, the output of the Egger's regression test (P < 0.05) should not be considered probative, and a small-study effect may be only suspected. Furthermore, the possibility that the efficacy of CBT over treatment as usual could be influenced by a small-study effect was taken into account in the CINeMa appraisal. Fifth, studies comparing psychotropic drugs head to head or against placebo were not searched, so this review cannot be informative on the efficacy and acceptability of antidepressants and benzodiazepines for panic disorder. We reasoned that studies allocating participants to pharmacotherapy versus placebo, without a psychotherapy arm, might substantially differ from studies with a psychotherapy arm, with a high risk of violating the transitivity assumption required for an NMA.^{Reference Salanti, Del Giovane, Chaimani, Caldwell and Higgins180,Reference Del Giovane, Cortese and Cipriani181} Sixth, most studies did not include patients with comorbid disorders, which might alter the external validity of the results. Last, the NMA approach is not free from technical and theoretical shortcomings, including risks of multiple statistical assumptions and the challenges in addressing the problem of intransitivity and inconsistency.^{Reference Cipriani, Higgins, Geddes and Salanti182}

Clinical and research implications

Shedding light on the most appropriate psychotherapies in terms of risk/benefit ratio is a priority that could reduce use of pharmacological strategies and discourage recourse to interventions not backed by a sufficient evidence base.^{Reference Lilienfeld183} The finding that CBT and short-term psychodynamic therapy may be regarded as reasonable first-line psychotherapies in the acute phase of panic disorder has clinical implications. In line with recommendations from current guidelines,^184–187 the present review strengthens the evidence base on the efficacy of CBT, as we found moderate quality of evidence pointing out that CBT has nearly 80% probability of being the best treatment available for panic disorder based on the SUCRA ranking convention. CBT ranked second to behavioural therapy, and the credibility of evidence for the efficacy of behavioural therapy was equal to that for CBT. Nonetheless, we found high confidence of evidence of the low acceptability of behavioural therapy compared with treatment as usual. The findings of the present review confirm the growing trend in favour of short-term psychodynamic psychotherapies as reliable first-line interventions for panic disorder^{Reference Pompoli, Furukawa, Imai, Tajika, Efthimiou and Salanti5} and in general for common mental disorders.^{Reference Abbass, Kisely, Town, Leichsenring, Driessen and De Maat188} Trials on manualised psychodynamic psychotherapy delivered short-term and relatively inexpensive interventions that are easily implemented after an adequate training.

Large, pragmatic and high-quality head-to-head studies comparing psychotherapies other than CBT are needed to overcome the paucity of evidence for some interventions and to test therapy working mechanisms, patient-defined outcomes and cost-effectiveness.^{Reference Cuijpers189,Reference Cuijpers190} As part of the results of this review, we abstracted information on the main characteristics, delivery modalities and reference manuals for each psychotherapy intervention, thus enhancing the understanding of treatment complexity, mechanism of actions and active ingredients of each therapy. This information will likely be beneficial to developers to inform updates to international and national guidelines from scientific organisations, to researchers planning future investigations and to practising clinicians for the ultimate goal of improving mental healthcare.