When treating patients with psychiatric disorders, clinicians and administrators should turn to randomised clinical trials to look for evidence-based interventions to guide clinical decision-making. ^{Reference Collins, Bowman, Landray and Peto1} Randomised clinical trials are generally thought to be the gold standard when evaluating the effects of any healthcare intervention. ^{Reference Higgins, Thomas, Chandler, Cumpston, Li and Page2} High-quality trials are conducted with methodological rigour to limit the risk of systematic and random errors and thereby increase the internal validity. However, the external validity, i.e. the generalisability of a trial, is equally important. The assessment of external validity of trials may be based on several factors, for example how the screening procedure was implemented, the number and type of eligibility criteria, the patient characteristics, how the intervention was implemented or the number and characteristics of the participants who discontinued the intervention. While these factors provide some insight into the external validity of a trial, a critical aspect is often overlooked: which patients are offered participation in randomised clinical trials? The selection process can introduce substantial problems with external validity, especially if certain patient groups are systematically less likely to be included in a trial, even though they frequently seek treatment in a real-world clinical setting. Hence, an arbitrary participant selection process in clinical trials can cause research participants to differ markedly from clinical populations, which negatively impacts the generalisability of the trial results. These challenges affect the research of many psychiatric disorders. In the following, we draw on out-patients with borderline personality disorder as an example.

Borderline personality disorder is a prevalent and burdensome psychiatric disorder often treated in out-patient psychotherapy clinics, ^{Reference Conklin and Westen3} for which the external validity of the evidence-based interventions may be questioned. A Cochrane review from 2020 found beneficial effects of psychological interventions for borderline personality disorder, but these results were based on low-certainty evidence. ^{Reference Storebø, Stoffers-Winterling, Völlm, Kongerslev, Mattivi and Jørgensen4} A previous systematic review found that the total proportion of eligible patients with borderline personality disorder who declined to enter a trial varied between 4.4 and 49.4%. ^{Reference Lana and Fernández-San Martín5} Hence, already at the screening stage, there is a risk of selection bias, as the patients who consent to participate in trials may potentially differ from patients who decline to participate. However, the proportion of patients who do not reach the trial screening stage is unclear.

Based on a screening of all the included trials from the most recent Cochrane review of psychological interventions for borderline personality disorder, ^{Reference Storebø, Stoffers-Winterling, Völlm, Kongerslev, Mattivi and Jørgensen4} we found that none of the included out-patient trials provided information about the time gap from referral to trial inclusion. However, eligible patients may potentially discontinue before they reach the screening and inclusion stage of a trial, particularly if the waiting period for the initial consultation is significant.

We assessed the proportion of out-patients with borderline personality disorder who were offered an initial consultation in the Mental Health Services of the Capital Region of Denmark from 2017 to 2023; the proportion of out-patients who attended the initial consultation; and the proportion of out-patients who subsequently initiated treatment (Fig. 1). Of the 3751 patients offered an initial consultation for treatment of borderline personality disorder at an out-patient clinic during this period, 11% did not attend the initial consultation. Only 65% of the patients offered initial consultation were also offered treatment, while 56% initiated treatment. Consequently, we can assume that between 11 and 44% of potentially eligible patients were not informed about the opportunity to participate in a randomised clinical trial, if such a trial was implemented at the clinic. Some of these patients may have been rightfully excluded because of failure to confirm the diagnosis. However, as they were offered an initial consultation, we must assume that a large proportion of these patients were potentially eligible.

Fig. 1

Sankey diagram of referred out-patients with borderline personality disorder in the Mental Health Services of the Capital Region of Denmark from 2017 to 2023.

When assessing the external validity of a trial, it may be relevant to consider the screening and enrolment process, but also to assess the proportion of eligible patients who do not reach the screening stage. Although there is often limited information about the pre-screening phase of trials, such information may offer important insights about the generalisability of trials. Figure 1 illustrates challenges with generalisability that may be invisible in most trial publications of borderline personality disorder: many patients are unintentionally not offered participation in trials, as they discontinue their clinical course before being screened for eligibility in out-patient psychological trials. As a result, we have limited knowledge about whether the evidence-based interventions for borderline personality disorder are effective for this unrepresented patient group. In addition, borderline personality disorder trials often face high drop-out rates during the study period, ^{Reference Iliakis, Ilagan and Choi-Kain6} but this estimate may be even higher if all eligible patients referred to the clinics were included. Ultimately, this raises the question of whether borderline personality disorder trials are generally conducted on a narrow subset of the broader population, resulting in what the Grading of Recommendations, Assessment, Development and Evaluation framework would refer to as indirectness of the trial results. ^{Reference Guyatt, Oxman, Kunz, Woodcock, Brozek and Helfand7}

While we still lack information about the comparability of the patients who discontinue their clinical course prior to being screened versus patients enrolled in the trials, we hypothesise that such patients potentially show different clinical characteristics, for example higher levels of impulsivity or clinical or functional instability. When trial participants do not accurately reflect the diversity of the real-world population, the assessed interventions may only be evidence-based for a subgroup of patients. This ultimately limits clinicians’ ability to make informed decisions and reduces patients’ chances of receiving effective care.

A potential solution could be to implement screening, informed consent collection, baseline assessments and randomisation procedures at the referral stage or right after the diagnosis has been confirmed, instead of waiting for the treatment to commence. Although this approach may present challenges, it could significantly enhance the understanding of external validity of borderline personality disorder trials. As a minimum, triallists should implement detailed screening logs at the referral stage to properly account for the external validity of the included trial population.