Evidence-based practice guidelines should improve patient care. Reference Grol and Grimshaw1,Reference Grol and Grimshaw2 To strengthen suicide prevention in Dutch mental healthcare, an evidence-based multidisciplinary practice guideline for the assessment and treatment of suicidal behaviour (PGSB) Reference van Hemert, Kerkhof, de Keijser and Verwey3 was issued in May 2012. The PGSB is based on various international guidelines on the assessment and treatment of suicidal behaviour 4–7 and two empirical reviews of the Scottish government. Reference McLean, Maxwell, Platt, Harris and Jepson8,Reference Leitner, Barr and Hobby9 The PGSB combines the stress–diathesis Reference Goldney10 and entrapment Reference Williams, Pollock and van Heeringen11 model to explain the onset and persistence of suicidal behaviour and offers concrete recommendations to clinicians. Internationally, a large number of psychiatric guidelines has been made available over the past 20 years. Reference Franx, Niesink, Swinkels, Burgers, Wensing and Grol12 However, adherence to these guidelines in mental healthcare has been incomplete Reference Weinmann, Koesters and Becker13–Reference van Splunteren and Hermens15 with the result that patients do not always receive appropriate care. Reference Franx, Niesink, Swinkels, Burgers, Wensing and Grol12 It is argued that the training of professionals in guideline recommendations may improve adherence to guidelines and thereby improve patient care. Reference Grol and Grimshaw1,Reference Baker, Reddish, Robertson, Hearnshawd: and Jones16,Reference Worrall, Angel, Chaulk, Clarke and Robbins17 A systematic review Reference Weinmann, Koesters and Becker13 described all studies addressing the implementation of psychiatric guidelines, and concluded that randomised controlled studies on the effect of educational strategies on patients' well-being and recovery are needed to adequately implement new evidence and improve patient care. Suicide prevention training has been shown to improve the knowledge, skills and attitudes towards suicidal behaviour of gatekeepers (teachers, general practitioners) Reference King and Smith18–Reference Chagnon, Houle, Marcoux and Renaud26 and mental health professionals. Reference Oordt, Jobes, Fonseca and Schmidt27,Reference Appleby, Morriss, Gask, Roland, Lewis and Perry28 Also, professional and gatekeeper training in diagnosis and treatment of depressive disorder, likely results in a reduction in suicide rates. Reference Matthieu, Cross, Batres, Flora and Knox24,Reference Rutz, Knorring and Wålinder29–Reference Knox, Litts, Talcott, Feig and Caine32 This might be explained by the strong association between depressive disorders and suicide. Reference Hawton and van Heeringen33 However, the actual effects of a specific suicide practice guideline training on suicidality of patients has not yet been examined in a randomised controlled trial (RCT).
Considering the relevance of the PGSB for Dutch mental heathcare, Reference Bool and Doeven34 and the need for evidence on the effectiveness of training professionals in guideline adherence, Reference Sinnema, Franx and Op 't Land35 the Dutch health funding agency ZONMW funded a cluster RCT on the structured training of professionals in recommendations of the PGSB called PITSTOP suicide (Professionals in Training to STOP suicide, Dutch trial register: NTR3092). The intervention consisted of an e-learning supported train-the-trainer programme (TtT-e), the content of which reflects the PGSB, and that was delivered to multidisciplinary teams of mental healthcare departments. We previously found that the TtT-e resulted in an approximately 10% improvement in confidence, knowledge and guideline adherence of mental healthcare professionals Reference de Beurs, de Groot, de Keijser, Van Duijn, de Winter and Kerkhof36 as assessed with self-report questionnaires. As practice guidelines reflect everyday practice, professionals already show high levels of guideline adherence without being trained. In a systematic review of 235 studies on guideline dissemination, 10% change has been found to be the maximum expected change when training experienced professionals. Reference Grimshaw, Thomas, MacLennan, Fraser, Ramsay and Vale37 In the current study, we hypothesised that individuals who were suicidal and who were treated by professionals who were trained using the TtT-e (intervention) would recover more quickly from suicidal ideation compared with patients treated by professionals who were not trained using the TtT-e (control) but received information on the release of the guideline via the usual methods (internet, conferences, workshops, etc.). Secondary outcomes were self-reported non-fatal suicide attempts and treatment satisfaction. As fostering a working relationship with the patient is the most important part of the TtT-e, we hypothesised that all patients, whether they present suicide ideation at baseline or not, would show more treatment satisfaction in the intervention condition than patients in the control condition.
Method
Design and sample recruitment
The PITSTOP suicide trial has been described in detail elsewhere. Reference de Beurs, de Groot, Bosmans, de Keijser, Mokkenstorm and Verwey38 In summary, the PITSTOP suicide trial was a multicentre cluster RCT. Clusters were care departments of mental healthcare institutions throughout The Netherlands. Departments were considered eligible for participation if they treated patients ⩾18 years of age and if professionals felt a need for training in suicide prevention skills. Patients of included departments were eligible if they were willing to provide written informed consent. Whether a patient was able to enter the study was left to the discretion of the staff. Patients who were deemed emotionally and/or cognitively unable to complete questionnaires were excluded.
Matching and randomisation
Eligible departments were matched in pairs on the basis of the main diagnostic DSM-IV category 39 of patients treated in the department, and on comparable average length of treatment. Reference de Beurs, de Groot, de Keijser, Verwey, Mokkenstorm and Twisk40 Members of matched pairs were randomly allocated to either implementation as usual (control condition) or the TtT-e plus implementation as usual (intervention condition). Binary randomisation was performed by an independent researcher of the Dutch Institute for Health and Care Research (EMGO) who was not involved in the study. Patients did not know whether they were in the intervention or the control condition.
Intervention
In the intervention condition, complete multidisciplinary teams of mental health professionals were trained in the application of the guideline via a 1-day small interactive group programme supported by e-learning modules (TtT-e). Reference de Beurs, de Groot, de Keijser, Verwey, Mokkenstorm and Twisk40 Personalised feedback was an important element of the training. The training was provided by peers, who were trained by experts/masters in the field of suicidology according to the PITSTOP training protocol. The train-the-trainer model is based on adult learning theory, Reference Knowles41 which states that adult learners learn more when the education is relevant to their context and draws on their previous experience, and on the diffusion of innovation theory, Reference Rogers42 which states that people adopt new information better through their trusted social networks. The TtT-e combines 1-day face-to-face training with an additional e-learning module; this form of blended learning is used extensively in medical education and has been found to be more effective compared with traditional instructor-based training. Reference Means, Toyama, Murphy and Baki43,Reference Pearce, Mann, Jones, van Buschbach, Olff and Bisson44 The PGSB recommendations served as the starting point to set the content of the TtT-e. The PGSB recommends systematic investigation of the suicidal condition of patients by using the Chronological Assessment of Suicidal Events (CASE) interview. Reference Shea45 Based on its outcome, the risk and protection factors for suicide of individual patients are weighted. Subsequently, structured diagnosis, treatment strategy and a safety protocol are determined. In the TtT-e, the CASE interview was the overall framework for each of four role plays in which one trainee acted as a patient who was suicidal and the other trainee interviewed the ‘patient’ via the CASE interview.
Data-collection procedure
If possible, patient data were collected via routine outcome monitoring (ROM), an online assessment by which data on the effectiveness of treatment in everyday clinical practice are systematically collected. Reference De Beurs, den Hollander-Gijsman, Van Rood, Van der Wee, Giltay and Van Noorden46 In mental healthcare institutions not using ROM, data were collected by graduate students and/or research assistants using paper-and-pencil questionnaires. In the intervention condition, as soon as all staff were trained, newly admitted patients were assessed at admission (T 0) and subsequently 3 months after admission (T 1). If a patient was discharged within 3 months, T 1 was arranged just before discharge. In the control departments, T 0 started at the time that the department was informed of the allocation outcome. At both T 0 and T 1, suicide ideation was measured with the first 19 items of the Beck Scale for Suicidal Ideation (BSS). Reference Beck, Kovacs and Weissman47 Total score ranged from 0 to 38, a higher score reflects stronger suicide ideation. In the user manual of the BSS it is stated that any positive response to any BSS item may reflect the presence of suicidal intention. Reference Beck and Steer48 Frequency of self-reported non-fatal suicide attempts were assessed at admission (‘Did you ever make one or more suicide attempts?’) and at T 1 (‘Did you make one or more suicide attempts between now and the first questionnaire?’). At T 1, we also assessed treatment satisfaction with four items that had been established to measure the quality of therapeutic alliance. The first two items were: ‘How satisfied are you overall with your treatment?’ and ‘How would you evaluate your relationship with your therapist?’ These two items were rated from zero to ten. The other two items were: ‘Was there any attention for your suicidal thoughts during treatment?’ and ‘How did your therapist deal with your suicidal thoughts?’ These two items were scored on a four-point scale ranging from 1, yes/well to 4, no/poor. Finally, the main DSM-IV diagnosis of each patient was collected from their electronic health record.
Written informed consent was obtained for all participants after the procedures had been fully explained. The study was approved by the medical ethical commission of the VU Medical Center (2011/151).
Patient recruitment, follow-up and sample size
The first patients were assessed on 20 January 2012. Final 3-month follow-up assessments were completed by 26 September 2013. For the primary outcome (change in suicide ideation) the number of patients that needed to be included was set at 423. This number is sufficient to find a small effect size (Cohen's d) of 0.3, assuming an alpha of 0.05 and the statistical power of 1 – β = 0.80. A correction of 20% for clustering of effects within departments was applied.
Statistical analyses
Differences in baseline characteristics of patients who dropped out and those who did not were analysed by using logistic regression.
Suicide ideation and non-fatal suicide attempts
In our study, we were interested in any change on the BSS, even in participants with the lowest score on the BSS. Therefore, for our primary analysis, we selected patients that scored ⩾1 on the BSS. Within this group of patients that scored ⩾1 on the BSS, baseline differences between the intervention and control group were described. Effects of the intervention on the BSS were analysed on an intention-to-treat basis by using multiple imputation for missing values. Reference Van Buuren49 We fitted a multilevel model with a random intercept at the department level. Score on the BSS at baseline was added as covariate and the randomisation condition was the between-participants factor. A multilevel logistic regression model was fitted to establish the effect of the intervention on self-reported non-fatal suicide attempt at T 1. A subgroup analysis was done by repeating the same analysis for patients diagnosed with a depressive disorder and scored ⩾1 on the BSS at baseline. Results were presented using regression coefficients (b) or odds ratios (ORs), 95% confidence intervals and P-values. Cohen's ds represented the effect size of TtT-e.
Treatment satisfaction
The first two items on treatment satisfaction were analysed by fitting linear multilevel models for all patients and, separately, for patients that scored ⩾1 on the BSS. Data were analysed on an intention-to-treat basis. Means, standard deviations and effect sizes are presented. Answers to the final two treatment satisfaction items were dichotomised by adding response option 1 and 2 into one response category (yes/well), and response 3 and 4 into another (no/poor). All analyses were done with SPSS 21.
Results
Figure 1 shows the flow of participants and departments through the trial, showing that 881 patients in 29 departments were included. Patients who were lost to follow-up were more likely allocated to the intervention group (OR = 1.96, 95% CI 1.45–2.68, P<0.01).
CONSORT 2010 flow diagram of the trial.
Intervention condition, implementation as usual plus the e-learning supported train-the-trainer programme; control condition, implementation as usual.
All 881 patients completed baseline assessments; 566 (64%) had a score on the BSS ⩾1 and 250 patients (28%) reported a history of at least one suicide attempt.
Within the sample of 566 patients with a score on the BSS ⩾1 at baseline, relatively more women were included in the intervention group compared with the control condition (Table 1). Also, the intervention condition contained more patients with a diagnosis of a personality disorder or an eating disorder, whereas the control group contained more patients with a depressive or a substance dependence disorder. Distribution of suicidal ideation and percentage of people who had made previous attempts within the suicidal sample were comparable between both groups.
Baseline characteristics of patients with a score on the Beck Scale for Suicidal Ideation ⩾1, split per condition
| Total (n = 566) |
Intervention
group (n = 312) |
Control
group (n = 254) |
n a | |
|---|---|---|---|---|
| Demographic characteristics | ||||
| Female, n (%) | 295 (55) | 160 (56) | 134 (53) | 540 |
| Age, years: mean (s.d.) | 42 (15) | 42 (15) | 41 (14) | 566 |
| Education, n (%) | 339 | |||
| Lower | 37 (11) | 21 (10) | 16 (12) | |
| Intermediate | 233 (69) | 145 (71) | 88 (65) | |
| Higher | 69 (20) | 37 (18) | 32 (24) | |
| Living with partner, n (%) | 133 (38) | 84 (40) | 49 (35) | 353 |
| Born in The Netherlands, n (%) | 253 (94) | 162 (95) | 91 (92) | 268 |
| Paid employment, n (%) | 47 (15) | 20 (10) | 27 (11) | 341 |
| Data collected with ROM, n (%) | 192 (34) | 98 (31) | 94 (37) | 566 |
| Clinical characteristics | ||||
| Suicidal thoughts, mean (s.d.) | 12 (9) | 13 (9) | 11 (8) | 566 |
| Attempted suicide, n (%) | 537 | |||
| Never | 303 (57) | 166 (55) | 137 (59) | |
| Once | 120 (22) | 66 (22) | 54 (23) | |
| More than once | 114 (21) | 72 (23) | 42 (18) | |
| Diagnosis, n (%) | 358 | |||
| Depression | 154 (41) | 75 (36) | 79 (50) | |
| Anxiety | 25 (7) | 12 (6) | 13 (8) | |
| Psychosis | 30 (8) | 18 (9) | 12 (8) | |
| Personality disorder | 62 (17) | 47 (23) | 15 (9) | |
| Substance dependence | 39 (11) | 9 (4) | 30 (19) | |
| Post-traumatic stress disorder | 21 (6) | 12 (6) | 9 (6) | |
| Eating disorder | 27 (7) | 26 (12) | 1 (6) | |
ROM, routine outcome monitoring.
a. Data are not available for all patients for all variables.
Suicide ideation results and non-fatal suicide attempts
Multilevel analysis showed no effect of the intervention on change in suicide ideation (b = −0.68, 95% CI −0.66 to 1.21, P>0.05) or on frequency of self-reported attempted suicide between baseline and follow-up (OR = 1.18, 95% CI 0.62 to 2.57, P>0.05). To our knowledge, no suicides took place during the follow-up period. However, in a subgroup of patients with a diagnosis of depression that also scored ⩾1 on the BSS at baseline (n = 154; intervention, 75, control, 79), a significant effect on change in suicide ideation between conditions was found: scores on the BSS decreased 8.4 points between baseline and follow-up in the intervention group, compared with a decrease of 4.8 in the control group (b = 3.41, 95% CI 0.38–5.93, P = 0.008, effect size 0.4 (Table 2)). No effect of the intervention on self-reported attempted suicide was found within this subgroup (OR = 1.18, 95% CI 0.63–2.52). For any other subgroup (anxiety, personality, psychosis), numbers were too small or too unbalanced for any significant testing (Table 2).
Mean decrease in Beck Scale for Suicidal Ideation (BSS) scores from baseline to follow-up and effect sizes
| Intervention group | Control group | Effect size |
|||||
|---|---|---|---|---|---|---|---|
| Mean (s.d.) | n | Mean (s.d.) | n | b (95% CI) | P | ||
| All patients with a score on the BSS ⩾1 | 4.2 (13.4) | 312 | 4.9 (10.5) | 254 | −0.68 (−0.66 to 1.21) | ns | |
| Patients with diagnoses of
depression and a score on the BSS ⩾ 1 |
8.4 (7.7) | 75 | 4.8 (7.9) | 79 | 3.41 (0.38 to 5.93) | 0.008 | 0.4 |
ns, not significant.
Treatment satisfaction
When analysing the total group of patients, no significant effect on overall satisfaction with treatment was found (Table 3). Satisfaction with the relationship with the therapist was higher in the control group when compared with the intervention condition (intervention group mean 6.9 (s.d. = 4.1), control condition mean 7.4 (s.d. = 3.9), b = 0.61, 95% CI 0.04–1.22, P = 0.03, effect size 0.4).
Mean treatment satisfaction at follow-up for all patients
| Intervention
group Mean (s.d.) (n = 484) |
Control
group Mean (s.d.) (n = 397) |
b (95% CI) | P | Effect size |
|
|---|---|---|---|---|---|
| Overall satisfaction with therapy, range (0–10) | 6.8 (4.4) | 6.8 (4.3) | −0.01 (−0.58 to 0.56) | ns | |
| Satisfaction with relationship with therapist (range 0–10) | 6.9 (4.1) | 7.4 (3.9) | 0.61 (0.04 to 1.22) | 0.03 | 0.4 |
ns, not significant.
Since >70% of the responses to the two items ‘Were suicidal thoughts addressed during treatment’ and ‘How did your therapist deal with your suicidal thoughts?’ were missing, we performed a complete case analysis only Reference Sterne, White, Carlin, Spratt, Royston and Kenward50 (Table 4). Suicidal thoughts were more likely to be discussed in the intervention group compared with the control group (b = 1.78 CI 95% 1.12–2.93, P = 0.02, risk ratio 1.3). No differences were found regarding the appraisal of the way the therapist dealt with suicidal ideations. When applying the same analysis for patients that scored ⩾1 on the BSS at baseline, no significant effects of the intervention on any of the four items was found.
n (%) yes/well at follow-up for completers only
| Intervention group, n/N (%) |
Control group n/N (%) |
Effect size | ||||
|---|---|---|---|---|---|---|
| OR (95% CI) | P | Risk ratio | Risk difference | |||
| Were suicidal thoughts addressed
during treatment? Yes |
85/154 (55) | 48/118 (41) | 1.78 (1.1–2.93) | 0.018 | 1.3 | 0.14 |
| How did your therapist deal with
your suicidal thoughts? Well |
102/133 (77) | 63/89 (71) | 1.33 (0.74–2.49) | ns | ||
ns, not significant.
Sensitivity analysis
When comparing the analysis of the imputed data and the complete case data, all results were comparable, expect for ‘Satisfaction with relationship with the therapist’. When using complete case analysis, no significant difference between intervention and control condition was found on that item (intervention group mean 7.1 (s.d. = 3.2), control group 7.5 (s.d. = 3.6), b = 0.41, 95% CI −0.32 to 1.16, P = 0.2).
Discussion
This study examined the additional effects on patients of an e-learning supported train-the-trainer programme aimed at training professionals in adherence to the Dutch guideline on the assessment and treatment of suicidal behaviour. The TtT-e was found to result in a 10% improvement in confidence, knowledge and guideline adherence among professionals. Reference de Beurs, de Groot, de Keijser, Van Duijn, de Winter and Kerkhof36 We tested whether patients benefited from this 10% improvement of professionals. For the total group of patients who were suicidal, no effect of the intervention was found on change in suicidal ideation or frequency of self-reported non-fatal suicide attempts at 3-month follow-up. As patients in the control group also recovered significantly from suicidal ideation, any additional effect of the TtT-e for the total group of patients who were suicidal was probably smaller than our sample size allowed us to detect. This is a common observation when implementing guidelines in psychiatry. Reference Weinmann, Koesters and Becker13 We did find a significant effect of our intervention for the group of 154 patients diagnosed with depression who scored positive on the BSS at baseline. This is in line with a multicentre trial that demonstrated the effectiveness of depression guideline training of general practitioners on decline in suicide ideation among elderly people with depression. Reference Alexopoulos, Reynolds, Bruce, Katz, Raue and Mulsant51 The effect of the intervention on only the subgroup of patients with depression might be explained by the focus of the TtT-e on making contact and discussing suicidality, which might be more appropriate for patients who are suicidal who have a depressive disorder and less so for those with, for example, a borderline or psychotic disorder.
No effect of our intervention was found on overall satisfaction with the treatment. Patients in both groups appeared to be quite satisfied with their treatment, making it difficult to achieve a significant increase in overall treatment satisfaction by training professionals in suicide guideline adherence. When analysing the imputed data, patients in the control group appear to be more satisfied with their therapist. As no significant effect was found when analysing completers-only data, and a sensitivity analysis found no other discrepancies between imputed data and completers only, we argue that this effect might be explained by the imputation of the data for a single item on subjective experience. Reference Sterne, White, Carlin, Spratt, Royston and Kenward50 An alternative explanation is that the additional focus on suicidal behaviour (which is the core element of the PGSB and therefore the TtT-e) during treatment may be perceived as uncomfortable or too confronting by patients, thereby resulting in less satisfaction with the therapist. It might also be that recently trained staff who were more confident, knowledgeable and adherent to the guideline (as a result of the TtT-e) were not so good at engaging patients in treatment – perhaps overconfident or more focused on the new technical aspects at the expense of being less sensitive to the person and the therapeutic relationship.
We have shown that patients reported more often that suicidality was discussed during treatment, which indicates that we were able to change the behaviour of professionals in individual treatment sessions. This is in line with another randomised study that found that general practitioners assessed more patients for suicide risk after a tailored depression guideline implementation. Reference Baker, Reddish, Robertson, Hearnshawd: and Jones16
Limitations and strengths
Patients who were lost to follow-up were more likely to be allocated to the intervention group. This might be explained by the fact that departments in the intervention condition seemed to be more motivated and successful in starting data collection, resulting in the inclusion of more patients at baseline. Some of these patients might have had more severe levels in pathology, and were therefore more likely to drop out at follow-up. Although mental healthcare institutional boards agreed on collecting patient data using ROM, only 34% of the data were collected via ROM. For various reasons, mostly technical and organisational, most institutions were not able to add our questionnaires to their existing ROM. As data collection via ROM is more systematic and on a larger scale when compared with data collection via paper-and-pencil questionnaires, we included fewer patients and had more missing values than we originally anticipated.
It was especially difficult for our research assistants to get access to the DSM-IV diagnosis of patients. This resulted in a large amount of missing patient diagnoses (37%) and a difference in proportion of patient diagnoses between the intervention and the control conditions. As the success of collecting diagnoses depended on factors unrelated to patient or professionals' characteristics, but rather on factors related to the managerial authorisation to access data within a department, we do not expect that the difference between conditions influenced our results. It did, however, mean that we were unable to test the effect of our intervention on subgroups other than patients with a diagnosis of depression. Therefore, the conclusion that our intervention has effects on patients with depression and not on other patients should be considered with these limitations in mind.
Budget cuts in Dutch mental healthcare were introduced just after our randomisation was completed. Various mental healthcare departments were shut down or did not have the resources anymore to fulfil the study requirements, resulting in a loss of 11 departments after randomisation, leading to less power. Also, during our data-collection period, government retrenchment for mental healthcare made mental healthcare less accessible for patients. Patients suddenly were forced to pay a considerable sum of money for treatment in specialised mental healthcare, which resulted in a decrease in the numbers of new patients being admitted to the psychiatric departments, resulting in less power.
An important focus of our intervention was making contact with patients who were suicidal, and paying more attention to suicidal ideation. Since most of the data (66%) were collected via paper-and-pencil instead of via the ROM, patients in both conditions might have experienced more attention being paid to their suicidal thoughts because of the assessment of suicidal ideation, making any effect of our intervention more difficult to detect. Finally, as part of our safety plan, when a patient showed increased suicidal ideation at baseline in either the control or the intervention group, we reported this to their therapist. This monitoring and supervision led to more attention being given to patients who were suicidal in both conditions, making it more difficult to find an effect of our intervention.
A strength of this study is its randomised controlled design, which is scarce in this field of research. Reference Weinmann, Koesters and Becker13 Also, the departments that were included are a good representation of the psychiatric departments in The Netherlands, Reference de Beurs, de Groot, de Keijser, Verwey, Mokkenstorm and Twisk40 making the results generalisable to other institutions. Finally, given the difficulty in collecting data from patients who are suicidal and admitted to mental healthcare, difficulties with ROM and the challenges of the ongoing budget cuts and reorganisations, the patient data collection in our trial can be regarded as quite successful. The large number of patients included in our study makes our findings more reliable and generalisable.
Implications and further study
This is the first randomised trial to examine the effect on patients of training professionals in adherence to the PGSB. We found that the structural training of professionals resulted in more attention being given to suicidality during treatment. This indicates that training in suicide guideline adherence can have an impact on actual clinical practice. Overall, we found no effect of our intervention on patients. We did find a significant effect on decrease in suicide ideation within a subgroup of patients with a diagnosis of depression. It appears that compared with implementation as usual, training professionals in adherence to suicide guidelines might have beneficial effects for patients who are depressed and suicidal. Future studies should investigate whether a more tailored programme, with a focus on the specific patient group in a department, would result in the same effect on, for example, patients with a personality disorder who are suicidal, as that currently found for patients who are depressed and suicidal.
Also, the relative effectiveness of the different elements of the TtT-e (the train the-trainer element, the face-to-face training, the e-learning module, the multidisciplinary element) has not been examined separately. Future studies may disentangle the effects of the different elements, so that more targeted programmes can be developed. Reference Cherry, Brown, Bethell, Neal and Shaw52 Finally, longitudinal studies should investigate the long-term effect of suicide guideline implementation on patients.
Considering the methodological and diagnostic issues discussed earlier, our study needs to be replicated. Ideally, data on suicidal ideation should be collected in a more systematic and less obtrusive manner via computerised outcome monitoring. However, implementation studies inevitably need to be done in a naturalistic sample, which has its limitations. In the current study, we managed to collect a large amount of data from multiple psychiatric departments among a heterogeneous sample of patients within a randomised design. Therefore, our findings offer the first evidence of effectiveness for patients who are depressed and suicidal of suicide guideline training of professionals.
Funding
This study is funded by The Dutch organisation for health research and development (ZonMW).
eLetters
No eLetters have been published for this article.