Skip to main content Accesibility Help
×
×
Home

Information:

  • Access
  • Open access

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Association between suicidal ideation and suicide: meta-analyses of odds ratios, sensitivity, specificity and positive predictive value
        Available formats
        ×
        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Association between suicidal ideation and suicide: meta-analyses of odds ratios, sensitivity, specificity and positive predictive value
        Available formats
        ×
        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Association between suicidal ideation and suicide: meta-analyses of odds ratios, sensitivity, specificity and positive predictive value
        Available formats
        ×
Export citation

Abstract

Background

The expression of suicidal ideation is considered to be an important warning sign for suicide. However, the predictive properties of suicidal ideation as a test of later suicide are unclear.

Aims

To assess the strength of the association between suicidal ideation and later suicide measured by odds ratio (OR), sensitivity, specificity and positive predictive value (PPV).

Method

We located English-language studies indexed in PubMed that reported the expression or non-expression of suicidal ideation among people who later died by suicide or did not. A random effects meta-analysis was used to assess the pooled OR, sensitivity, specificity and PPV of suicidal ideation for later suicide among groups of people from psychiatric and non-psychiatric settings.

Results

There was a moderately strong but highly heterogeneous association between suicidal ideation and later suicide (n = 71, OR = 3.41, 95% CI 2.59–4.49, 95% prediction interval 0.42–28.1, I2 = 89.4, Q-value = 661, d.f.(Q) = 70, P ≤0.001). Studies conducted in primary care and other non-psychiatric settings had similar pooled odds to studies of current and former psychiatric patients (OR = 3.86 v. OR = 3.23, P = 0.7). The pooled sensitivity of suicidal ideation for later suicide was 41% (95% CI 35–48) and the pooled specificity was 86% (95% CI 76–92), with high between-study heterogeneity. Studies of suicidal ideation expressed by current and former psychiatric patients had a significantly higher pooled sensitivity (46% v. 22%) and lower pooled specificity (81% v. 96%) than studies conducted in non-psychiatric settings. The PPV among non-psychiatric cohorts (0.3%, 95% CI 0.1%–0.5%) was significantly lower (Q-value = 35.6, P < 0.001) than among psychiatric samples (3.9%, 95% CI 2.2–6.6).

Conclusions

Estimates of the extent of the association between suicidal ideation and later suicide are limited by unexplained between-study heterogeneity. The utility of suicidal ideation as a test for later suicide is limited by a modest sensitivity and low PPV.

Declaration interest

M.M.L. and C.J.R. have provided expert evidence in civil, criminal and coronial matters. I.B.H. has been a Commissioner in Australia's National Mental Health Commission since 2012. He is the Co-Director, Health and Policy at the Brain and Mind Centre (BMC) University of Sydney. The BMC operates an early-intervention youth services at Camperdown under contract to Headspace. I.B.H. has previously led community-based and pharmaceutical industry-supported (Wyeth, Eli Lily, Servier, Pfizer, AstraZeneca) projects focused on the identification and better management of anxiety and depression. He is a Board Member of Psychosis Australia Trust and a member of Veterans Mental Health Clinical Reference group. He was a member of the Medical Advisory Panel for Medibank Private until October 2017. He is the Chief Scientific Advisor to, and an equity shareholder in, InnoWell. InnoWell has been formed by the University of Sydney and PricewaterhouseCoopers to administer the $30 M Australian Government Funded Project Synergy. Project Synergy is a 3-year programme for the transformation of mental health services through the use of innovative technologies.

Health professionals are expected to have the skills to assess and manage patients with suicidal thoughts.1 The presence of thoughts of suicide and the expression of suicidal ideation are signs of significant distress, and all patients who express suicidal ideation require a careful clinical assessment2, 3 leading to a comprehensive treatment plan.3 In addition, suicidal ideation is sometimes considered to be the most important sign of short-term suicide risk,4 and it has been argued that questions about suicidal ideation play a crucial role in suicide screening.5, 6

Despite the undoubted clinical importance of suicidal ideation, meta-analysis has only been used recently to estimate the statistical strength of suicidal ideation as a test for later suicide. Several meta-analyses have highlighted the modest strength of the association between suicidal ideation and later suicide710 and one has quantified the positive predictive value (PPV).8 To date, no meta-analysis has examined the sensitivity and specificity of suicidal ideation for suicide. Knowledge of the sensitivity of suicidal ideation for later suicide is particularly important because screening tests should be sufficiently sensitive, such that a high proportion of affected people screen positive.11 Moreover, existing meta-analyses of the association between suicide and suicidal ideation have been limited to studies with a cohort design,9 only included patients with particular diagnoses7 or included studies where suicidal ideation was established post-mortem using psychological autopsy methods.8

We report a meta-analysis of cohort and case–control studies that examined the presence or absence of expressed suicidal ideation among those who later did or did not die by suicide.

Aims and hypotheses

The first aim of this study was to calculate a comprehensive set of measures of prediction of suicidal ideation for later suicide, including of effect size as measured by the pooled odds ratio (OR), sensitivity, specificity, meta-analytically derived receiver operator curve (ROC) and area under the curve (AUC) and the PPV. The second aim was to explore the extent of, and moderators of, between-study heterogeneity in these predictive measures. We hypothesised that suicidal ideation would be a more sensitive and specific test for suicide in non-psychiatric settings, where patients with suicidal ideation can be expected to have fewer additional and potentially confounding risk factors.

Methods

We conducted a registered meta-analysis (Prospero 42017059236) according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (see supplementary Table 1 available at https://doi.org/10.1192/bjo.2018.88).12

Searches

Multiple search strategies were initially explored using the databases Medline, Embase and PsycINFO from inception to January 2017. These searches yielded an excessive number of titles when the term ‘suicide’ was used as a keyword (>200 000) or in the title (>70 000). Moreover, placing limits in the searches using relevant search terms (such as ‘ideation’ or ‘mortality’) missed numerous relevant papers that were known to the authors through earlier hand searching. However, PubMed indexed all but one English-language publication located by searches of multiple databases in earlier studies.7, 8 Therefore, and in order to obtain a large and representative sample of relevant studies, one author (M.M.L.) examined the titles of English-language publications with an accompanying abstract that contained variants of the single term ‘suicide’ (suicid*) in their title and were published in PubMed from inception to 14 September 2017 and hand searched the references list of relevant review articles.710, 13, 14 Two authors (M.M.L. and C.M.M.) winnowed the resulting abstracts and full-text publications (Fig. 1).

Fig. 1 Flow chart of searches.

Included studies

Studies were included if they reported the expression or non-expression of suicidal ideation among people who later died by suicide or did not, such that all individuals could be classified into the following groups: true positive (those with suicidal ideation and suicide), false positive (those with suicidal ideation but without suicide), false negative (those without suicidal ideation but with suicide) or true negative (those without suicidal ideation or suicide). Studies were included if they provided effect size or other data that was sufficient to enable the calculation of the numbers of people in the true positive, false positive, false negative and true negative categories.

We included studies of patients who had received psychiatric care and people recruited from non-psychiatric settings including primary healthcare, general population samples, military populations and prisons. We excluded studies in which the dependent variable was suicide attempts, or when suicidal ideation was assessed by psychological autopsy methods, when all the controls were deceased or all of the individuals had made suicide attempts. We excluded studies of patients with severe medical illnesses such as malignancies or human immunodeficiency virus.

Definitions of suicide and suicidal ideation

Two authors (M.M.L. and C.M.M.) independently extracted the numbers of individuals with suicidal ideation among those who had died by suicide and controls and the moderator variables. A third author (A.C.) re-examined the data points. The discrepant points were re-examined by two authors. We accepted the definition of suicidal ideation used in the primary research, acknowledging that the primary research might not have fully reflected differences in the way suicidal ideation was expressed. Studies were coded into four categories according to their definition of suicidal ideation: studies that reported a composite measure of suicidality (inclusive of both suicidal ideation and behaviour); studies that reported suicide plans (including suicide ‘threats’ as reported in the primary studies); studies that recorded a wish to die; and the majority of studies in which the nature of the suicidal ideation or suicidal thoughts was not specified.

Moderator variables

We collected data about potential moderators of the association between suicidal ideation and suicide; (a) whether the individuals were from psychiatric or non-psychiatric samples; (b) whether the samples consisted of hospital-treated patients; (c) the mean sample age; (d) the duration of follow-up after the assessment for suicidal ideation; (e) whether the study had a cohort or control design; (f) whether patients with suicidal behaviour were regarded as having suicidal ideation; (g) the year of study publication; (h) the proportion of individuals included with suicidal ideation; and (i) the incidence of suicide in the study.

Assessment of strength of reporting

Two researchers (M.M.L. and C.M.M.) independently assessed the reporting strength (and hence the risk of bias) of each primary study's research using a scale with four further moderator variables (scored 0–4) derived from the Newcastle Ottawa Scale.15 One point was allocated according to each of the following criteria: (a) use of a structured method to assess suicidal ideation; (b) collection of data about suicidal ideation in a method that was masked to the patient's suicide (either by a masking method or by electronic recording at the point of assessment); (c) if all the suicides were defined using a mortality database; and (d) inclusion of open verdicts as suicide. Studies counting open verdicts were rated as having stronger reporting because open verdicts are thought to be similar to suicides in some jurisdictions16 and because the presence of a history of suicidal ideation might make a coronial verdict of suicide more likely.

Data synthesis

Random effects meta-analysis was chosen a priori for all estimates (OR, sensitivity, specificity, PPV) because of the diversity of study populations and the differences in methods used in the primary research. Between-study heterogeneity in effect sizes was examined using I2, Q-value statistics and prediction intervals. The possibility of publication bias was assessed using a funnel plot and Egger's regression,17 and was quantified using Duval & Tweedie's trim and fill method.18 Subgroup analysis (random effects within subgroups) and random-effects meta-regression (method of moments) were used to explore the extent to which between-study heterogeneity was explained by categorical and continuous variables (including the year of publication and the proportion of all participants with suicidal ideation).

Moderator variables (including strength of reporting score items) that were significantly associated with between-study heterogeneity in sensitivity and specificity at P≤0.05 were included in multiple meta-regression models. Multiple meta-regression was not used to examine heterogeneity in ORs because of the absence of significant moderator variables. Comprehensive Meta-Analysis (Version 3, Biostat, Englewood NJ) was used for the main analysis and a meta-analytic estimate of the ROC and AUC was calculated using Meta-DiSc.19

Results

Searches and data extraction

The initial searches yielded 320 potentially relevant papers that were examined in full text. There were 70 papers,2089 reporting 71 studies in which true positive, false positive, false negative and true negative could be ascertained. Eight differences in the effect size in the independent data extraction were resolved by re-examination of the data (Table 1 and see supplementary Table 2 for the data used in meta-analysis).

Table 1 Included studies

The earliest study was published in 1961 and the latest in 2017. The median publication date was 2006. In total, 42 studies had a case–control design and 29 were cohort studies. Of the studies, 58 studies were of current or former psychiatric patients including 42 studies of current or former psychiatric in-patients.

In 52 studies suicidal ideation was defined clinically and 19 studies used a structured method. Twelve studies reported a composite measure of suicidality (inclusive of suicide attempts). Four studies reported on suicide plans and two studies recorded a wish to die rather than active suicidal ideation.

The 71 studies included in total of 4 669 303 individuals who had been assessed for suicidal ideation (mean 65 765, median 235) of whom 5 811 died by suicide (mean 81.8, median 49). In total there were 2082 people in the true positive, 125 034 in the false positive, 3729 in the false negative and 4 538 458 in the true negative groups.

Meta-analyses

Meta-analysis of odds ratios

The pooled OR of suicide associated with suicidal ideation was 3.41 (95% CI 2.59–4.49; 95% prediction interval (PI) 0.42–28.1). There was very marked between-study heterogeneity, I 2 89.4, Q-value = 661, d.f.(Q) = 70, P<0.001. The range of ORs was 0.28 to 81, the first quartile,  2.03, median, 2.96 and the third quartile  6.45 (Fig. 2).

Fig. 2 Forest plot of the odds of the association between suicidal ideation and suicide.

This was associated with a meta-analytic AUC of 0.676 (s.e. = 0.021) (Fig. 3). The funnel plot appeared symmetrical (Fig. 4) but an Egger's regression was borderline significant (intercept 1.55, t-value = 2.07, d.f. = 69, P = 0.04). Duval & Tweedie's trim and fill did not identify any hypothetically missing studies.

Fig. 3 Meta-analytic area under the curve.

Fig. 4 Funnel plot of the odds of the association between suicidal ideation and suicide.

Structured methods to assess suicidal ideation were associated with a non-significantly lower OR (OR = 2.38, 95% CI 1.14–4.99) than clinically defined suicidal ideation (OR = 3.72, 95% CI 2.96–4.67, Q-value = 1.28, d.f.(Q) = 1, P = 0.26). There was no evidence that studies that reported on a composite measure of suicidality spanning suicidal behaviour and suicidal ideation (OR = 3.09, 95% CI 2.02–4.72) differed from studies that simply reported on suicidal ideation (OR = 3.54, 95% CI 2.57–4.88, Q = 0.26, d.f.(Q) = 1, P = 0.61). Suicide plans were more strongly associated with suicide (four studies, OR = 8.51, 95% CI 5.51–13.06) than studies that reported a wish to die (two studies, OR = 3.01, 95% CI 1.49–6.06) and studies that did not specify the degree of intent or planning (65 studies, OR = 3.2, 95% CI 2.41–4.29, Q-value = 14.6, d.f.(Q) = 2, P = 0.001)

Studies that were considered to have stronger reporting on the basis of a total strength of reporting score of ≥2 had a non-significantly weaker association (Q-value = 3.58, d.f.(Q) = 1, P = 0.06) between suicidal ideation and later suicide (n = 37, OR = 2.70, 95% CI 1.81–4.01) than studies with a strength of reporting score of <2 (n = 34, OR = 4.41, 95% CI 3.20–6.08).

Between-study heterogeneity was not significantly explained by any of our predetermined moderator variables (Table 2). Studies of non-psychiatric samples (n = 13, OR = 3.86, 95% CI 2.67–3.97) had similar ORs to studies of psychiatric patients (n = 58, OR = 3.23, 95% CI 2.64–3.96; Q-value = 0.119, d.f.(Q) = 1, P = 0.7). The main result and the analyses of moderators were not sensitive to the exclusion of two non-psychiatric studies with very large sample sizes that when combined included over 90% of individuals and 3.5% of the study weight in the random-effects meta-analysis.36, 57

Table 2 Meta-regression of moderators of the odds ratio of suicidal ideation for later suicide

a. 58 studies in the analysis.

b. 68 studies included in the analysis.

Meta-analysis of sensitivity

The pooled sensitivity of suicidal ideation for suicide was 41% (95% CI 35–48). There was very high between-study heterogeneity, I 2 = 93.2, Q-value = 1033, d.f.(Q) = 70, P≤0.001. The sensitivity ranged from 0 to 97%, first quartile  25%, median 44% and the third quartile  67%. An Egger's regression was non-significant (intercept 0.60, t-value = 0.61, d.f. = 69, P = 0.54).

Studies of non-psychiatric populations had a significantly lower sensitivity (n = 13, sensitivity  22%, 95% CI 13–53) than studies among psychiatric patients (n = 58, sensitivity  46%, 95% CI 40–52; Q-value = 10.6, d.f.(Q) = 1, P = 0.001). This result was not sensitive to the exclusion of the two non-psychiatric studies with very large sample sizes.

Use of a mortality database and more recent publication were associated with a lower sensitivity. Sensitivity was increased among studies of patients in hospital and in studies with a higher proportion of individuals with suicidal ideation (Table 3). The proportion of people with suicidal ideation was the only factor that was independently associated with between-study heterogeneity in a multiple meta-regression (supplementary Table 3). The multiple meta-regression model suggested that the proportion of people with suicidal ideation was the only independent moderator and explained 58% of the between-study heterogeneity in sensitivity.

Table 3 Meta-regression of moderators of the sensitivity of suicidal ideation for later suicide

a. 58 studies included in the analysis.

b. 68 studies in the analysis.

Meta-analysis of specificity

The pooled specificity of suicidal ideation for later suicide was 86% (95% CI 76–92). There was very high between-study heterogeneity, I 2 = 99.9, Q-value >10 000, d.f. (Q) = 70, P≤0.001. The specificity ranged from 28 to 100%, first quartile 69%, median  83% and the third quartile  93%. An Egger's regression was non-significant (intercept  −7.27 t-value = 0.74, d.f. = 69, P = 0.46).

Studies of non-psychiatric populations had a higher specificity (n = 13, specificity 96%, 95% CI 84–99) than studies of psychiatric populations (n = 58, specificity  81%, 95% CI 76–85, Q-value = 4.78, d.f.(Q) = 1, P = 0.03). This result was not sensitive to the exclusion of the non-psychiatric studies with very large sample sizes.

Suicidal ideation was a less specific indicator of suicide among studies with a high proportion of people with suicidal ideation and among samples of hospital-treated patients. A multivariate model found that hospital treatment and the proportion of people with suicidal ideation in the study population were significant variables, explaining 79% of the between-study heterogeneity in specificity. (Table 4 and supplementary Table 4).

Table 4 Meta-regression tests of potential moderators of the specificity of suicidal ideation for later suicide

a. 58 studies in the analysis.

b. 68 studies included in the analysis.

Meta-analysis of PPV

The pooled PPV for suicidal ideation as a test for later suicide among the 29 cohort studies was 1.7% (95% CI 0.9–3.2) over an average duration of follow-up of 9.1 years. There was very high between-study heterogeneity, I 2 = 97.9, Q-value = 1186, d.f.(Q) = 28, P≤0.001. An Egger's regression was non-significant (intercept 2.81, t-value = 1.13, d.f. = 27, P = 0.27). The PPV among nine non-psychiatric cohorts (0.3% 95% CI 0.1–0.5) was significantly lower (Q-value = 35.6, d.f.(Q) = 1, P<0.001) than among psychiatric samples (3.9% 95% CI 2.2–6.6).

Discussion

Main findings

Clinicians sometimes rely on suicidal ideation as a crucial test for short-term suicide risk,4 and it has been argued that asking about suicidal ideation could form part of a screening test for later suicide.5, 6 Our finding of a pooled OR of suicide associated with suicidal ideation of 3.41 and the meta-analytic AUC of 0.676 indicate a statistical association between suicidal ideation and later suicide with moderate strength.90 However, this should be interpreted very cautiously because of the very high and largely unexplained heterogeneity between studies. Moreover, the low PPV of suicidal ideation for suicide, which flows from the low incidence of suicide outcomes, even among psychiatric cohorts, highlights the limitations of suicidal ideation as a practical test of future suicide.

We had hypothesised that suicidal ideation would be a more sensitive and specific test for suicide in non-psychiatric settings, where patients with suicidal ideation might have fewer other risk factors. We found no evidence that suicidal ideation is more strongly associated with suicide in non-psychiatric settings than in psychiatric settings. Contrary to our hypothesis we found that suicide ideation was a less sensitive test for later suicide in non-psychiatric settings, and that the prevalence of suicidal ideation is associated with the sensitivity for later suicide.

Our results can be compared with those of recent meta-analyses of the psychometric properties of suicide risk scales91, 92 and multivariate models of suicide risk,93 which found similar odds of suicide, sensitivity, specificity and PPV among studies conducted in psychiatric settings as well as similar between-study heterogeneity despite examining quite different study sets. Collectively, these studies highlight a high degree of uncertainty about the statistical strength of commonly used approaches to suicide risk assessment.

The current paper advances knowledge of the association between suicidal ideation and suicide by reporting the pooled sensitivity and specificity of suicidal ideation for suicide. The main finding is the limited sensitivity of suicidal ideation for suicide, such that approximately 60% of people who go on to die by suicide have not expressed suicidal ideation at a specified earlier time. In contrast, the non-expression of suicidal ideation is fairly specific to not dying by suicide in the study period, with only 14% of those not dying by suicide expressing suicidal ideation.

Implications

Knowledge of the sensitivity and specificity of suicidal ideation for suicide is important because any given OR can be achieved with differing trade-offs between sensitivity and specificity. In this study we found firm evidence that such trade-offs differ between studies, in part as a result of the proportion of individuals expressing suicidal ideation. The proportion of people with suicidal ideation was strongly associated with sensitivity and was inversely associated with specificity, such that similar odds of suicide were found in populations with higher rates of suicidal ideation (i.e. psychiatric patients) and lower rates of suicidal ideation (i.e. non-psychiatric populations).

In general, screening tests should be sensitive enough to capture most cases.11 Our study suggests that suicidal ideation is not sensitive enough to be very helpful as a stand-alone screening test for suicide in psychiatric or non-psychiatric settings, a limitation that is compounded by the modest specificity. Clinicians should not assume that patients experiencing mental distress without suicidal ideation are not at elevated risk of suicide. It is notable that our subgroup analysis of non-psychiatric settings found that nearly 80% of people who later die by suicide had not expressed suicidal ideation. Conversely suicidal ideation is somewhat specific to suicide, particularly in settings where the prevalence of suicidal ideation is low. In this sense, the presence of suicidal ideation conveys more salient information about later suicide than the absence of suicidal ideation.

The finding that the proportion of individuals with suicidal ideation was strongly associated with sensitivity and inversely associated with specificity requires some consideration. To some extent this result is a logical consequence of the definition of sensitivity as the proportion of all cases detected – hypothetically, an extremely sensitive test that included the slightest degree of suicidal ideation would likely select almost the entire population and would approach a 100% sensitivity. Although variation in the threshold test for suicidal ideation might have contributed to variation in the prevalence of suicidal ideation between studies in non-extreme examples with typical rates of suicidal ideation, a very wide variation in the proportion of people with suicidal ideation can be associated with a wide variation of sensitivities.

In addition to highlighting uncertainty in the strength of the association between suicidal ideation and suicide, this study highlights an important clinical dilemma about the trade-off between sensitivity and specificity. Although detailed questioning about suicidal ideation is often indicated, for example when there is suspicion of a suicide plan, and such questioning might bring about a greater sensitivity, it might be associated with a higher false positive rate than less detailed questioning.

Limitations

In addition to the main limitation posed by the extent of between-study heterogeneity, this meta-analysis has a number of other limitations. In most studies there was a lack of clarity about how suicidal ideation was determined and defined. However, this limitation might not have had very much impact on our results because we found little evidence that structured methods of assessing suicidal ideation predicted suicide more accurately than clinical methods. Although it is logical that there may be important threshold issues on the spectrum of severity of suicidal ideation – a possibility supported by our finding of an association between the rate of suicidal ideation and the sensitivity of suicidal ideation for later suicide – there were too few studies that reported different points on the spectrum of ideation to properly explore threshold issues. Another limitation is that we lacked any data about how factors such as gender, comorbid conditions and wider cultural factors might affect the association between suicidal ideation and suicide.

Our results should also be interpreted in light of the knowledge that suicidal ideation can fluctuate over short periods of time.94 The primary studies we included measured suicidal ideation at the beginning of a follow-up period, often a follow-up of years. In cases of individuals who died by suicide without earlier suicidal ideation it is likely that suicidal ideation emerged closer to the time of the suicide. It remains to be seen if real time or more temporally proximal measures of suicidal ideation will be more strongly associated with later suicide.95

In conclusion, enquiring about suicidal ideation will always be a central skill for mental health professionals, not because suicidal ideation is a meaningful forecast of suicide but because patients who express suicidal ideation are making important communications about their inner world and level of psychological distress. However, clinicians should be aware of the statistical limitations of ideation as a screening tool, and not be lured into either a false confidence generated by an absence of ideation, or determinism about the likelihood of suicide if it is present.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjo.2018.88.

References

1Weber, AN, Michail, M, Thompson, A, Fiedorowicz, JG. Psychiatric emergencies: assessing and managing suicidal ideation. Med Clin North Am 2017; 101: 553–71.
2Rives, W. Emergency department assessment of suicidal patients. Psychiatr Clin North Am 1999; 22: 779–87.
3Hawley, CJ, James, DV, Birkett, PL, Baldwin, DS, de Ruiter, MJ, Priest, RG. Suicidal ideation as a presenting complaint. Associated diagnoses and characteristics in a casualty population. Br J Psychiatry 1991; 159: 232–8.
4Morgan, HG, Stanton, R. Suicide among psychiatric in-patients in a changing clinical scene. Suicidal ideation as a paramount index of short-term risk. Br J Psychiatry 1997; 171: 561–3.
5Boudreaux, ED, Camargo, CA Jr, Arias, SA, Sullivan, AF, Allen, MH, Goldstein, AB, et al. Improving suicide risk screening and detection in the emergency department. Am J Prev Med 2016; 50: 445–53.
6Horowitz, LM, Snyder, D, Ludi, E, Rosenstein, DL, Kohn-Godbout, J, Lee, L, et al. Ask suicide-screening questions to everyone in medical settings: the asQ'em Quality Improvement Project. Psychosomatics 2013; 54: 239–47.
7Chapman, CL, Mullin, K, Ryan, CJ, Kuffel, A, Nielssen, O, Large, MM. Meta-analysis of the association between suicidal ideation and later suicide among patients with either a schizophrenia spectrum psychosis or a mood disorder. Acta Psychiatr Scand 2015; 131: 162–73.
8Hubers, AA, Moaddine, S, Peersmann, SH, Stijnen, T, van Duijn, E, van der Mast, RC, et al. Suicidal ideation and subsequent completed suicide in both psychiatric and non-psychiatric populations: a meta-analysis. Epidemiol Psychiatr Sci 2018; 27:186–98.
9Ribeiro, JD, Franklin, JC, Fox, KR, Bentley, KH, Kleiman, EM, Chang, BP, et al. Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: a meta-analysis of longitudinal studies. Psychol Med 2016; 46: 225–36.
10Franklin, JC, Ribeiro, JD, Fox, KR, Bentley, KH, Kleiman, EM, Huang, X, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychologic Bull 2017; 143: 187232.
11Wilson, J, Jungner, G. Principles and Practice of Screening for Disease. World Health Organization, 1968.
12Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6: e1000097.
13Large, M, Smith, G, Sharma, S, Nielssen, O, Singh, SP. Systematic review and meta-analysis of the clinical factors associated with the suicide of psychiatric in-patients. Acta Psychiatr Scand 2011; 124: 1829.
14Large, M, Sharma, S, Cannon, E, Ryan, C, Nielssen, O. Risk factors for suicide within a year of discharge from psychiatric hospital: a systematic meta-analysis. Aust N Z J Psychiatry 2011; 45: 619–28.
15Wells, GA, Shea, B, O'Connnell, D, Peterson, J, Welch, V, Losos, M, et al. The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analysis. The Ottawa Hospital Research Institute, 2013 (http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp).
16Linsley, KR, Schapira, K, Kelly, TP. Open verdict v. suicide - importance to research. Br J Psychiatry 2001; 178: 465–8.
17Egger, M, Davey Smith, G, Schneider, M, Minder, C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629–34.
18Duval, S, Tweedie, R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 2000; 56: 455–63.
19Zamora, J, Abraira, V, Muriel, A, Khan, K, Coomarasamy, A. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol 2006; 6: 31.
20Allebeck, P, Varla, A, Kristjansson, E, Wistedt, B. Risk factors for suicide among patients with schizophrenia. Acta Psychiatr Scand 1987; 76: 414–9.
21Al-Sayegh, H, Lowry, J, Polur, RN, Hines, RB, Liu, F, Zhang, J. Suicide history and mortality: a follow-up of a national cohort in the United States. Arch Suicide Res 2015; 19: 3547.
22Appleby, L, Dennehy, JA, Thomas, CS, Faragher, EB, Lewis, G. Aftercare and clinical characteristics of people with mental illness who commit suicide: a case-control study. Lancet 1999; 353: 1397–400.
23Baader-Matthei, T, Richter, P, Mundt, C. Suicides of psychiatric hospitalized patients and their risk factors. A Case-control study. Rev Chil Neuro-Psiquiat 2004; 42: 293316.
24Beck, AT, Brown, GK, Steer, RA, Dahlsgaard, KK, Grisham, JR. Suicide ideation at its worst point: a predictor of eventual suicide in psychiatric outpatients. Suicide Life Threat Behav 1999; 29: 19.
25Beisser, AR, Blanchette, JE. A study of suicides in a mental hospital. Dis Nerv Syst 1961; 22: 365–9.
26Berg, JE. Death by suicide long after electroconvulsive therapy. Is the sense of coherence test of Antonovsky a predictor of mortality from depression? Ment Illn 2010; 2: e3.
27Bertelsen, M, Jeppesen, P, Petersen, L, Thorup, A, Ohlenschlaeger, J, le Quach, P, et al. Suicidal behaviour and mortality in first-episode psychosis: the OPUS trial. Br J Psychiatry 2007; 191 (suppl 51): s1406.
28Bickley, H, Hunt, IM, Windfuhr, K, Shaw, J, Appleby, L, Kapur, N. Suicide within two weeks of discharge from psychiatric inpatient care: a case-control study. Psychiatr Serv 2013; 64: 653–9.
29Blumenthal, S, Bell, V, Neumann, NU, Schuttler, R, Vogel, R. Mortality and rate of suicide of first admission psychiatric patients. A 5–year follow-up of a prospective longitudinal study. Psychopathology 1989; 22: 50–6.
30Borg, SE, Stahl, M. Prediction of suicide. A prospective study of suicides and controls among psychiatric patients. Acta Psychiatr Scand 1982; 65: 221–32.
31Bradvik, L, Berglund, M. Risk factors for suicide in melancholia. A case-record evaluation of 89 suicides and their controls. Acta Psychiatr Scand 1993; 87: 306–11.
32Cheng, KK, Leung, CM, Lo, WH, Lam, TH. Risk factors of suicide among schizophrenics. Acta Psychiatr Scand 1990; 81: 220–4.
33Conlon, L, Garland, M, Prescott, P, Mannion, L, Leonard, M, Fahy, TJ. Psychiatric aftercare and suicide risk: a case-control study using blind rating. Arch Suicide Res 2007; 11: 291–5.
34Coryell, W, Young, EA. Clinical predictors of suicide in primary major depressive disorder. J Clin Psychatry 2005; 66: 412–7.
35Coryell, W, Kriener, A, Butcher, B, Nurnberger, J, McMahon, F, Berrettini, W, et al. Risk factors for suicide in bipolar I disorder in two prospectively studied cohorts. J Affect Disord 2016; 190: 15.
36Crandall, C, Fullerton-Gleason, L, Aguero, R, LaValley, J. Subsequent suicide mortality among emergency department patients seen for suicidal behavior. Acad Emerg Med 2006; 13: 435–42.
37De Hert, M, McKenzie, K, Peuskens, J. Risk factors for suicide in young people suffering from schizophrenia: a long-term follow-up study. Schizophr Res 2001; 47: 127–34.
38Didham, R, Dovey, S, Reith, D. Characteristics of general practitioner consultations prior to suicide: a nested case-control study in New Zealand. N Z Med J 2006; 119: U2358.
39Dingman, CW, McGlashan, TH. Discriminating characteristics of suicides. Chestnut Lodge follow-up sample including patients with affective disorder, schizophrenia and schizoaffective disorder. Acta Psychiatr Scand 1986; 74: 91–7.
40Dobscha, SK, Denneson, LM, Kovas, AE, Teo, A, Forsberg, CW, Kaplan, MS, et al. Correlates of suicide among veterans treated in primary care: case-control study of a nationally representative sample. J Gen Intern Med 2014; 29 (suppl 4): 853–60.
41Dong, JY, Ho, TP, Kan, CK. A case-control study of 92 cases of in-patient suicides. J Affect Disord 2005; 87: 91–9.
42Drake, RE, Gates, C, Cotton, PG. Suicide among schizophrenics: a comparison of attempters and completed suicides. Br J Psychiatry 1986; 149: 784–7.
43Dutta, R, Boydell, J, Kennedy, N, Van Os, J, Fearon, P, Murray, RM. Suicide and other causes of mortality in bipolar disorder: a longitudinal study. Psychol Med 2007; 37: 839–47.
44Farberow, NL, Shneidman, ES, Neuringer, C. Case history and hospitalization factors in suicides of neuropsychiatric hospital patients. J Nerv Ment Dis 1966; 142: 3244.
45Fernando, S, Storm, V. Suicide among psychiatric patients of a district general hospital. Psychol Med 1984; 14: 661–72.
46Flood, RA, Seager, CP. A retrospective examination of psychiatric case records of patients who subsequently committed suicide. Br J Psychiatry 1968; 114: 443–50.
47Fosse, R, Ryberg, W, Carlsson, MK, Hammer, J. Predictors of suicide in the patient population admitted to a locked-door psychiatric acute ward. PLoS One 2017; 12: e0173958.
48Fowler, R, Tsuang, MT, Kronfol, Z. Communication of suicidal intent and suicide in unipolar depression. J Affect Disord 1979; 1: 219–25.
49Fruehwald, S, Matschnig, T, Koenig, F, Bauer, P, Frottier, P. Suicide in custody: case-control study. Br J Psychiatry 2004; 185: 494–8.
50Funahashi, T, Ibuki, Y, Domon, Y, Nishimura, T, Akehashi, D, Sugiura, H. A clinical study on suicide among schizophrenics. Psychiatry Clin Neurosci 2000; 54: 173–9.
51Goldstein, RB, Black, DW, Nasrallah, A, Winokur, G. The prediction of suicide. Sensitivity, Specificity, and predictive value of a multivariate model applied to suicide among 1906 patients with affective disorders. Arch Gen Psychiatry 1991; 48: 418–22.
52Green, KL, Brown, GK, Jager-Hyman, S, Cha, J, Steer, RA, Beck, AT. The predictive validity of the beck depression inventory suicide item. J Clin Psychatry 2015; 76: 1683–6.
53Hoyer, EH, Licht, RW, Mortensen, PB. Risk factors of suicide in inpatients and recently discharged patients with affective disorders. A case-control study. Eur Psychiatry 2009; 24: 317–21.
54Hunt, IM, Kapur, N, Webb, R, Robinson, J, Burns, J, Turnbull, P, et al. Suicide in current psychiatric in-patients: a case-control study. The National Confidential Inquiry into Suicide and Homicide. Psychol Med 2007; 37: 831–7.
55Hunt, IM, Bickley, H, Windfuhr, K, Shaw, J, Appleby, L, Kapur, N. Suicide in recently admitted psychiatric in-patients: a case-control study. J Affect Disord 2013; 144: 123–8.
56Hunt, IM, Kapur, N, Webb, R, Robinson, J, Burns, J, Shaw, J, et al. Suicide in recently discharged psychiatric patients: a case-control study. Psychol Med 2009; 39: 443–9.
57Hyman, J, Ireland, R, Frost, L, Cottrell, L. Suicide incidence and risk factors in an active duty US military population. Am J Public Health 2012; 102 (suppl 1): S13846.
58Kan, CK, Ho, TP, Dong, JY, Dunn, EL. Risk factors for suicide in the immediate post-discharge period. Soc Psychiatry Psychiatr Epidemiol 2007; 42: 208–14.
59Kessler, RC, Warner, CH, Ivany, C, Petukhova, MV, Rose, S, Bromet, EJ, et al. Predicting suicides after psychiatric hospitalization in US army soldiers: the army study to assess risk and resilience in servicemembers (Army STARRS). JAMA Psychiatry 2015; 72: 4957.
60Khang, YH, Kim, HR, Cho, SJ. Relationships of suicide ideation with cause-specific mortality in a longitudinal study of South Koreans. Suicide Life Threat Behav 2010; 40: 465–75.
61Khanra, S, Mahintamani, T, Bose, S, Khess, C, Umesh, S, Ram, D. Inpatient suicide in a psychiatric hospital: a nested case-control study. Indian J Psychol Med 2016; 38: 571–6.
62Kim, HM, Smith, EG, Ganoczy, D, Walters, H, Stano, CM, Ilgen, MA, et al. Predictors of suicide in patient charts among patients with depression in the Veterans Health Administration health system: importance of prescription drug and alcohol abuse. J Clin Psychatry 2012; 73: e126975.
63Kjelsberg, E, Neegaard, E, Dahl, AA. Suicide in adolescent psychiatric inpatients: incidence and predictive factors. Acta Psychiatr Scand 1994; 89: 235–41.
64Kleiman, EM, Liu, RT. Prospective prediction of suicide in a nationally representative sample: religious service attendance as a protective factor. Br J Psychiatry 2014; 204: 262–6.
65Li, J, Ran, MS, Hao, Y, Zhao, Z, Guo, Y, Su, J, et al. Inpatient suicide in a Chinese psychiatric hospital. Suicide Life Threat Behav 2008; 38: 449–55.
66Lin, SK, Hung, TM, Liao, YT, Lee, WC, Tsai, SY, Chen, CC, et al. Protective and risk factors for inpatient suicides: a nested case-control study. Psychiatry Res 2014; 217: 54–9.
67Lopez-Morinigo, JD, Ayesa-Arriola, R, Torres-Romano, B, Fernandes, AC, Shetty, H, Broadbent, M, et al. Risk assessment and suicide by patients with schizophrenia in secondary mental healthcare: a case-control study. BMJ Open 2016; 6: e011929.
68Louzon, SA, Bossarte, R, McCarthy, JF, Katz, IR. Does suicidal ideation as measured by the PHQ-9 predict suicide among VA patients? Psychiatr Serv 2016; 67: 517–22.
69Lukaschek, K, Baumert, J, Krawitz, M, Erazo, N, Forstl, H, Ladwig, KH. Determinants of completed railway suicides by psychiatric in-patients: case-control study. Br J Psychiatry 2014; 205: 398406.
70McEwan, T, Mullen, P, MacKenzie, R. Suicide among stalkers. J Forens Psychiatry Psychol 2010; 4: 514–20.
71Mock, CN, Grossman, DC, Mulder, D, Stewart, C, Koepsell, TS. Health care utilization as a marker for suicidal behavior on an American Indian Reservation. J Gen Intern Med 1996; 11: 519–24.
72Motto, JA, Bostrom, A. Empirical indicators of near-term suicide risk. Crisis 1990; 11: 52–9.
73Murphy, GE, Wetzel, RD. The lifetime risk of suicide in alcoholism. Arch Gen Psychiatry 1990; 47: 383–92.
74Park, EH, Hong, N, Jon, DI, Hong, HJ, Jung, MH. Past suicidal ideation as an independent risk factor for suicide behaviours in patients with depression. Int J Psychiatry Clin Pract 2017; 21: 24–8.
75Powell, J, Geddes, J, Deeks, J, Goldacre, M, Hawton, K. Suicide in psychiatric hospital in-patients. Risk factors and their predictive power. Br J Psychiatry 2000; 176: 266–72.
76Roy, A. Suicide in chronic schizophrenia. Br J Psychiatry 1982; 141: 171–7.
77Salama, AA. Depression and suicide in schizophrenic patients. Suicide Life Threat Behav 1988; 18: 379–84.
78Sani, G, Tondo, L, Koukopoulos, A, Reginaldi, D, Kotzalidis, GD, Koukopoulos, AE, et al. Suicide in a large population of former psychiatric inpatients. Psychiatry Clin Neurosci 2011; 65: 286–95.
79Shah, AK, Ganesvaran, T. Inpatient suicides in an Australian mental hospital. Aust N Z J Psychiatry 1997; 31: 291–8.
80Sharma, V, Persad, E, Kueneman, K. A closer look at inpatient suicide. J Affect Disord 1998; 47: 123–9.
81Simon, GE, Rutter, CM, Peterson, D, Oliver, M, Whiteside, U, Operskalski, B, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv 2013; 64: 1195–202.
82Sinclair, JM, Harriss, L, Baldwin, DS, King, EA. Suicide in depressive disorders: a retrospective case-control study of 127 suicides. J Affect Disord 2005; 87: 107–13.
83Sinclair, JM, Mullee, MA, King, EA, Baldwin, DS. Suicide in schizophrenia: a retrospective case-control study of 51 suicides. Schizophrenia Bull 2004; 30: 803–11.
84Spiessl, H, Hubner-Liebermann, B, Cording, C. Suicidal behaviour of psychiatric in-patients. Acta Psychiatr Scand 2002; 106: 134–8.
85Stephens, JH, Richard, P, McHugh, PR. Suicide in patients hospitalized for schizophrenia: 1913–1940. J Nerv Ment Dis 1999; 187: 10–4.
86Thong, JY, Su, AH, Chan, YH, Chia, BH. Suicide in psychiatric patients: case-control study in Singapore. Aust N Z J Psychiatry 2008; 42: 509–19.
87Weiser, M, Goldberg, S, Werbeloff, N, Fenchel, D, Reichenberg, A, Shelef, L, et al. Risk of completed suicide in 89,049 young males assessed by a mental health professional. Eur Neuropsychopharmacol 2016; 26: 341–9.
88Werbeloff, N, Dohrenwend, BP, Levav, I, Haklai, Z, Yoffe, R, Large, M, et al. Demographic, behavioral, and psychiatric risk factors for suicide. Crisis 2016; 37: 104–11.
89Yim, PH, Yip, PS, Li, RH, Dunn, EL, Yeung, WS, Miao, YK. Suicide after discharge from psychiatric inpatient care: a case-control study in Hong Kong. Aust N Z J Psychiatry 2004; 38: 6572.
90Rosenthal, J. Qualitative descriptors of strength of association and effect size. J Soc Serv Res 1996; 21: 3757.
91Carter, G, Milner, A, McGill, K, Pirkis, J, Kapur, N, Spittal, MJ. Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales. Br J Psychiatry 2017; 210: 387395.
92Chan, MK, Bhatti, H, Meader, N, Stockton, S, Evans, J, O'Connor, RC, et al. Predicting suicide following self-harm: systematic review of risk factors and risk scales. Br J Psychiatry 2016; 209: 277–83.
93Large, M, Kaneson, M, Myles, N, Myles, H, Gunaratne, P, Ryan, C. Meta-analysis of longitudinal cohort studies of suicide risk assessment among psychiatric patients: heterogeneity in results and lack of improvement over time. PLoS One 2016; 11: e0156322.
94Williams, CL, Davidson, JA, Montgomery, I. Impulsive suicidal behavior. J Clin Psychol 1980; 36: 90–4.
95Kleiman, EM, Turner, BJ, Fedor, S, Beale, EE, Huffman, JC, Nock, MK. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J Abnorm Psychol 2017; 126: 726–38.