Mistakes are inevitable in any branch of medicine, but psychiatry is a particularly risky business (Reference HollowayHolloway, 1997). When psychiatrists get it wrong there may serious consequences for their patients, the clinical team and the wider public. The Government introduced a series of initiatives in the 1990s: the Care Programme Approach (1990), the supervision register (Department of Health & Home Office, 1994) and supervised discharge (Secretary of State for Health, 1997). One of the main purposes of this legislation was to minimise the risk psychiatric patients pose to the community. Future service provision will be shaped by clinical governance and the National Service Framework for Mental Health (Secretary of State for Health, 1997), and evaluation and management of risk will become increasingly important.
Evaluating risk is part of everyday practice. For example, we often need to weigh the risk of harmful behaviour against the benefits of discharging individuals from hospital, or the risk of side-effects against the potential therapeutic benefits of a course of treatment. Holloway (1997) highlights one day's risk decisions in the life of a general psychiatrist. Psychiatric practice could be likened to a minefield. The question is, how do we negotiate it? How do we turn Òan impossible burden” into Òa demanding but intellectually rewarding task”(Reference HollowayHolloway, 1998)? In this paper I will discuss the concept of risk and different approaches to evaluating it, as well as considering some of the practical aspects of assessing and managing risk.
What is risk?
The word ‘risk’ came into the English language from Spanish or Portuguese, where it referred to sailing into uncharted waters (Reference WilliamsWilliams, 1999). Latterly, it has been used by insurance and finance companies to express how likely a policy holder is to make a claim. Risk is simply the probability or likelihood of a particular event occurring. Perhaps because of human nature, it usually refers to an unwelcome outcome (e.g. disease onset, relapse, death or disability), but it is just as valid a concept for welcome outcomes (prevention, remission or recovery). When discussing risk, the outcome of interest needs to be clearly specified. On an individual level this might include the risk of suicide, side-effects from medication, offending behaviour or violence and aggression. Alternatively, we might wish to consider wider outcomes, such as the potential effects on the public or even the risks to the reputation of individual clinicians and the standing of the clinical service (Reference SnowdenSnowden, 1997).
Risk tends to be expressed in binary terms, with a person said to be at either ‘high risk’ or ‘low risk’ of a particular disorder. This is because most medical decisions and treatments are binary in nature. Our prescribing options for lithium in the prophylaxis of bipolar disorder (to prescribe or not prescribe) depend on our assessment of the patient's risk of relapse (high or low). In fact, classifications of high- and low-risk groups may be largely arbitrary, since most risks are distributed continuously throughout populations (Reference RoseRose, 1992). However, it has been argued that such dimensional approaches to risk are not clinically useful (Reference Kraemer, Kazdin and OffordKraemer et al, 1997).
Risk as a dynamic concept
Risk is not static. It varies between populations and across age ranges. For example, marriage is a risk factor for suicide among teenage girls, but a protective factor among adult women (Reference Bancroft, Skrimshire and ReynoldsBancroft et al, 1975; Reference HawtonHawton, 1986). Even for an individual, identified risks can increase or decrease over time, and the nature of the risks may change (Reference SnowdenSnowden, 1997). This has important implications for risk assessment, which may be no more than a ‘snapshot’ at one particular moment in time. Risk also varies depending on the stage of the disorder we are considering. The risk factors for a first onset of major depressive disorder are different from those for relapse, which in turn differ from those for remission or recovery.
Expressing risk
The degree of risk can be expressed in several ways. A clinical classification might simply use categories (high and low risk). Measures of risk that can be calculated from epidemiological studies include attributable risk (the disease rate in persons exposed to the risk factor of interest minus the disease rate in unexposed persons) and relative risk (the ratio of disease rate in exposed persons to those not exposed). Attributable risk is the measure of association that is most relevant for making decisions about individuals, but relative risks are more generally available because they can be estimated from a wider range of study designs. Relative risks can be misleading. For example, the relative risk of suicide in the year following self-harm is 100 (Reference Hawton and FaggHawton & Fagg 1988). This seems impressive, but because suicide is a rare outcome it means that if self-harm is used as a basis on which to predict future suicide, the prediction will be correct less than 1% of the time. The population attributable risk is the rate of disease (or outcome of interest) in the population minus the rate that would apply if all of the population were unexposed. It measures the potential impact of control measures in a population. For example, it has been calculated that if the USA were able to prevent schizophrenia completely, the amount of crime in the population would decrease by only 3% (Wesseley, 1997).
Quantifying risk
Research suggests that we are not terribly proficient at quantifying and predicting risk. For example, when predicting violent behaviour one study shows that mental health professionals are wrong 95% of the time (Reference EnnisEnnis, 1972). It could be argued that we might be better off simply assuming no one will have the (generally rare) outcome of interest, in which case we will be right more often than not (Reference GrubinGrubin, 1997). It may also be that specialist skills are not a prerequisite for accurate risk prediction. Teachers have been found to do almost as well as psychiatrists in simulated assessments (Reference Quinsey and AmbtmanQuinsey & Ambtman, 1979). Central to the task of improving the risk assessment skills of clinicians has been the debate regarding the relative merits of actuarial and clinical methods of risk evaluation.
Actuarial approaches to risk
Statisticians use ‘actuarial’ to refer to mathematical techniques they apply to set insurance premiums and pension schemes. In the psychiatric risk assessment literature the term has been used to describe any mathematical means of combining information (Reference BuchananBuchanan, 1999).
A recent study of suicide in hospital in-patients provides a good example of an actuarial approach to risk (Reference Powell, Geddes and DeeksPowell et al, 2000). When 112 in-patient suicides were compared with 112 controls, five factors were found to distinguish particularly strongly between the two groups: recent bereavement, presence of delusions, suicidal ideation, chronic mental illness and a family history of suicide. Ideally, these risk factors would then have been used to predict suicide in an independent sample, but the authors tested the predictive power of the factors in the sample from which they were derived. This is less than ideal, but the results are still of interest (Fig. 1). Using a cut-off of two or more risk factors to denote ‘high risk’ of suicide, the specificity of the test (an indication of how well it identifies non-suicides) is very good, at 89/90 or 99%. However, its sensitivity (an indication of how well it identifies suicides) is poor, with only 26/97 or 27% being correctly identified. In other words, we will fail to identify almost 75% of suicides using this test. Of course, we can lower our cut-off so that high-risk individuals are defined as having just one or more risk factors, but this will give an unacceptable number of false positives (non-suicidal patients identified as high risk).
(a) sensitivity (a/(a+c)), specificity (b/(b+d)) and predictive value (a/(a+b)) of a diagnostic test to assess the risk of suicide. (b) Powell et al's (2000) results for psychiatric in-patients
This illustrates a very important point regarding actuarial methods in psychiatry: their usefulness is limited because most of our assessment tools have limited specificity and sensitivity, and the outcomes we are interested in have a low incidence. This means that the positive predictive value of actuarial assessments (the proportion of individuals identified as high risk who are actually high risk) is often disappointingly low, and many patients would need to be treated to prevent one adverse outcome. For example, figures presented by Geddes (1999) indicate that even with a ‘perfect’ treatment (i.e. one that is 100% effective) and a very good risk assessment tool (one with 80% sensitivity and specificity) we would have to treat over 100 high-risk patients following their discharge from hospital to prevent a single suicide.
There are other problems with actuarial techniques (Reference GrubinGrubin, 1997). These include a lack of flexibility, poor generalisability across patient groups and limited utility in everyday practice because clinicians often have only fragmentary information on which to base clinical decisions. Actuarial approaches also fail to take into account the individual circumstances of the patient and to provide an explanation of the behaviour.
Newer tools for actuarial assessment have overcome some of these difficulties. One such tool is the iterative classification tree (ICT) for assessing violence risk (Reference Monahan, Steadman and ApplebaumMonahan et al, 2000). In a USA study, 106 risk factors in four domains (personal, historical, situational, clinical) were measured at baseline in 939 in-patients. The patients were followed up for 20 months after discharge to investigate which factors were predictive of future violence. The abridged version of the ICT is designed for use in routine practice, and Box 1 shows some of the factors that it considers when predicting risk. The ICT is based on an interactive and contingent model of violence that permits consideration of combinations of risk factors in order to classify a person as high or low risk. The questions are asked according to a tree structure (determined in the original study by statistical consideration of the clustering of risk factors and patients). All subjects are asked a first question. The answer to this determines the next question on the tree and so on until each person is classified into a risk category. One of the advantages over traditional approaches is that this model explicitly acknowledges the fact that violence is an outcome that can be reached by multiple routes. The ICT seems to be superior to existing actuarial risk assessment tools, with only 15% of patients being assigned to an incorrect risk category, but it is still unclear whether the findings are generalisable to the practice of psychiatry in other health care settings.
Factors to consider when assessing violence risk using the iterative classification tree (Reference Monahan, Steadman and ApplebaumMonahan et al, 2000)
Previous convictions and seriousness of offence
Recent violent fantasies
Drug use by father
Motor impulsiveness
Substance misuse
Legal status
Major psychiatric disorder
Diagnosis of schizophrenia
Anger reaction
Employment status
Recent violence
Loss of consciousness
History of parental fighting
Clinical approaches to risk
Some view the clinical method of risk assessment as merely an informal and unsystematic version of actuarial approaches and describe clinical experience as Òa prestigious synonym for anecdotal evidence” (Reference Grove and MeehlGrove & Meehl, 1996). It has been argued that clinicians either allocate patients to reference classes or assign weights to variables and combine these weights mathematically, just as is done using actuarial tables, but that the clinicans do it much less well.
In much of the research literature clinical risk assessment is defined by a process of exclusion and refers to any means of prediction that is not mathematical in nature. It is much more than this. It is a person-specific assessment, which takes into account past behaviour and the context in which it occurred (Reference VinestockVinestock, 1996). It refers to a Òbalanced summary of prediction derived from knowledge of the individual, the present circumstances and what is known of the disorder from which he is suffering” (Department of Health & Home Office, 1994). Central to this approach is a detailed understanding of the person's underlying mental state and psychopathology. The risks in each individual case need to be identified and then assessed in terms of frequency and severity. The assessment process should be multi-disciplinary and information should be obtained from all available sources (Reference SnowdenSnowden, 1997).
Clinical approaches undoubtedly have drawbacks. When clinicians are asked to predict adverse outcomes they almost invariably overpredict. This is probably because of the relative consequences of the two types of mistake that could be made (Reference BuchananBuchanan, 1999). For example, in the current risk-averse climate, most psychiatrists would probably rather detain someone who turns out not to be violent (a false positive) than discharge someone who subsequently commits a violent act (a false negative). This inevitably raises a number of ethical issues regarding the erosion of patients' civil liberties. How many false positives is society prepared to accept in the interests of ‘safety’?
Reports from recent inquiries give us a valuable insight into situations where clinical risk assessment has failed to prevent disasters in mental health services (i.e. false negatives). Some of the reasons for failure are listed in Box 2 (Reference Lipsedge and VincentLipsedge, 1995).
Why things go wrong with risk assessment (Reference Lipsedge and VincentLipsedge, 1995)
Failure to lend sufficient weight to reports by carers and members of the public about disturbed behaviour
Undue emphasis on the civil liberties of patients
Failure properly to implement the Mental Health Act
Tendency to take a cross-sectional rather than a long-term view of risk
Failure to share information in the best interests of the patient
Practical risk assessment
Perhaps the most sensible approach to risk assessment would be to combine both actuarial and clinical methods, using our knowledge of risk factors from epidemiological studies as well as the patient's individual characteristics, in order to inform our clinical decisions. Perhaps surprisingly this approach has not received much support in the literature, and the consensus seems to be that we need to come down on one side of the fence or the other. This may be short-sighted. Even if actuarial approaches become more sophisticated and categories of risk factor better defined, clinical skills will continue to play an important role in determining whether a patient falls into a particular category or not. As the phenomenon being measured becomes more unusual, the level of clinical skill required to measure it increases (Reference BuchananBuchanan, 1999).
A risk assessment framework
It may be that the role of risk assessment is not about making an accurate prediction, but about making informed, defensible decisions (Reference Grounds and CrichtonGrounds, 1995). The content of risk assessment varies according to the adverse outcome being assessed (e.g. the risk of harm to self or others) and has been comprehensively covered elsewhere (Reference Alberg, Hatfield and HuxleyAlberg, et al, 1996; Royal College of Psychiatrists, 1996; Reference VinestockVinestock, 1996).
Moore (1995) suggests a framework that can be applied to most risk assessments in psychiatry (Box 3). The behaviour to be predicted must be defined and each behaviour should be assessed individually, as each is likely to involve different risk factors. The assessing team needs to distinguish clearly between the probability (likelihood) of the behaviour and its cost (potentially serious consequences). Assessors must be aware of the possible sources of error in the assessment, arising from the patient, the assessors themselves and the context of the assessment. The interaction of internal factors (e.g. attitudes, drives, needs, controls) and external environmental factors (e.g. demands, constraints, stressors) in producing the target behaviour should be considered. A check should be made of whether all necessary information has been gathered. If it has not, what additional information is needed and where might it be obtained? A longitudinal perspective should be adopted, with some prediction of the factors and circumstances likely to increase or decrease future risk. Key interventions should be planned and a decision made about whether to involve other professional groups.
A risk assessment framework (Reference MooreMoore, 1995)
Define the behaviour to be predicted
Distinguish between the probability and the cost of the behaviour
Be aware of the possible sources of error in the assessment
Take into account the influence of both internal and external factors on the behaviour
Check that all the necessary information has been gathered
Predict factors likely to increase or decrease future risk
Identify when other professionals or agencies need to be involved
Plan key interventions
Too many decisions, too little time
Sometimes the realities of clinical practice mean that we do not have as much time as we would wish to make clinical decisions. We often have to act in less than ideal circumstances with less than comprehensive information. Three questions might help us to prioritise risk decisions (Reference Taylor, Hirsch and WeinbergerTaylor 1995): What is the seriousness of the risk? What is the imminence of the risk? What is the probability of the risk becoming actual?
Managing risk
In its simplest sense, clinical risk management describes the development of strategies to reduce the severity and frequency of identified risks and adverse outcomes (Reference VincentVincent, 1995). In psychiatry this might involve a clear record of the role and responsibilities of each professional involved with a patient, clinical audit of adverse incidents and predetermined plans of action when concerns arise (Reference SnowdenSnowden, 1997). Good-quality medical records, thorough note-keeping and open communication are key aspects of effective risk management, as is compliance with statutory requirements. Clinical guidelines, checklists and management protocols have the potential to reduce the risk associated with practice. On an organisational level, risk management requires a service that is adequately staffed with personnel who are fully trained in the current best clinical practice. Risks also need to be shared, with junior staff having adequate access to consultant advice.
Risk substitution
It could be argued that minimising a risk by effective risk management simply leads to a compensatory increase in other risk factors, and so is largely a waste of time. For example, it may be that changes to gun laws in the USA in the 1980s led to a compensatory increase in other methods of suicide. An alternative to this ‘risk substitution hypothesis’ is the ‘risk reduction hypothesis’. It has been estimated that 6700 suicides were prevented by the detoxification of domestic gas in the UK in the 1960s. These concepts are impossible to disentangle (Reference Gunnell and FrankelGunnell & Frankel, 1994) and there is little firm evidence one way or the other. However, policy-makers continue to be persuaded by the risk reduction hypothesis, as evidenced by recent legislation to limit the over-the-counter availability of paracetamol in the UK.
Evaluating risks and making decisions
Much of clinical practice involves us in evaluating and balancing the risks of different courses of action. ‘Decision analysis’ was originally developed in the business world and is an explicit quantitative approach to examining difficult decisions. Its aim is to enable clinicians to make the best decision for individuals or groups of patients (Reference HatcherHatcher, 1995).
Decision analysis involves assigning a probability to each potential clinical outcome. These probabilities represent the doctor's ‘best guess’ about how likely a particular outcome is, and are based on clinical features and data from relevant clinical studies. A utility is also assigned to each outcome. This represents a patient's preference for one clinical outcome over another. Utilities are given values from 0 to 1 (0=worse possible outcome, e.g. death; 1=best possible outcome, e.g. perfect health). The various decision points and their consequences, along with their associated probabilities and utilities, are mapped to form a decision tree. This is a visual representation of the decisions available. The probabilities and utilities of the various decisions are combined to determine the best treatment. The best treatment is the one with the highest expected utility (i.e. the treatment most likely to provide the optimal outcome for the patient). Figure 2 shows a simplified decision tree for prescribing antipsychotics in first episode schizophrenia with respect to the risk of tardive dyskinesia and recurrence of the disorder. In this example, the expected utility (value) of using antipsychotics or not is calculated by summing the product of probabilities and utility for all outcomes distal to the first branch of the tree. So the expected utility of not using antipsychotics is (0.7 × 0.45) + (0.3 × 1.0)=0.615; that of using antipsychotics is (0.15 × 0.58 × 0) + (0.15 × 0.42 × 0.75) + (0.85 × 0.58 × 0.45) + (0.85 × 0.42 × 1)=0.626. In this case it is better to treat with antipsychotics than not to.
Decision tree for prescribing antipsychotics in first-episode schizophrenia with respect to risk of tardive dyskinesia and recurrence of disorder (adapted from Hatcher, 1995). Probabilities (P) of different outcomes are based on research evidence and the clinician's ‘best guess’. Utilities were determined in this case by asking three clinicians. Utility=0 indicates the worse possible outcome, and utility=1 the best possible outcome
The use of decision analysis to evaluate risks in psychiatry has its problems. The quality of the psychiatric research database is poor – the evidence to guide us in certain situations simply does not exist. There may be difficulties in generalising decisions to other patients. The value, or utility, ascribed to outcomes is arbitrary and, especially when considering risk, may vary between those involved. The utility to the patient of a particular outcome may differ from its utility to the clinical team or wider society. Nevertheless, decision analysis might prove a useful adjunct to current methods of risk evaluation and certainly has a role to play in teaching, audit and research (Reference HatcherHatcher, 1995).
Alternatives to a ‘high-risk’ approach
Psychiatry is currently preoccupied with the concept of risk, and practice is based on what is essentially a ‘high-risk’ strategy. This involves applying risk assessment to a population, and then focusing intervention on those identified as being at greatest risk of poor outcomes (Reference RoseRose, 1992). The advantages of this strategy are clear. It offers a cost-effective use of resources, since only a minority of individuals is targeted. It avoids unnecessary and potentially harmful treatment of those who do not need it. It leads to interventions that are more likely to be tailored to the needs of the individual. Unfortunately, there are serious problems with an exclusively high-risk strategy. First, those in the lower-risk, higher-volume group are excluded even though they represent a larger absolute number of those with poor outcomes and, therefore, a greater population burden. Any intervention targeted at the high-risk group alone may make only a small contribution to reducing the overall incidence of the disease in the population. Second, instruments screening for the high risk may be expensive, impractical or lack predictive power. Third, the distinction between high- and low-risk groups may well be an artificial one.
The current management of deliberate self-harm is an example of a high-risk strategy, and it illustrates the disadvantages of this approach (Reference Kapur and HouseKapur & House, 1998). First, targeting high-risk groups results in large numbers of supposedly low-risk individuals being ‘defined out’ of care, reducing the impact of suicide prevention strategies. For example, those identified as being at high risk using one risk assessment scale account for only 26% of cases of future suicidal behaviour, the much larger low-risk group accounting for the remainder. With intervention restricted to the high-risk group, even assuming that it is totally effective (which is improbable), we will reduce the overall rate of suicidal behaviour by at most one-quarter. Second, available risk measures are poor at predicting repeated suicidal behaviour (positive predictive value 25% at best). Third, suicide risk may be continuously distributed in a population, in which case a dichotomous distinction between high- and low-risk groups is not valid.
The population-based strategy of prevention is an alternative to the high-risk approach, and it involves targeting whole populations rather than just vulnerable individuals. This is potentially powerful because of the number of individuals involved; even very small population shifts can have large effects. To use an example from general medicine, a fall of just 3% in the mean population blood pressure reduces the population prevalence of hypertension by 25% (Reference RoseRose, 1992). Population approaches are also radical since they seek to address underlying causes of a phenomenon rather than just its external manifestations. Again using the example of deliberate self-harm, a population strategy might involve: limiting the availability of methods of self-harm; a review of economic and social policy; and changing our clinical management to offer interventions to everyone following an episode of deliberate self-harm and not just those at high risk.
However, the implementation of population approaches may be unacceptable or prohibitively expensive, and is not always compatible with our medical model of managing health problems. A combination of population and high-risk strategies might be a more feasible and effective option (Reference Kapur and HouseKapur & House, 1998). For deliberate self-harm, this could take the form of providing a basic level of intervention for all patients (e.g. an emergency card listing contact numbers) and then using clinical and actuarial methods of risk assessment to identify patients who may benefit from more intensive treatment (e.g. interpersonal or problem-solving therapy).
Conclusion
In psychiatry, as in other fields of medicine, there has been a temptation to view risk as an entirely negative concept, but risk has its positive aspects as well (Reference HollowayHolloway, 1997). Risk can be viewed as a mobilising force for change in society (Reference WilliamsWilliams, 1999), and no risk may mean no progress. This is all very well, but it does not help us much on a practical level. What skills do we need to evaluate and manage risks and how might we best impart these to trainees? The traditional apprenticeship model is quite good at teaching the necessary clinical skills, provided that the senior clinician explains how decisions are made and encourages a questioning attitude. However, a clinical skills apprenticeship is not enough. It needs supplementing. Psychiatrists should have knowledge of epidemiological, actuarial and population perspectives of risk in order to deal more effectively with the risks that are an inherent part of clinical practice.
Multiple choice questions
-
1. Risk:
-
a refers to the probability of a particular event occurring
-
b is a static concept
-
c is always negative
-
d is usually expressed in categorical terms
-
e is relevant to psychiatric practice.
-
-
2. Actuarial approaches to risk assessment:
-
a consider the combined effects of groups of risk factors
-
b are extremely good at predicting risk
-
c may lack clinical relevance
-
d have been shown to be superior to simple clinical judgement
-
e are continuing to evolve.
-
-
3. Clinical approaches to risk assessment:
-
a are based purely on guesswork
-
b result in overestimates of risk
-
c are extremely good at predicting risk
-
d consider individual factors and past history
-
e are incompatible with actuarial methods.
-
-
4. Risk management:
-
a is not applicable to psychiatry
-
b is not influenced by staff or resource levels
-
c may be enhanced by clinical guidelines
-
d may involve the investigation of adverse incidents
-
e may be improved by use of decision analysis
-
-
5. Population-based strategies of risk prevention:
-
a target vulnerable individuals only
-
b are more powerful than high-risk approaches
-
c may be expensive to implement
-
d may not be compatible with the medical model of health care
-
e involve assigning individuals to high- or low-risk groups
-
MCQ answers
| 1 | 2 | 3 | 4 | 5 | |||||
| a | T | a | T | a | F | a | F | a | F |
| b | F | b | F | b | T | b | F | b | T |
| c | F | c | T | c | F | c | T | c | T |
| d | T | d | T | d | T | d | T | d | T |
| e | T | e | T | e | F | e | T | e | F |
eLetters
No eLetters have been published for this article.