Delphi process

The Delphi process recognises the value of experts' opinions, experience and intuition when full scientific knowledge is lacking. ^{Reference Linstone and Turoff3} It was developed, at the beginning of the cold war, as a systematic, interactive way for a panel of independent experts to forecast the impact of technology on warfare. ^{Reference Dalkey and Helmer4} A carefully selected group of experts answer surveys in two or more rounds. After each round, a facilitator provides an anonymous summary of the experts' views and their comments, allowing all participants to compare these with their own. The aim is that during this iterative process the range of the answers will decrease and the group will converge towards the ‘correct’ answer. ^{Reference Skulmoski, Hartman and Krahn5} Use of the Delphi process has resulted in widely accepted outcomes, including guidelines in the health field, without evidence of social conformism. ^{Reference Langlands, Jorm, Kelly and Kitchener6,Reference Campbell, Cantrill and Roberts7}

Statement development

The results of systematic reviews of the evidence in this field along with existing guidelines and components widely cited as of potential benefit (e.g. specific formal interventions) following disasters were used to develop 96 statements. The statements covered six different domains of disaster management: planning, initial response (first week), early response (first month), response between 1 and 3 months, human resources and specific interventions. An example of a statement from each domain is shown in Appendix 1. A full list of the statements is available from the author on request. The online survey tool Survey Monkey (www.surveymonkey.com/) was used to present the statements with clear instructions to indicate level of agreement or disagreement using a nine-point scale where five was neutral. Participants were also asked to provide comments regarding their score for each statement. An example is given in Fig. 1.

Fig. 1

Example of a statement in round one.

In the second round, the survey comprised the same statements but this time with summary statistics indicating the number and percentage of participants who rated each score and the mean score for each statement. A summary of the comments was inserted underneath each statement for participants to consider when completing the second round. Some statements were amended slightly for clarification as a result of round one comments. An example is shown in Fig. 2. Participants were also sent a list of the full comments and their own scores. Fifteen new statements suggested by participants during round one were added to the end of the round two survey.

Fig. 2

Example of a statement in round two.

Any statements that had achieved a positive consensus (mean score >7 and 70% participants scoring 7 or above) were removed for round three along with any statements that were unlikely to achieve a positive consensus (those with a mean score of <6). Statements that scored between 6 and 7 were retained to determine whether positive consensus could be achieved. These statements were presented in the same format as the round two statements to illustrate the summary statistics and comments of the second round. The statements included for the first time in round two were retained for round three to allow participants the opportunity to reconsider their responses in the light of the comments of other participants and the summary statistics.

Participants

Potential participants were identified by the TENTS partners as having an expertise in the trauma field. They included researchers, clinicians, emergency planners, providers of psychosocial care following traumatic events and disaster survivors using services. The aim was to invite key individuals from countries throughout Europe and internationally recognised experts to participate in the survey.

Analysis

Basic statistics were generated by the Survey Monkey software. SPSS version 14 for Windows was used to determine which statements achieved positive consensus. The comments were summarised by J.I.B. to include the themes that emerged.

Consultation

Following completion of the three rounds of the Delphi process, the statements that achieved positive consensus were used to compile draft guidelines. These were circulated to the participants to comment on. The comments received were summarised and then discussed at a meeting of the TENTS partners to agree the final wording of the guidelines.

Round one

One hundred and twenty-two potential participants were invited to take part in the first round. Of these, 106 (87%) responded and 17 (16.3%) identified themselves as researchers, 49 (47.1%) as researcher–clinicians, 20 (19.2%) as clinicians, 9 (8.7%) as policy makers, 3 (2.9%) as service users and 8 (7.5%) as others. Online Table DS1 shows the number of respondents by country.

There did not appear to be significant differences in agreement with statements between respondents from different backgrounds or between respondents from different areas of the world. Fifty-three (55%) of the original 96 statements achieved good consensus in round one and 15 new statements were formulated from respondents' suggestions. Twelve (12.5%) statements were amended slightly to clarify their meaning as a result of the comments.

Round two

Round two was sent to the 106 round one respondents and 92 (87%) completed it. Figure 3 shows the mean scores for rounds one and two showing an improvement in consensus for round two. Sixty-two (65%) statements had achieved positive consensus, 8 (8%) were in the 6–7 range and the remaining 26 (27%) had a mean score of less than 6. Six statements had a mean score of less than 3 and were considered to have achieved negative consensus. Thirteen (87%) of the 15 new statements achieved positive consensus. The three statements that participants agreed most strongly with were: ‘the psychosocial care plan should be incorporated into the overall disaster plan’; ‘the initial response requires practical, pragmatic support provided in an empathic manner’; and ‘community self-help and social support should be facilitated’. The three statements that participants disagreed most strongly with were: ‘mental health professionals have no role in the initial response’; ‘individuals should be discouraged from giving detailed accounts of their experiences’; ‘all responses should provide formal single session early interventions such as psychological debriefing’.

Fig. 3

Changes in mean scores for the 96 statements between rounds one and two.

Consultation

The draft guidelines developed from the Delphi process comprised a brief introductory paragraph followed by six sections that covered: planning, preparation and management; general components; and specific components to be included within the first week, within the first month, 1 to 3 months after the disaster and beyond 3 months. Thirty-three (31%) of the 106 round one respondents commented on the draft guidelines. The feedback received was very positive and included some suggested changes that primarily concerned the wording of the recommendations. The TENTS partners agreed that no statements should be altered in a way that contradicted what had been agreed during the Delphi process. Some minor changes to the wording of some statements were made. Some concerns were raised by partners that a number of the recommendations focused on individuals despite there being a strong consensus for a community-based approach. This was addressed by adding the following sentence to the introductory paragraph of the guidelines: ‘Several of the components focus on individuals but all aspects of psychosocial care should only be provided with full consideration of individuals’ wider social environment, especially their families and communities'. Appendix 2 lists the principal recommendations of the final guidelines, which are available in full on the TENTS website (www.tentsproject.eu).

Main findings

The absence of high-quality research to inform the development of evidence-based guidelines is a major problem and can result in guidelines that are little more than the unchallenged views of a few people. The Delphi process allowed the development of a consensus of items that should be included in a psychosocial care guideline following disasters by an independent group of 106 experts from 25 different countries. The process allowed individuals to challenge the views of others in a non-threatening, anonymous manner, and it was apparent that many participants' views changed to a degree as a result of the process. There were also statements where the consensus remained low suggesting that participants were not adjusting their responses to achieve conformity. The process allowed views to be aired regarding several issues that have been widely debated in the area, for example the use of single session psychological debriefing, producing some results that were not predictable, for example opposition to screening for everyone affected, as well as confirming a consensus that is largely consistent with what appears to be the prevailing view in this area.

Early universal intervention

The result of the Delphi process was a strong consensus against early application of formal intervention universally for all individuals involved in traumatic events. This concurs with findings of recent research ^{Reference Rose, Bisson, Churchill and Wessely8,Reference Roberts, Kitchiner, Kenardy and Bisson9} and the considerable caution being expressed towards emerging interventions such as Psychological First Aid ¹⁰ and strong opposition to the inclusion of single session individual psychological debriefing. There is some evidence that debriefing can be associated with increased symptoms among trauma victims following some traumatic events. ^{Reference Bisson, Jenkins, Alexander and Bannister11,Reference Hobbs, Mayou, Harrison and Worlock12}

Screening

Screening those who might have been affected by a traumatic event for symptoms has been advocated despite studies that fail to support its implementation. ^{Reference Rona, Jones, French, Hooper and Wessely13} Positive outcomes from a screening programme implemented after the July 2005 bombings in London have been reported although the researchers emphasised caution in drawing definitive conclusions regarding the effectiveness of screening programmes. ^{Reference Brewin, Scragg, Robertson, Thompson, d'Ardenne and Ehlers14} The Delphi process resulted in consensus against formal screening of everyone affected. Participants commented that there was absence of evidence for effectiveness of screening, potential organisational difficulties and that scarce resources could be better allocated. Interestingly, the statement ‘formal screening should not occur but helpers should be aware of the importance of identifying individuals with significant difficulties’ achieved positive consensus in round two having not done so in round one.

Pharmacological assessment and management

The statement ‘all responses should provide access to pharmacological assessment and management’ did not achieve consensus after two rounds. The result is contrary to the widespread practice of prescribing medication to those who are affected by traumatic events. ^{Reference Mellman, Clark and Peacock15} The comments about this statement suggested that participants were aware of the limited evidence base for pharmacological agents in the treatment of PTSD in the early response phases and their recommendation as a second-line therapy for PTSD. ¹ There is evidence for a small positive effect of medication for chronic PTSD and a better evidence base for other post-traumatic disorders such as depression.

Psychoeducation

The Delphi results showed a strong consensus for education being an important element in early management of those affected by disaster. Participants commented that by providing information about the consequences of trauma and education to the affected communities, the focus could be moved away from symptoms and towards resilience and natural human resources. This is against the findings of several randomised controlled trials that have failed to demonstrate a positive effect for education or providing self-help materials following traumatic events. ^{Reference Brewin, Andrews and Valentine18,Reference Ozer, Best, Lipsey and Weiss19} Yet, the role of education in trauma and disaster is an area that has provoked a range of different views, and the Delphi process reflected that.

Social support

The participants demonstrated strong support for social approaches in helping those affected, which is consistent with findings that the presence of PTSD is associated with perceived lack of social support. ^{Reference Brewin, Andrews and Valentine18,Reference Ozer, Best, Lipsey and Weiss19} However, no randomised controlled trials have addressed the role of social support in recovery from traumatic events. The Delphi process resulted in strong agreement with the five intervention principles proposed by Hobfoll and colleagues ^{Reference Hobfoll, Watson, Bell, Bryant, Brymer and Friedman2} that are supported by indirect evidence from various sources: promotion of sense of safety, sense of self- and community-efficacy, connectedness, calming and hope.

Stepped care

The guidelines are consistent with a stepped/stratified model of care, i.e. that routine formal psychological or pharmacological intervention for all affected individuals is not appropriate. Instead, individuals who develop symptoms may benefit from interventions, particularly trauma-focused CBT within the first 3 months for acute stress disorder and acute PTSD. ¹ The guidelines also deal with the very practical issue that not all symptomatic individuals benefit or can engage with trauma-focused CBT, by recommending the availability of alternatives with an evidence base for the treatment of chronic PTSD as second-line treatments (for example eye movement desensitisation and reprocessing and stress management). ¹

Strengths and limitations

A consensus approach to the development of guidelines should never replace a genuinely evidence-based approach or restrict efforts to achieve this. However, there is a requirement for psychosocial guidelines following disaster and we consider our approach the next best option, i.e. developing a consensus of rarely evidence-based, but usually evidence-informed, statements in a systematic manner. The Delphi process cannot develop high-level evidence but does avoid several potential problems encountered with non-systematic consensus development, for example failure to fully consider the limited evidence available, bias towards certain views and the dominance of certain individuals that often occurs in face-to-face group discussions. The high retention rate of participants across the three rounds was a major positive feature of this study, as was the care taken to systematically identify participants from a range of backgrounds and geographical areas as opposed to solely identifying individuals who belonged to a specific network or distinct professional background.

Implications for practice

The guidelines will evolve as our knowledge develops but represent the 2008 state of the art. They are much shorter than many recent documents and should be more accessible as a result. They are also written in a manner that is easy to translate into other languages. The results of the Delphi process strongly endorsed that such guidelines should not be mandatory. However, there was a consensus that planning in advance for disasters should occur along with training a workforce to respond appropriately to them. Such planning has long been in place for the non-psychosocial elements of responses to disaster and appears to result in good practice. The adoption of versions of these guidelines adapted for specific countries is likely to enhance preparedness for disasters and major incidents in the future.

Principal recommendations of the TENTS guidelines

Multi-agency planning group: every area should have a multi-agency psychosocial care planning group that includes mental health professionals with expertise in traumatic stress who have a designated responsibility for psychosocial care following disasters and major incidents. Individuals affected by disasters or major incidents should also be represented.

Psychosocial care plan: every area should have guidelines on the provision of psychosocial care in emergencies (a psychosocial care plan) that are incorporated into the overall disaster/major incident plan and regularly updated.

Training and supervision: all care providers should have undergone formal training and receive ongoing training, support and supervision.

Response: the response should promote a sense of safety, self and community efficacy/empowerment, connectedness, calm and hope.

Responses should provide general support, access to social support, physical support and psychological support.

Formal early interventions for all: provision of specific formal interventions such as single session individual psychological debriefing for everyone affected should not occur.

Social support: efforts should be made to identify the correct supportive resources (e.g. family, community, school, friends, etc).

One-stop shop: a humanitarian assistance centre/one-stop shop should be established where a range of services potentially required can be based.

Symptomatic individuals: individuals with psychosocial difficulties should be formally assessed by a trained professional with consideration for their physical, psychological and social needs before receiving any specific intervention.

Trauma-focused cognitive–behavioural therapy (CBT): treatment with trauma-focused CBT should be available for individuals with acute stress disorder or acute post-traumatic stress disorder (PTSD).

Other evidence-based treatments: other treatments with an evidence base for chronic PTSD should be available for individuals with acute PTSD when trauma-focused CBT is not available or is not tolerated.

Evidence-based interventions for individuals with other mental health difficulties should be available.

Long-term coordinated planning and cooperation: detailed planning should occur with local authorities/governments and existing services to fund and provide appropriate extra provision to support local services for several years following the disaster.