The suffering experiences that prompt patients with psychiatric conditions to consider ending their life by means of suicide or medical assistance in dying has been understudied and remains insufficiently understood.Reference Tossani1–Reference Verrocchio, Carrozzino, Marchetti, Andreasson, Fulcheri and Bech3 Medical assistance in dying has been legally justifiable in some states in the USA and in several countries for the terminally ill.Reference Dyer, White and García Rada4,Reference Quill, Back and Block5 However, in the Netherlands, Belgium and Luxembourg, euthanasia requests can be legally granted for non-terminally ill patients, including those with psychiatric conditions. In Belgium, the ability to grant euthanasia requests requires that patients experience constant and unbearable suffering stemming from one or more somatic or mental disorders, without any (reasonable) prospect of improvement.6 The number of patients with psychiatric conditions who died by means of euthanasia has increased annually, particularly from 2008 onwards: from 5 patients in the years 2002 to 2007, to 72 between 2008 and 2012, and to 181 between 2013 and 2017.Reference Dierickx, Deliens, Cohen and Chambaere7,8
Shortcomings of existing instruments to directly capture ‘mental suffering’
Whether or not psychiatric euthanasia requests are justifiable remains controversial, with particular emphasis on the challenges involved in comprehending, assessing and evaluating unbearable mental suffering in these patients.Reference Braeckman, Ravelingien and Boudry9–12 Several instruments have been devised and are used to assess suffering-related constructs such as bodily pain,Reference Stanhope13,Reference Bodian C, Freedman, Hossain, Eisenkraft and Beilin14 mental painReference Tossani1,Reference Shneidman15 and the association between them.Reference Sartorius16 However, suffering cannot be reduced to one single aspect (for example painReference Sensky17), as it involves many other social, societal and existential aspects of life.Reference Loeser18–Reference Verhofstadt, Thienpont and Peters22 Therefore, instruments have been developed to assess suffering from other instruments used for broader concepts such as quality of life and well-being, based on the assumption that low scores on these indices may represent a high level of suffering.Reference Bruno, Bernheim, Ledoux, Pellas, Demertzi and Laureys23 However, there is clinical evidence suggesting that this may be invalid: among patients who display stable scores of their quality of life over time, in their quality of life over time, some patients requested euthanasia, whereas others did not.Reference Ruijs, Onwuteaka-Philipsen, van der Wal and Kerkhof24
Although an intrinsic aim of healthcare practice is to alleviate patients' suffering, suffering is seldom addressed comprehensively, regularly overlooking patients' perspectives in clinical and scientific settings.Reference Sensky17,Reference Ruijs, Onwuteaka-Philipsen, van der Wal and Kerkhof24–Reference Mak and Elwyn27 Over the last several decades, a growing interest in the concept of suffering has led to the development of suffering scales, assessing psychosocial, social and existential aspects of suffering. However, as focus on the topic has been primarily from a clinician's perspective, only a few of the tools designed to assess suffering have been developed in the end-of-life context, with a target population consisting of advanced or terminally ill patients, primarily suffering from somatic disorders.Reference Ruijs, Onwuteaka-Philipsen, van der Wal and Kerkhof24,Reference Krikorian, Limonero and Corey28 Hence, although it is crucial that the items of an instrument accurately and comprehensively represent the specific topic of interest and assessment goals, and thus reflect the specific target population and context, insufficient research has been undertaken in this field.
Towards a new assessment tool to capture suffering in psychiatric patients, with and without a euthanasia request
Ruijs and colleaguesReference Ruijs, Onwuteaka-Philipsen, van der Wal and Kerkhof24,Reference Ruijs, Kerkhof, van der Wal and Onwuteaka-Philipsen29,Reference Ruijs, Goedhart, Kerkhof, van der Wal and Onwuteaka-Philipsen30 were the first to develop and test the State of Suffering-V (SOS-V), an assessment tool that directly addresses unbearable suffering in the end-of-life context. However, as it was developed for terminally ill cancer patients, it may not be a valid tool for patients with psychiatric conditions who may present with less apparent biomedical conditions, while still experiencing a high level of suffering, a construct that may be apparent to the patient, yet less salient to physicians.
Unfortunately, there is no generally accepted theoretical model of suffering nor definition of unbearable suffering in the end-of-life context.Reference Dees, Vernooij-Dassen, Dekkers and Van Weel2 Moreover, the existing definition does not include the perspectives on unbearable suffering of patients with psychiatric conditions. To date, only two studies, both qualitative, have addressed this group's perspectives on unbearable suffering,Reference Verhofstadt, Thienpont and Peters22,Reference Dees, Vernooij-Dassen, Dekkers, Vissers and van Weel31 one of which exclusively dealt with the suffering experiences of patients with psychiatric conditions who have made a request for euthanasia.Reference Verhofstadt, Thienpont and Peters22 This study yielded 44 terms that participants used to describe the nature and extent of their suffering,Reference Verhofstadt, Thienpont and Peters22 a useful starting point in the development a new assessment instrument.
Given that unbearable suffering is an important condition for legally granting euthanasia requests, yet suffering and death requests remain uncomfortable topics of discussion during physician–patient consultations,Reference Dees, Vernooij-Dassen, Dekkers, Elwyn, Vissers and van Weel25 support is sorely needed. In order to aid in practice, an assessment instrument (similar to the SOS-V, but tailored for patients with psychiatric conditions) that can help professionals and patients discuss the intensity and duration of patients' suffering experiences needs to be carefully developed and standardised. Given the inherent subjective nature of suffering experiences, it is paramount that the items comprising such an instrument have high cognitive validity (achieved when all items are interpreted by the target population as intended by its developers and end-users, and free from bias introduced by, for example, social desirabilityReference Willis32–Reference Buers, Triemstra, Bloemendal, Zwijnenberg, Hendriks and Delnoij36). Studies assessing cognitive validity for new and already validated measurement instruments typically find such measurement instruments lacking in both respects.Reference Plass and Peters37–Reference Van Kessel, Hendriks, van der Hoek and Plass41
This study therefore aims to optimise the cognitive validity of an initial item pool that was derived from a qualitative study.Reference Verhofstadt, Thienpont and Peters22 The resulting item pool can then serve as a starting point for further study as well as the development of an instrument that can assist professionals and patients in discussing patients' suffering experiences.
Method
Research design
We applied Rattray & Jones' eight stages in developing an assessment instrumentReference Rattray and Jones42 and Willis' cognitive interviewing protocolReference Willis43 to design the procedure used for conducting the face-to-face cognitive interviews with the patients with psychiatric conditions, with or without a euthanasia request. The questionnaire was developed in eight consecutive steps, which were undertaken in a series of three phases: phase one (three steps) involved the generation and scale construction based on relevant literature; phase two (subsequent three steps) concerned the evaluation of item answers, including the relevance of the answers produced, including stylistic and formal criteria, all followed by basic statistical analyses; phase three (final two steps) focused on the development of a final version of the instrument, to be immediately tested with regard to additional psychometric qualities if and only if the variable can be considered a unidimensional construct (e.g. simple mental and cognitive processes, emotions and beliefs). As suffering seems to be a multidimensional construct, a fully-fledged validation study to further optimise psychometric qualities or to determine cut-off scores would not be deemed ethically or scientifically appropriate. The full procedures undertaken in this study, including the first three steps of Rattray & Jones' eight-step plan, that were followed in order to define both the need for, as well as the structure of, the new questionnaire are detailed in supplementary Box 1 available at https://doi.org/10.1192/bjo.2019.25 (and are described in detail in the supplementary documents ‘NEOSi development’ that have been posted in the Open Science Framework (OSF) repository accompanying this paper (https://osf.io/bhde3/).
Original item pool
The working title of the initial item pool on suffering was the ‘Nature and Extent Of Suffering indices’ (NEOSi). It contained 71 items that were based on the domains and descriptors of suffering, as identified in the qualitative study that produced the item pool.Reference Verhofstadt, Thienpont and Peters22 Two separate, but related, item pools map the nature (NOSi) and extent (EOSi) of suffering experiences.
NOSi
This first item pool contained 62 items examining the frequency and intensity of different aspects of unbearable suffering. Out of consideration for the target population's vulnerability, these items were then sorted into eight clusters, which were then ranked from ‘concrete’ and ‘emotionally safe’ to ‘abstract’ and ‘potentially emotional’. Specifically, these clusters were:
(a) medical complaints (for example ‘Clearly demonstrable physical complaints, such as gastrointestinal infections, cardiac arrhythmias but also hearing loss.’);
(b) problems with former or current therapies and diagnostics (for example ‘Negative experiences with diagnostics (diagnosed falsely or too late)’);
(c) physician–patient communication problems (for example ‘The feeling that physicians do not completely understand you and your experience with suffering’);
(d) the procedure for granting euthanasia requests (for example ‘The fact that the euthanasia procedure takes quite some time and energy’);
(e) financial and work-related problems (for example ‘Financial worries (low income or debt)’);
(f) social problems (for example ‘Conflicts with important others (partner, kids, family, friends)’);
(g) traumas (for example ‘The experience of a sexual trauma during childhood’); and
(h) feelings and fears (for example ‘The feeling that your suffering experiences have deprived you of your dignity as a human being’).
Each cluster was titled and its questions prefaced. Responses to each item were registered using two side-by-side Likert scales. The first scale measured frequency and consisted of five answer options using the labels ‘never’, ‘yearly or less’, ‘monthly’, ‘weekly’, and ‘daily’. The second scale measured intensity and consisted of four answer options using the labels ‘non-existent’, ‘slightly disturbing’, ‘deeply disturbing’ and ‘overwhelming’. Each cluster was followed by an invitation to answer an open-ended question regarding whether the participant experienced additional contributors related to a given experience associated with suffering.
EOSi
This item pool contained nine questions combined with visual analogue scales (VASs) to assess the extent of patients' overall evaluation of their suffering experiences with respect to intensity (for example ‘How would you describe your suffering in general?’ anchored by ‘bearable’ versus ‘unbearable’), duration (for example ‘How long have you been you suffering?’, with anchors ‘short term’ versus ‘long term’), chronicity (for example ‘How often do you suffer from the symptoms of your disorder(s)?’, with anchors ‘sporadically’ versus ‘continuously’), and perspective (for example ‘How do you feel concerning your situation?’, with anchors ‘hopeless’ versus ‘hopeful’). Each VAS used a slider with a minimum value of 0 and a maximum value of 100. Higher scores indicated more intense and salient suffering. The full lists of original, adjusted and final items are available in the OSF repository.
Procedure
Recruitment strategy
In order to be eligible to participate in this study, participants had to be legally competent and Dutch-speaking adults. Individuals presenting with acute grief, signs of acute substance misuse, psychosis and dementia were excluded. Potential participants were recruited via two approaches. First, M.V. contacted patients with psychiatric diagnoses through her broad social circle; second, an independent psychiatrist recruited potential participants with at least one psychiatric diagnosis and a currently active or withdrawn euthanasia request.
Cognitive interview procedures
The first round of cognitive interviews took place from August to September of 2016. Before the start of the cognitive interview, four initial questions were asked: age, gender, diagnosis and whether the participant had requested euthanasia. M.V. conducted all interviews with study participants. A psychiatrist was always on call (but not present), to offer medical and emotional support if needed.
Two cognitive interviewing methods were combined. The basic paradigm was the ‘think-aloud’ method, in which each participant was asked to read out loud, comment on and discuss the items, and, subsequently, had the option to pick a (non-) numeric score that represented her answer to the item. The theoretical evidence for this approach is based on Ericsson & Simon's workReference Ericsson and Simon44 in which a distinction is made between the cognitive processes of (a) working memory for concurrent reasoning and (b) long-term memory for retrospective reasoning.
The goal of think-aloud research is to get a deeper insight into the processes of working memory. However, as not all information reaches our working memory, because of its limited capacity to store information, and as working memory has the tendency to be overruled by new information, only verbally expressed information that follows very rapidly after a thought process can be perceived as the most accurate reflection of participants' thoughts. However, with regard to this methodology, Ericsson & Simon warned that the repeated practice of a task might lead to automaticity before thought processes can be reported.Reference Ericsson and Simon44
Therefore, the think-aloud technique was combined with ‘probing’, where the researcher asked additional questions regarding how items and answer categories were understood, interpreted and evaluated, whether the questions and answer options were precise and easy to answer, and whether there were omissions or ambiguities.Reference Buers, Triemstra, Bloemendal, Zwijnenberg, Hendriks and Delnoij36 The following issues were probed when no information was spontaneously provided: (a) relevance, (b) interpretation, (c) clarity, (d) linguistic correctness, (e) sequence of clusters, questions and answer options and layout in general, and (f) social-desirability answering risk.
As is generally the case, iterative rounds of interviews needed to be scheduled before data saturation and cognitive validity had been reached (i.e. no new information gained after at least three successive interviews).Reference Willis32,Reference Willis43,Reference Francis, Johnston, Robertson, Glidewell, Entwistle and Eccles45 Finally, each participant was asked to provide their general opinion on the items, research topic and method and willingness to participate in follow-up research. Each remark from the participants was noted in order to enable adjustments to the research procedures in future studies. When suggestions or opinions of participants differed, the opinion of the majority was considered when implementing changes. However, all remarks were noted for closer investigation in a second round of cognitive interviews, which took place from October to December 2016, in which slight changes to the interviewing procedure were implemented (see Results and OSF repository).
To ensure participants' privacy, cognitive interviews were not recorded, but instead written down as carefully and literally as possible. During transcription of these notes, attention was paid to participants' anonymity. Transcripts were then saved in a folder on a secure server and coded in QualiCoder (qualitative data coding software).46 All participants were offered the opportunity to review the manuscript and – if applicable – to correct their quotes.
Analyses and criteria
Cognitive interviews were systematically analysed according to the criteria of WillisReference Willis47 and are schematically represented in the OSF repository. The following overarching themes were used as guidelines: (a) item interpretation (with item interpretation issues as subthemes), (b) item formulation (with vagueness and nuances as subthemes), (c) language (with typing error, word choice and grammar as subthemes), (d) sequence (with cluster sequence and item sequence as subthemes), (e) answer categories (with frequency and intensity as subthemes), (f) introduction, (g) lay out, and (h) opinion (with NEOSi, research topic and research method as subthemes). The theme coding model with codes representing the key findings were first labelled as brief keyword-type subthemes and then covered in overarching themes.
Ethics
This research project received provisional ethical approval from the ethics review board on research (cETO) of the Open University (reference U2016/03311/FRO, pending approval by a Belgian ethics review board) and definite approval of the Medical Ethics Committee of the Vrije Universiteit Brussel (reference B.U.N. 143201628847).
Results
Participants in the first round of cognitive interviews
The participants were nine adults between 35 and 76 years of age. All the participants with a euthanasia request were women, as well as one out of four without. Depression (n = 6) and autism spectrum disorders (n = 3) were the most common diagnoses. The interviews lasted between 60 and 210 min (see the OSF repository).
NOSi: feedback on items
Of the 62 NOSi items, (potential) problems were identified in 54 items (87%), mostly with regard to precision of interpretation, inadequate formulation of item and answer options, imprecision in language or the need for more accessible language, formal and structural aspects and omission of 20 relevant aspects of suffering.
Participants suggested the following adjustments:
(a) reformulate items in order to make them univocal;
(b) split items when certain elements represented various components of suffering;
(c) include fewer, other or no examples when items were clear by themselves, to avoid invoking a specific mindset;
(d) include relevant examples to broaden participants' mindset or to clarify the item subject; and
(e) add new items that contribute to suffering experiences.
Finally, participants provided feedback concerning language and formal aspects, including word choice, grammar and typographical errors. Furthermore, participants commented on the need for overt, rather than vague, statements or questions. For example, ‘personal experiences’ was used to cautiously describe ‘traumas’, although the majority of participants found this caution inappropriate because it insufficiently acknowledged crucial components of their suffering.
Regarding the options offered regarding the frequency of suffering, participants expressed a need for more precision and nuance: the gap between ‘daily’ and ‘weekly’ and, especially, the gap between ‘monthly’ and ‘yearly’ was considered too large. Moreover, the answer option ‘never’ was susceptible to more than one interpretation. For example, on the item ‘Negative experiences in psychodiagnostics (wrong diagnoses or diagnosed too late), endorsing ‘never’ could either mean never having experienced this, or never having been diagnosed at all. Participants suggested including ‘not applicable/never’. As for intensity, participants also expressed a need for more precision and nuance, based on the gap between ‘deeply disturbing’ and ‘overwhelming’. (See the OSF repository for a full overview of all problems and adjustments.)
NOSi: additional feedback
Participants' feedback on the introduction and structural aspects of the NOSi suggested it would be useful to indicate the number of items per cluster and to alter the cluster and item sequence. Participants suggested putting key themes (such as ‘traumas’ and ‘social problems’) at the beginning of the questionnaire and dividing the cluster ‘social problems’ into ‘social problems with (important) others’ and ‘societal problems’ because these imply different consequences. The participants suggested merging the clusters ‘physician–patient communication problems’ and ‘treatment and diagnostics’ because of their perceived interrelatedness. The last cluster ‘feelings and fears’, contrary to the other clusters, was not perceived as a stand-alone suffering category.
EOSi: feedback on questions and anchors
Participants provided suggestions on how to improve eight of the nine questions and/or their anchors, given that item formulation was perceived as too vague (for example ‘all sorts of problems’) or subjective (for example when asked for the duration of suffering with the anchors ‘short versus long’ some participants expressed the need to correct for age whereas others did not). Suggestions were made to reformulate questions and include other anchors.
Issues with lay out and word choice in the introductory text were also identified by the participants. The use of ‘future life perspectives’ was perceived as inappropriate in euthanasia requests and it was suggested that this could be altered to the more neutral wording of ‘future expectations’. Finally, one participant refused to use the slider, instead using a grade from 0 to 10 to answer the negatively formulated anchor option. Afterwards, this participant suggested implementing this scoring system in the NOSi as well (see the OSF repository for an overview in Dutch, including illustrative quotations, leading to an adjusted NEOSi).
Opinions concerning the NEOSi in general
All participants, except one, perceived all items as relevant and – when not applicable to themselves – applicable to others. Some participants spontaneously described their most candid suffering experiences with terms such as ‘hopelessness’, ‘being tired of life’, ‘being through with life’, ‘loss of dignity’ and ‘suicide or self-destruction’. All participants appreciated the opportunity to clarify their suffering experiences with open-ended questions. Two participants indicated their opposition to filling out the NEOSi online, strongly preferring verbally conducted interviews (for example because of dyslexia). One participant advised including an additional open question at the end of the questionnaire to ensure no aspects of suffering had been overlooked. Furthermore, the general question ‘Have you requested euthanasia?’ also gave rise to differences in interpretation. Although the supervising physicians only referred participants with a current euthanasia request, some participants stated that they did not request euthanasia, as they perceived their euthanasia request as something preliminary if they had not yet officially put their request in writing, not yet discussed it with important others or if it had been withdrawn or not (yet) declared eligible. We, therefore, decided to include the following extra answer options for this question: ‘Yes/no, still considering’, ‘Yes, but it has not (yet) been declared admissible’, ‘Yes, the euthanasia procedure is ongoing’, ‘Yes, in the past, but it has been declined’, ‘Yes, in the past, but I withdrew it’ and ‘No’, to be cognitively tested during the second round of interviews.
Opinions on the research topic and method
All participants vocalised an appreciation for both the topic of study and the methodology used. Specifically, participants expressed appreciation for the length, explaining that other, shorter questionnaires were often perceived as insufficiently addressing the core of the matter. In addition, the broad scope of the NEOSi was appreciated, as well as the fact that patients were involved as ‘essential experiential experts’ at this early stage of research. Participants also expressed their hope that the NEOSi could eliminate the taboo of unbearable mental suffering. Moreover, participants with a euthanasia request also declared that the NEOSi gave them the feeling that they were being taken seriously and their suffering acknowledged. Two participants revealed they had shared issues with the interviewer that had not (yet) been discussed in detail with the physicians involved in their euthanasia request.
Finally, participants explained the cognitive interviews were not perceived as emotionally strenuous, but rather as comforting because they offered the possibility to talk openly, and without reservations, about their suffering experiences and – when applicable – their euthanasia request. Because these results were inconsistent with both our and the consulted ethical committees' prior expectations, we contacted the supervising physician, responsible for participant recruitment before, and well-being after, the study, who confirmed this finding. All participants were willing to participate in the follow-up study for further item-pool improvements.
Second round of cognitive interviews: changes in methods
A second round of cognitive interviews was deemed essential to further optimise the cognitive validity of the substantially altered NEOSi in order to detect whether the problems with it were resolved adequately, whether new problems surfaced and to explore broad-based acceptance of form and content.
The recruitment process was focused on recruiting more male participants with a euthanasia request, as the first-round population who had made a euthanasia request had all been women. The participants consisted of ten adults (six men and four women), five of whom had participated in the first round of cognitive interviews. The participants in the first-round study that could not participate were willing to participate at a later time but declined the invitation because of acute health or familial problems or the need for a mindset completely focused on rehabilitation. Of the participants with a euthanasia request (n = 6), one had only recently requested euthanasia (n = 1), others still had a euthanasia procedure that was ongoing (n = 2) or had recently been granted (n = 1), or had indefinitely withdrawn their euthanasia request (n = 2). For more information on the participants and recruitment procedure, see the OSF repository.
The feedback from the first round had been used to develop a total of six versions of the NOSi. Each contained 92 items, 20 of which were new and had been added based on participants' feedback in the first round. In one version, the items were alphabetically ranked; in another version, items were clustered according to the five domains that were identified in the qualitative study on unbearable suffering of patients with psychiatric conditions with a euthanasia request (medical problems, personal problems, interpersonal problems, societal problems and existential problems).Reference Verhofstadt, Thienpont and Peters22 These two item orderings were combined with three response registration formats: Likert scales, VASs or text-entry fields to enter a number from 0 to 10.
The EOSi still contained nine questions assessing the extent of suffering experiences with respect to intensity, duration, chronicity and perspectives, but with two response registration formats: one with a five-point Likert scale and one with a VAS. Hence, six NOSi versions and two EOSi versions were reviewed. For detailed information on when each NOSi and EOSi version was presented and discussed, see our OSF repository.
During the cognitive interviews, the same cognitive interviewing techniques and criteria were employed as in the first cognitive interview round. Additionally, to test social desirability, the participants were asked if their answers would be similar if asked in a different context, such as online or with a different interviewer.
As opposed to the first round of cognitive interviews, when a participant made a suggestion or remark, this was probed further in later interviews with all subsequent participants, immediately before the next item, in order to gauge data saturation and hence, ultimately higher cognitive validity. At the end of the cognitive interview, whether and why there was a need to include an open question was asked by M.V. As opposed to the first round of cognitive interviews, a broad support base on item relevance, content and sequence, as well as preference and convenience concerning layout out and sequence was sought. We set the following criteria for data saturation: we would make final adjustments to the NEOSi items in case 1, respondent expressed content-related feedback that was approved by at least 3 other respondents. We would make adjustments to the layout and item sequence: in case 1, respondent expressed her preference and 7 other respondents agreed. If these criteria for adjustment were not met, alterations to the NEOSi were to be made according to the majority's preferences and a third round of cognitive interviews would have to be organised.
Second round of cognitive interviews: results
NOSi: feedback on items
In general, participants perceived the items in the adjusted NOSi to be sufficiently easily understood, sensitive to delicate nuances and complete. All participants recognised the aspects of suffering in themselves and/or in others. However, suggestions for possible improvements were made in 35 of 92 (38%) NOSi items on the grounds of potential issues with interpretation, formulation, language and missing relevant aspects of suffering.
As for interpretation, feedback was given on items that were perceived as possibly insensitive to nuances, although it did not lead to misunderstanding between participants. For example, item 48, ‘thoughts about suicide or other self-destructive behaviours (e.g. cutting, burning or other self-injuries)’ was commented on as follows:
‘I know what you mean and to me it's clear, but please pay attention don't miss anything with people that are taking alcohol or drugs. (…) Besides, self-harm can also include poor, bad or not eating, don't forget that. You can also neglect taking care of yourself. Always working overtime could be seen as self-harm behaviour as well. You can also be self-destructive by means of self-isolation, as the more people are actively involved in your social network, the more chance you have of being protected when things might go the wrong way. Now your item examples only contain active behaviours, but that doesn't cover self-destructive behaviour, as a lack of actions can be self-destructive as well. Maybe you can include additional examples as “poor diet or malnutrition”, “too many drugs”, etc.’ (Participant with a euthanasia request)
Participants suggested making the following adjustments: (a) reformulate items more accurately and precisely; (b) include other examples to broaden participants' mindset; and (c) add new items that relevantly contribute to the NOSi. Five items were added during probing (n = 2) or at the end of the NOSi questionnaire (n = 3). Although these new items were not perceived as key criteria for patients' own unbearable suffering, they were added to the adjusted NEOSi.
The most commonly reported problem with the NEOSi was that the items specifically related to procedures involved in legally granting patients' euthanasia requests were perceived as potentially inappropriate because: (a) answer options were dependent on the specific phase of the euthanasia procedure; (b) applicability outside the euthanasia context; (c) possibility of provoking a euthanasia request, or (d) potentially too shocking (for one participant without a euthanasia request). Suggestions were made and approved by the next participants to further improve the NOSi via: (a) item reformulation, (b) merging items, (c) splitting items, or (d) removing items.
In general, all participants agreed with the content of both answer categories, although the gap between ‘one or more per month’ and ‘at most, once or more per year’ for frequency was found to be too large and poorly formulated. As for intensity, there was broad support (n = 9) that the NOSi version should use a grade from 0 to 10 because it was (a) more sensitive to nuance and (b) the value behind answer options was clearer and more insightful.
NOSi: additional feedback
Participants gave feedback on the introduction and structural aspects of the NOSi that included: (a) a more explicit time frame mentioned at the introduction per survey question in order to ensure that respondents have as little doubt as possible when they answer the survey questions, (b) a more logical item sequence, (c) layout, and (d) removal of small inaccuracies (for example too much white space).
Broad support (n = 8) for the clustered version was substantiated as a way to avoid: (a) the tendency to look back and detect possible double items, (b) difficulties in answering the items precisely because the sensitivity in nuances could only be detected when looking back to similar items, and (c) suddenly and abruptly changing from items on general aspects of suffering, for example going back and forth between questions concerning general irritations versus deep, personal emotions and experiences was perceived as too exhaustive or confusing.
‘Sometimes you can only see the nuances between items if they are clustered together. Now I often wonder: Haven't I already answered it? (…) It is confusing to me. I would stick with the items on physicians and other aid workers, then the items on people in the social inner circle, then items on society and finally the microcosm-like items. Now it's a jumble and that's why I find it difficult to answer.’ (Participant without a euthanasia request)
EOSi: feedback on questions and anchors
Participants suggested reformulation of four out of ten anchors, especially concerning the duration of suffering experiences. Feedback was given on notions of problems and difficulties in life preceding the awareness that these notions could be designated as suffering. Other anchors were still perceived as too vague (‘short versus long’) or not befitting a suffering-related questionnaire (‘promising perspectives’). Participants' suggestions to reformulate anchors were approved by the next participants. As for word choice and layout, no comments were made.
Four participants strongly preferred a Likert scale over a VAS, explaining that it: (a) allowed more precise answers, (b) avoided the risk of respondents just drawing a line without thinking and (c) was easy to answer, based on some of the participants' observation that they would need to have used a ruler in order to give a correct answer. Four others slightly preferred the VAS as it was perceived as less ‘categorical’ than a five-point Likert scale because of the sensitivity of nuance if and only if the adjusted EOSi had a small vertical line to indicate the middle. Two other participants indicated no preference, as they stated that both scales were easy to comprehend and utilise to convey their responses. The participant who declined to use a VAS and instead give a report mark in the first round was now more in favour of the Likert-scale method. (See the OSF repository for a concise or complete overview of all problems and suggestions in Dutch.)
Additional feedback on the NEOSI
Participants who had also taken part in the first round commented that the adjusted NEOSi contained (a) fewer items that were subject to misinterpretations, (b) clear, accurate, concise and nuanced items, (c) answer categories that were easier to answer and more accurate, nuanced and complete. Most participants declared both a few times during, as well as after the interview, that the sensitivity for nuances was the most positive improvement, for example the difference between ‘hopeless’ versus ‘without prospect’.
‘It's good that you pull these apart, because they are slightly though really different in their essence. Hopeless is more like a feeling: it could refer to something depressing or another screwed up feeling. But “without prospect” refers more to a context, a situation and as such contains something more rational, calculated or something like that. Something that you can evaluate over time, in a thoughtful way, well-considered or something of the sort, while hopelessness is more a feeling that suddenly you can be overwhelmed with.’ (Participant without a euthanasia request)
Participants declared that nuances within the (phrasing/scoring) allowed greater variability within their responses to particular items and thus, their suffering experiences could be more accurately addressed (such as not only gauging a disorder, but also deeper, underlying, existential feelings). For example, one participant with a mood disorder stated that the item ‘feeling/conviction that you are a burden to others' can be interpreted through a spectrum of intensities, with a general distaste for burdening others at one end and a true symptom of depression at the other, where an individual may feel, ‘not being worthy to breathe and take oxygen out of the air, as others seem to have more right to it’ (participant without a euthanasia request).
In addition, three participants recommended the researcher during or after the cognitive interview that the NEOSi should be used to assess the progress of suffering over time by means of identifying the possible influence of a temporary episode inherent in a fluctuating disorder (i.e. multiple measurements). Moreover, participants suggested that the NEOSi should be used to assess differences in the nature and extent of suffering in patients with and without a psychiatric diagnosis, as well as in patients with psychiatric conditions with and without a euthanasia request. For instance, a possible distinction had been reported in items 35 (not or no longer having a role, function or meaning in life) and 39 (being tired of or done with life) as well as items related to negative experiences in someone's personal background, support with mental health problems and social contact.
The fact that the adjusted NEOSi contained 30% more items than the initial version was positively perceived as exhaustively addressing various forms of experiences with suffering. Nevertheless, every participant preferred to include an open question at the end, even participants who had no further information to include. There were two underlying rationales: to guarantee that no aspects of suffering or additional feedback were missed, and that emotional steam could be let off when needed. During and after the cognitive interview, each participant with a euthanasia request spontaneously pointed out there were in total up to 44 NOSi items that were included that were crucial aspects of unbearable suffering, and as a potential consequence, of a euthanasia request. Finally, participants also declared they would give the same honest answers regardless of whether the NEOSi was administered verbally, in writing or online.
Feedback on the research topic and method
All the original and new participants appreciated being involved in the research topic and method, as expressed in the initial study. During, as well as after, the cognitive interview, one patient with a euthanasia request said the NEOSi provided an insight into many aspects of life that were still positive, providing the individual with extra motivation to pursue alternative treatment strategies.
‘Gradually I realised how lucky I am. I do have great physicians with whom I can talk openly. I do have friends and from an economic point of view, I don't have financial difficulties. It's like I told you on the phone, yesterday. The first time I came here, I saw other people in the corridor that were, how should I explain it… living more on the margin. Don't get me wrong, no offence, but you could see poverty by means of having lost everything. You can see it in people, sometimes, that they have lost everything. I'm not like them. And now by commenting on this questionnaire I realise again what I still have, that despite everything, how well off I actually am.’ (Participant with a euthanasia request)
Discussion
Major findings
During the first round of cognitive interviews, participants suggested making changes to 62 of the 71 initial NEOSi items because of problems related to content, form and language. By comprehensively amending the questions and presenting different versions of the NEOSi for feedback and sharing this feedback with the next participant during a second round of cognitive interviews, there was broad support for the adjustments and commonly shared preferences on clustered items, item sequence and logical structure. This enhanced the cognitive validity of the final item pool.
The value of cognitive validation studies and cognitive interviewing techniques
In addition, when participants compared the length of the NEOSi with other, often shorter questionnaires, the shorter versions were often perceived as not sufficiently addressing the core of the matter, which may suggest that these surveys were lacking in content validity. In general, the results of the first and second round of cognitive interviews are in line with other cognitive validation endeavours showing that items and answer options (in both new and validated instruments) are often interpreted differently by the developers and the target population, and as such this may interfere with proper measurements.Reference Plass and Peters37,Reference Maters, Sanderman, Kim and Coyne39–Reference Van Kessel, Hendriks, van der Hoek and Plass41
This also underpins the need for cognitive validation studies to detect and eliminate these obstacles. This endeavour should be an aim, and an essential component, of subsequent quantitative (validation) studies.Reference Plass and Peters37,Reference Peters, Dima, Plass, Crutzen, Gibbons and Doyle38 Cognitive interviews can also identify end-users' preferences and facilitates developing an end-user-friendly questionnaire. For example, initially, a short VAS scale had been chosen to assess the extent of suffering, as this tool has often been used to research pain, perceived quality of life and changes in medical treatment effects. It also generates high face validity when directly examining patients' experiences.Reference Miller, Duncan, Brown, Sparks and Claud48 Although participants declared both versions were appropriate and easy to answer, some had a strong preference for the Likert scale and this was therefore chosen by the research team for the final version of the EOSi.
The methodology of cognitive interviewing resulted in rich, anecdotal evidence, encouraging participants to give extensive feedback on aspects such as item interpretation and sequence, layout and missing items until data saturation had been achieved.Reference Francis, Johnston, Robertson, Glidewell, Entwistle and Eccles45 Long and comprehensive face-to-face interviews, taking place in a serene atmosphere, allowed the interviewer to bond with the respondents while also keeping an eye on their body language, resulting in a number of opportunities to delve further into their experiences, opinions and feelings.
Stigma and prejudice: the value of engaging vulnerable patients in research endeavours
As a result of the scarcity of studies of patients with psychiatric conditions who have made requests for euthanasia, there was a lack of knowledge about how participants would cognitively and emotionally react during this research project. Our results show that participants acknowledged the study's value and relevance in reducing the negative stigma associated with psychiatric euthanasia requests. They greatly appreciated being involved in the early phase of instrument development, which was reflected in the fact that participants were willing to participate in a follow-up study. Participants with euthanasia requests declared that the nature and duration of the cognitive interview offered them a degree of consolation as they could talk openly, while being taken seriously, about their experience with suffering unbearably. This reaction to our study is in accordance with the results of an interview study on respondents' satisfaction, summarised in terms of being heard and making meaningful, relevant contributions via trustful, respectful communication.Reference Wenemark, Hollman Frisman, Svensson and Kristenson49 These findings are remarkable as both the research team and ethics committees had concerns about possible negative consequences for this highly vulnerable patient group.
Moreover, the fact that these patients did not conceal certain aspects of their suffering but clearly discussed even the most sensitive issues, emphasises the value and necessity of involving this particular target population in further research endeavours. This result aligns with previous studies that showed not only a lack of adverse long-term effects in participants involved in psychiatric research (only a minority showed more distress immediately afterwards), but positive, rather than negative, reactions to the study itself.Reference Jorm, Kelly and Morgan50 Although their results were based on small sample sizes, a recent meta-analysis showed that exposure to suicide-related content paradoxically even reduced suicidal ideation and attempts. In particular, interview-based studies had a positive impact on respondents (such as a decrease in distress).Reference Blades, Stritzke, Page and Brown51
The validity of this tool in a variety of populations of patients with psychiatric conditions should also be considered. Some authors consider a patient's euthanasia request as a symptom of suicidality, necessarily rooted in a patient's (underlying) depression.Reference van der Lee52,Reference van der Lee53 However, systematic review on suicide revealed that although high levels of mental pain indeed contribute to a higher risk for suicidal tendency, it does so independently from depression.Reference Verrocchio, Carrozzino, Marchetti, Andreasson, Fulcheri and Bech3 Moreover, empirical evidence shows that in most terminally ill patients with a euthanasia request, no depressive disorder was found.Reference van der Lee53–Reference Ganzini55 The available evidence, albeit scarce, consistently shows that not all patients with a psychiatric condition who makes a euthanasia request have mood disorders.Reference Thienpont, Verhofstadt, Van Loon, Distelmans, Audenaert and De Deyn56 Moreover, some of these patients, even after having their euthanasia requests granted, withdrew their requests, whereas others still die by suicide.Reference Thienpont, Verhofstadt, Van Loon, Distelmans, Audenaert and De Deyn56 These data are in line with this study, in which four patients with a euthanasia request did not have a mood disorder whereas three patients presenting with such a disorder had not requested euthanasia.
Limitations and implications for further research endeavours
It should be noted that because a convenience sample of patients with psychiatric conditions was used (i.e. either selected from the researcher's broad inner circle or recruited by a supervising psychiatrist), it is possible that the item pool's apparent cognitive validity does not hold in a wider target population. It remains vital to attend to participants' well-being before, during and after future data-collection efforts. This problem can be addressed by administering the NEOSi face to face. In addition, as sampling was relatively limited, additional items as well as additional enhancements in item formulations may emerge in future research.
Future research will be able to explore the validity of these statements by exploring how answers to surveys online, in person or via other media may differ. It should be considered, however, that even in the case of patients giving the same consistent answers – irrespective of the NEOSi being presented in a paper pencil version, online or in any other medias form – whether or not each (or any) respondent is perfectly able to introspectively assess a given question with complete accuracy. It is inherent to self-report questionnaires that respondents' answers often do not fully correspond to their true feelings, thoughts and actual behavior. If the NEOSi were used in clinical practice, the potential danger of respondents not being able to introspectively assess a given question with complete accuracy could lead to certain responses being given and perhaps a social desirability bias depending on the patient's situation. As for patients asking a physician for euthanasia, they might feel the need to exaggerate their suffering and portray it as more frequently occuring and intense.
Although the study protocol had set a duration of approximately 1 h for interviews, according to existing guidelines,Reference Willis47 some interviews lasted more than 2 h as a result of the many personal memories, feelings and thoughts shared with the interviewer. Future research should take this into account when research concerns a sensitive topic that is regularly considered taboo, as participants may use this opportunity to discuss the topic in an open, non-judgemental setting. The interviews were not recorded to ensure participant's privacy and candid conversation, which also may have affected participants' openness as well as accurate conversation reproduction (note that any quotes used in this text were verified by the respective participants).
In short
With regards to content, the results are consistent with the observation that suffering is not limited to the psychological or physical symptoms of the disorder.Reference Verhofstadt, Thienpont and Peters22,Reference Dees, Vernooij-Dassen, Dekkers, Vissers and van Weel31,Reference Anderson57 It is noteworthy that participants stressed the importance of a good patient–physician relationship, characterised by open, empathic and transparent communication, with respect to patients' perspectives and opinions on suffering and, if applicable, a wish to die. Many of these patient needs have been raised in former research endeavours in the context of euthanasia.Reference Dees, Vernooij-Dassen, Dekkers, Elwyn, Vissers and van Weel25,Reference Back, Starks, Hsu, Gordon, Bharucha and Pearlman58
In our study the fact that one participant who had an ongoing euthanasia request declared that the NEOSi provided insights into many positive aspects of life, leading them to reconsider further treatment options, anecdotally and paradoxically suggests that a comprehensive discussion of the nature and extent of suffering may have preventive effects. This may imply that in a clinical context, shying away from a patients experiences of suffering and eventual wish to die during a consultation with a patient may have the unintended effect of contributing to their suffering and wish to die.
The item pool resulting from this study (available in its entirety in the OSF) can aid understanding about the multidimensional construct of unbearable suffering and can aid professional–patient interactions in clinical practice. A comprehensive discussion on the nature and extent of suffering during physician–patient consultations may have preventive and curative effects. However, as suffering is a multidimensional and complex construct, a purely medical approach might be insufficient, especially in patients with psychiatric conditions. Some correlates of suffering (for example social, socioeconomic and financial difficulties) indicate the need for a broad medical, societal and politic debate. The cognitively valid item pool we present can facilitate these discussions and help achieve a deeper understanding of the entire experience and meaning of unbearable suffering, extending the medical perspective.
As suffering does not seem to be a unidimensional psychological construct and thus difficult to ‘measure’, a fully fledged psychometric quantitative follow-up study to further optimise psychometric qualities (such as the COSMIN checklistReference Mokkink, Terwee and Patrick59) or to determine cut-off scores is ethically and scientifically not appropriate. As the concept of suffering has not been sufficiently examined and is, thus, poorly understood, more qualitative research is required in order to obtain deeper and clearer insight into the construct of suffering and its (underlying) properties. For example, in depth-interviews with patients who persist in and patients who withdraw their euthanasia requests, and interviews with patients, their relatives and physicians on suffering can reveal important insights into the overall concept of suffering.
Once a clear insight into the concept of suffering, specifically within the domain of end-of-life care, has been obtained, further quantitative research might reveal other aspects of suffering that were previously missing. Moreover, as our participants might not be representative of the whole spectrum of suffering in patients with psychiatric conditions, in terms of determinants such as marital status, socioeconomic status and the influence of cultural/religious differences, further research may shed light on the clinical, personal and social backgrounds of patients with psychiatric conditions requesting euthanasia and the (differences in the) nature and extent of their experienced unbearable suffering.
Finally, a longitudinal research design would enable further exploration of changes in suffering experiences over time and differences in patterns of suffering between and within groups. Specifically, comparing the experience of suffering in patients with and without euthanasia requests, as well as in patients who have had their euthanasia requests denied, rejected, still under review or granted, may help us learn more about the relationship between the suffering and the potential outcomes of a euthanasia request.
Funding
M.V. is funded by the Research Foundation Flanders via research project (G017818N) and a PhD fellowship (1162618N).
Acknowledgements
The authors wish to thank all participants for sharing their thoughts and feelings with the interviewer. We also want to acknowledge the psychiatrist, Dr Lieve Thienpont, of the Flemish end-of-life centre ‘Vonkel’ for patient recruitment and follow-up, logistics services and encouragement. Finally, we wish to acknowledge the valuable language editing of Michelle Leisner, a graduate student at Boston University. Complete information is available via the Open Science Framework repository at https://osf.io/bhde3.
Supplementary material
Supplementary material is available online at https://doi.org/10.1192/bjo.2019.25.
Open access
eLetters
No eLetters have been published for this article.