Skip to main content Accessibility help
×
×
Home

The Negative Effects Questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments

  • Alexander Rozental (a1) (a2), Anders Kottorp (a3), David Forsström (a4), Kristoffer Månsson (a1) (a5) (a6), Johanna Boettcher (a7), Gerhard Andersson (a1) (a8), Tomas Furmark (a6) and Per Carlbring (a5) (a9)...
Abstract
Background:

Psychological treatments provide many benefits for patients with psychiatric disorders, but research also suggests that negative effects might occur from the interventions involved. The Negative Effects Questionnaire (NEQ) has previously been developed as a way of determining the occurrence and characteristics of such incidents, consisting of 32 items and six factors. However, the NEQ has yet to be examined using modern test theory, which could help to improve the understanding of how well the instrument works psychometrically.

Aims:

The current study investigated the reliability and validity of the NEQ from both a person and item perspective, establishing goodness-of-fit, item bias, and scale precision.

Method:

The NEQ was distributed to 564 patients in five clinical trials at post-treatment. Data were analysed using Rasch analysis, i.e. a modern test theory application.

Results:

(1) the NEQ exhibits fairness in testing across sociodemographics, (2) shows comparable validity for a final and condensed scale of 20 instead of 32 items, (3) uses a rating scale that advances monotonically in steps of 0 to 4, and (4) is suitable for monitoring negative effects on an item-level.

Conclusions:

The NEQ is proposed as a useful instrument for investigating negative effects in psychological treatments, and its newer shorter format could facilitate its use in clinical and research settings. However, further research is needed to explore the relationship between negative effects and treatment outcome, as well as to test it in more diverse patient populations.

Copyright
Corresponding author
*Corresponding author. Email: alexander.rozental@ki.se
References
Hide All
Andersson, G. (2016). Internet-delivered psychological treatments. Annual Review of Clinical Psychology, 12, 157179. doi: https://doi.org/10.1146/annurev-clinpsy-021815-093006
Andersson, G., Carlbring, P. and Hadjistavropoulos, H. D. (2017). Internet-based cognitive behavior therapy. In Hofmann, S. G., and Asmundson, G. J. G. (eds), The Science of Cognitive Behavioral Therapy (pp. 531549). San Diego, CA: Academic Press.10.1016/B978-0-12-803457-6.00021-0
Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561573. doi: https://doi.org/10.1007/BF02293814
Boettcher, J., Rozental, A., Andersson, G. and Carlbring, P. (2014). Side effects in internet-based interventions for social anxiety disorder. Internet Interventions, 1, 311. doi: https://doi.org/10.1016/j.invent.2014.02.002
Bond, T. G. and Fox, C. M. (2013). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. New Jersey, NJ: Psychology Press.10.4324/9781410614575
Boswell, J. F., Kraus, D. R., Miller, S. D. and Lambert, M. J. (2015). Implementing routine outcome monitoring in clinical practice: benefits, challenges, and solutions. Psychotherapy Research, 25, 619. doi: https://doi.org/10.1080/10503307.2013.817696
Castonguay, L. G., Boswell, J. F., Constantino, M. J., Goldfried, M. R. and Hill, C. E. (2010). Training implications of harmful effects of psychological treatments. American Psychologist, 65, 3449. doi: https://doi.org/10.1037/a0017330
Crawford, M. J., Thana, L., Farquharson, L., Palmer, L., Hancock, E., Bassett, P., et al. (2016). Patient experience of negative effects of psychological treatment: results of a national survey. British Journal of Psychiatry, 208, 260265. doi: https://doi.org/10.1192/bjp.bp.114.162628
Fisher, W. (1992). Reliability, separation, strata statistics. Rasch Measurement Transactions, 6, 238. doi: https://doi.org/
Hansen, N. B., Lambert, M. J. and Forman, E. M. (2002). The psychotherapy dose-response effect and its implications for treatment delivery services. Clinical Psychology: Science and Practice, 9, 329343. doi: https://doi.org/10.1093/clipsy.9.3.329
Hofmann, S. G., Asnaani, A., Vonk, I. J. J., Sawyer, A. T. and Fang, A. (2012). The efficacy of cognitive behavioral therapy: a review of meta-analyses. Cognitive Therapy and Research, 36, 427440. doi: https://doi.org/10.1007/s10608-012-9476-1
Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50, 537567. doi: https://doi.org/10.1146/annurev.psych.50.1.537
Ladwig, I., Rief, W. and Nestoriuc, Y. (2014). What are the risks and side effects to psychotherapy? – development of an Inventory for the assessment of Negative Effects of Psychotherapy (INEP). Verhaltenstherapie, 24, 252263. doi: https://doi.org/10.1159/000367928
Lerdal, A., Kottorp, A., Gay, C., Aouizerat, B. E., Lee, K. A. and Miaskowski, C. (2016). A Rasch analysis of assessments of morning and evening fatigue in oncology patients using the Lee Fatigue Scale. Journal of Pain and Symptom Management, 51, 10021012. doi: https://doi.org/10.1016/j.jpainsymman.2015.12.331
Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3, 85106.
Linacre, J. M. (2004). Rasch model estimation: further topics. Journal of Applied Measurement, 5, 95110.
Linacre, J. M. (2005). Rasch dichotomous model vs. one-parameter logistic model. Rasch Measurement Transactions, 19, 1032.
Lindner, P., Frykheden, O., Forsström, D., Andersson, E., Ljótsson, B., Hedman, E., et al. (2016). The Brunnsviken Brief Quality of life scale (BBQ): development and psychometric evaluation. Cognitive Behaviour Therapy, 45, 182195. doi: https://doi.org/10.1080/16506073.2016.1143526
Loerinc, A. G., Meuret, A. E., Twohig, M. P., Rosenfield, D., Bluett, E. J. and Craske, M. G. (2015). Response rates for CBT for anxiety disorders: need for standardized criteria. Clinical Psychology Review, 42, 7282. doi: https://doi.org/10.1016/j.cpr.2015.08.004
Lovibond, P. F. and Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33, 335343. doi: https://doi.org/10.1016/0005-7967(94)00075-U
Löwe, B., Kroenke, K., Herzog, W. and Gräfe, K. (2004). Measuring depression outcome with a brief self-report instrument: sensitivity to change of the Patient Health Questionnaire (PHQ-9). Journal of Affective Disorders, 81, 6166. doi: https://doi.org/10.1016/S0165-0327(03)00198-8
Mantel, N. (1963). Chi-square tests with one degree of freedom; extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690700. doi: https://doi.org/10.2307/2282717
McHugh, R. K. and Barlow, D. H. (2010). The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. American Psychologist, 65, 7384. doi: https://doi.org/10.1037/a0018121
Mechler, J. and Holmqvist, R. (2016). Deteriorated and unchanged patients in psychological treatment in Swedish primary care and psychiatry. Nordic Journal of Psychiatry, 70, 1623. doi: https://doi.org/10.3109/08039488.2015.1028438
Moritz, S., Fieker, M., Hottenrott, B., Seeralan, T., Cludius, B., Kolbeck, K., et al. (2015). No pain, no gain? Adverse effects of psychotherapy in obsessive-compulsive disorder and its relationship to treatment gains. Journal of Obsessive-Compulsive and Related Disorders, 5, 6166. doi: https://doi.org/10.1016/j.jocrd.2015.02.002
Muris, P. and Merckelbach, H. (1996). A comparison of two spider fear questionnaires. Journal of Behavior Therapy and Experimental Psychiatry, 27, 241244. doi: https://doi.org/10.1016/S0005-7916(96)00022-5
Parker, G., Fletcher, K., Berk, M. and Paterson, A. (2013). Development of a measure quantifying adverse psychotherapeutic ingredients: the Experiences of Therapy Questionnaire (ETQ). Psychiatry Research, 206, 293301. doi: https://doi.org/10.1016/J.Psychres.2012.11.026
Patomella, A-H., Tham, K. and Kottorp, A. (2006). P-drive: assessment of driving performance after stroke. Journal of Rehabilitation Medicine, 38, 273279. doi: https://doi.org/10.1080/16501970600632594
Raîche, G. (2005). Critical eigenvalue sizes in standardized residual principal components analysis. Rasch Measurement Transactions, 19, 1012.
Rheker, J., Beisel, S., Kräling, S. and Rief, W. (2017). Rate and predictors of negative effects of psychotherapy in psychiatric and psychosomatic inpatients. Psychiatry Research, 254, 143150. doi: https://doi.org/10.1016/j.psychres.2017.04.042
Rozental, A., Andersson, G., Boettcher, J., Ebert, D. D., Cuijpers, P., Knaevelsrud, C., et al. (2014). Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interventions, 1, 1219. doi: https://doi.org/10.1016/j.invent.2014.02.001
Rozental, A., Boettcher, J., Andersson, G., Schmidt, B. and Carlbring, P. (2015). Negative effects of Internet interventions: a qualitative content analysis of patients’ experiences with treatments delivered online. Cognitive Behaviour Therapy, 44, 223236. doi: https://doi.org/10.1080/16506073.2015.1008033
Rozental, A., Kottorp, A., Boettcher, J., Andersson, G. and Carlbring, P. (2016). Negative effects of psychological treatments: an exploratory factor analysis of the Negative Effects Questionnaire for monitoring and reporting adverse and unwanted events. PLoS One, 11, e0157503. doi: https://doi.org/10.1371/journal.pone.0157503
Rozental, A., Magnusson, K., Boettcher, J., Andersson, G. and Carlbring, P. (2017). For better or worse: an individual patient data meta-analysis of deterioration among participants receiving internet-based cognitive behavior therapy. Journal of Consulting and Clinical Psychology, 85, 160177. doi: https://doi.org/10.1037/ccp0000158
Shea, T. L., Tennant, A. and Pallant, J. F. (2009). Rasch model analysis of the Depression, Anxiety and Stress Scales (DASS). BMC Psychiatry, 9, 21. doi: https://doi.org/10.1186/1471-244X-9-21
Smith, A. B., Rush, R., Fallowfield, L. J., Velikova, G. and Sharpe, M. (2008). Rasch fit statistics and sample size considerations for polytomous data. BMC Medical Research Methodology, 8(33). doi: https://doi.org/10.1186/1471-2288-8-33
Spector, P. E. (1992). Summated Rating Scale Construction: An Introduction. Newbury Park, CA: Sage.10.4135/9781412986038
Spitzer, R. L., Kroenke, K., Williams, J. B. W. and Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder. Archives of Internal Medicine, 166, 1092. doi: https://doi.org/
Strupp, H. H. and Hadley, S. W. (1977). A tripartite model of mental health and therapeutic outcomes. With special reference to negative effects in psychotherapy. The American Psychologist, 32, 187196. doi: https://doi.org/10.1037/0003-066X.32.3.187
Suh, C. S., Strupp, H. H. and O’Malley, S. S. (1986). The Vanderbilt process measures: the Psychotherapy Process Scale (VPPS) and the Negative Indicators Scale (VNIS). In Greenberg, L. S. (ed), The Psychotherapeutic Process: A Research Handbook (pp. 285323). New York, NY, USA: Guilford Press.
Vlaescu, G., Alasjö, A., Miloff, A., Carlbring, P. and Andersson, G. (2016). Features and functionality of the Iterapi platform for internet-based psychological treatment. Internet Interventions, 6, 107114. doi: https://doi.org/10.1016/j.invent.2016.09.006
Waugh, R. and Chapman, E. (2004). An analysis of dimensionality using factor analysis (true-score theory) and Rasch measurement: what is the difference? Which method is better? Journal of Applied Measurement, 6, 8099.
Wright, B. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97116. doi: https://doi.org/10.1111/j.1745-3984.1977.tb00031.x
Wright, B. (1996). Comparing Rasch measurement and factor analysis. Structural Equation Modeling: A Multidisciplinary Journal, 3, 324. doi: https://doi.org/10.1080/10705519609540026
Wright, B. and Masters, G. N. (1982). Rating Scale Analysis: Rasch Measurement. Chicago, IL: MESA Press.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Behavioural and Cognitive Psychotherapy
  • ISSN: 1352-4658
  • EISSN: 1469-1833
  • URL: /core/journals/behavioural-and-cognitive-psychotherapy
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed