Hostname: page-component-8448b6f56d-cfpbc Total loading time: 0 Render date: 2024-04-24T05:46:26.241Z Has data issue: false hasContentIssue false
  • English
  • Français

A survey of statistical methods utilized for analysis of randomized controlled trials of behavioral interventions

Published online by Cambridge University Press:  06 June 2023

Rebecca Tutino Open the ORCID record for Rebecca Tutino [Opens in a new window] ,
Elizabeth Schofield ,
Rebecca M. Saracino ,
Leah Walsh ,
Emma Straus  and
Christian J. Nelson
Rebecca Tutino*
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA Department of Psychology, Fordham University, Bronx, NY, USA
Elizabeth Schofield
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Rebecca M. Saracino
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Leah Walsh
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA Department of Psychology, Fordham University, Bronx, NY, USA
Emma Straus
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Christian J. Nelson
Affiliation:
Department of Psychiatry and Behavioral Sciences, Memorial Sloan Kettering Cancer Center, New York, NY, USA
*
Corresponding author: Rebecca Tutino; Email: tutinor@mskcc.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives

Given the many statistical analysis options used for randomized controlled trials (RCTs) of behavioral interventions and the lack of clear guidance for analysis selection, the present study aimed to characterize the predominate statistical analyses utilized in RCTs in palliative care and behavioral research and to highlight the relative strengths and weaknesses of each of these methods as guidance for future researchers and reform.

Methods

All RCTs published between 2015 and 2021 were systematically extracted from 4 behavioral medicine journals and analyzed based on prespecified inclusion criteria. Two independent raters classified each of the manuscripts into 1 of 5 RCT analysis strategies.

Results

There was wide variation in the methods used. The 2 most prevalent analyses for RCTs were longitudinal modeling and analysis of covariance. Application of method varied significantly by sample size.

Significance of results

Each statistical analysis presents its own unique strengths and weaknesses. The information resulting from this research may prove helpful for researchers in palliative care and behavioral medicine in navigating the variety of statistical methods available. Future discussion around best practices in RCT analyses is warranted to compare the relative impact of interventions in a more standardized way.

Keywords

StatisticsRandomized control trialANOVAhierarchical level modelingBehavioral clinical trials
Type
Original Article
Information
Palliative & Supportive Care , Volume 22 , Issue 2 , April 2024 , pp. 221 - 225
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bacon, SL, Lavoie, KL, Ninot, G, et al. Spring for the International Behavioural Trials Network (2015) An international perspective on improving the quality and potential of behavioral clinical trials. Current Cardiovascular Reports 9(1), 1–6.Google Scholar
Breitbart, W, Pessin, H, Rosenfeld, B, et al. (2018) Individual meaning-centered psychotherapy for the treatment of psychological and existential distress: A randomized controlled trial in patients with advanced cancer. Cancer 124(15), 3231–3239.CrossRefGoogle ScholarPubMed
Breznau, N, Rinke, EM, Wuttke, A, et al. (2022) Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proceedings of the National Academy of Sciences of the United States of America 119(44) .Google ScholarPubMed
Funderburk, JS, Shepardson, RL, Wray, J, et al. (2018) Behavioral medicine interventions for adult primary care settings: A review. Families, Systems, & Health 36(3), 368–399.10.1037/fsh0000333CrossRefGoogle ScholarPubMed
IBM Corp (2020) IBM SPSS statistics for windows (Version 27.0) [Computer software], Armonk, NY: IBM Corp.Google Scholar
Kalish, LA and Begg, CB (1985) Treatment allocation methods in clinical trials: A review. Statistics in Medicine 4, 129–144.10.1002/sim.4780040204CrossRefGoogle ScholarPubMed
Marszalek, JM, Barber, C, Kohlhart, J, et al. (2011) Sample size in psychological research over the past 30 years. Perceptual and Motor Skills 112, 331–348.10.2466/03.11.PMS.112.2.331-348CrossRefGoogle ScholarPubMed
Maxwell, SE, Delaney, HD and Kelley, K (2017) Designing Experiments and Analyzing Data: A Model Comparison Perspective. New York: Psychology Press, Taylor & Francis Group.CrossRefGoogle Scholar
Nature (2019) It’s time to talk about ditching statistical significance. Nature 567, .Google Scholar
Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349 (6251), .Google Scholar
Penzien, DB, Andrasik, F, Freidenberg, BM, et al. (2005) Guidelines for trials of behavioral treatments for recurrent headache: American Headache Society Behavioral Clinical Trials Workgroup. Headache: The Journal of Head and Face Pain 45, S110–S32.Google ScholarPubMed
Peterson, D and Panofsky, A (2023) Metascience as a scientific social movement. Minerva, 1–28.CrossRefGoogle Scholar
Rudestam, K and Newton, R (2012) Statistical Consultant: Answers to Your Data Analysis Questions. Thousand Oaks, CA: Sage Publications.Google Scholar
Sibbald, B and Roland, M (1998) Understanding controlled trials. Why are randomized controlled trials important? BMJ 316, .Google Scholar
Vickers, AJ (2001) The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: A simulation study. BMC Medical Research Methodology 1, .CrossRefGoogle Scholar
Vickers, AJ (2005a) Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Medical Research Methodology 5, 1–12.CrossRefGoogle ScholarPubMed
Vickers, AJ (2005b) Analysis of variance is easily misapplied in the analysis of randomized trials: A critique and discussion of alternative statistical approaches. Psychosomatic Medicine 67, 652–655.CrossRefGoogle ScholarPubMed
Warne, RT (2014) A primer on multivariate analysis of variance (MANOVA) for behavioral scientists. Practical Assessment, Research & Evaluation 19 1-10 .Google Scholar
Wasserstein, RL and Lazar, NA (2016) The ASA statement on p-values: Context, process, and purpose. The American Statistician 70(2), 129–133.10.1080/00031305.2016.1154108CrossRefGoogle Scholar
Wilkinson, L Task Force on Statistical Inference, American Psychological Association, Science Directorate (1999) Statistical methods in psychology journals: Guidelines and explanations. American Psychologist 54(8), 594–604.CrossRefGoogle Scholar
Yeung, V, Sharpe, L, Geers, A, et al. (2020) Choice, expectations, and the placebo effect for sleep difficulty. Annals of Behavioral Medicine 54, 94–107.Google ScholarPubMed