Skip to main content Accessibility help
×
Home

One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments

  • Patrick Onghena (a1), Bart Michiels (a1), Laleh Jamshidi (a1), Mariola Moeyaert (a2) and Wim Van den Noortgate (a1)...

Abstract

This paper presents a unilevel and multilevel approach for the analysis and meta-analysis of single-case experiments (SCEs). We propose a definition of SCEs and derive the specific features of SCEs’ data that have to be taken into account when analysing and meta-analysing SCEs. We discuss multilevel models of increasing complexity and propose alternative and complementary techniques based on probability combining and randomisation test wrapping. The proposed techniques are demonstrated with real-life data and corresponding R code.

Copyright

Corresponding author

Address for correspondence: Patrick Onghena, Faculty of Psychology and Educational Sciences, KU Leuven, University of Leuven, Tiensestraat 102, BE-3000 Leuven, Belgium. E-mail: patrick.onghena@kuleuven.be

References

Hide All
Aiken, L.S., West, S.G., & Pitts, S.C. (2003). Multiple linear regression. In Schinka, J. & Velicer, W. (Eds.), Comprehensive handbook of psychology: Vol. 2. Research methods in psychology (pp. 481507. New York, NY: Wiley.
Anumendem, N.D., De Fraine, B., Onghena, P., & Van Damme, J. (2013). The impact of coding time on the estimation of school effects. Quality & Quantity, 47, 10211040. doi:10.1007/s11135-011-9581-3.
Araujo, A., Julious, S., & Senn, S. (2016). Understanding variation in sets of N-of-1 trials. PLoS ONE, 11 (12), e0167167. doi:10.1371/journal.pone.0167167.
Baek, E., Moeyaert, M., Petit-Bois, M., Beretvas, S., Van den Noortgate, W., & Ferron, J. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24, 590606. doi:10.1080/09602011.2013.835740.
Barlow, D.H., Nock, M.K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Pearson.
Bates, D.M. (2010). Lme4: Mixed-effects modeling with R. Springer. Retrieved from http://lme4.r-forge.r-project.org/book/.
Beretvas, S.N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129141. doi:10.1080/17489530802446302.
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012). The nuts and bolts of PROSPERO: An international prospective register of systematic reviews. Systematic Reviews, 1, 2. doi:10.1186/2046-4053-1-2.
Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley. doi:10.1002/ 9780470743386.ch43.
Box, G.E.P., Jenkins, G.M., Reinsel, G.C., & Ljung, G.M. (2016). Time series analysis: Forecasting and control (5th ed.). New York, NY: Wiley.
Brosnan, J., Moeyaert, M., Brooks Newsome, K., Healy, O., Heyvaert, M., Onghena, P., & Van den Noortgate, W. (2017). Multilevel analysis of multiple-baseline data evaluating precision teaching as an intervention for improving fluency in foundational reading skills for at risk readers. Exceptionality. doi:10.1080/09362835.2016.1238378.
Bulté, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467478. doi:10.3758/BRM.40.2.467.
Bulté, I., & Onghena, P. (2009). Randomization tests for multiple baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477485. doi:10.3758/BRM.41.2.477.
Bulté, I., & Onghena, P. (2013). The Single-Case Data Analysis package: Analysing single-case experiments with R software. Journal of Modern Applied Statistical Methods, 12, 450478. doi:10.22237/jmasm/1383280020.
Carey, T.A., & Stiles, W.B. (2016). Some problems with randomized controlled trials and some viable alternatives. Clinical Psychology & Psychotherapy, 23, 8795. doi: 10.1002/cpp.1942.
Cassell, D.L. (2002). A randomization-test wrapper for SAS® PROCs. SAS User's Group International Proceedings, 27, 108111. Retrieved from http://www.lexjansen.com/wuss/2002/WUSS02023.pdf.
Chen, L.-T., Peng, C.-Y.J., & Chen, M.-E. (2015). Computing tools for implementing standards for single-case designs. Behavior Modification, 39, 835869. doi: 10.1177/0145445515603706.
Cohen, L.L., Feinstein, A., Masuda, A., & Vowles, K.E. (2014). Single-case research design in pediatric psychology: Considerations regarding data analysis. Journal of Pediatric Psychology, 39, 124137. doi:10.1093/jpepsy/jst065.
Cools, W., Van den Noortgate, W., & Onghena, P. (2009). Design efficiency for imbalanced multilevel data. Behavior Research Methods, 41, 192203. doi:10.3758/BRM.41.1.192.
Cools, W., Van den Noortgate, W., & Onghena, P. (2008). ML-DEs: A program for designing efficient multilevel studies. Behavior Research Methods, 40, 236249. doi:10.3758/BRM.40.1.236.
Cooper, H., Hedges, L.V., & Valentine, J.C. (Eds.) (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY: Russell Sage Foundation.
Dart, E.H., & Radley, K.C. (2017). The impact of ordinate scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105118. doi:10.1016/j.jsp.2017.03.008.
de Vries, R.M., & Morey, R.D. (2013). Bayesian hypothesis testing for single-subject designs. Psychological Methods, 18, 165185. doi: 10.1037/ a0031037.
Douglas, J.M., Knox, L., De Maio, C., & Bridge, H. (2015). Improving communication-specific coping after traumatic brain injury: Evaluation of a new treatment using single-case experimental design. Brain Impairment, 15, 190201. doi: 10.1017/BrImp.2014.25.
Dugard, P., File, P., & Todman, J. (2011). Single-case and small-N experimental designs. New York, NY: Routledge Academic.
Edgington, E.S. (1967). Statistical inference from N=1 experiments. The Journal of Psychology, 65, 195199.
Edgington, E.S. (1972). An additive method for combining probability values from independent experiments. The Journal of Psychology, 80, 351363. doi:10.1080/00223980.1972.9924813.
Edgington, E.S. (1975). Randomization tests for one-subject operant experiments. Journal of Psychology, 90, 5768. doi:10.1080/00223980.1975.9923926.
Edgington, E.S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235251. doi:10.2307/1164966.
Edgington, E.S. (1996). Randomized single-subject experimental designs. Behaviour Research and Therapy, 34, 567574. doi:10.1016/ 0005-7967(96)00012-5.
Edgington, E.S., & Onghena, P. (2007). Randomization tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.
Eysenck, H.J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517. Retrieved from https://doi.org/10.1037/0003-066X.33.5.517.a.
Fay, M. (2010). Exact or asymptotic permutation tests: The perm package (version 1.0-0.0) in R. Retrieved from https://cran.r-project.org/web/packages/perm/perm.pdf.
Ferron, J.M., & Levin, J.R. (2014). Single-case permutation and randomization statistical tests: Present status, promising new developments. In Kratochwill, T.R. & Levin, J.R. (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 153183). Washington, DC: American Psychological Association.
Ferron, J.M., Farmer, J.L., & Owens, C.M. (2010). Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches. Behavior Research Methods, 42, 930943. doi:10.3758/BRM.42.4.930.
Fisher, R.A. (1925). Statistical methods for research workers. Edinburgh, UK: Oliver and Boyd.
Fisher, R.A. (1938). Presidential address. Sankhyā: The Indian Journal of Statistics, 4, 1417.
Gast, D.L., & Ledford, J.R. (2014). Single case research methodology: Applications in special education and behavioral sciences (2nd ed.). New York, NY: Routledge.
Gentile, J.R., Roden, A.H., & Klein, R.D. (1972). An analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193198.
Guyatt, G., Jaeschke, R., & McGinn, T. (2002). PART 2B1: Therapy and validity. N-of-1 randomized controlled trials. In Guyatt, G., Rennie, D., Meade, M.O., & Cook, D.J. (Eds.), Users’ guides to the medical literature (pp. 275290). New York, NY: McGraw-Hill.
Harrington, M., & Velicer, W.F. (2015). Comparing visual and statistical analysis in single-case studies using published studies. Multivariate Behavioral Research, 50, 162183. doi:10.1080/00273171.2014.973989.
Hartley, H.O., & Hocking, R.R. (1971). The analysis of incomplete data. Biometrics, 27, 783823.
Heyvaert, M., & Onghena, P. (2014). Randomization tests for single-case experiments: State of the art, state of the science, and state of the application. Journal of Contextual Behavioral Science, 3, 5164. doi:10.1016/j.jcbs.2013.10.002.
Heyvaert, M., Hannes, K., & Onghena, P. (2017). Using mixed methods research synthesis for literature reviews. Thousand Oaks, CA: Sage.
Heyvaert, M., Maes, B., & Onghena, P. (2010). A meta-analysis of intervention effects on challenging behaviour among persons with intellectual disabilities. Journal of Intellectual Disability Research, 54, 634649. doi:10.1111/j.1365-2788.2010.01291.x.
Heyvaert, M., Maes, B., & Onghena, P. (2013). Mixed methods research synthesis: Definition, framework, and potential. Quality & Quantity, 47, 659676. doi:10.1007/s11135-011-9538-6.
Heyvaert, M., Maes, B., Van den Noortgate, W., Kuppens, S., & Onghena, P. (2012). A multilevel meta-analysis of single-case and small-n research on interventions for reducing challenging behavior in persons with intellectual disabilities. Research in Developmental Disabilities, 33, 766780. doi:10.1016/j.ridd.2011.10.010.
Heyvaert, M., Moeyaert, M., Verkempynck, P., Van den Noortgate, W., Vervloet, M., Ugille, M., & Onghena, P. (2017). Testing the intervention effect in single-case experiments: A Monte Carlo simulation study. Journal of Experimental Education, 85, 175196. doi:10.1080/00220973.2015.1123667.
Heyvaert, M., Wendt, O., Van den Noortgate, W., & Onghena, P. (2015). Randomization and data-analysis items in quality standards for single-case experimental studies. Journal of Special Education, 49, 146156. doi:10.1177/0022466914525239.
Horner, R.H., Carr, E.G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165179. doi:10.1177/001440290507100203.
Huitema, B.E., & McKean, J.W. (2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60, 3858. doi:10.1177/00131640021970358.
Jacobson, K.H. (2017). Introduction to health research methods (2nd ed.). Burlington, MA: Jones & Bartlett.
Jamshidi, L., Heyvaert, M., Declercq, L., Fernández Castilla, B., Ferron, J. M., Moeyaert, M., . . . Van den Noortgate, W. (2017). Review of single-subject experimental design meta-analyses and reviews: 1985–2015. Manuscript submitted.
Jones, R.R., Weinrot, R., & Vaught, R.S. (1978). Effects of serial dependence on the agreement between visual and statistical inferences. Journal of Applied Behavioral Analysis, 11, 277283. doi:10.1901/jaba.1978.11-277.
Kaptchuk, T.J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54, 541549. doi:10.1016/S0895-4356(00)00347-4.
Kazdin, A.E. (2011). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
Koch, G.G., & Gillings, D.B. (1984). Inference, design based vs. model based. In Johnson, N.L. & Kotz, S. (Eds.), Encyclopedia of statistical sciences, vol. 4 (pp. 8488). New York, NY: Wiley.
Kratochwill, T.R., & Levin, J.R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124144. doi:10.1037/a0017736.
Kratochwill, T.R., & Levin, J.R. (2014). Meta- and statistical analysis of single-case intervention research data: Quantitative gifts and a wish list. Journal of School Psychology, 52, 231235. doi:10.1016/j.jsp.2014.01.003.
Kratochwill, T.R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010 ). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
Kratochwill, T.R., Hitchcock, J.H., Horner, R.H., Levin, J.R., Odom, S.L., Rindskopf, D.M., & Shadish, W.R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 2638. doi:10.1177/0741932512452794.
Kravitz, R.L., & Duan, N. (2014). The DEcIDE Methods Center N-of-1 Guidance Panel. In Duan, N., Eslick, I., Gabler, N.B., Kaplan, H.C., Kravitz, R.L., Larson, E.B., Pace, W.D., Schmid, C.H., Sim, I., & Vohra, S. (Eds.), Design and implementation of N-of-1 trials: A user's guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from https://effectivehealthcare.ahrq.gov/topics/n-1-trials/research-2014-5/.
Kuppens, S., & Onghena, P. (2010). Are there enough pieces to unravel the puzzle? A method to determine sufficiency in single-case research synthesis. Annual meeting of the american educational research association (AERA). Denver, 30 April–4 May 2010.
Kuppens, S., & Onghena, P. (2012). Sequential meta-analysis to determine the sufficiency of cumulative knowledge: The case of early intensive behavioral intervention for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 6, 168176. doi:10.1016/j.rasd.2011.04.002.
Kuppens, S., Heyvaert, M., Van den Noortgate, W., & Onghena, P. (2011). Sequential meta-analysis of single-case experimental data. Behavior Research Methods, 43, 720729. doi:10.3758/s13428-011-0080-1.
Lakens, D., Hilgard, J., & Staaks, J. (2016). On the reproducibility of meta-analyses: Six practical recommendations. BMC Psychology, 4, 24. doi:10.1186/s40359-016-0126-3.
Lane, J.D., & Gast, D.L. (2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24, 445463. doi:10.1080/09602011.2013.815636.
Levin, J.R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13 (2), 252. doi:10.22237/jmasm/1414814460.
Little, R.J.A., & Rubin, D.B. (1987). Statistical analysis with missing data. New York, NY: Wiley.
Lobo, M.A., Moeyaert, M., Cunha, A.B., & Babik, I. (2017). Single-case design, analysis, and quality assessment for intervention research. Journal of Neurologic Physical Therapy, 41, 187197. doi:10.1097/NPT.0000000000000187.
Manolov, R., & Moeyaert, M. (2017a). How can single-case data be analyzed? Software resources, tutorial, and reflections on analysis. Behavior Modification, 41, 179228. doi:10.1177/0145445516664307.
Manolov, R., & Moeyaert, M. (2017b). Recommendations for choosing single-case data analytical techniques. Behavior Therapy, 48, 97114. doi:10.1016/j.beth.2016.04.008.
Manolov, R., & Solanas, A. (2009). Percentage of nonoverlapping corrected data. Behavior Research Methods, 41, 12621271. doi:10.3758/BRM.41.4.1262.
McCullagh, P., & Nelder, J. (1989). Generalized linear models (2nd ed.). Boca Raton, FL: Chapman and Hall/CRC.
McDonald, S., & Davidson, K.W. (2016). Using N-of-1 methodology to study or change health-related behaviour. The European Health Psychologist, 18, 3842.
Michiels, B., & Onghena, P. (2017). Nonparametric meta-analysis for single-case research: Confidence intervals for combined effect sizes. Manuscript submitted for publication.
Michiels, B., Heyvaert, M., Meulders, A., & Onghena, P. (2017). Confidence intervals for single-case effect size measures based on randomization test inversion. Behavior Research Methods, 49, 363381. doi: 10.3758/s13428-016-0714-4.
Moeyaert, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-subject experimental data. Journal of School Psychology, 52, 191211. doi:10.1016/j.jsp.2013.11.003.
Moeyaert, M., Rindskopf, D., Onghena, P., & Van den Noortgate, W. (2017). Multilevel modeling of single-case data: A comparison of maximum likelihood and Bayesian estimation. Psychological Methods. doi:10.1037/met0000136.
Moeyaert, M., Ugille, M., Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2016). Methods for dealing with multiple outcomes in meta-analysis: A comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology: Theory & Practice. doi:10.1080/13645579.2016. 1252189.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2013a). The three-level synthesis of standardized single-subject experimental data: A Monte Carlo simulation study. Multivariate Behavioral Research, 48, 719748. doi:10.1080/00273171.2013.816621.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2013b). Three-level analysis of single-case experimental data: Empirical validation. Journal of Experimental Education, 82, 121. doi:10.1080/00220973.2012.745470.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2014). The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-case experimental design research. Behavior Modification, 38, 665704. doi:10.1177/0145445514535243.
Moeyaert, M., Ugille, M., Ferron, J., Onghena, P., Heyvaert, M, & Van den Noortgate, W. (2015). Estimating intervention effects across different types of single-subject experimental designs: Empirical illustration. School Psychology Quarterly, 25, 191211. doi:10.1037/spq0000068.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & the PRISMA group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. PLoS Medicine, 6 (7), e1000097. doi:10.1371/journal.pmed1000097.
Molenaar, P.C.M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement: Interdisciplinary Research and Perspectives, 2, 201211. doi:10.1207/s15366359mea0204_1.
Molenaar, P.C.M., & Campbell, C.G. (2009). The new person-specific paradigm in psychology. Current Directions in Psychology, 18, 112117. doi:10.1111/j.1467-8721.2009.01619.x.
Moore, D.S., McCabe, G.P., & Craig, B.A. (2017). Introduction to the practice of statistics (9th ed.). New York, NY: D. H. Freeman.
Onghena, P. (1992). Randomization tests for extensions and variations of ABAB single-case experimental designs: A rejoinder. Behavioral Assessment, 14, 153171.
Onghena, P. (2005). Single-case designs. In Everitt, B. & Howell, D. (Eds.), Encyclopedia of statistics in behavioral science, vol. 4 (pp. 18501854). Chichester, UK: Wiley.
Onghena, P. (2007). N-of-1 randomized clinical trials. In Berger, V. (Ed.), Design and analysis of randomized clinical trials: Design, analysis & theory. London: Henry Stewart Talks Ltd. Retrieved from https://hstalks.com/t/555/.
Onghena, P. (2016). Randomization in N-of-1 clinical trials: Is it possible to draw causal inferences from single-patient data? In Berger, V. (Ed.), The risk of bias in randomized clinical trials. London: Henry Stewart Talks Ltd. Retrieved from https://hstalks.com/bs/3311/.
Onghena, P., & Edgington, E.S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21, 5668.
Onghena, P., & Struyve, C. (2015). Case studies. In Balakrishnan, N., Brandimarte, P., Everitt, B., Molenberghs, G., Piegorsch, W., & Ruggeri, F. (Eds.), Wiley StatsRef: Statistics reference online (pp. 15). Chichester, UK: Wiley. doi:10.1002/9781118445112.stat06656.pub2.
Onghena, P., Vlaeyen, J.W.S., & de Jong, J. (2007). Randomized replicated single-case experiments: Treatment of pain-related fear by graded exposure in vivo. In Sawilowsky, S. (Ed.), Real data analysis (pp. 387396). Charlotte, NC: Information Age Publishing.
Ottenbacher, K.J. (1993). Interrater agreement of visual analysis in single-subjects designs: Quantitative review and analysis. American Journal of Mental Retardation, 98, 135142.
Ottoboni, K., Lewis, F., & Salmaso, L. (2017). A comparison of parametric and permutation tests for regression analysis of randomized experiments. arXiv:1702.04851v1.
Peat, J. (2002). Health science research: A handbook of quantitative methods. London, UK: Sage.
Perdices, M., & Tate, R.L. (2009). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychological Rehabilitation, 19, 904927. doi:10.1080/09602010903040691.
Pesarin, F., & Salmaso, L. (2010). Permutation tests for complex data: Theory, applications and software. Chichester, UK: Wiley.
Pinheiro, J.C., & Bates, D.M. (2000). Mixed-effects models in S and S-PLUS. New York: Springer.
Piwek, L., Ellis, D. A., Andrews, S., & Joinson, A. (2016). The rise of consumer health wearables: Promises and barriers. PLoS Medicine, 13 (2), e1001953. doi:10.1371/journal.pmed.1001953.
Punja, S., Schmid, C.H., Hartling, L., Urichuk, L., Nikles, C.J., & Vohra, S. (2016). To meta-analyze or not to meta-analyze? A combined meta-analysis of N-of-1 trial data with RCT data on amphetamines and methylphenidate for pediatric ADHD. Journal of Clinical Epidemiology, 76, 7681. doi:10.1016/j.jclinepi.2016.03.021.
Pustejovsky, J.E., Hedges, L.V., & Shadish, W.R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39, 368393. doi:10.3102/1076998614547577.
Rosenthal, R. (1978). Combining the results of independent studies. Psychological Bulletin, 85, 185193. doi:10.1037/0033-2909.85.1.185.
Sackett, D.L., Rosenberg, W.M., Gray, J.A., Haynes, R.B., & Richardson, W.S. (1996). Evidence based medicine: What it is and what it isn't. BMJ, 312, 7172. doi:10.1136/bmj.312.7023.71.
Scargle, J.D. (2000). Publication bias: The “file-drawer” problem in scientific inference. Journal of Scientific Exploration, 14, 91106.
Schlosser, R.W., Lee, D.L., & Wendt, O. (2008). Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics. Evidence-Based Communication Assessment and Intervention, 2, 163187. doi:10.1080/17489530802505412.
Schork, N.J. (2015). Personalized medicine: Time for one-person trials. Nature, 520 (7549), 609611. doi:10.1038/520609a.
Scruggs, T.E., & Mastropieri, M.A. (2013). PND at 25: Past, present, and future trends in summarizing single-subject research. Remedial and Special Education, 34, 919. doi:10.1177/0741932512440730.
Senn, S.J. (2017, 8 February 2017). Consult two medics and you'll get two opinions but consult two statisticians and you could easily get three #thewonderofstats [Twitter moment]. Retrieved from https://twitter.com/stephensenn/status/829593923123343363.
Shadish, W.R. (2014). Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology, 52, 109122. Retrieved from http://doi.org/10.1016/j.jsp.2013.11.009.
Shadish, W.R., & Sullivan, K.J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971980. doi:10.3758/s13428-011-0111-y.
Shadish, W.R., Kyse, E.N., & Rindskopf, D.M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18, 385405. doi:10.1037/a0032964.
Shadish, W.R., Rindskopf, D.M., & Boyajian, J.G. (2016). Single-case experimental design yielded an effect estimate corresponding to a randomized controlled trial. Journal of Clinical Epidemiology, 76, 8288. doi:10.1016/j.jclinepi.2016.01.035.
Shadish, W.R., Zelinsky, N.A., Vevea, J.L., & Kratochwill, T.R. (2016). A survey of publication practices of single-case design researchers when treatments have small or large effects. Journal of Applied Behavioral Analysis, 49, 656673. doi: 10.1002/jaba.308.
Shamseer, L., Sampson, M., Bukutu, C., Schmid, C.H., Nikles, J., Tate, R., . . . & the CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015: Explanation and elaboration. British Medical Journal, 350, h1793. doi:10.1136/bmj/h1793.
Shapiro, M.B. (1966). The single case in clinical-psychological research. The Journal of General Psychology, 74, 323. doi:10.1080/ 00221309.1966.9710306.
Shea, B.J., Hamel, C., Wells, G.A., Bouter, L.M., Kristjansson, E., Grimshaw, J., . . . Boers, M. (2009). AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology, 62, 10131020. doi:10.1016/j.jclinepi.2008.10.009.
Shine, L.C., & Bower, S.M. (1971). A one-way analysis of variance for single-subject designs. Educational and Psychological Measurement, 31, 105113. doi:10.1177/001316447103100108.
Sidman, M. (1952). A note on functional relations obtained from group data. Psychological Bulletin, 49, 263269. doi:10.1037/h0063643.
Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 13591366. doi:10.1177/0956797611417632.
Smith, J.D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510550. doi:10.1037/a0029312.
Solmi, F., & Onghena, P. (2014). Combining P-values in replicated single-case experiments with multivariate outcome. Neuropsychological Rehabilitation, 24, 607633. doi:10.1080/09602011.2014.881747.
Solomon, B.G. (2014). Violations of assumptions in school-based single-case data: Implications for the selection and interpretation of effect sizes. Behavior Modification, 38, 477496. doi:10.1177/0145445513510931.
Sterba, S.K. (2009). Alternative model-based and design-based frameworks for inference from samples to populations: From polarization to integration. Multivariate Behavioral Research, 44, 711740. doi:10.1080/00273170903333574.
Strube, M.J., & Miller, R.H. (1986). Comparison of power rates for combined probability procedures: A simulation study. Psychological Bulletin, 99, 407415. doi:10.1037/0033-2909. 99.3.407.
Tate, R. L., Aird, V., & Taylor, C. (2013). Bringing single-case methodology into the clinic to enhance evidence-based practices. Brain Impairment, 13, 347359. doi:10.1017/BrImp.2012.32.
Tate, R.L., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W., . . . Vohra, S. (2016a). The single-case reporting guideline in BEhavioural Interventions (SCRIBE) 2016: Explanation and elaboration. Archives of Scientific Psychology, 4, 1031. doi:10.1037/arc0000027.
Tate, R.L., Perdices, M., Rosenkoetter, U., Shadish, W., Vohra, S., Barlow, D. H., . . . Wilson, B. (2016b). The single-case reporting guideline In BEhavioural interventions (SCRIBE) 2016 statement. Archives of Scientific Psychology, 4, 19. doi:10.1037/arc0000026.
Tate, R.L., Perdices, M., Rosenkoetter, U., Wakim, D., Godbee, K., Togher, L., & McDonald, S. (2013). Revision of a method quality rating scale for single-case experimental designs and N-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23, 619638. Retrieved from https://doi.org/10.1080/09602011.2013.824383.
Terrin, N., Schmid, C.H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22, 21132126. doi:10.1002/sim.1461.
Tierney, J.F., Vale, C., Riley, R., Smith, C.T., Stewart, L., Clarke, M., & Rovers, M. (2015). Individual participant data (IPD) meta-analyses of randomised controlled trials: Guidance on their use. PLoS Medicine, 12 (7), e1001855. doi:10.1371/journal.pmed.1001855.
Tukey, J.W. (1969). Analyzing data: Sanctification or detective work? American Psychologist, 24, 8391. doi:10.1037/h0027108.
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., . . . Moher, D. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database of Systematic Reviews, 11, MR000030. Retrieved from https://doi.org/10.1002/14651858.mr000030.pub2.
Ugille, M., Moeyaert, M., Beretvas, S., Ferron, J., & Van den Noortgate, W. (2012). Multilevel meta-analysis of single-subject experimental designs: A simulation study. Behavior Research Methods, 44, 12441254. doi:10.3758/s13428-012-0213-1.
Vallverdú, J. (2016). Bayesians versus frequentists: A philosophical debate on statistical reasoning. New York, NY: Springer.
Van den Noortgate, W., & Onghena, P. (2003a). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325346.
Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments & Computers, 35, 110. doi:10.3758/BF03195492.
Van den Noortgate, W., & Onghena, P. (2003c). Multilevel meta-analysis: a comparison with traditional meta-analytical procedures. Educational and Psychological Measurement, 63, 765790. doi:10.1177/0013164403251027.
Van den Noortgate, W., & Onghena, P. (2007). The aggregation of single-case results using hierarchical models. Behavior Analyst Today, 8, 196208. doi:10.1037/h0100613.
Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-Based Communication Assessment and Intervention, 2, 142151. doi:10.1080/17489530802505362.
Van den Noortgate, W., López-López, J., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45, 576594. doi:10.3758/s13428-012-0261-6.
Van den Noortgate, W., López-López, J., Marín-Martínez, F., & Sánchez-Meca, J. (2015). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47, 12741294. doi:10.3758/s13428-014-0527-2.
Vanderkerken, L., Heyvaert, M., Maes, B., & Onghena, P. (2013). Psychosocial interventions for reducing vocal challenging behaviour in persons with autistic disorder: A multilevel meta-analysis of single-case experiments. Research in Developmental Disabilities, 34, 45154533. doi:10.1016/j.ridd.2013.09.030.
Velicer, W.F., & Molenaar, P. (2013). Time series analysis. In Schinka, J. & Velicer, W.F. (Eds.), Handbook of psychology: Research methods in psychology (2nd ed., Vol. 50, pp. 628660). New York: Wiley.
Vohra, S., Shamseer, L., Sampson, M., Bukutu, C., Schmid, C.H., Tate, R., . . . & the CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. British Medical Journal, 350, h1738. doi:10.1136/bmj/h1738.
Wheeler, B., & Torchiano, M. (2016). Permutation tests for linear models: the lmPerm package (version 2.1.0) in R. Retrieved from https://cran.r-project.org/web/packages/lmPerm/lmPerm.pdf.
Wicherts, J.M., Veldkamp, C.L.S., Augusteijn, H.E.M., Bakker, M., van Aert, R.C.M., & van Assen, M.A. L.M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. doi:10.3389/fpsyg.2016.01832.

Keywords

Related content

Powered by UNSILO
Type Description Title
UNKNOWN
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 1

 Unknown (295 bytes)
295 bytes
UNKNOWN
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 2

 Unknown (12 bytes)
12 bytes
UNKNOWN
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 3

 Unknown (7 KB)
7 KB
UNKNOWN
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 4

 Unknown (121 bytes)
121 bytes
UNKNOWN
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 5

 Unknown (142 bytes)
142 bytes

One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments

  • Patrick Onghena (a1), Bart Michiels (a1), Laleh Jamshidi (a1), Mariola Moeyaert (a2) and Wim Van den Noortgate (a1)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.