Skip to main content Accessibility help

One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments

  • Patrick Onghena (a1), Bart Michiels (a1), Laleh Jamshidi (a1), Mariola Moeyaert (a2) and Wim Van den Noortgate (a1)...


This paper presents a unilevel and multilevel approach for the analysis and meta-analysis of single-case experiments (SCEs). We propose a definition of SCEs and derive the specific features of SCEs’ data that have to be taken into account when analysing and meta-analysing SCEs. We discuss multilevel models of increasing complexity and propose alternative and complementary techniques based on probability combining and randomisation test wrapping. The proposed techniques are demonstrated with real-life data and corresponding R code.


Corresponding author

Address for correspondence: Patrick Onghena, Faculty of Psychology and Educational Sciences, KU Leuven, University of Leuven, Tiensestraat 102, BE-3000 Leuven, Belgium. E-mail:


Hide All
Aiken, L.S., West, S.G., & Pitts, S.C. (2003). Multiple linear regression. In Schinka, J. & Velicer, W. (Eds.), Comprehensive handbook of psychology: Vol. 2. Research methods in psychology (pp. 481507. New York, NY: Wiley.
Anumendem, N.D., De Fraine, B., Onghena, P., & Van Damme, J. (2013). The impact of coding time on the estimation of school effects. Quality & Quantity, 47, 10211040. doi:10.1007/s11135-011-9581-3.
Araujo, A., Julious, S., & Senn, S. (2016). Understanding variation in sets of N-of-1 trials. PLoS ONE, 11 (12), e0167167. doi:10.1371/journal.pone.0167167.
Baek, E., Moeyaert, M., Petit-Bois, M., Beretvas, S., Van den Noortgate, W., & Ferron, J. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24, 590606. doi:10.1080/09602011.2013.835740.
Barlow, D.H., Nock, M.K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Pearson.
Bates, D.M. (2010). Lme4: Mixed-effects modeling with R. Springer. Retrieved from
Beretvas, S.N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129141. doi:10.1080/17489530802446302.
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012). The nuts and bolts of PROSPERO: An international prospective register of systematic reviews. Systematic Reviews, 1, 2. doi:10.1186/2046-4053-1-2.
Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R. (2009). Introduction to meta-analysis. Chichester, UK: Wiley. doi:10.1002/ 9780470743386.ch43.
Box, G.E.P., Jenkins, G.M., Reinsel, G.C., & Ljung, G.M. (2016). Time series analysis: Forecasting and control (5th ed.). New York, NY: Wiley.
Brosnan, J., Moeyaert, M., Brooks Newsome, K., Healy, O., Heyvaert, M., Onghena, P., & Van den Noortgate, W. (2017). Multilevel analysis of multiple-baseline data evaluating precision teaching as an intervention for improving fluency in foundational reading skills for at risk readers. Exceptionality. doi:10.1080/09362835.2016.1238378.
Bulté, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467478. doi:10.3758/BRM.40.2.467.
Bulté, I., & Onghena, P. (2009). Randomization tests for multiple baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477485. doi:10.3758/BRM.41.2.477.
Bulté, I., & Onghena, P. (2013). The Single-Case Data Analysis package: Analysing single-case experiments with R software. Journal of Modern Applied Statistical Methods, 12, 450478. doi:10.22237/jmasm/1383280020.
Carey, T.A., & Stiles, W.B. (2016). Some problems with randomized controlled trials and some viable alternatives. Clinical Psychology & Psychotherapy, 23, 8795. doi: 10.1002/cpp.1942.
Cassell, D.L. (2002). A randomization-test wrapper for SAS® PROCs. SAS User's Group International Proceedings, 27, 108111. Retrieved from
Chen, L.-T., Peng, C.-Y.J., & Chen, M.-E. (2015). Computing tools for implementing standards for single-case designs. Behavior Modification, 39, 835869. doi: 10.1177/0145445515603706.
Cohen, L.L., Feinstein, A., Masuda, A., & Vowles, K.E. (2014). Single-case research design in pediatric psychology: Considerations regarding data analysis. Journal of Pediatric Psychology, 39, 124137. doi:10.1093/jpepsy/jst065.
Cools, W., Van den Noortgate, W., & Onghena, P. (2009). Design efficiency for imbalanced multilevel data. Behavior Research Methods, 41, 192203. doi:10.3758/BRM.41.1.192.
Cools, W., Van den Noortgate, W., & Onghena, P. (2008). ML-DEs: A program for designing efficient multilevel studies. Behavior Research Methods, 40, 236249. doi:10.3758/BRM.40.1.236.
Cooper, H., Hedges, L.V., & Valentine, J.C. (Eds.) (2009). The handbook of research synthesis and meta-analysis (2nd ed.). New York, NY: Russell Sage Foundation.
Dart, E.H., & Radley, K.C. (2017). The impact of ordinate scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105118. doi:10.1016/j.jsp.2017.03.008.
de Vries, R.M., & Morey, R.D. (2013). Bayesian hypothesis testing for single-subject designs. Psychological Methods, 18, 165185. doi: 10.1037/ a0031037.
Douglas, J.M., Knox, L., De Maio, C., & Bridge, H. (2015). Improving communication-specific coping after traumatic brain injury: Evaluation of a new treatment using single-case experimental design. Brain Impairment, 15, 190201. doi: 10.1017/BrImp.2014.25.
Dugard, P., File, P., & Todman, J. (2011). Single-case and small-N experimental designs. New York, NY: Routledge Academic.
Edgington, E.S. (1967). Statistical inference from N=1 experiments. The Journal of Psychology, 65, 195199.
Edgington, E.S. (1972). An additive method for combining probability values from independent experiments. The Journal of Psychology, 80, 351363. doi:10.1080/00223980.1972.9924813.
Edgington, E.S. (1975). Randomization tests for one-subject operant experiments. Journal of Psychology, 90, 5768. doi:10.1080/00223980.1975.9923926.
Edgington, E.S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235251. doi:10.2307/1164966.
Edgington, E.S. (1996). Randomized single-subject experimental designs. Behaviour Research and Therapy, 34, 567574. doi:10.1016/ 0005-7967(96)00012-5.
Edgington, E.S., & Onghena, P. (2007). Randomization tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.
Eysenck, H.J. (1978). An exercise in mega-silliness. American Psychologist, 33, 517. Retrieved from
Fay, M. (2010). Exact or asymptotic permutation tests: The perm package (version 1.0-0.0) in R. Retrieved from
Ferron, J.M., & Levin, J.R. (2014). Single-case permutation and randomization statistical tests: Present status, promising new developments. In Kratochwill, T.R. & Levin, J.R. (Eds.), Single-case intervention research: Methodological and statistical advances (pp. 153183). Washington, DC: American Psychological Association.
Ferron, J.M., Farmer, J.L., & Owens, C.M. (2010). Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches. Behavior Research Methods, 42, 930943. doi:10.3758/BRM.42.4.930.
Fisher, R.A. (1925). Statistical methods for research workers. Edinburgh, UK: Oliver and Boyd.
Fisher, R.A. (1938). Presidential address. Sankhyā: The Indian Journal of Statistics, 4, 1417.
Gast, D.L., & Ledford, J.R. (2014). Single case research methodology: Applications in special education and behavioral sciences (2nd ed.). New York, NY: Routledge.
Gentile, J.R., Roden, A.H., & Klein, R.D. (1972). An analysis of variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193198.
Guyatt, G., Jaeschke, R., & McGinn, T. (2002). PART 2B1: Therapy and validity. N-of-1 randomized controlled trials. In Guyatt, G., Rennie, D., Meade, M.O., & Cook, D.J. (Eds.), Users’ guides to the medical literature (pp. 275290). New York, NY: McGraw-Hill.
Harrington, M., & Velicer, W.F. (2015). Comparing visual and statistical analysis in single-case studies using published studies. Multivariate Behavioral Research, 50, 162183. doi:10.1080/00273171.2014.973989.
Hartley, H.O., & Hocking, R.R. (1971). The analysis of incomplete data. Biometrics, 27, 783823.
Heyvaert, M., & Onghena, P. (2014). Randomization tests for single-case experiments: State of the art, state of the science, and state of the application. Journal of Contextual Behavioral Science, 3, 5164. doi:10.1016/j.jcbs.2013.10.002.
Heyvaert, M., Hannes, K., & Onghena, P. (2017). Using mixed methods research synthesis for literature reviews. Thousand Oaks, CA: Sage.
Heyvaert, M., Maes, B., & Onghena, P. (2010). A meta-analysis of intervention effects on challenging behaviour among persons with intellectual disabilities. Journal of Intellectual Disability Research, 54, 634649. doi:10.1111/j.1365-2788.2010.01291.x.
Heyvaert, M., Maes, B., & Onghena, P. (2013). Mixed methods research synthesis: Definition, framework, and potential. Quality & Quantity, 47, 659676. doi:10.1007/s11135-011-9538-6.
Heyvaert, M., Maes, B., Van den Noortgate, W., Kuppens, S., & Onghena, P. (2012). A multilevel meta-analysis of single-case and small-n research on interventions for reducing challenging behavior in persons with intellectual disabilities. Research in Developmental Disabilities, 33, 766780. doi:10.1016/j.ridd.2011.10.010.
Heyvaert, M., Moeyaert, M., Verkempynck, P., Van den Noortgate, W., Vervloet, M., Ugille, M., & Onghena, P. (2017). Testing the intervention effect in single-case experiments: A Monte Carlo simulation study. Journal of Experimental Education, 85, 175196. doi:10.1080/00220973.2015.1123667.
Heyvaert, M., Wendt, O., Van den Noortgate, W., & Onghena, P. (2015). Randomization and data-analysis items in quality standards for single-case experimental studies. Journal of Special Education, 49, 146156. doi:10.1177/0022466914525239.
Horner, R.H., Carr, E.G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165179. doi:10.1177/001440290507100203.
Huitema, B.E., & McKean, J.W. (2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60, 3858. doi:10.1177/00131640021970358.
Jacobson, K.H. (2017). Introduction to health research methods (2nd ed.). Burlington, MA: Jones & Bartlett.
Jamshidi, L., Heyvaert, M., Declercq, L., Fernández Castilla, B., Ferron, J. M., Moeyaert, M., . . . Van den Noortgate, W. (2017). Review of single-subject experimental design meta-analyses and reviews: 1985–2015. Manuscript submitted.
Jones, R.R., Weinrot, R., & Vaught, R.S. (1978). Effects of serial dependence on the agreement between visual and statistical inferences. Journal of Applied Behavioral Analysis, 11, 277283. doi:10.1901/jaba.1978.11-277.
Kaptchuk, T.J. (2001). The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology, 54, 541549. doi:10.1016/S0895-4356(00)00347-4.
Kazdin, A.E. (2011). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
Koch, G.G., & Gillings, D.B. (1984). Inference, design based vs. model based. In Johnson, N.L. & Kotz, S. (Eds.), Encyclopedia of statistical sciences, vol. 4 (pp. 8488). New York, NY: Wiley.
Kratochwill, T.R., & Levin, J.R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124144. doi:10.1037/a0017736.
Kratochwill, T.R., & Levin, J.R. (2014). Meta- and statistical analysis of single-case intervention research data: Quantitative gifts and a wish list. Journal of School Psychology, 52, 231235. doi:10.1016/j.jsp.2014.01.003.
Kratochwill, T.R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010 ). Single-case designs technical documentation. Retrieved from
Kratochwill, T.R., Hitchcock, J.H., Horner, R.H., Levin, J.R., Odom, S.L., Rindskopf, D.M., & Shadish, W.R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 2638. doi:10.1177/0741932512452794.
Kravitz, R.L., & Duan, N. (2014). The DEcIDE Methods Center N-of-1 Guidance Panel. In Duan, N., Eslick, I., Gabler, N.B., Kaplan, H.C., Kravitz, R.L., Larson, E.B., Pace, W.D., Schmid, C.H., Sim, I., & Vohra, S. (Eds.), Design and implementation of N-of-1 trials: A user's guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality. Retrieved from
Kuppens, S., & Onghena, P. (2010). Are there enough pieces to unravel the puzzle? A method to determine sufficiency in single-case research synthesis. Annual meeting of the american educational research association (AERA). Denver, 30 April–4 May 2010.
Kuppens, S., & Onghena, P. (2012). Sequential meta-analysis to determine the sufficiency of cumulative knowledge: The case of early intensive behavioral intervention for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 6, 168176. doi:10.1016/j.rasd.2011.04.002.
Kuppens, S., Heyvaert, M., Van den Noortgate, W., & Onghena, P. (2011). Sequential meta-analysis of single-case experimental data. Behavior Research Methods, 43, 720729. doi:10.3758/s13428-011-0080-1.
Lakens, D., Hilgard, J., & Staaks, J. (2016). On the reproducibility of meta-analyses: Six practical recommendations. BMC Psychology, 4, 24. doi:10.1186/s40359-016-0126-3.
Lane, J.D., & Gast, D.L. (2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24, 445463. doi:10.1080/09602011.2013.815636.
Levin, J.R., Ferron, J. M., & Gafurov, B. S. (2014). Improved randomization tests for a class of single-case intervention designs. Journal of Modern Applied Statistical Methods, 13 (2), 252. doi:10.22237/jmasm/1414814460.
Little, R.J.A., & Rubin, D.B. (1987). Statistical analysis with missing data. New York, NY: Wiley.
Lobo, M.A., Moeyaert, M., Cunha, A.B., & Babik, I. (2017). Single-case design, analysis, and quality assessment for intervention research. Journal of Neurologic Physical Therapy, 41, 187197. doi:10.1097/NPT.0000000000000187.
Manolov, R., & Moeyaert, M. (2017a). How can single-case data be analyzed? Software resources, tutorial, and reflections on analysis. Behavior Modification, 41, 179228. doi:10.1177/0145445516664307.
Manolov, R., & Moeyaert, M. (2017b). Recommendations for choosing single-case data analytical techniques. Behavior Therapy, 48, 97114. doi:10.1016/j.beth.2016.04.008.
Manolov, R., & Solanas, A. (2009). Percentage of nonoverlapping corrected data. Behavior Research Methods, 41, 12621271. doi:10.3758/BRM.41.4.1262.
McCullagh, P., & Nelder, J. (1989). Generalized linear models (2nd ed.). Boca Raton, FL: Chapman and Hall/CRC.
McDonald, S., & Davidson, K.W. (2016). Using N-of-1 methodology to study or change health-related behaviour. The European Health Psychologist, 18, 3842.
Michiels, B., & Onghena, P. (2017). Nonparametric meta-analysis for single-case research: Confidence intervals for combined effect sizes. Manuscript submitted for publication.
Michiels, B., Heyvaert, M., Meulders, A., & Onghena, P. (2017). Confidence intervals for single-case effect size measures based on randomization test inversion. Behavior Research Methods, 49, 363381. doi: 10.3758/s13428-016-0714-4.
Moeyaert, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2014). From a single-level analysis to a multilevel analysis of single-subject experimental data. Journal of School Psychology, 52, 191211. doi:10.1016/j.jsp.2013.11.003.
Moeyaert, M., Rindskopf, D., Onghena, P., & Van den Noortgate, W. (2017). Multilevel modeling of single-case data: A comparison of maximum likelihood and Bayesian estimation. Psychological Methods. doi:10.1037/met0000136.
Moeyaert, M., Ugille, M., Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2016). Methods for dealing with multiple outcomes in meta-analysis: A comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology: Theory & Practice. doi:10.1080/13645579.2016. 1252189.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2013a). The three-level synthesis of standardized single-subject experimental data: A Monte Carlo simulation study. Multivariate Behavioral Research, 48, 719748. doi:10.1080/00273171.2013.816621.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2013b). Three-level analysis of single-case experimental data: Empirical validation. Journal of Experimental Education, 82, 121. doi:10.1080/00220973.2012.745470.
Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S., & Van den Noortgate, W. (2014). The influence of the design matrix on treatment effect estimates in the quantitative analyses of single-case experimental design research. Behavior Modification, 38, 665704. doi:10.1177/0145445514535243.
Moeyaert, M., Ugille, M., Ferron, J., Onghena, P., Heyvaert, M, & Van den Noortgate, W. (2015). Estimating intervention effects across different types of single-subject experimental designs: Empirical illustration. School Psychology Quarterly, 25, 191211. doi:10.1037/spq0000068.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & the PRISMA group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. PLoS Medicine, 6 (7), e1000097. doi:10.1371/journal.pmed1000097.
Molenaar, P.C.M. (2004). A manifesto on psychology as idiographic science: Bringing the person back into scientific psychology, this time forever. Measurement: Interdisciplinary Research and Perspectives, 2, 201211. doi:10.1207/s15366359mea0204_1.
Molenaar, P.C.M., & Campbell, C.G. (2009). The new person-specific paradigm in psychology. Current Directions in Psychology, 18, 112117. doi:10.1111/j.1467-8721.2009.01619.x.
Moore, D.S., McCabe, G.P., & Craig, B.A. (2017). Introduction to the practice of statistics (9th ed.). New York, NY: D. H. Freeman.
Onghena, P. (1992). Randomization tests for extensions and variations of ABAB single-case experimental designs: A rejoinder. Behavioral Assessment, 14, 153171.
Onghena, P. (2005). Single-case designs. In Everitt, B. & Howell, D. (Eds.), Encyclopedia of statistics in behavioral science, vol. 4 (pp. 18501854). Chichester, UK: Wiley.
Onghena, P. (2007). N-of-1 randomized clinical trials. In Berger, V. (Ed.), Design and analysis of randomized clinical trials: Design, analysis & theory. London: Henry Stewart Talks Ltd. Retrieved from
Onghena, P. (2016). Randomization in N-of-1 clinical trials: Is it possible to draw causal inferences from single-patient data? In Berger, V. (Ed.), The risk of bias in randomized clinical trials. London: Henry Stewart Talks Ltd. Retrieved from
Onghena, P., & Edgington, E.S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21, 5668.
Onghena, P., & Struyve, C. (2015). Case studies. In Balakrishnan, N., Brandimarte, P., Everitt, B., Molenberghs, G., Piegorsch, W., & Ruggeri, F. (Eds.), Wiley StatsRef: Statistics reference online (pp. 15). Chichester, UK: Wiley. doi:10.1002/9781118445112.stat06656.pub2.
Onghena, P., Vlaeyen, J.W.S., & de Jong, J. (2007). Randomized replicated single-case experiments: Treatment of pain-related fear by graded exposure in vivo. In Sawilowsky, S. (Ed.), Real data analysis (pp. 387396). Charlotte, NC: Information Age Publishing.
Ottenbacher, K.J. (1993). Interrater agreement of visual analysis in single-subjects designs: Quantitative review and analysis. American Journal of Mental Retardation, 98, 135142.
Ottoboni, K., Lewis, F., & Salmaso, L. (2017). A comparison of parametric and permutation tests for regression analysis of randomized experiments. arXiv:1702.04851v1.
Peat, J. (2002). Health science research: A handbook of quantitative methods. London, UK: Sage.
Perdices, M., & Tate, R.L. (2009). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychological Rehabilitation, 19, 904927. doi:10.1080/09602010903040691.
Pesarin, F., & Salmaso, L. (2010). Permutation tests for complex data: Theory, applications and software. Chichester, UK: Wiley.
Pinheiro, J.C., & Bates, D.M. (2000). Mixed-effects models in S and S-PLUS. New York: Springer.
Piwek, L., Ellis, D. A., Andrews, S., & Joinson, A. (2016). The rise of consumer health wearables: Promises and barriers. PLoS Medicine, 13 (2), e1001953. doi:10.1371/journal.pmed.1001953.
Punja, S., Schmid, C.H., Hartling, L., Urichuk, L., Nikles, C.J., & Vohra, S. (2016). To meta-analyze or not to meta-analyze? A combined meta-analysis of N-of-1 trial data with RCT data on amphetamines and methylphenidate for pediatric ADHD. Journal of Clinical Epidemiology, 76, 7681. doi:10.1016/j.jclinepi.2016.03.021.
Pustejovsky, J.E., Hedges, L.V., & Shadish, W.R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39, 368393. doi:10.3102/1076998614547577.
Rosenthal, R. (1978). Combining the results of independent studies. Psychological Bulletin, 85, 185193. doi:10.1037/0033-2909.85.1.185.
Sackett, D.L., Rosenberg, W.M., Gray, J.A., Haynes, R.B., & Richardson, W.S. (1996). Evidence based medicine: What it is and what it isn't. BMJ, 312, 7172. doi:10.1136/bmj.312.7023.71.
Scargle, J.D. (2000). Publication bias: The “file-drawer” problem in scientific inference. Journal of Scientific Exploration, 14, 91106.
Schlosser, R.W., Lee, D.L., & Wendt, O. (2008). Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics. Evidence-Based Communication Assessment and Intervention, 2, 163187. doi:10.1080/17489530802505412.
Schork, N.J. (2015). Personalized medicine: Time for one-person trials. Nature, 520 (7549), 609611. doi:10.1038/520609a.
Scruggs, T.E., & Mastropieri, M.A. (2013). PND at 25: Past, present, and future trends in summarizing single-subject research. Remedial and Special Education, 34, 919. doi:10.1177/0741932512440730.
Senn, S.J. (2017, 8 February 2017). Consult two medics and you'll get two opinions but consult two statisticians and you could easily get three #thewonderofstats [Twitter moment]. Retrieved from
Shadish, W.R. (2014). Analysis and meta-analysis of single-case designs: An introduction. Journal of School Psychology, 52, 109122. Retrieved from
Shadish, W.R., & Sullivan, K.J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971980. doi:10.3758/s13428-011-0111-y.
Shadish, W.R., Kyse, E.N., & Rindskopf, D.M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18, 385405. doi:10.1037/a0032964.
Shadish, W.R., Rindskopf, D.M., & Boyajian, J.G. (2016). Single-case experimental design yielded an effect estimate corresponding to a randomized controlled trial. Journal of Clinical Epidemiology, 76, 8288. doi:10.1016/j.jclinepi.2016.01.035.
Shadish, W.R., Zelinsky, N.A., Vevea, J.L., & Kratochwill, T.R. (2016). A survey of publication practices of single-case design researchers when treatments have small or large effects. Journal of Applied Behavioral Analysis, 49, 656673. doi: 10.1002/jaba.308.
Shamseer, L., Sampson, M., Bukutu, C., Schmid, C.H., Nikles, J., Tate, R., . . . & the CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015: Explanation and elaboration. British Medical Journal, 350, h1793. doi:10.1136/bmj/h1793.
Shapiro, M.B. (1966). The single case in clinical-psychological research. The Journal of General Psychology, 74, 323. doi:10.1080/ 00221309.1966.9710306.
Shea, B.J., Hamel, C., Wells, G.A., Bouter, L.M., Kristjansson, E., Grimshaw, J., . . . Boers, M. (2009). AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology, 62, 10131020. doi:10.1016/j.jclinepi.2008.10.009.
Shine, L.C., & Bower, S.M. (1971). A one-way analysis of variance for single-subject designs. Educational and Psychological Measurement, 31, 105113. doi:10.1177/001316447103100108.
Sidman, M. (1952). A note on functional relations obtained from group data. Psychological Bulletin, 49, 263269. doi:10.1037/h0063643.
Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 13591366. doi:10.1177/0956797611417632.
Smith, J.D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510550. doi:10.1037/a0029312.
Solmi, F., & Onghena, P. (2014). Combining P-values in replicated single-case experiments with multivariate outcome. Neuropsychological Rehabilitation, 24, 607633. doi:10.1080/09602011.2014.881747.
Solomon, B.G. (2014). Violations of assumptions in school-based single-case data: Implications for the selection and interpretation of effect sizes. Behavior Modification, 38, 477496. doi:10.1177/0145445513510931.
Sterba, S.K. (2009). Alternative model-based and design-based frameworks for inference from samples to populations: From polarization to integration. Multivariate Behavioral Research, 44, 711740. doi:10.1080/00273170903333574.
Strube, M.J., & Miller, R.H. (1986). Comparison of power rates for combined probability procedures: A simulation study. Psychological Bulletin, 99, 407415. doi:10.1037/0033-2909. 99.3.407.
Tate, R. L., Aird, V., & Taylor, C. (2013). Bringing single-case methodology into the clinic to enhance evidence-based practices. Brain Impairment, 13, 347359. doi:10.1017/BrImp.2012.32.
Tate, R.L., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W., . . . Vohra, S. (2016a). The single-case reporting guideline in BEhavioural Interventions (SCRIBE) 2016: Explanation and elaboration. Archives of Scientific Psychology, 4, 1031. doi:10.1037/arc0000027.
Tate, R.L., Perdices, M., Rosenkoetter, U., Shadish, W., Vohra, S., Barlow, D. H., . . . Wilson, B. (2016b). The single-case reporting guideline In BEhavioural interventions (SCRIBE) 2016 statement. Archives of Scientific Psychology, 4, 19. doi:10.1037/arc0000026.
Tate, R.L., Perdices, M., Rosenkoetter, U., Wakim, D., Godbee, K., Togher, L., & McDonald, S. (2013). Revision of a method quality rating scale for single-case experimental designs and N-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23, 619638. Retrieved from
Terrin, N., Schmid, C.H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22, 21132126. doi:10.1002/sim.1461.
Tierney, J.F., Vale, C., Riley, R., Smith, C.T., Stewart, L., Clarke, M., & Rovers, M. (2015). Individual participant data (IPD) meta-analyses of randomised controlled trials: Guidance on their use. PLoS Medicine, 12 (7), e1001855. doi:10.1371/journal.pmed.1001855.
Tukey, J.W. (1969). Analyzing data: Sanctification or detective work? American Psychologist, 24, 8391. doi:10.1037/h0027108.
Turner, L., Shamseer, L., Altman, D. G., Weeks, L., Peters, J., Kober, T., . . . Moher, D. (2012). Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database of Systematic Reviews, 11, MR000030. Retrieved from
Ugille, M., Moeyaert, M., Beretvas, S., Ferron, J., & Van den Noortgate, W. (2012). Multilevel meta-analysis of single-subject experimental designs: A simulation study. Behavior Research Methods, 44, 12441254. doi:10.3758/s13428-012-0213-1.
Vallverdú, J. (2016). Bayesians versus frequentists: A philosophical debate on statistical reasoning. New York, NY: Springer.
Van den Noortgate, W., & Onghena, P. (2003a). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325346.
Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments & Computers, 35, 110. doi:10.3758/BF03195492.
Van den Noortgate, W., & Onghena, P. (2003c). Multilevel meta-analysis: a comparison with traditional meta-analytical procedures. Educational and Psychological Measurement, 63, 765790. doi:10.1177/0013164403251027.
Van den Noortgate, W., & Onghena, P. (2007). The aggregation of single-case results using hierarchical models. Behavior Analyst Today, 8, 196208. doi:10.1037/h0100613.
Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-Based Communication Assessment and Intervention, 2, 142151. doi:10.1080/17489530802505362.
Van den Noortgate, W., López-López, J., Marín-Martínez, F., & Sánchez-Meca, J. (2013). Three-level meta-analysis of dependent effect sizes. Behavior Research Methods, 45, 576594. doi:10.3758/s13428-012-0261-6.
Van den Noortgate, W., López-López, J., Marín-Martínez, F., & Sánchez-Meca, J. (2015). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47, 12741294. doi:10.3758/s13428-014-0527-2.
Vanderkerken, L., Heyvaert, M., Maes, B., & Onghena, P. (2013). Psychosocial interventions for reducing vocal challenging behaviour in persons with autistic disorder: A multilevel meta-analysis of single-case experiments. Research in Developmental Disabilities, 34, 45154533. doi:10.1016/j.ridd.2013.09.030.
Velicer, W.F., & Molenaar, P. (2013). Time series analysis. In Schinka, J. & Velicer, W.F. (Eds.), Handbook of psychology: Research methods in psychology (2nd ed., Vol. 50, pp. 628660). New York: Wiley.
Vohra, S., Shamseer, L., Sampson, M., Bukutu, C., Schmid, C.H., Tate, R., . . . & the CENT group (2015). CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. British Medical Journal, 350, h1738. doi:10.1136/bmj/h1738.
Wheeler, B., & Torchiano, M. (2016). Permutation tests for linear models: the lmPerm package (version 2.1.0) in R. Retrieved from
Wicherts, J.M., Veldkamp, C.L.S., Augusteijn, H.E.M., Bakker, M., van Aert, R.C.M., & van Assen, M.A. L.M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. doi:10.3389/fpsyg.2016.01832.


Related content

Powered by UNSILO
Type Description Title
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 1

 Unknown (295 bytes)
295 bytes
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 2

 Unknown (12 bytes)
12 bytes
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 3

 Unknown (7 KB)
7 KB
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 4

 Unknown (121 bytes)
121 bytes
Supplementary materials

Onghena et al supplementary material
Onghena et al supplementary material 5

 Unknown (142 bytes)
142 bytes

One by One: Accumulating Evidence by using Meta-Analytical Procedures for Single-Case Experiments

  • Patrick Onghena (a1), Bart Michiels (a1), Laleh Jamshidi (a1), Mariola Moeyaert (a2) and Wim Van den Noortgate (a1)...


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed.