Meta-analysis

doi:10.1017/9781009170123.028

27 - Meta-analysis

from Part IV - Understanding What Your Data Are Telling You About Psychological Processes

Published online by Cambridge University Press: 12 December 2024

Judith A. Hall and

Edited by

Tessa West and

Harry T. Reis: Affiliation:
University of Rochester, New York
Tessa West: Affiliation:
New York University
Charles M. Judd: Affiliation:
University of Colorado Boulder

Book contents

Get access

Summary

Meta-analysis is the quantitative analysis of results of a research literature. Typically, meta-analysis is paired with a systematic review that fully documents the search process, inclusion and exclusion criteria, and study characteristics. A key feature of meta-analysis is the calculation of effect sizes – metric-free indices of study outcome that allow the mathematical combination of effects across studies. The methodological literature on meta-analysis has grown rapidly in recent years, yielding an abundance of resources and sophisticated analytic techniques. These developments are improvements to the field but can also be overwhelming to new aspiring meta-analysts. This chapter therefore aims to demystify some of that complexity, offering conceptual explanations instead of mathematical formulas. We aim to help readers who have not conducted a meta-analysis before to get started, as well as to help those who simply want to be intelligent consumers of published meta-analyses.

Keywords

meta-analysis research synthesis literature review systematic review

Information

Type: Chapter
Information: Handbook of Research Methods in Social and Personality Psychology , pp. 678 - 704

DOI: https://doi.org/10.1017/9781009170123.028 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Albajes-Eizagirre, A., Solanes, A., and Radua, J. (2019). Meta-analysis of non-statistically significant unreported effects. Statistical Methods in Medical Research, 28(12), 3741–3754.10.1177/0962280218811349CrossRef Google Scholar PubMed

American Psychological Association. (2020). Meta-analysis reporting standards (MARS), https://apastyle.apa.org/jars/quant-table-9.pdf.Google Scholar

Bartoš, F., Maier, M., Quintana, D. S., and Wagenmakers, E. J. (2022). Adjusting for publication bias in JASP and R: Selection models, PET-PEESE, and robust Bayesian meta-analysis. Advances in Methods and Practices in Psychological Science, 5(3), 25152459221109259.10.1177/25152459221109259CrossRef Google Scholar

Becker, B. J. (2005). Failsafe N or file-drawer number. In Rothstein, H. R., Sutton, A. J., and Borenstein, M. (eds.) Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. John Wiley and Sons, Ltd.Google Scholar

Borenstein, M. (2019). Common Mistakes in Meta-analysis and How to Avoid Them. Biostat, Inc.Google Scholar

Borenstein, M., Hedges, L. V., Higgins, J. P. T., and Rothstein, H. R. (2009). Introduction to Meta-analysis. John Wiley and Sons.10.1002/9780470743386CrossRef Google Scholar

Borenstein, M., Higgins, J. P. T., Hedges, L. V., and Rothstein, H. R. (2017). Basics of meta-analysis: I² is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5–18.10.1002/jrsm.1230CrossRef Google Scholar

Bushman, B. J., and Wang, M. C. (2009). Vote-counting procedures in meta-analysis. In Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.) The Handbook of Research Synthesis and Meta-analysis. Russell Sage Foundation.Google Scholar

Button, K. S., Ioannidis, J., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., and Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376.10.1038/nrn3475CrossRef Google Scholar PubMed

Carter, E. C., Schönbrodt, F. D., Gervais, W. M., and Hilgard, J. (2019). Correcting for bias in psychology: A comparison of meta-analytic methods. Advances in Methods and Practices in Psychological Science, 2(2), 115–144.10.1177/2515245919847196CrossRef Google Scholar

Cheung, M. W. L. (2014). Fixed‐ and random‐effects meta‐analytic structural equation modeling: Examples and analyses in R. Behavioral Research Methods, 46(1), 29–40.10.3758/s13428-013-0361-yCrossRef Google Scholar PubMed

Cinar, O., Umbanhowar, J., Hoeksema, J. D., and Viechtbauer, W. (2021). Using information‐theoretic approaches for model selection in meta‐analysis. Research Synthesis Methods, 12(4), 537–556.10.1002/jrsm.1489CrossRef Google Scholar PubMed

Cleophas, T. J. M., and Zwinderman, A. H. (2017). Modern Meta-analysis: Review and Update of Methodologies. Springer.10.1007/978-3-319-55895-0CrossRef Google Scholar

Coburn, K. M., and Vevea, J. L. (2019). Weightr: Estimating weight-function models for publication bias. R package version 2.0.2, https://CRAN.R-project.org/package=weightr.Google Scholar

Cohen, J. (1989). Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Lawrence Erlbaum Associates.Google Scholar

Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.). (2019). The Handbook of Research Synthesis and Meta-analysis. Russell Sage Foundation.10.7758/9781610448864CrossRef Google Scholar

Cooper, H. M. (2016). Research Synthesis and Meta-analysis: A Step-by-Step Approach, 5th ed. Sage.Google Scholar

Del Re, A. C., and Flückiger, C. (2016). Meta-analysis. In Norcross, J. C., VandenBos, G. R., Freedheim, D. K., and Olatunji, B. O. (eds.) APA Handbook of Clinical Psychology: Theory and Research. American Psychological Association.Google Scholar

Dickens, L. R., and Robins, R. W. (2022). Pride: A meta-analytic project. Emotion, 22(5), 1071–1087.10.1037/emo0000905CrossRef Google Scholar

DiMatteo, M. R. (2004). Social support and patient adherence to medical treatment: A meta-analysis. Health Psychology, 23, 207–218.10.1037/0278-6133.23.2.207CrossRef Google Scholar PubMed

Donnelly, K., and Twenge, J. M. (2017). Masculine and feminine traits on the Bem Sex-Role Inventory, 1993–2012: A cross-temporal meta-analysis. Sex Roles, 76(9), 556–565.10.1007/s11199-016-0625-yCrossRef Google Scholar

Duval, S. (2005). The trim and fill method. In Rothstein, H. R., Sutton, A. J., and Borenstein, M. (eds.) Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. John Wiley and Sons.Google Scholar

Eagly, A. H., and Steffen, V. J. (1986). Gender and aggressive behavior: A meta-analytic review of the social psychological literature. Psychological Bulletin, 100(3), 309–330.10.1037/0033-2909.100.3.309CrossRef Google Scholar PubMed

Eysenck, H. J. (1978). An exercise in mega-silliness. American Psychologist, 33(5), 517.10.1037/0003-066X.33.5.517.aCrossRef Google Scholar

Freudenberg, M., Albohn, D. N., Kleck, R. E., Adams, R. B., Jr., and Hess, U. (2020). Emotional stereotypes on trial: Implicit emotion associations for young and old adults. Emotion, 20(7), 1244–1254.10.1037/emo0000626CrossRef Google Scholar PubMed

Goh, J. X., Hall, J. A., and Rosenthal, R. (2016). Mini meta-analysis of your own studies: Some arguments on why and a primer on how. Social and Personality Psychology Compass, 10, 535–549.10.1111/spc3.12267CrossRef Google Scholar

Hall, J. A. (1978). Gender effects in decoding nonverbal cues. Psychological Bulletin, 85, 845–857.10.1037/0033-2909.85.4.845CrossRef Google Scholar

Hall, J. A. (2006). How big are nonverbal sex differences? The case of smiling and nonverbal sensitivity. In Dindia, K. and Canary, D. J. (eds.) Sex Differences and Similarities in Communication. Lawrence Erlbaum Associates Publishers.Google Scholar

Hall, J. A., Coats, E. J., and Smith LeBeau, L. (2005). Nonverbal behavior and the vertical dimension of social relations: A meta-analysis. Psychological Bulletin, 131, 898–924.10.1037/0033-2909.131.6.898CrossRef Google Scholar PubMed

Hall, J. A., and Rosenthal, R. (2018). Choosing between random effects models in meta-analysis: Units of analysis and the generalizability of obtained results. Social and Personality Psychology Compass, 12(10), article e12414.10.1111/spc3.12414CrossRef Google Scholar

Hedges, L. V. (2009). Statistical considerations. In Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.) The Handbook of Research Synthesis and Meta-analysis, 2nd ed. Russell Sage Foundation.Google Scholar

Hedges, L. V., and Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9(4), 426–445.10.1037/1082-989X.9.4.426CrossRef Google Scholar PubMed

Hedges, L. V., and Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486–504.10.1037/1082-989X.3.4.486CrossRef Google Scholar

Higgins, J. P., Savović, J., Page, M. J., Elbers, R. G., and Sterne, J. A. (2019). Assessing risk of bias in a randomized trial. In Cochrane Handbook for Systematic Reviews of Interventions, 205–228, at https://training.cochrane.org/handbook.Google Scholar

Iyengar, S., and Greenhouse, J. B. (1988). Selection models and the file drawer problem. Statistical Science, 3(1), 109–135.Google Scholar

Johnson, B. T. (2021). Toward a more transparent, rigorous, and generative psychology. Psychological Bulletin, 147(1), 1–15.10.1037/bul0000317CrossRef Google Scholar

Johnson, B. T., and Eagly, A. H. (2014). Meta-analysis of research in social and personality psychology. In Reis, H. T. and Judd, C. M. (eds.) Handbook of Research Methods in Social and Personality Psychology. Cambridge University Press.Google Scholar

Kenny, D. A., and Judd, C. M. (2019). The unappreciated heterogeneity of effect sizes: Implications for power, precision, planning of research, and replication. Psychological Methods, 24(5), 578–589.10.1037/met0000209CrossRef Google Scholar PubMed

Konrath, S. H., O’Brien, E. H., and Hsing, C. (2011). Changes in dispositional empathy in American college students over time: A meta-analysis. Personality and Social Psychology Review, 15(2), 180–198.10.1177/1088868310377395CrossRef Google Scholar PubMed

Kossmeier, M., Tran, U. S., and Voracek, M. (2020). Charting the landscape of graphical displays for meta-analysis and systematic reviews: A comprehensive review, taxonomy, and feature analysis. BMC Medical Research Methodology, 20(1), 1–24.10.1186/s12874-020-0911-9CrossRef Google Scholar PubMed

Kvarven, A., Strømland, E., and Johannesson, M. (2019). Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature Human Behavior, 4, 423–434.10.1038/s41562-019-0787-zCrossRef Google Scholar PubMed

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(26), 863, DOI:10.3389/fpsyg.2013.00863.CrossRef Google Scholar

Lakens, D., Hilgard, J., and Staaks, J. (2016). On the reproducibility of meta-analyses: Six practical recommendations. BMC Psychology, 4(1), 1–10.10.1186/s40359-016-0126-3CrossRef Google Scholar PubMed

Lipsey, M. W., and Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48(12), 1181–1209.10.1037/0003-066X.48.12.1181CrossRef Google Scholar PubMed

Lipsey, M. W., and Wilson, D. B. (2001). Practical Meta-analysis. Sage.Google Scholar PubMed

López‐López, J. A., Page, M. J., Lipsey, M. W., and Higgins, J. P. T. (2018). Dealing with effect size multiplicity in systematic reviews and meta‐analyses. Research Synthesis Methods, 9(3), 336–351.10.1002/jrsm.1310CrossRef Google Scholar PubMed

López‐López, J. A., van den Noortgate, W., Tanner‐Smith, E. E., Wilson, S. J., and Lipsey, M. W. (2017). Assessing meta‐regression methods for examining moderator relationships with dependent effect sizes: A Monte Carlo simulation. Research Synthesis Methods, 8(4), 435–450.10.1002/jrsm.1245CrossRef Google Scholar PubMed

Lovakov, A., and Agadullina, E. R. (2021). Empirically derived guidelines for effect size interpretation in social psychology. European Journal of Social Psychology, 51(3), 485–504.10.1002/ejsp.2752CrossRef Google Scholar

McShane, B. B., Böckenholt, U., and Hansen, K. T. (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11(5), 730–749.10.1177/1745691616662243CrossRef Google Scholar PubMed

Marks‐Anglin, A., and Chen, Y. (2020). A historical review of publication bias. Research Synthesis Methods, 11(6), 725–742.10.1002/jrsm.1452CrossRef Google Scholar PubMed

Mathur, M. B., and VanderWeele, T. J. (2020). Sensitivity analysis for publication bias in meta‐analyses. Journal of the Royal Statistical Society: Series C (Applied Statistics), 69(5), 1091–1119.Google Scholar PubMed

Mathur, M. B., and VanderWeele, T. J. (2021). Estimating publication bias in meta‐analyses of peer‐reviewed studies: A meta‐meta‐analysis across disciplines and journal tiers. Research Synthesis Methods, 12(2), 176–191.10.1002/jrsm.1464CrossRef Google Scholar PubMed

Miller, D. I., Nolla, K. M., Eagly, A. H., and Uttal, D. H. (2018). The development of children’s gender-science stereotypes: A meta-analysis of five decades of U.S. Draw-A-Scientist studies. Child Development, 89(6), 1943–1955.10.1111/cdev.13039CrossRef Google Scholar

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and the Group, PRISMA. (2009) Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6(7), e1000097.10.1371/journal.pmed.1000097CrossRef Google Scholar PubMed

Mosteller, F. M., and Bush, R. R. (1954). Selected quantitative techniques. In Lindzey, G. (ed.) Handbook of Social Psychology, vol. 1, Theory and Method. Addison-Wesley.Google Scholar

Collaboration, Open Science. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.10.1126/science.aac4716CrossRef Google Scholar

Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., … McKenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ, 372, n160.10.1136/bmj.n160CrossRef Google Scholar PubMed

Page, M. J., Moher, D., and McKenzie, J. E. (2022). Introduction to PRISMA 2020 and implications for research synthesis methodologists. Research Synthesis Methods, 13(2), 156–163.10.1002/jrsm.1535CrossRef Google Scholar PubMed

Polanin, J. R., Hennessy, E. A., and Tanner-Smith, E. E. (2017). A review of meta-analysis packages in R. Journal of Educational and Behavioral Statistics, 42(2), 206–242.10.3102/1076998616674315CrossRef Google Scholar

Polanin, J. R., Hennessy, E. A., and Tsuji, S. (2020). Transparency and reproducibility of meta-analyses in psychology: A meta-review. Perspectives on Psychological Science, 15(4), 1026–1041.10.1177/1745691620906416CrossRef Google Scholar PubMed

Polanin, J. R., Pigott, T. D., Espelage, D. L., and Grotpeter, J. K. (2019). Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses. Research Synthesis Methods, 10(3), 330–342.10.1002/jrsm.1354CrossRef Google Scholar

Pustejovsky, J. E., and Rodgers, M. A. (2019). Testing for funnel plot asymmetry of standardized mean differences. Research Synthesis Methods, 10(1), 57–71.10.1002/jrsm.1332CrossRef Google Scholar PubMed

Pustejovsky, J. E., and Tipton, E. (2022). Meta-analysis with robust variance estimation: Expanding the range of working models. Prevention Science, 23, 425–438.10.1007/s11121-021-01246-3CrossRef Google Scholar PubMed

Rathbone, J., Hoffmann, T., and Glasziou, P. (2015). Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Systematic Reviews, 4(1), 1–7.10.1186/s13643-015-0067-6CrossRef Google Scholar PubMed

Razpurker-Apfeld, I., and Shamoa-Nir, L. (2021). Is an outgroup welcome with open arms? Approach and avoidance motor activations and outgroup prejudice. Journal of Experimental Psychology: Applied, 27(2), 417–429.Google Scholar

Rosenthal, R. (1966). Experimenter Effects in Behavioral Research. Appleton-Century-Crofts.Google Scholar

Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.10.1037/0033-2909.86.3.638CrossRef Google Scholar

Rosenthal, R. (1994). Parametric measures of effect size. In Cooper, H. and Hedges, L. V. (eds.) The Handbook of Research Synthesis. Russell Sage Foundation.Google Scholar

Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bulletin, 118, 183–192.10.1037/0033-2909.118.2.183CrossRef Google Scholar

Rosenthal, R., Rosnow, R. L., and Rubin, D. B. (2000). Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach. Cambridge University Press.Google Scholar

Rosenthal, R., and Rubin, D. B. (1979). Comparing significance levels of independent studies. Psychological Bulletin, 86, 1165–1168.10.1037/0033-2909.86.5.1165CrossRef Google Scholar

Schild, A. H., and Voracek, M. (2015). Finding your way out of the forest without a trail of bread crumbs: development and evaluation of two novel displays of forest plots. Research Synthesis Methods, 6(1), 74–86.10.1002/jrsm.1125CrossRef Google Scholar PubMed

Schlegel, K., Boone, R. T., and Hall, J. A. (2017). Individual differences in interpersonal accuracy: A multi-level meta-analysis to assess whether judging other people is one skill or many. Journal of Nonverbal Behavior, 41, 103–137.10.1007/s10919-017-0249-0CrossRef Google Scholar

Schlegel, K., Palese, T., Mast, M. S., Rammsayer, T. H., Hall, J. A., and Murphy, N. A. (2020). A meta-analysis of the relationship between emotion recognition ability and intelligence. Cognition and Emotion, 34(2), 329–351.10.1080/02699931.2019.1632801CrossRef Google Scholar PubMed

Schmid, C. H., Stijnen, T., and White, I. R. (eds.) (2021). Handbook of Meta-analysis. Routledge/Taylor and Francis Group.Google Scholar

Schmidt, F. L., Le, H., and Oh, I. (2009). Correcting for the distorting effects of study artifacts in meta-analysis. In Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.) The Handbook of Research Synthesis and Meta-analysis, 2nd ed. Russell Sage Foundation.Google Scholar

Schmidt, F. L., Oh, I.-S., and Hayes, T. L. (2009). Fixed- versus random-effects models in meta-analysis: Model properties and an empirical comparison of differences in results. British Journal of Mathematical and Statistical Psychology, 62(1), 97–128.10.1348/000711007X255327CrossRef Google Scholar

Shuster, J. J, Guo, J. D., and Skyler, J. S. (2012). Meta‐analysis of safety for low event‐rate binomial trials. Research Synthesis Methods, 3, 30–50.10.1002/jrsm.1039CrossRef Google Scholar PubMed

Siddaway, A. P., Wood, A. M., and Hedges, L. V. (2019). How to do a systematic review: A best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annual Review of Psychology, 70, 747–770.10.1146/annurev-psych-010418-102803CrossRef Google Scholar

Siegel, M., Eder, J. S. N., Wicherts, J. M., and Pietschnig, J. (2021). Times are changing, bias isn’t: A meta-meta-analysis on publication bias detection practices, prevalence rates, and predictors in industrial/organizational psychology. Journal of Applied Psychology, 107(11), 2013–2039.10.1037/apl0000991CrossRef Google Scholar PubMed

Simonsohn, U., Nelson, L. D., and Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534–547.10.1037/a0033242CrossRef Google Scholar

Simonsohn, U., Simmons, J., and Nelson, L. (2017, June 15). Why p-curve excludes ps > .05 (blog post), https://datacolada.org/61.+.05+(blog+post),+https://datacolada.org/61.>Google Scholar

Simonsohn, U., Simmons, J., and Nelson, L. (2018, January 8). P-curve handles heterogeneity just fine (blog post), https://datacolada.org/67.Google Scholar

Smith, M. L., and Glass, G.V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32(9), 752–760.10.1037/0003-066X.32.9.752CrossRef Google Scholar PubMed

Stanley, T. D., and Doucouliagos, H. (2014). Meta‐regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60–78.10.1002/jrsm.1095CrossRef Google Scholar PubMed

Sterne, J. A. C., Becker, B. J., and Egger, M. (2005). The funnel plot. In Rothstein, H. R., Sutton, A. J., and Borenstein, M. (eds.) Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. John Wiley & Sons.Google Scholar

Tanner-Smith, E. E., and Tipton, E. (2014). Robust variance estimation with dependent effect sizes: Practical considerations including a software tutorial in Stata and SPSS. Research Synthesis Methods, 5(1), 13–30.10.1002/jrsm.1091CrossRef Google Scholar PubMed

Tanner-Smith, E. E., Tipton, E., and Polanin, J. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2, 85–112.10.1007/s40865-016-0026-5CrossRef Google Scholar

Taylor, J. A., Pigott, T., and Williams, R. (2022). Promoting knowledge accumulation about intervention effects: Exploring strategies for standardizing statistical approaches and effect size reporting. Educational Researcher, 51(1), 72–80.10.3102/0013189X211051319CrossRef Google Scholar

Thomas, J., and Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45, at https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-8-45.10.1186/1471-2288-8-45CrossRef Google Scholar PubMed

Tipton, E., Pustejovsky, J. E., and Ahmadi, H. (2019). A history of meta-regression: Technical, conceptual, and practical developments between 1974 and 2018. Research Synthesis Methods, 10(2), 161–179.10.1002/jrsm.1338CrossRef Google Scholar PubMed

Tucker-Drob, E. M., Brandmaier, A. M., and Lindenberger, U. (2019). Coupled cognitive changes in adulthood: A meta-analysis. Psychological Bulletin, 145(3), 273–301.10.1037/bul0000179CrossRef Google Scholar PubMed

Valentine, J. C. (2009). Judging the quality of primary research. In Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.) The Handbook of Research Synthesis and Meta-analysis, 2nd ed. Russell Sage Foundation.Google Scholar

Valentine, J. C. (2012). Meta-analysis. In Cooper, H., Camic, P. M., Long, D. L., Panter, A. T., Rindskopf, D., and Sher, K. J. (eds.). APA Handbook of Research Methods in Psychology, vol. 3. American Psychological Association.Google Scholar

Valentine, J. C., Pigott, T. D., and Rothstein, H. R. (2010). How many studies do you need? A primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics, 35(2), 215–247.10.3102/1076998609346961CrossRef Google Scholar

van Aert, R. C. M., Wicherts, J. M., and van Assen, M. A. L. M. (2016). Conducting meta-analyses based on p-values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713–729.10.1177/1745691616650874CrossRef Google Scholar PubMed

van Lissa, C. J. (2020). Small sample meta-analyses: Exploring heterogeneity using MetaForest. In Van De Schoot, R. and Miočević, M. (eds.) Small sample size solutions (open access): A guide for applied researchers and practitioners. CRC Press, www.crcpress.com/Small-Sample-SizeSolutions-Open-Access-A-Guide-for-Applied-Researchers/Schoot-Miocevic/p/book/9780367222222.Google Scholar

Vevea, J. L. and Hedges, L. V. (1995). A general linear model for estimating effect size in the presence of publication bias. Psychometrika, 60(3), 419–435.10.1007/BF02294384CrossRef Google Scholar

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.10.18637/jss.v036.i03CrossRef Google Scholar

Waffenschmidt, S., Knelangen, M., Sieben, W., Bühn, S., and Pieper, D. (2019). Single screening versus conventional double screening for study selection in systematic reviews: A methodological systematic review. BMC Medical Research Methodology, 19(1), 1–9.10.1186/s12874-019-0782-0CrossRef Google Scholar

Wells, G., Shea, B., O’Connell, D., Peterson, J., Welch, V., Losos, M., and Tugwell, P. (2000). The Newcastle–Ottawa Scale (NOS) for assessing the quality of non-randomized studies in meta-analysis, www.ohri.ca/programs/clinical_epidemiology/oxford.asp.Google Scholar

Clearinghouse, What Works. (2022). What Works Clearinghouse Procedures and Standards Handbook, Version 5.0. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education, https://ies.ed.gov/ncee/wwc/handbooks.Google Scholar

White, H. D. (2009). Scientific communication and literature retrieval. In Cooper, H., Hedges, L. V., and Valentine, J. C. (eds.) The Handbook of Research Synthesis and Meta-analysis, 2nd ed. Russell Sage Foundation.Google Scholar

Zuckerman, M., Li, C., and Hall, J. A. (2016). When men and women differ in self-esteem and when they don’t: A meta-analysis. Journal of Research in Personality, 64, 34–51.10.1016/j.jrp.2016.07.007CrossRef Google Scholar

Zuckerman, M., Silberman, J., and Hall, J. A. (2013). The relation between intelligence and religiosity: A meta-analysis and some proposed explanations. Personality and Social Psychology Review, 17, 325–354.10.1177/1088868313497266CrossRef Google Scholar PubMed

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the HTML of this chapter is currently unknown and may be updated in the future.