Skip to main content
×
×
Home

Empirical Benchmarks for Interpreting Effect Size Variability in Meta-Analysis

  • Brenton M. Wiernik (a1), Jack W. Kostal (a2), Michael P. Wilmot (a2), Stephan Dilchert (a3) and Deniz S. Ones (a2)...
Extract

Generalization in meta-analyses is not a dichotomous decision (typically encountered in papers using the Q test for homogeneity, the 75% rule, or null hypothesis tests). Inattention to effect size variability in meta-analyses may stem from a lack of guidelines for interpreting credibility intervals. In this commentary, we describe two methods for making practical interpretations and determining whether a particular SDρ represents a meaningful level of variability.

Copyright
Corresponding author
Correspondence concerning this article should be addressed to Brenton M. Wiernik, Department of Developmental, Personality and Social Psychology, Ghent University, Henri Dunantlaan 2, 9000 Gent, Belgium. Email: wiernik@workpsy.ch
References
Hide All
Berry, C. M., Ones, D. S., & Sackett, P. R. (2007). Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology, 92 (2), 410424. https://doi.org/10.1037/0021-9010.92.2.410
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-analytic review. Journal of Management, 36 (4), 10651105. https://doi.org/10/ftwgxk
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100 (2), 431449. http://doi.org/10/bnw8
Chang, C.-H., Ferris, D. L., Johnson, R. E., Rosen, C. C., & Tan, J. A. (2012). Core self-evaluations: A review and evaluation of the literature. Journal of Management, 38 (1), 81128. https://doi.org/10/dbd9nb
Chiaburu, D. S., Oh, I.-S., Berry, C. M., Li, N., & Gardner, R. G. (2011). The five-factor model of personality traits and organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 96 (6), 11401166. https://doi.org/10.1037/a0024004
Christian, M. S., Garza, A. S., & Slaughter, J. E. (2011). Work engagement: A quantitative review and test of its relations with task and contextual performance. Personnel Psychology, 64 (1), 89136. https://doi.org/10/c6b58z
Credé, M., Tynan, M. C., & Harms, P. D. (2016). Much ado about grit: A meta-analytic synthesis of the grit literature. Journal of Personality and Social Psychology. Advance online publication. https://doi.org/10.1037/pspp0000102
Gignac, G. E., & Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 7478. http://doi.org/10/f84bhv
Hemphill, J. F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58 (1), 7879. http://doi.org/10/fb38g8
Judge, T. A., Heller, D., & Mount, M. K. (2002). Five-factor model of personality and job satisfaction: A meta-analysis. Journal of Applied Psychology, 87 (3), 530541. https://doi.org/10.1037//0021-9010.87.3.530
Judge, T. A., Rodell, J. B., Klinger, R. L., Simon, L. S., & Crawford, E. R. (2013). Hierarchical representations of the five-factor model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives. Journal of Applied Psychology, 98 (6), 875925. https://doi.org/10/bdbb
Ng, T. W. H., Eby, L. T., Sorensen, K. L., & Feldman, D. C. (2005). Predictors of objective and subjective career success: A meta-analysis. Personnel Psychology, 58 (2), 367408. https://doi.org/10/dw64z6
Paterson, T. A., Harms, P. D., Steel, P., & Credé, M. (2016). An assessment of the magnitude of effect sizes: Evidence from 30 years of meta-analysis in management. Journal of Leadership & Organizational Studies, 23 (1), 6681. http://doi.org/10/bjz9
Raju, N. S., Burke, M. J., Normand, J., & Langlois, G. M. (1991). A new meta-analytic approach. Journal of Applied Psychology, 76 (3), 432446. http://doi.org/10/dcrgkf
Schmidt, F. L. (2008). Meta-analysis: A constantly evolving research integration tool. Organizational Research Methods, 11 (1), 96113. http://doi.org/10/drwrb2
Steel, P. D. G. (2007). The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure. Psychological Bulletin, 133 (1), 6594. https://doi.org/10.1037/0033-2909.133.1.65
Steel, P., Schmidt, J., & Shultz, J. (2008). Refining the relationship between personality and subjective well-being. Psychological Bulletin, 134 (1), 138161.
Steel, P. D. G., Kammeyer-Mueller, J., & Paterson, T. A. (2015). Improving the meta-analytic assessment of effect size variance with an informed Bayesian prior. Journal of Management, 41 (2), 718743. http://doi.org/10/b6rc
Tett, R. P., Hundley, N. A., & Christiansen, N. D. (2017). Meta-analysis and the myth of generalizability. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10 (3), 421–456.
Thomas, J. P., Whitman, D. S., & Viswesvaran, C. (2010). Employee proactivity in organizations: A comparative meta-analysis of emergent proactive constructs. Journal of Occupational and Organizational Psychology, 83 (2), 275300. https://doi.org/10.1348/096317910x502359
Vacha-Haase, T., Tani, C. R., Kogan, L. R., Woodall, R. A., & Thompson, B. (2001). Reliability generalization: Exploring reliability variations on MMPI/MMPI-2 validity scale scores. Assessment, 8 (4), 391401. http://doi.org/10.1177/107319110100800404
Wilmot, M. P. (2017). Personality and its impacts across the behavioral sciences: A quantitative review of meta-analytic findings. Doctoral dissertation. University of Minnesota, Minneapolis, MN.
Zimmerman, R. D. (2008). Understanding the impact of personality traits on individuals' turnover decisions: A meta-analytic path model. Personnel Psychology, 61 (2), 309348. https://doi.org/10.1111/j.1744-6570.2008.00115.x
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Industrial and Organizational Psychology
  • ISSN: 1754-9426
  • EISSN: 1754-9434
  • URL: /core/journals/industrial-and-organizational-psychology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed