Skip to main content Accessibility help
×
×
Home

Exploring the Relationship of a Gamified Assessment with Performance

  • Ioannis Nikolaou (a1), Konstantina Georgiou (a1) and Vasiliki Kotsasarlidou (a1)

Abstract

Our study explores the validity of a game-based assessment method assessing candidates’ soft skills. Using self-reported measures of performance, (job performance, Organizational Citizenship Behaviors (OCBs), and Great Point Average (GPA), we examined the criterion-related and incremental validity of a game-based assessment, above and beyond the effect of cognitive ability and personality. Our findings indicate that a game-based assessment measuring soft skills (adaptability, flexibility, resilience and decision making) can predict self-reported job and academic performance. Moreover, a game-based assessment can predict academic performance above and beyond personality and cognitive ability tests. The effectiveness of gamification in personnel selection is discussed along with research and practical implications introducing recruiters and HR professionals to an innovative selection technique.

Copyright

Corresponding author

*Correspondence concerning this article should be addressed to Ioannis Nikolaou. Athens University of Economics and Business. Department of Management Science and Technology. 104 34 Athens (Greece). E-mail: inikol@aueb.gr

Footnotes

Hide All
How to cite this article:

Nikolaou, I., Georgiou, K., & Kotsasarlidou, V. (2018). Exploring the relationship of a game-based assessment with performance. The Spanish Journal of Psychology, 21. e6. Doi:10.1017/SJP.2019.5

Footnotes

References

Hide All
Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016). Gamifying recruitment, selection, training, and performance management: Game-thinking in human resource management. In Davis, D. & Gangadharbatla, H. (Eds.), Emerging research and trends in gamification (pp. 140165). Hershey, PA: IGI Global.
Avey, J. B., Reichard, R. J., Luthans, F., & Mhatre, K. H. (2011). Meta-analysis of the impact of positive psychological capital on employee attitudes, behaviors, and performance. Human Resource Development Quarterly, 22(2), 127152. https://doi.org/10.1002/hrdq.20070
Barrick, M. R., Mount, M. K., & Judge, T. A. (2001). Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9(1-2), 930. https://doi.org/10.1111/1468-2389.00160
Bhattacharya, M., Gibson, D. E., & Doty, D. H. (2005). The effects of flexibility in employee skills, employee behaviors, and human resource practices on firm performance. Journal of Management, 31(4), 622640. https://doi.org/10.1177/0149206304272347
Brackett, M. A., Rivers, S. E., Reyes, M. R., & Salovey, P. (2012). Enhancing academic performance and social and emotional competence with the RULER feeling words curriculum. Learning and Individual Differences, 22(2), 218224. https://doi.org/10.1016/j.lindif.2010.10.002
Cascio, W. F., & Montealegre, R. (2016). How technology is changing work and organizations. Annual Review of Organizational Psychology and Organizational Behavior, 3(1), 349375. https://doi.org/10.1146/annurev-orgpsych-041015-062352
Chamorro-Premuzic, T., Akhtar, R., Winsborough, D., & Sherman, R. A. (2017). The datafication of talent: How technology is advancing the science of human potential at work. Current Opinion in Behavioral Sciences, 18, 1316. https://doi.org/10.1016/j.cobeha.2017.04.007
Chamorro-Premuzic, T., & Furnham, A. (2008). Personality, intelligence and approaches to learning as predictors of academic performance. Personality and Individual Differences, 44(7), 15961603. https://doi.org/10.1016/j.paid.2008.01.003
Chiabur, D. S., Oh, I.-S., Berry, C. M., Li, N., & Gardner, R. G. (2011). The five-factor model of personality traits and organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology, 96, 11401166. https://doi.org/10.1037/a0024004
Condon, D. M., & Revelle, W. (2014). The international cognitive ability resource: Development and initial validation of a public-domain measure. Intelligence, 43, 5264. https://doi.org/10.1016/j.intell.2014.01.004
Côte, S., & Miners, C. T. H. (2006). Emotional intelligence, cognitive intelligence, and job performance. Administrative Science Quarterly, 51(1), 128. https://doi.org/10.2189/asqu.51.1.1
Fetzer, M., McNamara, J., & Geimer, J. L. (2017). Gamification, serious games and personnel selection. In Goldstein, H. W., Pulakos, E. D., Passmore, J., & Semedo, C. (Eds.), The Wiley Blackwell handbook of the psychology of recruitment, selection and employee retention (pp. 293309). West Sussex, UK: John Wiley & Sons Ltd.
Georgiou, K., Nikolaou, I., & Gouras, A. (2017). Serious gaming in employees’ selection process. In I. Nikolaou, Alliance for Organizational Psychology Invited Symposium-The impact of technology on recruitment and selection: An international perspective. Paper presented at the 32nd Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, USA.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1), 8496. https://doi.org/10.1016/j.jrp.2005.08.007
Golden, W., & Powell, P. (2000). Towards a definition of flexibility: In search of the Holy Grail? Omega, 28(4), 373384. https://doi.org/10.1016/S0305-0483(99)00057-2
Gray, A., (2016). The 10 skills you need to thrive in the Fourth Industrial Revolution. Retrieved from The World Economic Forum website: https://www.weforum.org/agenda/2016/01/the-10-skills-you-need-to-thrive-in-the-fourth-industrial-revolution
Hamtiaux, A., Houssemand, C., & Vrignaud, P. (2013). Individual and career adaptability: Comparing models and measures. Journal of Vocational Behavior, 83(2), 130141. https://doi.org/10.1016/j.jvb.2013.03.006
International Cognitive Ability Resource (ICAR) (2014). [Public-domain assessment tool]. Retrieved from https://icar-project.com/
Kehoe, J. F. (2000). Managing selection in changing organizations: Human resource strategies. San Francisco, CA: Jossey-Bass Publ.
Kwan, H.-K., & Mao, Y. (2011). The role of citizenship behavior in personal learning and work–family enrichment. Frontiers of Business Research in China, 5(1), 96120. https://doi.org/10.1007/s11782-011-0123-6
Lim, B.-C., & Ployhart, R. E. (2006). Assessing the convergent and discriminant validity of Goldberg’s international personality item pool: A multitrait-multimethod examination. Organizational Research Methods, 9(1), 2954. https://doi.org/10.1177/1094428105283193
Luthans, F. (2002). The need for and meaning of positive organizational behavior. Journal of Organizational Behavior, 23(6), 695706. https://doi.org/10.1002/job.165
Martin, A. J., & Marsh, H. W. (2006). Academic resilience and its psychological and educational correlates: A construct validity approach. Psychology in the Schools, 43(3), 267281. https://doi.org/10.1002/pits.20149
Martin, A. J., & Marsh, H. W. (2008). Academic buoyancy: Towards an understanding of students’ everyday academic resilience. Journal of School Psychology, 46(1), 5383. https://doi.org/10.1016/j.jsp.2007.01.002
McKinsey & Company (Producer). (2017). The digital future of work: What skills will be needed? [Video]. Available from https://www.youtube.com/watch?v=UV46n44jnoA
Miller, D., & Lee, J. (2001). The people make the process: Commitment to employees, decision making, and performance. Journal of Management, 27(2), 163189. https://doi.org/10.1177%2F014920630102700203
Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R., Murphy, K., & Schmitt, N. (2007). Are we getting fooled again? Coming to terms with limitations in the use of personality tests for personnel selection. Personnel Psychology, 60(4), 10291049. https://doi.org/10.1111/j.1744-6570.2007.00100.x
Nelson, G. D. (1984). Assessment of health decision making skills of adolescents. Retrieved from ERIC database (ED252774).
Nikolaou, I., & Robertson, I. T. IV. (2001). The Five-Factor model of personality and work behavior in Greece. European Journal of Work and Organizational Psychology, 10(2), 161186. https://doi.org/10.1080/13594320143000618
Ones, D. S., Dilchert, S., & Viswesvaran, C. (2012). Cognitive abilities. In Schmitt, N. (Ed.), The Oxford handbook of personnel assessment and selection (pp. 179224). New York, NY: Oxford University Press.
Oostrom, J. K., Born, M. P., & van der Molen, H. T. (2013). Webcam tests in personnel selection. In Derks, D. & Bakker, A. (Eds.), The psychology of digital media at work (pp. 166180). USA & Canada: Psychology Press.
Parker, J. D. A., Creque, R. E., Barnhart, D. L., Harris, J. I., Majeski, S. A., Wood, L. M., ... Hogan, M. J. (2004). Academic achievement in high school: Does emotional intelligence matter? Personality and Individual Differences, 37(7), 13211330. https://doi.org/10.1016/j.paid.2004.01.002
Paul, H., Bamel, U. K., & Garg, P. (2016). Employee resilience and OCB: Mediating effects of organizational commitment. Vikalpa, 41(4), 308324. https://doi.org/10.1177/0256090916672765
Picchi, A. (2016, August 31). Do you have the “soft skills” employers badly need? CBSNews.com . Retrieved from https://www.cbsnews.com/news/do-you-have-the-soft-skills-employers-badly-need/
Ployhart, R. E., & Bliese, P. D. (2006). Individual adaptability (I–ADAPT) theory: Conceptualizing the antecedents, consequences, and measurement of individual differences in adaptability. In Shawn Burke, C., Pierce, Linda G., & Salas, Eduardo (Eds.) Advances in human performance and cognitive engineering research (Vol. 6, 339). Bingley, UK: Emerald Group Publishing Limited.
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879903. https://doi.org/10.1037/0021-9010.88.5.879
Poropat, A. E. (2009). A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin, 135(2), 322338. https://doi.org/10.1037/a0014996
Pransky, G., Finkelstein, S., Berndt, E., Kyle, M., Mackell, J., & Tortorice, D. (2006). Objective and self-report work performance measures: A comparative analysis. International Journal of Productivity and Performance Management, 55(5), 390399. https://doi.org/10.1108/17410400610671426
Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence, 35(1), 8392. https://doi.org/10.1016/j.intell.2006.05.004
Roth, P. L., BeVier, C. A., Switzer, F. S. III., & Schippmann, J. S. (1996). Meta-analyzing the relationship between grades and job performance. Journal of Applied Psychology, 81(5), 548556. https://doi.org/10.1037/0021-9010.81.5.548
Ryan, A. M., & Ployhart, R. E. (2014). A century of selection. Annual Review of Psychology, 65(1), 693717. https://doi.org/10.1146/annurev-psych-010213-115134
Schmitt, N. (2014). Personality and cognitive ability as predictors of effective performance at work. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 4565. https://doi.org/10.1146/annurev-orgpsych-031413-091255
Shute, V. (2015, August). Stealth assessment in video games. Paper presented at the Australian Council for Educational Research Research “Conference Learning Assessments: Designing the future conference”. Melbourne, Australia.
Shute, V. J., Ventura, M., Bauer, M., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning. In Ritterfeld, U., Cody, M., & Vordered, P. (Eds.), Serious games: Mechanisms and effects (pp. 295321). New York and London: Routledge, Taylor & Francis.
Smith, C. A., Organ, D. W., & Near, J. P. (1983). Organizational citizenship behavior: Its nature and antecedents. Journal of Applied Psychology, 68(4), 653663. https://psycnet.apa.org/doi/10.1037/0021-9010.68.4.653
Somech, A. (2010). Participative decision making in schools: A mediating-moderating analytical framework for understanding school and teacher outcomes. Educational Administration Quarterly, 46(2), 174209. https://doi.org/10.1177/1094670510361745
Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287305. https://doi.org/10.1177/1094428110369842
Strenze, T. (2007). Intelligence and socioeconomic success: A meta-analytic review of longitudinal research. Intelligence, 35(5), 401426. https://doi.org/10.1016/j.intell.2006.09.004
van Iddekinge, C. H., Lanivich, S. E., Roth, P. L., & Junco, E. (2016). Social media for selection? Validity and adverse impact potential of a Facebook-based assessment. Journal of Management, 42(7), 18111835. https://doi.org/10.1177/0149206313515524
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

The Spanish Journal of Psychology
  • ISSN: 1138-7416
  • EISSN: 1988-2904
  • URL: /core/journals/spanish-journal-of-psychology
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed