Book contents
- Technology and Measurement around the Globe
- Educational and Psychological Testing in a Global Context
- Technology and Measurement around the Globe
- Copyright page
- Contents
- Contributors
- Introduction
- Part I Foundations
- Part II Global Perspectives on Key Methods/Topics
- Chapter 4 Social Media Assessments around the Globe
- Chapter 5 Game-Based Assessment around the Globe
- Chapter 6 Mobile Sensing around the Globe
- Part III Regional Focus
- Index
- References
Chapter 5 - Game-Based Assessment around the Globe
International Differences in Privacy, Legality, and Applicant Reaction Concerns
from Part II - Global Perspectives on Key Methods/Topics
Published online by Cambridge University Press: 08 November 2023
Book contents
- Technology and Measurement around the Globe
- Educational and Psychological Testing in a Global Context
- Technology and Measurement around the Globe
- Copyright page
- Contents
- Contributors
- Introduction
- Part I Foundations
- Part II Global Perspectives on Key Methods/Topics
- Chapter 4 Social Media Assessments around the Globe
- Chapter 5 Game-Based Assessment around the Globe
- Chapter 6 Mobile Sensing around the Globe
- Part III Regional Focus
- Index
- References
Summary
Despite promising early evidence for the validity of well-designed game-based assessments (GBAs) for employee selection, the interaction between the complexity of games and their use in international and cross-cultural contexts is unknown. To address this, this paper presents a descriptive, qualitative study examining the perspectives of both GBA vendors and organizational stakeholders related to cross-cultural issues unique to GBAs related to 1) privacy, 2) legality, and 3) applicant reactions. Overall, privacy and legality concerns appeared similar for GBAs as with other assessment methods, although certain common characteristics of GBAs amplify common concerns. Applicant reactions appeared more positive to GBAs across national borders and cultures than traditional assessments, although some international differences were reported. Other cross-cultural topics raised included international differences in the conflation of GBA and artificial intelligence, in the importance of mobile-first design, and in the ability of GBAs to provide a more language-agnostic experience than other assessment types.
Keywords
- Type
- Chapter
- Information
- Technology and Measurement around the Globe , pp. 157 - 175Publisher: Cambridge University PressPrint publication year: 2023
References
Armstrong, M. B., Ferrell, J., Collmus, A. B., & Landers, R. N. (2016). Correcting misconceptions about gamification of assessment: More than SJTs and badges. Industrial and Organizational Psychology, 9(3), 671–677. https://doi.org/10.1017/iop.2016.69CrossRefGoogle Scholar
Auer, E. M., Mersy, G., Marin, S., Blaik, J., & Landers, R. N. (2022). Using machine learning to model trace behavioral data from a game-based assessment. International Journal of Selection and Assessment, 30(1), 82–102. https://doi.org/10.1111/ijsa.12363CrossRefGoogle Scholar
Bhatia, S., & Ryan, A. M. (2018). Hiring for the win: Game-based assessment in employee selection. In Dulebohn, J. H. & Stone, D. L. (Eds.), The brave new world of eHRM 2.0 (pp. 81–110). IAP Information Age Publishing.Google Scholar
Bina, S., Mullins, J. K., & Petter, S. (2021). Examining game-based approaches in human resources recruitment and selection: A literature review and research agenda. In Proceedings of the 54th Hawaii International Conference on System Sciences (pp. 1325–1334).CrossRefGoogle Scholar
Bjogvinsson, E., Ehn, P., & Hillgren, P.-A. (2012). Design things and design thinking: Contemporary participatory design challenges. Design Issues, 28(3), 101–116. https://doi.org/10.1162/DESI_a_00165CrossRefGoogle Scholar
Cubrich, M., King, R., Mracek, D., Strong, J., Hassenkamp, K., Vaughn, D., & Dudley, N. (2021). Examining the criterion-related validity evidence of LinkedIn profile elements in an applied sample. Computers in Human Behavior, 120, 106742. https://doi.org/10.1016/j.chb.2021.106742CrossRefGoogle Scholar
Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011). Gamification: Using game-design elements in non-gaming contexts. In CHI’11 extended abstracts on human factors in computing systems (pp. 2425–2428). ACM Press.Google Scholar
Grelle, D. M., & Gutierrez, S. L. (2019). Developing device-equivalent and effective measures of complex thinking with an information processing framework and mobile first design principles. Personnel Assessment and Decisions, 5(3). https://doi.org/10.25035/pad.2019.03.004CrossRefGoogle Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. The Behavioral and Brain Sciences, 33(2–3), 61–135. https://doi.org/10.1017/S0140525X0999152XCrossRefGoogle ScholarPubMed
Howison, J., Wiggins, A., & Crowston, K. (2011). Validity issues in the use of social network analysis with digital trace data. Journal of the Association for Information Systems, 12(12). https://doi.org/10.17705/1jais.00282CrossRefGoogle Scholar
Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. Proceedings of the AAAI Workshop on Challenges in Game AI, 4(1), 1722–1726.Google Scholar
Ip, C. (2018, May 4). To find a job, play these games. Engadget. https://www.engadget.com/2018/05/04/pymetrics-gamified-recruitment-behavioral-tests/Google Scholar
Jackson, G. T., Grace, L., Inglese, P., Wain, J., & Hone, R. (2018). Awkward Annie: Game-based assessment of English pragmatic skills. In Cheok, A., Inami, M., & Romão, T. (Eds.), Advances in computer entertainment technology. ACE 2017. Lecture Notes in Computer Science, Vol. 10714. Springer. https://doi.org/10.1007/978-3-319-76270-8_54Google Scholar
Jones, S. E. (2008). The meaning of video games: Gaming and textual strategies. Routledge.CrossRefGoogle Scholar
Kapp, K. M., Valtchanov, D., & Pastore, R. (2020). Enhancing motivation in workplace training with casual games: A twelve month field study of retail employees. Educational Technology Research and Development, 68(5), 2263–2284. https://doi.org/10.1007/s11423-020-09769-2CrossRefGoogle Scholar
Landers, R. N., Armstrong, M. B., Collmus, A. B., Mujcic, S., & Blaik, J. (2021). Theory-driven game-based assessment of general cognitive ability: Design theory, measurement, prediction of performance, and test fairness. Journal of Applied Psychology, 107(10). https://doi.org/10.1037/apl0000954Google ScholarPubMed
Landers, R. N., Auer, E. M., Collmus, A. B., & Armstrong, M. B. (2018). Gamification science, its history and future: Definitions and a research agenda. Simulation & Gaming, 49(3), 315–337. https://doi.org/10.1177/1046878118774385CrossRefGoogle Scholar
Landers, R. N., & Behrend, T. S. (2023). Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models. American Psychologist, 78(1), 36–49. https://doi.org/10.1037/amp0000972CrossRefGoogle ScholarPubMed
Landers, R. N., & Sanchez, D. R.(2022). Game-based, gamified, and gamefully design assessments for employee selection: Definitions, distinctions, design, and validation. International Journal of Selection and Assessment, 30(1), 1–13. https://doi.org/10.1111/ijsa.12376CrossRefGoogle Scholar
Landers, R. N., Tondello, G. F., Kappen, D. L., Collmus, A. B., Mekler, E. D., & Nacke, L. (2019). Defining gameful experience as a psychological state caused by gameplay: Replacing the term ‘gamefulness’ with three distinct constructs. International Journal of Human Computer Studies, 127, 81–94. https://doi.org/10.1016/j.ijhcs.2018.08.003CrossRefGoogle Scholar
Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D. R., & Brown, R. A. (2002). Evaluation of a behavioral measure of risk taking: The balloon analogue risk task (BART). Journal of Experimental Psychology: Applied, 8(2), 75–84. https://doi.org/10.1037//1076-898x.8.2.75Google ScholarPubMed
Michael, D. R., & Chen, S. L. (2005). Serious games: Games that educate, train, and inform. Thomson Course Technology.Google Scholar
Mislevy, R. J., Oranje, A., Bauer, M. I., von Davier, A., Hao, J., Corrigan, S., Hoffman, E., DiCerbo, K., & John, M. (2014). Psychometric considerations in game-based assessment. GlassLab. https://www.envisionexperience.com/~/media/files/blog/glasslab- psychometrics.pdf?la=enGoogle Scholar
Mollick, E. R., & Rothbard, N. (2014). Mandatory fun: Consent, gamification and the impact of games at work. The Wharton School Research Paper Series. https://ssrn.com/abstract=2277103Google Scholar
Myors, B., Lievens, F., Schollaert, E., Van Hoye, G., Cronshaw, S. F., Mladinic, A., … Sackett, P. R. (2008). International perspectives on the legal environment for selection. Industrial and Organizational Psychology, 1(2), 206–246. https://doi.org/10.1111/j.1754-9434.2008.00040.xCrossRefGoogle Scholar
Orji, R., Mandryk, R. L., & Vassileva, J. (2017). Improving the efficacy of games for change using personalization models. ACM Transactions on Computer-Human Interaction (TOCHI), 24(5), 1–22. https://doi.org/10.1145/3119929CrossRefGoogle Scholar
Pitoyo, M. D., Sumardi, S., & Asib, A. (2019). Gamification based assessment: A test anxiety reduction through game elements in Quizizz platform. International Online Journal of Education and Teaching (IOJET), 6(3), 456–471. http://iojet.org/index.php/IOJET/article/view/626Google Scholar
Plattner, H., Meinel, C., & Leifer, L. (2011). Design thinking: Understand, improve, apply. Springer-Verlag.Google Scholar
Shen, W., Sackett, P. R., Lievens, F., Schollaert, E., Van Hoye, G., Steiner, D. D., … Cook, M. (2017). Updated perspectives on the international legal environment for selection. In Farr, J. L., Tippins, N. T., Borman, W. C., Chan, D., Coovert, M. D., Jacobs, R., … & Schneider, B. (Eds.), Handbook of employee selection (2nd ed., pp. 659–677). Routledge. https://doi.org/10.4324/9781315690193-29CrossRefGoogle Scholar
Tippins, N. T. (2015). Technology and assessment in selection. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 551–582. https://doi.org/10.1146/annurev-orgpsych-031413-091317CrossRefGoogle Scholar