Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-gbm5v Total loading time: 0 Render date: 2025-01-03T15:59:33.106Z Has data issue: false hasContentIssue false

Chapter 5 - Game-Based Assessment around the Globe

International Differences in Privacy, Legality, and Applicant Reaction Concerns

from Part II - Global Perspectives on Key Methods/Topics

Published online by Cambridge University Press:  08 November 2023

By
Richard N. Landers ,
Vivien Lee  and
Chulin Chen
Edited by
Louis Tay ,
Sang Eun Woo  and
Tara Behrend
Louis Tay
Affiliation:
Purdue University, Indiana
Sang Eun Woo
Affiliation:
Purdue University, Indiana
Tara Behrend
Affiliation:
Purdue University, Indiana
Get access

Summary

Despite promising early evidence for the validity of well-designed game-based assessments (GBAs) for employee selection, the interaction between the complexity of games and their use in international and cross-cultural contexts is unknown. To address this, this paper presents a descriptive, qualitative study examining the perspectives of both GBA vendors and organizational stakeholders related to cross-cultural issues unique to GBAs related to 1) privacy, 2) legality, and 3) applicant reactions. Overall, privacy and legality concerns appeared similar for GBAs as with other assessment methods, although certain common characteristics of GBAs amplify common concerns. Applicant reactions appeared more positive to GBAs across national borders and cultures than traditional assessments, although some international differences were reported. Other cross-cultural topics raised included international differences in the conflation of GBA and artificial intelligence, in the importance of mobile-first design, and in the ability of GBAs to provide a more language-agnostic experience than other assessment types.

Keywords

game-based assessmentgamesgamificationcross-culturalinternationalprivacylegalityapplicant reactions
Type
Chapter
Information
Technology and Measurement around the Globe , pp. 157 - 175
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Armstrong, M. B., Ferrell, J., Collmus, A. B., & Landers, R. N. (2016). Correcting misconceptions about gamification of assessment: More than SJTs and badges. Industrial and Organizational Psychology, 9(3), 671–677. https://doi.org/10.1017/iop.2016.69CrossRefGoogle Scholar
Auer, E. M., Mersy, G., Marin, S., Blaik, J., & Landers, R. N. (2022). Using machine learning to model trace behavioral data from a game-based assessment. International Journal of Selection and Assessment, 30(1), 82–102. https://doi.org/10.1111/ijsa.12363CrossRefGoogle Scholar
Bhatia, S., & Ryan, A. M. (2018). Hiring for the win: Game-based assessment in employee selection. In Dulebohn, J. H. & Stone, D. L. (Eds.), The brave new world of eHRM 2.0 (pp. 81–110). IAP Information Age Publishing.Google Scholar
Bina, S., Mullins, J. K., & Petter, S. (2021). Examining game-based approaches in human resources recruitment and selection: A literature review and research agenda. In Proceedings of the 54th Hawaii International Conference on System Sciences (pp. 1325–1334).CrossRefGoogle Scholar
Bjogvinsson, E., Ehn, P., & Hillgren, P.-A. (2012). Design things and design thinking: Contemporary participatory design challenges. Design Issues, 28(3), 101–116. https://doi.org/10.1162/DESI_a_00165CrossRefGoogle Scholar
Caillois, R. (2011). Man, play, and games. University of Illinois Press.Google Scholar
Cubrich, M., King, R., Mracek, D., Strong, J., Hassenkamp, K., Vaughn, D., & Dudley, N. (2021). Examining the criterion-related validity evidence of LinkedIn profile elements in an applied sample. Computers in Human Behavior, 120, 106742. https://doi.org/10.1016/j.chb.2021.106742CrossRefGoogle Scholar
Deterding, S., Sicart, M., Nacke, L., O’Hara, K., & Dixon, D. (2011). Gamification: Using game-design elements in non-gaming contexts. In CHI’11 extended abstracts on human factors in computing systems (pp. 2425–2428). ACM Press.Google Scholar
Grelle, D. M., & Gutierrez, S. L. (2019). Developing device-equivalent and effective measures of complex thinking with an information processing framework and mobile first design principles. Personnel Assessment and Decisions, 5(3). https://doi.org/10.25035/pad.2019.03.004CrossRefGoogle Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. The Behavioral and Brain Sciences, 33(2–3), 61–135. https://doi.org/10.1017/S0140525X0999152XCrossRefGoogle ScholarPubMed
Howison, J., Wiggins, A., & Crowston, K. (2011). Validity issues in the use of social network analysis with digital trace data. Journal of the Association for Information Systems, 12(12). https://doi.org/10.17705/1jais.00282CrossRefGoogle Scholar
Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. Proceedings of the AAAI Workshop on Challenges in Game AI, 4(1), 1722–1726.Google Scholar
Ip, C. (2018, May 4). To find a job, play these games. Engadget. https://www.engadget.com/2018/05/04/pymetrics-gamified-recruitment-behavioral-tests/Google Scholar
Jackson, G. T., Grace, L., Inglese, P., Wain, J., & Hone, R. (2018). Awkward Annie: Game-based assessment of English pragmatic skills. In Cheok, A., Inami, M., & Romão, T. (Eds.), Advances in computer entertainment technology. ACE 2017. Lecture Notes in Computer Science, Vol. 10714. Springer. https://doi.org/10.1007/978-3-319-76270-8_54Google Scholar
Jones, S. E. (2008). The meaning of video games: Gaming and textual strategies. Routledge.CrossRefGoogle Scholar
Kapp, K. M., Valtchanov, D., & Pastore, R. (2020). Enhancing motivation in workplace training with casual games: A twelve month field study of retail employees. Educational Technology Research and Development, 68(5), 2263–2284. https://doi.org/10.1007/s11423-020-09769-2CrossRefGoogle Scholar
Landers, R. N., Armstrong, M. B., Collmus, A. B., Mujcic, S., & Blaik, J. (2021). Theory-driven game-based assessment of general cognitive ability: Design theory, measurement, prediction of performance, and test fairness. Journal of Applied Psychology, 107(10). https://doi.org/10.1037/apl0000954Google ScholarPubMed
Landers, R. N., Auer, E. M., Collmus, A. B., & Armstrong, M. B. (2018). Gamification science, its history and future: Definitions and a research agenda. Simulation & Gaming, 49(3), 315–337. https://doi.org/10.1177/1046878118774385CrossRefGoogle Scholar
Landers, R. N., & Behrend, T. S. (2023). Auditing the AI auditors: A framework for evaluating fairness and bias in high stakes AI predictive models. American Psychologist, 78(1), 36–49. https://doi.org/10.1037/amp0000972CrossRefGoogle ScholarPubMed
Landers, R. N., & Sanchez, D. R.(2022). Game-based, gamified, and gamefully design assessments for employee selection: Definitions, distinctions, design, and validation. International Journal of Selection and Assessment, 30(1), 1–13. https://doi.org/10.1111/ijsa.12376CrossRefGoogle Scholar
Landers, R. N., Tondello, G. F., Kappen, D. L., Collmus, A. B., Mekler, E. D., & Nacke, L. (2019). Defining gameful experience as a psychological state caused by gameplay: Replacing the term ‘gamefulness’ with three distinct constructs. International Journal of Human Computer Studies, 127, 81–94. https://doi.org/10.1016/j.ijhcs.2018.08.003CrossRefGoogle Scholar
Lejuez, C. W., Read, J. P., Kahler, C. W., Richards, J. B., Ramsey, S. E., Stuart, G. L., Strong, D. R., & Brown, R. A. (2002). Evaluation of a behavioral measure of risk taking: The balloon analogue risk task (BART). Journal of Experimental Psychology: Applied, 8(2), 75–84. https://doi.org/10.1037//1076-898x.8.2.75Google ScholarPubMed
Michael, D. R., & Chen, S. L. (2005). Serious games: Games that educate, train, and inform. Thomson Course Technology.Google Scholar
Mislevy, R. J., Oranje, A., Bauer, M. I., von Davier, A., Hao, J., Corrigan, S., Hoffman, E., DiCerbo, K., & John, M. (2014). Psychometric considerations in game-based assessment. GlassLab. https://www.envisionexperience.com/~/media/files/blog/glasslab- psychometrics.pdf?la=enGoogle Scholar
Mollick, E. R., & Rothbard, N. (2014). Mandatory fun: Consent, gamification and the impact of games at work. The Wharton School Research Paper Series. https://ssrn.com/abstract=2277103Google Scholar
Myors, B., Lievens, F., Schollaert, E., Van Hoye, G., Cronshaw, S. F., Mladinic, A., … Sackett, P. R. (2008). International perspectives on the legal environment for selection. Industrial and Organizational Psychology, 1(2), 206–246. https://doi.org/10.1111/j.1754-9434.2008.00040.xCrossRefGoogle Scholar
Orji, R., Mandryk, R. L., & Vassileva, J. (2017). Improving the efficacy of games for change using personalization models. ACM Transactions on Computer-Human Interaction (TOCHI), 24(5), 1–22. https://doi.org/10.1145/3119929CrossRefGoogle Scholar
Pitoyo, M. D., Sumardi, S., & Asib, A. (2019). Gamification based assessment: A test anxiety reduction through game elements in Quizizz platform. International Online Journal of Education and Teaching (IOJET), 6(3), 456–471. http://iojet.org/index.php/IOJET/article/view/626Google Scholar
Plattner, H., Meinel, C., & Leifer, L. (2011). Design thinking: Understand, improve, apply. Springer-Verlag.Google Scholar
Rowe, P. (1986). Design thinking. The MIT Press.Google Scholar
Shen, W., Sackett, P. R., Lievens, F., Schollaert, E., Van Hoye, G., Steiner, D. D., … Cook, M. (2017). Updated perspectives on the international legal environment for selection. In Farr, J. L., Tippins, N. T., Borman, W. C., Chan, D., Coovert, M. D., Jacobs, R., … & Schneider, B. (Eds.), Handbook of employee selection (2nd ed., pp. 659–677). Routledge. https://doi.org/10.4324/9781315690193-29CrossRefGoogle Scholar
Tippins, N. T. (2015). Technology and assessment in selection. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 551–582. https://doi.org/10.1146/annurev-orgpsych-031413-091317CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×