Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-11-22T17:05:19.709Z Has data issue: false hasContentIssue false

Chapter 9 - A European Perspective on Psychometric Measurement Technology

from Part III - Regional Focus

Published online by Cambridge University Press:  08 November 2023

By
Nigel Guenole ,
Cicek Svensson ,
Bart Wille ,
Kristina Aloyan  and
Peter Saville
Edited by
Louis Tay ,
Sang Eun Woo  and
Tara Behrend
Louis Tay
Affiliation:
Purdue University, Indiana
Sang Eun Woo
Affiliation:
Purdue University, Indiana
Tara Behrend
Affiliation:
Purdue University, Indiana
Get access

Summary

The quality of psychological assessment processes in talent management is influenced by our choices about which measurement technologies to use. Technology with relevance to assessing talent is also advancing at great speed in many domains. These advances include processing power and speed, human computer interaction research, and machine learning and artificial intelligence. Given these rapid developments, it is an appropriate time to pause and take stock of how emerging assessment approaches (e.g., game-based assessment) that leverage these new developments are used, relative to more traditional approaches such as questionnaires and interviews. To achieve this objective, we report here on a survey of European assessment practitioners. We ask about the technology they use for psychological assessment, the constructs they measure with those approaches, and the levels of organisations they are used at. We also asked about how traditional approaches are being enhanced with technology and about practitioner perceptions of the reliability, validity and adverse impact and privacy of their technological choices.

Keywords

psychometric testingpsychological assessmenttalent managementtechnology
Type
Chapter
Information
Technology and Measurement around the Globe , pp. 271 - 307
Publisher: Cambridge University Press
Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, N., Salgado, J. F., & Hülsheger, U. R. (2010). Applicant reactions in selection: Comprehensive meta-analysis into reaction generalization versus situational specificity. International Journal of Selection and Assessment, 18(3), 291–304. https://doi.org/10.1111/j.1468-2389.2010.00512.xCrossRefGoogle Scholar
Arthur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center dimensions. Personnel Psychology, 56(1), 125–153. https://doi.org/10.1111/j.1744-6570.2003.tb00146.xCrossRefGoogle Scholar
Arvey, R. D., & Campion, J. E. (1982). The employment interview: A summary and review of recent research. Personnel Psychology, 35(2), 281–322. https://doi.org/10.1111/j.1744-6570.1982.tb02197.xCrossRefGoogle Scholar
Becton, J. B., Walker, J. H., Gilstrap, J. B., & Schwager, P. H. (2019, June 13). Social media snooping on job applicants: The effects of unprofessional social media information on recruiter perceptions. Personnel Review, 48(5), 1261–1280. https://doi.org/10.1108/PR-09-2017-0278CrossRefGoogle Scholar
Berry, C. M., Sackett, P. R., & Landers, R. N. (2007). Revisiting interview–cognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 60(4), 837–874. https://doi.org/10.1111/j.1744-6570.2007.00093.xCrossRefGoogle Scholar
Catano, V. M., Brochu, A., & Lamerson, C. D. (2012). Assessing the reliability of situational judgment tests used in high-stakes situations. International Journal of Selection and Assessment, 20(3), 333–346. https://doi.org/10.1111/j.1468-2389.2012.00604.xCrossRefGoogle Scholar
Chamorro-Premuzic, T., & Furnham, A. (2010). The psychology of personnel selection. Cambridge University Press. https://doi.org/10.1017/CBO9780511819308CrossRefGoogle Scholar
Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143–159. https://doi.org/10.1037/0021-9010.82.1.143CrossRefGoogle ScholarPubMed
CIPD. (2020). Resourcing and talent planning survey 2020. https://www.cipd.org/globalassets/media/knowledge/knowledge-hub/reports/resourcing-and-talent-planning-2020_tcm18-85530.pdfGoogle Scholar
Collins, M. W., & Morris, S. B. (2008). Testing for adverse impact when sample size is small. The Journal of Applied Psychology, 93(2), 463–471. https://doi.org/10.1037/0021-9010.93.2.463CrossRefGoogle ScholarPubMed
Dipboye, R. L. (1994). Structured and unstructured selection interviews: Beyond the job-fit model. Research in Personnel and Human Resources Management, 12, 79–123. https://www.academia.edu/download/44957092/STRUCTURED_AND_UNSTRUCTURED_SELECTION_IN20160421-2666-opzc8p.pdfGoogle Scholar
Drasgow, F. (2015). Technology and testing: Improving educational and psychological measurement. Routledge.CrossRefGoogle Scholar
Fetzer, M., McNamara, J., & Geimer, J. L. (2017). Gamification, serious games and personnel selection. In Goldstein, H. W., Pulakos, E. D., Semedo, C., & Passmore, J. (Eds.), The Wiley Blackwell handbook of the psychology of recruitment, selection and employee retention (pp. 293–309). Wiley.CrossRefGoogle Scholar
Georgiou, K., Gouras, A., & Nikolaou, I. (2019). Gamification in employee selection: The development of a gamified assessment. International Journal of Selection and Assessment, 27(2), 91–103. https://doi.org/10.1111/ijsa.12240CrossRefGoogle Scholar
Guenole, N., Chernyshenko, O. S., Stark, S., Cockerill, T., & Drasgow, F. (2013). More than a mirage: A large-scale assessment centre with more dimension variance than exercise variance. Journal of Occupational and Organizational Psychology, 86(1), 5–21. https://doi.org/10.1111/j.2044-8325.2012.02063.xCrossRefGoogle Scholar
Guenole, N., Chernyshenko, O., Stark, S., & Drasgow, F. (2015). Are predictions based on situational judgement tests precise enough for feedback in leadership development? European Journal of Work and Organizational Psychology, 24(3), 433–443. https://doi.org/10.1080/1359432X.2014.926890CrossRefGoogle Scholar
Guenole, N., Chernyshenko, O. S., & Weekly, J. (2017). On designing construct driven situational judgment tests: Some preliminary recommendations. International Journal of Testing, 17(3), 234–252. https://doi.org/10.1080/15305058.2017.1297817CrossRefGoogle Scholar
Harris, M. M. (1989). Reconsidering the employment interview: A review of recent literature and suggestions for future research. Personnel Psychology, 42(4), 691–726. https://doi.org/10.1111/j.1744-6570.1989.tb00673.xCrossRefGoogle Scholar
Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta‐analysis. Personnel Psychology, 57(3), 639–683. https://doi.org/10.1111/j.1744-6570.2004.00003.xCrossRefGoogle Scholar
Hickman, L., Bosch, N., Ng, V., Saef, R., Tay, L., & Woo, S. E. (2021). Automated video interview personality assessments: Reliability, validity, and generalizability investigations. Journal of Applied Psychology, 107(8), 1323–1351. https://doi.org/10.1037/apl0000695CrossRefGoogle ScholarPubMed
Hickman, L., Saef, R., Ng, V., Woo, S. E., Tay, L., & Bosch, N. (2021). Developing and evaluating language‐based machine learning algorithms for inferring applicant personality in video interviews. Human Resource Management Journal. https://doi.org/10.1111/1748-8583.12356CrossRefGoogle Scholar
Howland, A. C., Rembisz, R., Wang-Jones, T. S., Heise, S. R., & Brown, S. (2015). Developing a virtual assessment center. Consulting Psychology Journal: Practice and Research, 67(2), 110–126. https://doi.org/10.1037/cpb0000034CrossRefGoogle Scholar
Huffcutt, A. I., Culbertson, S. S., & Weyhrauch, W. S. (2013). Employment interview reliability: New meta-analytic estimates by structure and format. International Journal of Selection and Assessment, 21(3), 264–276. https://doi.org/10.1111/ijsa.12036CrossRefGoogle Scholar
International Taskforce on Assessment Center Guidelines. (2015). Guidelines and ethical considerations for assessment center operations. Journal of Management, 41(4), 1244–1273. https://doi.org/10.1177/0149206314567780Google Scholar
International Test Commission. (2005). ITC guidelines on computer-based and internet delivered testing, version 1.0. https://www.intestcom.org/files/guideline_computer_based_testing.pdfGoogle Scholar
Judge, T. A., Cable, D. M., & Higgins, C. A. (2000). The employment interview: A review of recent research and recommendations for future research. Human Resource Management Review, 10(4), 383–406. https://doi.org/10.1016/S1053-4822(00)00033-4CrossRefGoogle Scholar
Kasten, N., & Freund, P. A. (2016). A meta-analytical multilevel reliability generalization of situational judgment tests (SJTs). European Journal of Psychological Assessment: Official Organ of the European Association of Psychological Assessment, 32(3), 230–240. https://doi.org/10.1027/1015-5759/a000250CrossRefGoogle Scholar
Lance, C. E. (2008). Why assessment centers do not work the way they are supposed to. Industrial and Organizational Psychology, 1(1), 84–97. https://doi.org/10.1111/j.1754-9434.2007.00017.xCrossRefGoogle Scholar
Landers, R. N., Armstrong, M. B., Collmus, A. B., Mujcic, S., & Blaik, J. (2022). Theory-driven game-based assessment of general cognitive ability: Design theory, measurement, prediction of performance, and test fairness. Journal of Applied Psychology, 107(10), 1655–1677. https://doi.org/10.1037/apl0000954CrossRefGoogle ScholarPubMed
Landers, R. N., Auer, E. M., & Abraham, J. (2020). Gamifying a situational judgment test with immersion and control game elements: Effects on applicant reactions and construct validity. Journal of Managerial Psychology, 35(4), 225–239. https://doi.org/10.1108/JMP-10-2018-0446CrossRefGoogle Scholar
Law, S. J., Bourdage, J., & O’Neill, T. A. (2016). To fake or not to fake: Antecedents to interview faking, warning instructions, and its impact on applicant reactions. Frontiers in Psychology, 7, Article 1771. https://doi.org/10.3389/fpsyg.2016.01771CrossRefGoogle ScholarPubMed
Levashina, J., & Campion, M. A. (2007). Measuring faking in the employment interview: Development and validation of an interview faking behavior scale. Journal of Applied Psychology, 92(6), 1638–1656. https://doi.org/10.1037/0021-9010.92.6.1638CrossRefGoogle ScholarPubMed
Levashina, J., Hartwell, C. J., Morgeson, F. P., & Campion, M. A. (2014). The structured employment interview: Narrative and quantitative review of the research literature. Personnel Psychology, 67(1), 241–293. https://doi.org/10.1111/peps.12052CrossRefGoogle Scholar
Levinson, M. (2010). Social networking ever more critical to job search success. CIO. https://www.cio.com/article/280120/careers-staffing-social-networking-ever-more-critical-to-job-search-success.htmlGoogle Scholar
Lievens, F. (2006). The ITC guidelines on computer-based and internet-delivered testing: Where do we go from here? International Journal of Testing, 6(2), 189–194. https://doi.org/10.1207/s15327574ijt0602_7CrossRefGoogle Scholar
Lievens, F. (2009). Assessment centres: A tale about dimensions, exercises, and dancing bears. European Journal of Work and Organizational Psychology, 18(1), 102–121. https://doi.org/10.1080/13594320802058997CrossRefGoogle Scholar
Lievens, F. (2017). Construct-driven SJTs: Toward an agenda for future research. International Journal of Testing, 17(3), 269–276. https://doi.org/10.1080/15305058.2017.1309857CrossRefGoogle Scholar
Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college admissions: Illustrating the importance of matching predictor and criterion construct domains. Journal of Applied Psychology, 90(3), 442–452. https://doi.org/10.1037/0021-9010.90.3.442CrossRefGoogle ScholarPubMed
Lievens, F., & Sackett, P. R. (2007). Situational judgment tests in high-stakes settings: Issues and strategies with generating alternate forms. Journal of Applied Psychology, 92(4), 1043–1055. https://doi.org/10.1037/0021-9010.92.4.1043CrossRefGoogle ScholarPubMed
Lukacik, E.-R., Bourdage, J. S., & Roulin, N. (2020). Into the void: A conceptual model and research agenda for the design and use of asynchronous video interviews. Human Resource Management Review, 32(1), Article 100789. https://doi.org/10.1016/j.hrmr.2020.100789Google Scholar
Macan, T. (2009). The employment interview: A review of current studies and directions for future research. Human Resource Management Review, 19(3), 203–218. https://doi.org/10.1016/j.hrmr.2009.03.006CrossRefGoogle Scholar
Macan, T. H., Avedon, M. J., Paese, M., & Smith, D. E. (1994). The effects of applicants’ reactions to cognitive ability tests and an assessment center. Personnel Psychology, 47(4), 715–738. https://doi.org/10.1111/j.1744-6570.1994.tb01573.xCrossRefGoogle Scholar
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86(4), 730–740. https://doi.org/10.1037/0021-9010.86.4.730CrossRefGoogle ScholarPubMed
McDaniel, M. A., Whetzel, D. L., Schmidt, F. L., & Maurer, S. D. (1994). The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79(4), 599–616. https://doi.org/10.1037/0021-9010.79.4.599CrossRefGoogle Scholar
McDaniel Cabrera, M. A., & Nguyen, N. T. (2001). Situational judgment tests: A review of practice and constructs assessed. International Journal of Selection and Assessment, 9(1–2), 103–113. https://doi.org/10.1111/1468-2389.00167CrossRefGoogle Scholar
Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114(3), 449–458. https://doi.org/10.1037/0033-2909.114.3.449CrossRefGoogle Scholar
Melchers, K. G., & Basch, J. M. (2021). Fair play? Sex‐, age‐, and job‐related correlates of performance in a computer‐based simulation game. International Journal of Selection and Assessment, 30(1), 48–61. https://doi.org/10.1111/ijsa.12337CrossRefGoogle Scholar
Melchers, K. G., Roulin, N., & Buehl, A.-K. (2020). A review of applicant faking in selection interviews. International Journal of Selection and Assessment, 28(2), 123–142. https://doi.org/10.1111/ijsa.12280CrossRefGoogle Scholar
Meriac, J. P., Hoffman, B. J., Woehr, D. J., & Fleisher, M. S. (2008). Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings. Journal of Applied Psychology, 93(5), 1042–1052. https://doi.org/10.1037/0021-9010.93.5.1042CrossRefGoogle ScholarPubMed
Morelli, N., Potosky, D., Arthur, W., Jr, & Tippins, N. (2017). A call for conceptual models of technology in I-O psychology: An example from technology-based talent assessment. Industrial and Organizational Psychology, 10(4), 634–653. https://doi.org/10.1017/iop.2017.70CrossRefGoogle Scholar
Moscoso, S. (2000). Selection interview: A review of validity evidence, adverse impact and applicant reactions. International Journal of Selection and Assessment, 8(4), 237–247. https://doi.org/10.1111/1468-2389.00153CrossRefGoogle Scholar
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of Applied Psychology, 75(6), 640–647. https://doi.org/10.1037/0021-9010.75.6.640CrossRefGoogle Scholar
Naim, I., Tanveer, M. I., Gildea, D., & Hoque, M. (2015). Automated analysis and prediction of job interview performance. http://arxiv.org/abs/1504.03425Google Scholar
Nawaz, N., & Gomes, A. M. (2020). Artificial intelligence chatbots are new recruiters. International Journal of Advanced Computer Science and Applications, 10(9). https://doi.org/10.2139/ssrn.3521915Google Scholar
Nye, C. D., Do, B.-R., Drasgow, F., & Fine, S. (2008). Two-step testing in employee selection: Is score inflation a problem? International Journal of Selection and Assessment, 16(2), 112–120. https://doi.org/10.1111/j.1468-2389.2008.00416.xCrossRefGoogle Scholar
Posthuma, R. A., Morgeson, F. P., & Campion, M. A. (2002). Beyond employment interview validity: A comprehensive narrative review of recent research and trends over time. Personnel Psychology, 55(1), 1–81. https://doi.org/10.1111/j.1744-6570.2002.tb00103.xCrossRefGoogle Scholar
Reddock, C. M., Auer, E. M., & Landers, R. N. (2020). A theory of branched situational judgment tests and their applicant reactions. Journal of Managerial Psychology, 35(4), 255–270. https://doi.org/10.1108/JMP-10-2018-0434CrossRefGoogle Scholar
Robertson, I. T., & Smith, M. (2001). Personnel selection. Journal of Occupational and Organizational Psychology, 74(4), 441–472. https://doi.org/10.1348/096317901167479CrossRefGoogle Scholar
Roth, P. L., Bobko, P., Van Iddekinge, C. H., & Thatcher, J. B. (2016). Social media in employee-selection-related decisions: A research agenda for uncharted territory. Journal of Management, 42(1), 269–298. https://doi.org/10.1177/0149206313503018CrossRefGoogle Scholar
Roth, P. L., & Huffcutt, A. I. (2013). A meta-analysis of interviews and cognitive ability. Journal of Personnel Psychology, 12(4), 157–169. https://doi.org/10.1027/1866-5888/a000091CrossRefGoogle Scholar
Ryan, A. M., & Ployhart, R. E. (2014). A century of selection. Annual Review of Psychology, 65, 693–717. https://doi.org/10.1146/annurev-psych-010213-115134CrossRefGoogle ScholarPubMed
Sackett, P. R., & Lievens, F. (2008). Personnel selection. Annual Review of Psychology, 59, 419–450. https://doi.org/10.1146/annurev.psych.59.103006.093716CrossRefGoogle ScholarPubMed
Salgado, J. F., & Moscoso, S. (2002). Comprehensive meta-analysis of the construct validity of the employment interview. European Journal of Work and Organizational Psychology, 11(3), 299–324. https://doi.org/10.1080/13594320244000184CrossRefGoogle Scholar
Scott, J. C., Bartram, D., & Reynolds, D. H. (2017). Next generation technology-enhanced assessment: Global perspectives on occupational and workplace testing. Cambridge University Press.CrossRefGoogle Scholar
Suen, H.-Y., Chen, M. Y.-C., & Lu, S.-H. (2019). Does the use of synchrony and artificial intelligence in video interviews affect interview ratings and applicant attitudes? Computers in Human Behavior, 98, 93–101. https://doi.org/10.1016/j.chb.2019.04.012CrossRefGoogle Scholar
Thornton, G. C., & Gibbons, A. M. (2009). Validity of assessment centers for personnel selection. Human Resource Management Review, 19(3), 169–187. https://doi.org/10.1016/j.hrmr.2009.02.002CrossRefGoogle Scholar
Thornton, G. C., III, & Rupp, D. E. (2006). Assessment centers in human resource management: Strategies for prediction, diagnosis, and development. Lawrence Erlbaum Associates.CrossRefGoogle Scholar
Tippins, N. T., & Adler, S. (2011). Technology-enhanced assessment of talent. Jossey-Bass.CrossRefGoogle Scholar
Van Iddekinge, C. H., Lanivich, S. E., Roth, P. L., & Junco, E. (2016). Social media for selection? Validity and adverse impact potential of a Facebook-based assessment. Journal of Management, 42(7), 1811–1835. https://doi.org/10.1177/0149206313515524CrossRefGoogle Scholar
Wade, J. T., Roth, P. L., Thatcher, J. B., & Dinger, M. (2020). Social media and selection: Political issue similarity, liking, and the moderating effect of social media platform. The Mississippi Quarterly, 44(3), 1301–1357. https://doi.org/10.25300/misq/2020/14119CrossRefGoogle Scholar
Weekley, J. A., Hawkes, B., Guenole, N., & Ployhart, R. E. (2015). Low-fidelity simulations. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 295–322. https://doi.org/10.1146/annurev-orgpsych-032414-111304CrossRefGoogle Scholar
Weekley, J. A., & Jones, C. (1999). Further studies of situational tests. Personnel Psychology, 52(3), 679–700. https://doi.org/10.1111/j.1744-6570.1999.tb00176.xCrossRefGoogle Scholar
Whetzel, D. L., McDaniel, M. A., & Nguyen, N. T. (2008). Subgroup differences in situational judgment test performance: A meta-analysis. Human Performance, 21(3), 291–309. https://doi.org/10.1080/08959280802137820CrossRefGoogle Scholar
Woods, S. A., Ahmed, S., Nikolaou, I., Costa, A. C., & Anderson, N. R. (2020). Personnel selection in the digital age: A review of validity and applicant reactions, and future research challenges. European Journal of Work and Organizational Psychology, 29(1), 64–77. https://doi.org/10.1080/1359432X.2019.1681401CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×