Hostname: page-component-669899f699-vbsjw Total loading time: 0 Render date: 2025-05-06T23:25:01.922Z Has data issue: false hasContentIssue false
  • English
  • Français

E Pluribus Unum? Why criteria should be multimethod and multirater

Published online by Cambridge University Press:  22 August 2024

Jeffrey M. Cucina Open the ORCID record for Jeffrey M. Cucina [Opens in a new window]  and
Theodore L. Hayes Open the ORCID record for Theodore L. Hayes [Opens in a new window]
Jeffrey M. Cucina*
Affiliation:
U.S. Customs and Border Protection, Washington, DC, USA
Theodore L. Hayes
Affiliation:
U.S. Customs and Border Protection, Arlington, VA, USA
*
Corresponding author: Jeffrey M. Cucina; Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'
Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 17 , Issue 3 , September 2024 , pp. 314 - 318
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of Society for Industrial and Organizational Psychology

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

The views expressed in this paper are those of the authors and do not necessarily reflect the views of U.S. Customs and Border Protection, the U.S. Federal Government, or any agency of the U.S. Federal Government.

References

Abrahams, N. M., Mendoza, J. L., & Held, J. D. (2015). Interpreting the correlation (validity) coefficient. In Held, J. D., Carretta, T. R., Johnson, J. W., & McCloy, R. A. (Eds.), Technical guidance for conducting ASVAB validation/standards studies in the U.S. Navy (pp. 37–49). Navy Personnel Research, Studies, and Technology (Technical Report NPRST-TR-15-2).Google Scholar
Al Ali, O. E., Garner, I., & Magadley, W. (2012). An exploration of the relationship between emotional intelligence and job performance in police organizations. Journal of Police and Criminal Psychology, 27(1), 1–8. https://doi.org/10.1007/s11896-011-9088-9 Google Scholar
Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14(2), 87–10.Google Scholar
Cooper, W. H., & Richardson, A. J. (1986). Unfair comparisons. Journal of Applied Psychology, 71(2), 179–184. https://doi.org/10.1037/0021-9010.71.2.179 Google Scholar
Cucina, J. M., Burtnick, S. K., De la Flor, M. E., Walmsley, P. T., & Wilson, K. J. (2024). Meta-analytic validity of cognitive ability for hands-on military job proficiency. Intelligence, 104, 101818. https://doi.org/10.1016/j.intell.2024.101818 Google Scholar
Cucina, J. M., Martin, N. R., Vasilopoulos, N. L., & Thibodeaux, H. F. (2012). Self-serving bias effects on job analysis ratings. Journal of Psychology: Interdisciplinary and Applied, 146(5), 1–21.Google ScholarPubMed
Foster, J., Steel, P., Harms, P., O’Neill, T., & Wood, D. (2024). Selection tests work better than we think they do, and have for years. Industrial and Organizational Psychology, 17.Google Scholar
Funder, D. C., & Ozer, D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168.Google Scholar
Grubb, A. D. (2011). Promotional assessment at the FBI: How the search for a high-tech solution led to a low-tech simulation. In Adler, S. & Tippins, N. T. (Eds.), Technology-enhanced assessment of talent (pp. 293–306). Jossey-Bass.Google Scholar
Hayes, T. L., McElreath, J., & Reilly, S. M. (2022). The criterion-related validity of logic-based tests of reasoning for personnel selection. In Hayes, T. L. (chair), The validity of logic-based measurement for selection and promotion decisions. Symposium at the 17th Annual SIOP Conference.Google Scholar
Hunter, J. E. (1983a). A causal analysis of cognitive ability, job knowledge, job performance, and supervisor ratings. In Landy, F., Zedeck, S., & Cleveland, J. (Ed.), Performance measurement and theory (pp. 257–266). Erlbaum.Google Scholar
Hunter, J. E. (1983b). Test validation for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). U.S. Department of Labor.Google Scholar
Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge, and job performance. Journal of Vocational Behavior, 29, 340–362.Google Scholar
Kerr, S. (1975). On the folly of rewarding A, while hoping for B. Academy of Management Journal, 18(4), 769–783.Google ScholarPubMed
MacLane, C. N., Cucina, J. M., Busciglio, H. H., & Su, C. (2020). Supervisory opportunity to observe moderates criterion-related validity estimates. International Journal of Selection and Assessment, 28(1), 55–67. https://psycnet.apa.org/doi/10.1111/ijsa.12267 Google Scholar
Martin, R., Guillaume, Y., Thomas, G., Lee, A., & Epitropaki, O. (2016). Leader-member exchange (LMX) and performance: A meta-analysis. Personnel Psychology, 69(1), 67–121. https://doi-org.proxygw.wrlc.org/10.1111/peps.12100 Google Scholar
McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43(2), 335–354. https://doi.org/10.1111/j.1744-6570.1990.tb01562.x Google Scholar
Murphy, K. R., & Cleveland, J. N. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Sage.Google Scholar
OʼBoyle, E. H. Jr, Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P. A. (2011). The relation between emotional intelligence and job performance: A meta-analysis. Journal of Organizational Behavior, 32(5), 788–818. https://doi.org/10.1002/job.714 Google Scholar
Paullin, C., Putka, D. J., Tsacoumis, S., & Colberg, M. (2010). Using a logic-based measurement approach to measure cognitive ability. In Paullin, C. (chair), Cognitive ability testing: Exploring new models, methods, and statistical techniques. Symposium conducted at the 25th Annual SIOP Conference.Google Scholar
Ree, M. J., Earles, J. A., & Teachout, M. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79, 518–524.Google Scholar
Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107, 2040–2068. https://doi.org/10.1037/apl0000994 Google ScholarPubMed
Salgado, J. F., & Moscoso, S. (2019). Meta-analysis of the validity of general mental ability for five performance criteria: Hunter and Hunter, 1984 revisited. Frontiers in Psychology, 10, 476186.Google ScholarPubMed
Schmidt, F. L. (2013). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 100 years of research. Invited presentation at the 2013 Fall Event of the Personnel Testing Council/Metropolitan Washington.Google Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274. https://doi.org/10.1037/0033-2909.124.2.262 Google Scholar
Schmidt, F. L., Hunter, J. E., Outerbridge, A. N., & Trattner, M. H. (1986). The economic impact of job selection methods on size, productivity, and payroll costs of the federal work force: An empirically based demonstration. Personnel Psychology, 39, 1–29.Google Scholar
U.S. General Accounting Office. (2003). Equal employment opportunity: Hiring, promotion, and discipline processes at DEA (report #GAO-03-413). Author.Google Scholar
van Rijn, P., & Payne, S. S. (1980). Criterion related validity research base for the D.C. firefighter selection test. Personnel Research and Development Center, U.S. Office of Personnel Management.Google Scholar