Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-25T19:28:38.396Z Has data issue: false hasContentIssue false
  • English
  • Français

Content Validation Is Useful for Many Things, but Validity Isn't One of Them

Published online by Cambridge University Press:  07 January 2015

Kevin R. Murphy
Kevin R. Murphy*
Affiliation:
Pennsylvania State University
*
E-mail: [email protected], Address: Kevin R. Murphy, Department of Psychology, Pennsylvania State University, University Park, PA 16802.
Get access Rights & Permissions [Opens in a new window]

Abstract

Content-oriented validation strategies establish the validity of selection tests as predictors of performance by comparing the content of the tests with the content of the job. These comparisons turn out to have little if any bearing on the predictive validity of selection tests. There is little empirical support for the hypothesis that the match between job content and test content influences validity, and there are often structural factors in selection (e.g., positive correlations among selection tests) that strongly limit the possible influence of test content on validity. Comparisons between test content and job content have important implications for the acceptability of testing, the defensibility of tests in legal proceedings, and the transparency of test development and validation, but these comparisons have little if any bearing on validity.

Type
Focal Article
Information
Industrial and Organizational Psychology , Volume 2 , Issue 4 , December 2009 , pp. 453 - 464
Copyright
Copyright © Society for Industrial and Organizational Psychology 2009 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, N., Born, M. P. H., & Cunningham-Snell, N. (2001). Recruitment and selection: Applicant perspectives and outcomes. In Anderson, N., Ones, D. S., Senangil, H. K., & Viswesvaran, C. (Eds.), Handbook of industrial, work and organizational psychology: Personnel Psychology (Vol. 1, pp. 200–218). London: Sage.Google Scholar
Anderson, N., & Witvliet, C. (2008). Fairness reactions to personnel selection methods: An international comparison between the Netherlands, the United States, France, Spain, Portugal, and Singapore. International Journal of Selection and Assessment, 16, 1–13.Google Scholar
Berry, C. M., Sackett, P. R., & Landers, R. N. (2007). Revisiting interview–cognitive ability relationships: Attending to specific range restriction mechanisms in meta-analysis. Personnel Psychology, 60, 837–874.Google Scholar
Binning, J. F. (2007). Validation strategies. In Rogelberg, S. G. (Ed.), Encyclopedia of industrial and organizational psychology (Vol. 2, pp. 859–864). Thousand Oaks, CA: Sage.Google Scholar
Bobko, P., Roth, P. L., & Potosky, D. (1999). Derivation and implications of a meta-analytic matrix incorporating cognitive ability, alternative predictors, and job. Personnel Psychology, 60, 561–589.Google Scholar
Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, 87–100.Google Scholar
Carrier, M. R., Dalessio, A. T., & Brown, S. H. (1990). Correspondence between estimates of content and criterion-related validity values. Personnel Psychology, 43, 85–100.Google Scholar
Chan, D., Schmitt, N., Jennings, D., Clause, C., & Delbridge, K. (1998). Applicant perceptions of test fairness: Integrating justice and self-serving bias perspectives. International Journal of Selection and Assessment, 6, 232–239.Google Scholar
Conway, J. M., & Peneno, G. M. (1999). Comparing structured interview question types: Construct validity and applicant reactions. Journal of Business and Psychology, 13, 485–506.Google Scholar
Cortina, J. M., Goldstein, N. B., Payne, S. C., Davison, H. K., & Gilliland, S. W. (2000). The incremental validity of interview scores over and above cognitive ability and conscientiousness scores. Personnel Psychology, 53, 325–351.Google Scholar
Dalessio, A. T. & Silverhart, T. A. (1994). Combining biodata test and interview information. Predicting decisions and performance criteria. Personnel Psychology, 47, 303–315.Google Scholar
Dye, D. A., Reck, M., & McDaniel, M. A. (1993). The validity of job knowledge measures. International Journal of Selection and Assessment, 1, 153–157.Google Scholar
Dwight, S. A., & Alliger, G. M. (1997). Reaction to overt integrity test items. Educational and Psychological Measurement, 57, 937–948.Google Scholar
Earles, J. A., & Ree, M. J. (1992). The predictive validity of the ASVAB for training grades. Educational and Psychological Measurement, 52, 721–725.Google Scholar
Elkins, T. J., & Phillips, J. S. (2000). Job context, selection decision outcome, and perceived fairness of selection tests: Biodata as an illustrative case. Journal of Applied Psychology, 85, 479–484.Google Scholar
Fleishman, E. A., & Mumford, M. D. (1991). Evaluating classifications of job behavior: A construct validation of ability requirement scales. Personnel Psychology, 44, 523–575.Google Scholar
Fleishman, E. A., & Quaintance, M. K. (1984). Taxonomies of human performance: The description of human tasks. New York: Academic Press.Google Scholar
Gilliland, S. W. (1993). The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review, 18, 694–734.Google Scholar
Gilliland, S. W. (1994). Effects of procedural and distributive justice on reactions to a selection system. Journal of Applied Psychology, 79, 691–701.Google Scholar
Gilliland, S. W. (1995). Fairness from the applicant's perspective: Reactions to employee selection procedures. International Journal of Selection and Assessment, 3, 11–19.Google Scholar
Goldstein, I. L., & Zedeck, S. (1996). Content validation. In Barrett, R. (Ed.), Fair employment strategies in human resource management (pp. 27–37). Westport, CT: Quorum Books.Google Scholar
Goldstein, I. L., Zedeck, S., & Schneider, B. (1993). An exploration of the job analysis-content validity process. In Schmitt, N. & Borman, W. C. (Eds.), Personnel selection in organizations. San Francisco: Jossey-Bass.Google Scholar
Guion, R. M. (1977). Content validity–The source of my discontent. Applied Psychological Measurement, 1, 1–10.Google Scholar
Guion, R. M. (1978a). “Content validity” in moderation. Personal Psychology, 31, 205–213.Google Scholar
Guion, R. M. (1978b). Scoring content domain samples: The problem of fairness. Journal of Applied Psychology, 63, 499–506.Google Scholar
Guion, R. M. (1998). Assessment, measurement, and prediction for personnel selection. Mahwah, NJ: Erlbaum.Google Scholar
Hartigan, J. A., & Wigdor, A. K. (1989). Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery. Washington, DC: National Academy Press.Google Scholar
Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004). Applicant reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639–683.Google Scholar
Himelfarb, R., & Perotti, R. (2004). Principle over politics?: The domestic policy of the George H. W. Bush presidency. New York: Greenwood.Google Scholar
Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996). A meta-analytic investigation of cognitive ability employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology, 81, 459–473.Google Scholar
Hunter, J. E. (1982, June). What is the validity of a content valid test? Presented at meeting of International Personnel Management Association, Minneapolis.Google Scholar
Hunter, J. E. (1986). Cognitive ability, cognitive aptitudes, job knowledge and job performance. Journal of Vocational Behavior, 29, 340–362.Google Scholar
Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72–98.Google Scholar
Jeanneret, P. R., & Strong, M. H. (2003). Linking O*NET job information to job requirement predictors: An O*NET application. Personnel Psychology, 56, 465–492.Google Scholar
Jensen, A. R. (1980). Bias in mental testing. New York: Free Press.Google Scholar
Jones, G. E., & Ree, M. J. (1998). Aptitude test score validity: No moderating effect due to job ability requirement differences. Educational and Psychological Measurement, 58, 284–294.Google Scholar
McDaniel, M. A., Morgeson, F. P., Finnegan, E. B., Campion, M. A., & Braverman, E. P. (2001). Use of situational judgment tests to predict job performance: A clarification of the literature. Journal of Applied Psychology, 86, 730–740.Google Scholar
Mosoco, S. (2000). Selection interview: A review of validity evidence, adverse impact and applicant reactions. International Journal of Selection and Assessment, 8, 237–247.Google Scholar
Murphy, K. R., & Davidshofer, C. O. (2005). Psychological testing: Principles and applications (6th ed.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
Murphy, K. R., Dzieweczynski, J. L., & Yang, Z. (2009). Positive manifold limits the relevance of content-matching strategies for validating selection test batteries. Journal of Applied Psychology, 94, 1018–1031.Google Scholar
Peterson, N. G., Wise, L. L., Arabian, J., & Hoffman, R. G. (2001). Synthetic validation and validity generalization when empirical validation is not possible. In Campbell, J. & Knapp, D. (Eds.), Exploring the upper limits of personnel selection and classification (pp. 411–451). Mahwah, NJ: Erlbaum.Google Scholar
Potosky, D., Bobko, P., & Roth, P. L. (2005). Forming composites of cognitive ability and alternate measures to predict job performance and reduce adverse impact: Corrected estimated and realistic expectations, International Journal of Selection and Assessment, 13, 304–315.Google Scholar
Prediger, D. (1987). Validity of the new Armed Services Vocational Aptitude Battery job cluster scores in career planning. Career Development Quarterly, 36, 113–125.Google Scholar
Ree, M. J., & Carretta, T. R. (2002). G2k. Human Performance, 15, 3–23.Google Scholar
Ree, M. J., & Earles, J. A. (1991). Predicting training success: Not much more than g . Personnel Psychology, 44, 321–332.Google Scholar
Ree, M. J., & Earles, J. A. (1992). Intelligence is the best predictor of job performance. Current Directions in Psychological Science, 1, 86–89.Google Scholar
Ree, M. J., Earles, J. A., & Teachout, M. S. (1994). Predicting job performance: Not much more than g . Journal of Applied Psychology, 79, 518–524.Google Scholar
Reilly, R. R., & Chao, G. T. (1982). Validity and fairness of some alternate employee selection procedures. Personnel Psychology, 35, 1–67.Google Scholar
Roth, P. L., Bobko, P., & McFarland, L. A. (2005). A meta-analysis of work sample test validity: Updating and integrating some classic literature. Personnel Psychology, 58 (4), 1009–1038.Google Scholar
Ryan, A. M., & Huth, M. (2008). Not much more than platitudes? A critical look at applicant reaction research. Human Resource Management Review, 18, 119–132.Google Scholar
Ryan, A. M., & Ployhart, R. E. (2000). Applicants' perceptions of selection procedures and decisions: A critical review and agenda for the future. Journal of Management, 26, 565–606.Google Scholar
Sackett, P. R. (1987). Assessment centers and content validity: Some neglected issues. Personnel Psychology, 40, 13–25.Google Scholar
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.Google Scholar
Schmitt, N., & Chan, D. (2004). An agenda for future research on applicant reactions to selection procedures: A construct-oriented approach. International Journal of Selection and Assessment, 12, 9–23.Google Scholar
Schmitt, N., Gooding, R. Z., Noe, R. D., & Kirsch, M. (1984). Meta-analyses of validity studies published between 1964 and 1982 and the investigation of study characteristics. Personnel Psychology, 37, 407–422.Google Scholar
Schmitt, N., Rogers, W., Chan, D., Sheppard, L., & Jennings, D. (1997). Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology, 82, 719–730.Google Scholar
Society for Industrial and Organizational Psychology, Inc. (2003). Principles for the validation and use of personnel selection procedures (4th ed.). Bowling Green, OH: Author.Google Scholar
Society for Industrial and Organizational Psychology (1987). Principles for the validation and use of personnel selection procedures (3rd ed.). College Park, MD: Author.Google Scholar
Truxillo, D. M., Steiner, D. D., & Gilliland, S. W. (2004). The importance of organizational justice in personnel selection: Defining when selection fairness really matters. International Journal of Selection and Assessment, 13, 39–53.Google Scholar
Uniform Guidelines on Employee Selection Procedures. (1978). 29 C.F.R. 1607.Google Scholar