Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-12-02T01:46:33.124Z Has data issue: false hasContentIssue false
  • English
  • Français

Can Synthetic Validity Methods Achieve Discriminant Validity?

Published online by Cambridge University Press:  07 January 2015

Frank L. Schmidt  and
In-Sue Oh
Frank L. Schmidt*
Affiliation:
University of Iowa
In-Sue Oh
Affiliation:
Virginia Commonwealth University
*
E-mail: [email protected], Address: Department of Management and Organizations, Tippie College of Business, University of Iowa, W236 John Pappajohn Business Building, University of Iowa, IA 52242-1994
Get access Rights & Permissions [Opens in a new window]

Extract

Our focus is on the difficulties that synthetic validity encounters in attempting to achieve discriminant validity and the implications of these difficulties. Johnson et al. (2010) acknowledge the potential problems involved in attaining discriminant validity in synthetic validity. For example, they report that Peterson et al. (2001), Johnson (2007), and other synthetic validity studies report failure to achieve discriminant validity. What this failure means is that a synthetic validity equation developed to predict validity for Job A does as well in predicting validity for Jobs B, C, D, and so forth as it does for Job A. Johnson et al. then go on to propose that this problem might be overcome by careful attention to both the criterion and predictor sides of synthetic validity. We question whether their proposals can be made to work.

Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 3 , Issue 3 , September 2010 , pp. 344 - 350
Copyright
Copyright © Society for Industrial and Organizational Psychology 2010 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

Department of Management and Organizations, Tippie College of Business, University of Iowa

**

In-Sue Oh, Department of Management, School of Business, Virginia Commonwealth University.

References

Bertua, C., Anderson, N., & Salgado, J. F. (2005). The predictive validity of cognitive ability tests: A U.K. meta-analysis. Journal of Occupational and Organizational Psychology, 78, 387–409.Google Scholar
Cattin, P. (1980). Estimation of the predictive power of a regression model. Journal of Applied Psychology, 65, 401–414.Google Scholar
Dudley, N., Orvis, K., Lebiecki, J., & Cortina, J. (2006). A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits. Journal of Applied Psychology, 91, 40–57.Google Scholar
Gutenberg, R. L., Arvey, R. D., Osburn, H. G., & Jeanneret, P. R. (1983). The moderating effects of decision-making/information processing job dimensions on test validities. Journal of Applied Psychology, 68, 602–608.Google Scholar
Hoffman, C. C., Holden, L. M., & Gale, K. (2000). So many jobs, so little “N”: Applying expanded validation models to support generalization of cognitive test validity. Personnel Psychology, 53, 955–991.Google Scholar
Hoffman, C. C., & McPhail, S. M. (1998). Exploring options for supporting test use in situations precluding local validation. Personnel Psychology, 51, 987–1003.Google Scholar
Hulsheger, U. R., Maier, G. W., & Stumpp, T. (2007). Validity of general mental ability for the prediction of job performance and training success in Germany: A meta-analysis. International Journal of Selection and Assessment, 15, 3–18.Google Scholar
Hunter, J. E. (1983). Validity generalization for 12,000 jobs: An application of synthetic validity and validity generalization to the General Aptitude Test Battery (GATB). Washington, DC: U.S. Department of Labor, Employment Service.Google Scholar
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.) Newbury Park, CA: Sage.Google Scholar
Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594–612.Google Scholar
Johnson, J. W. (2007). Synthetic validity: A technique of use (finally). In McPhail, S. M.(Ed.), Alternative validation strategies: Developing new and leveraging existing validity evidence (pp. 122–158). San Francisco: Jossey-Bass.Google Scholar
Johnson, J. W., Steel, P., Scherbaum, C. A., Hoffman, C. C., Richard Jeanneret, P., & Foster, J. (2010). Validation is like motor oil: Synthetic is better. Industrial and Organizational Psychology: Perspectives on Science and Practice, 3, 305–328.Google Scholar
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo in a multidimensional forced choice performance evaluation scale. Journal of Applied Psychology, 65, 507–516.Google Scholar
Morris, D. C., Hoffman, C. C., & Shultz, K. S. (2003, April). A comparison of job components validity estimates to meta-analytic validity estimates. Poster presented at the 18th Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.Google Scholar
Mount, M. K., Judge, T. A., Scullen, S. E., Sytsma, M. R., & Hezlett, S. A. (1998). Trait, rater, and level effects in 360-degree performance ratings. Personnel Psychology, 51, 557–576.Google Scholar
Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980). Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. Journal of Applied Psychology, 65, 373–406.Google Scholar
Peterson, N. G., Wise, L. L., Arabian, J., & Hoffman, R. G. (2001). Synthetic validation and validity generalization: When empirical validation is not possible. In Campbell, J. P. & Knapp, D. J. (Eds.), Exploring the limits of personnel selection and classification (pp. 411–451). Mahwah, NJ: Erlbaum.Google Scholar
Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & De Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, 573–605.Google Scholar
Schmidt, F. L., & Hunter, J. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199–223.Google Scholar
Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1981). Task difference and validity of aptitude tests in selection: A red herring. Journal of Applied Psychology, 66, 166–185.Google Scholar
Schmidt, F. L., Law, K., Hunter, J. E., Rothstein, H. R., Pearlman, K., & McDaniel, M. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology, 78, 3–13.Google Scholar
Schmidt, F. L., Shaffer, J. A., & Oh, I.-S. (2008). Increased accuracy of range restriction corrections: Implications for the role of personality and general mental ability in job and training performance. Personnel Psychology, 61, 827–868.Google Scholar
Steel, P., & Kammeyer-Mueller, J. (2009). Using a meta-analytic perspective to enhance job component validation. Personnel Psychology, 62, 533–552.Google Scholar
Verive, J. M., & McDaniel, M. A. (1996). Short-term memory tests in personnel selection: Low adverse impact and high validity. Intelligence, 23, 15–32.Google Scholar
Viswesvaran, C., Schmidt, F. L., & Ones, D. S. (2005). Is there a general factor in ratings of job performance? A meta-analytic framework for disentangling substantive and error influences. Journal of Applied Psychology, 90, 108–131.Google Scholar