Hostname: page-component-586b7cd67f-t7fkt Total loading time: 0 Render date: 2024-11-26T11:45:44.620Z Has data issue: false hasContentIssue false
  • English
  • Français

Generalizability Versus Situational Specificity in Adverse Impact Analysis: Issues in Data Aggregation

Published online by Cambridge University Press:  30 August 2017

Elizabeth Howard ,
Scott B. Morris  and
Eric Dunleavy
Elizabeth Howard*
Affiliation:
Illinois Institute of Technology
Scott B. Morris
Affiliation:
Illinois Institute of Technology
Eric Dunleavy
Affiliation:
DCI Consulting Group
*
Correspondence concerning this article should be addressed to Elizabeth Howard, Illinois Institute of Technology, 3105 S. Dearborn, Chicago, IL 60616. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Extract

Tett, Hundley, and Christiansen (2017) argue that the concept of validity generalization in meta-analysis is a myth, as the variability of the effect size appears to decrease with increasing moderator specificity such that the level of precision needed to deem an estimate “generalizable” is actually reached at levels of situational specificity that are so high as to (paradoxically) refute an inference of generalizability. This notion highlights the need to move away from claiming that effects are either “generalizable” or “situationally specific” and instead look more critically and less dichotomously at degrees of generalizability, or effect size variability.

Type
Commentaries
Information
Industrial and Organizational Psychology , Volume 10 , Issue 3 , September 2017 , pp. 495 - 501
Copyright
Copyright © Society for Industrial and Organizational Psychology 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Arthur, W., Doverspike, D., Barrett, G. V., & Miguel, R. (2013). Chasing the Title VII holy grail: The pitfalls of guaranteeing adverse impact elimination. Journal of Business and Psychology, 28 (4), 473–485.CrossRefGoogle Scholar
Biddle, D. A. (2011). Adverse impact and test validation: A practitioner's handbook (3rd ed.). Scottsdale, AZ: Infinity.Google Scholar
Bielby, W. T., & Coukos, P. (2007). “Statistical dueling” with unconventional weapons: What courts should know about experts in employment discrimination class actions. Emory Law Journal, 56, 1563–1612.Google Scholar
Brannick, M. T. (2001). Implications of empirical Bayes meta-analysis for test validation. Journal of Applied Psychology, 86, 468–480.CrossRefGoogle ScholarPubMed
Breslow, N. E., & Day, N. E. (1980). Statistical methods in cancer research, Volume 1: The analysis of case-control studies (Vol. 32). Lyon, France: IARC Scientific.Google Scholar
Cohen, D. B., Aamodt, M. G., & Dunleavy, E. M. (2010). Technical advisory committee report on best practices in adverse impact analyses. Washington, DC: Center for Corporate Equality.Google Scholar
Dunleavy, E., Morris, S., & Howard, E. (2015). Measuring adverse impact in employee selection decisions. In Hanvey, C. & Sady, K. (Eds.), Practitioner's guide to legal issues in organizations (pp. 1–26). Cham, Switzerland: Springer International Publishing.Google Scholar
Fleiss, J. L., & Berlin, J. A. (2009). Effect sizes for dichotomous data. In Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 237–254). New York: Russell Sage Foundation.Google Scholar
Gutman, A., Koppes, L. L., & Vodanovich, S. J. (2010). EEO law and personnel practices. Abingdon, UK: Psychology Press.CrossRefGoogle Scholar
Huang, J., & Morris, S. B. (2013, April). HGLM and Mantel–Haenszel tests for adverse impact. Poster presented at the 28th Annual Conference of the Society for Industrial and Organizational Psychology, Houston, TX.Google Scholar
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719–748.Google ScholarPubMed
Mehri, C., & Lieder, M. (2017). Addressing the ever increasing standards for statistical evidence: A plaintiff attorney's perspective. In Morris, S. B. and Dunleavy, E. M. (Eds.), Adverse impact analysis (pp. 298–329). New York: Routledge.Google Scholar
Morris, S. B. (2001). Sample size required for adverse impact analysis. Applied H.R.M. Research, 6, 13–32.Google Scholar
Morris, S. B., Dunleavy, E. M., & Lee, M. (2017). Many 2x2 tables: Understanding multiple events in adverse impact analyses. In Morris, S. B. and Dunleavy, E. M. (Eds.), Adverse impact analysis (pp. 147–166). New York, NY: Routledge.Google Scholar
Ross, D. B., & Merrill, G. (2017). Disparate impact, trial by statistics: Thoughts from a defense attorney's perspective. In Morris, S. B. and Dunleavy, E. M. (Eds.), Adverse impact analysis (pp. 330–348). New York Routledge.Google Scholar
Tett, R. P., Hundley, N. A., & Christiansen, N. D. (2017). Meta-analysis and the myth of generalizability. Industrial and Organizational Psychology: Perspectives on Science and Practice, 10 (3), 421–456.CrossRefGoogle Scholar
Wal-Mart Stores, Inc. v. Dukes, 131 S. Ct. 2541 (2011).Google Scholar