Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-22T20:51:10.289Z Has data issue: false hasContentIssue false

10 - Measurement Validity in the Social and Behavioral Sciences: Some “Whys” and “Hows”

from Part II - Important Methodological Considerations

Published online by Cambridge University Press:  12 December 2024

By
John J. Skowronski
Edited by
John E. Edlund  and
Austin Lee Nichols
John E. Edlund
Affiliation:
Rochester Institute of Technology, New York
Austin Lee Nichols
Affiliation:
Central European University, Vienna
Get access

Summary

This chapter argues that research-focused social and behavioral scientists also need to be good research technicians. This statement reflects the belief that this technical skill is needed because an accurate understanding of the social and behavioral sciences depends crucially on the use of valid measures of variables that are of interest and importance. The chapter also argues that the establishment of measurement validity is not an easy task, requiring researchers to gather evidence for measurement validity diligently, persistently, and constantly. Described in the chapter are some ways in which such evidence can be obtained and some of the pitfalls that confront researchers when they evaluate their evidence.

Type
Chapter
Information
The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
Volume 2: Performing Research
 , pp. 211 - 231
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ajzen, I. (2005). The influence of attitudes on behavior. In Albarracín, D., Johnson, B. T., & Zanna, M. P. (eds.), The Handbook of Attitudes (pp. 173–221). Lawrence Erlbaum Associates.Google Scholar
American Psychological Association (2019, August). Depression Assessment Instruments. www.apa.org/depression-guideline/assessmentGoogle Scholar
Bode, B., King, A., Russell-Jones, D., & Billings, L. K. (2021). Leveraging advances in diabetes technologies in primary care: A narrative review. Annals of Medicine, 53(1), 805–816. https://doi.org/10.1080/07853890.2021.1931427CrossRefGoogle ScholarPubMed
Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061–1071. https://doi.org/10.1037/0033-295X.111.4.1061CrossRefGoogle ScholarPubMed
Buntins, M., Buntins, K., & Eggert, F. (2017). Clarifying the concept of validity: From measurement to everyday language. Theory & Psychology, 27(5), 703–710. https://doi.org/10.1177/0959354317702256CrossRefGoogle Scholar
Camargo, S. L., Herrera, A. N., & Traynor, A. (2018). Looking for a consensus in the discussion about the concept of validity: A Delphi study. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 14(4), 146–155. https://doi.org/10.1027/1614-2241/a000157CrossRefGoogle Scholar
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the Multitrait-Multimethod Matrix. Psychological Bulletin, 56(2), 81–105. http://dx.doi.org/10.1037/h0046016CrossRefGoogle ScholarPubMed
Chartrand, T. L., & Lakin, J. L. (2013). The antecedents and consequences of human behavioral mimicry. Annual Review of Psychology, 64, 285–308. https://doi.org/10.1037/h0046016CrossRefGoogle ScholarPubMed
Combe, G. (1835). The Constitution of Man, Considered in Relation to External Objects. John Anderson Jr.Google Scholar
Combe, G. (1851). A System of Phrenology. Benjamin B. Mussey.Google Scholar
Conan Doyle, A. (1981). The Original Illustrated Sherlock Holmes. Castle Books.Google Scholar
Costa-Gomes, M.A., Ju, Y., & Li, J. (2019). Role reversal consistency: An experimental study of the golden rule. Economic Inquiry, 57(1), 685–704. https://doi.org/10.1111/ecin.12708CrossRefGoogle Scholar
Duckworth, A. L., Peterson, C., Matthews, M. D., & Kelly, D. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 92(6), 1087–1101. https://doi.org/10.1037/0022-3514.92.6.1087CrossRefGoogle ScholarPubMed
Eling, P., & Finger, S. (eds.) (2021). Gall, Spurzheim, and the Phrenological Movement: Insights and Perspectives. Routledge.CrossRefGoogle Scholar
Falk, C. F., Heine, S. J., Takemura, K., Zhang, C. X., & Hsu, C.-W. (2015). Are implicit self-esteem measures valid for assessing individual and cultural differences? Journal of Personality, 83, 56–68. https://doi.org/10.1111/jopy.12082CrossRefGoogle ScholarPubMed
Foster, S. L., & Cone, J. D. (1995). Validity issues in clinical assessment. Psychological Assessment, 7(3), 248–260. https://doi.org/10.1037/1040-3590.7.3.248CrossRefGoogle Scholar
Fuentes, A., Ackermann, R. R., Athreya, S., Bolnik, D., Lasisi, T., Lee, S., et al. (2019). AAPA statement on race and racism. American Journal of Physical Anthropology, 169 (3), 400–402. https://doi.org/10.1002/ajpa.23882CrossRefGoogle ScholarPubMed
Gardner, M. (1957). From bumps to handwriting. In Gardner, Fads and Fallacies in the Name of Science. Dover.Google Scholar
Ghandnoosh, N. (2014, September 3). Race and punishment: Racial perceptions of crime and support for punitive policies. The Sentencing Project. www.sentencingproject.org/publications/race-and-punishment-racial-perceptions-of-crime-and-support-for-punitive-policiesGoogle Scholar
Goodman, A. H., Moses, Y. T., & Jones, J. L. (2020). Race: Are We So Different? 2nd ed. John Wiley & Sons / American Anthropological Association.Google Scholar
Greenblatt, S. H. (1995). Phrenology in the science and culture of the 19th century. Neurosurgery, 37(4), 790–805. https://doi.org/10.1227/00006123-199510000-00025CrossRefGoogle ScholarPubMed
Herrnstein, R. J., & Murray, C. (1994). The Bell Curve: Intelligence and class Structure in American Life. Free Press.Google Scholar
Hollander, B. (1902). Scientific Phrenology: Being a Practical Mental Science and Guide to Human Character. Grant Richards.Google Scholar
Inzlicht, M., McKay, L., & Aronson, J. (2006). Stigma as ego depletion: How being the target of prejudice affects self-control. Psychological Science, 17(3), 262–269. https://doi.org/10.1111/j.1467-9280.2006.01695.xCrossRefGoogle ScholarPubMed
Jones, N. (2020). Carbon dating, the archaeological workhorse, is getting a major reboot. Nature News, May 19. https://doi.org/10.1038/d41586-020-01499-yCrossRefGoogle Scholar
Lilienfeld, S. O. (2017). Microaggressions: Strong claims, inadequate evidence. Perspectives on Psychological Science, 12(1), 138–169. https://doi.org/10.1177/1745691616659391CrossRefGoogle Scholar
Lissitz, Robert W. (ed.). (2009). The Concept of Validity: Revisions, New Directions, and Applications. IAP Information Age.Google Scholar
Milner, J. S. (1994). Assessing physical child abuse risk: The Child Abuse Potential Inventory. Clinical Psychology Review, 14(6), 547–583. https://doi.org/10.1016/0272-7358(94)90017-5CrossRefGoogle Scholar
Milner, J. S. (2022a). The Child Abuse Potential Inventory: Manual. Psytec.Google Scholar
Milner, J. S. (2022b). An Interpretive Manual for the Child Abuse Potential Inventory, rev. ed. Psytec.Google Scholar
Milner, J. S., & Crouch, J. L. (2012). Psychometric characteristics of translated versions of the Child Abuse Potential Inventory. Psychology of Violence, 2(3), 239–259. https://doi.org/10.1037/a0026957CrossRefGoogle Scholar
Milner, J. S., & Crouch, J. L. (2017). Child physical abuse risk assessment: Parent and family evaluations. In Campbell, J. C. & Messing, J. T. (eds.), Assessing Dangerousness: Domestic Violence Offenders and Child Abusers, 3rd ed. (pp. 55–88). Springer.Google Scholar
Parker Jones, O., Alfaro-Almagro, F., & Jbabdi, S. (2018). An empirical, 21st century evaluation of phrenology. Cortex, 106, 28–35. https://doi.org/10.1016/j.cortex.2018.04.011CrossRefGoogle ScholarPubMed
Ponnock, A., Muenks, K., Morell, M., Seung Yang, J., Gladstone, J. R., & Wigfield, A. (2020). Grit and conscientiousness: Another jangle fallacy. Journal of Research in Personality, 89, 1–5. https://doi.org/10.1016/j.jrp.2020.104021CrossRefGoogle Scholar
Rudman, L. A., Greenwald, A. G., Mellott, D. S., & Schwartz, J. L. K. (1999). Measuring the automatic components of prejudice: Flexibility and generality of the Implicit Association Test. Social Cognition, 17(4), 437–465. https://doi.org/10.1521/soco.1999.17.4.437CrossRefGoogle Scholar
Sackett, P. R, Putka, D. J, & McCloy, R. A. (2012). The concept of validity and the process of validation. In Schmitt, N. (ed.), The Oxford Handbook of Personnel Assessment and Selection (pp. 91–118). Oxford University Press.CrossRefGoogle Scholar
Schimmack, U. (2021). The Implicit Association Test: A method in search of a construct. Perspectives on Psychological Science, 16(2), 396–414. https://doi.org/10.1177/1745691619863798CrossRefGoogle Scholar
Sue, D. W., Capodilupo, C. M., Torino, G. C., Bucceri, J. M., Holder, A. M. B., Nadal, K. L., & Esquilin, M. (2007). Racial microaggressions in everyday life: Implications for clinical practice. American Psychologist, 62(4), 271–286. https://doi.org/10.1037/0003-066X.62.4.271CrossRefGoogle ScholarPubMed
Thomas, C., & Skowronski, J. J. (2020). The study of microaggressive behavior: Reflections on the construct, construct-relevant research, and possible future research. In Nadler, J. T. & Voyles, E. C. (eds.), Stereotypes: The Incidence and Impacts of Bias (pp. 45–69). Praeger.CrossRefGoogle Scholar
Thompson, C. E. (2021). An Organ of Murder: Crime, Violence, and Phrenology in Nineteenth-Century America. Rutgers University Press.CrossRefGoogle Scholar
Tynan, M. C. (2021). Deconstructing Grit’s validity: The case for revising Grit measures and theory. In van Zyl, L. E., Olckers, C., & van der Vaart, E. (eds.), Multidisciplinary perspectives on Grit: Contemporary Theories, Assessments, Applications and Critiques (pp. 137–155). Springer.CrossRefGoogle Scholar
Vazire, S., Schiavone, S. R., & Bottesini, J. G. (2022). Credibility beyond replicability: Improving the four validities in psychological science. Current Directions in Psychological Science, 31(2), 162–168. https://doi.org/10.1177/09637214211067779CrossRefGoogle Scholar
Vintage News Daily. (2017, September 20). Bad invention: Psychograph, a phrenology machine to measure the shape of your head from the early 20th century. https://vintagenewsdaily.com/bad-invention-psychograph-a-phrenology-machine-to-measure-the-shape-of-your-head-from-the-early-20th-centuryGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×