Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-22T19:32:35.618Z Has data issue: false hasContentIssue false

3 - Replication in Social and Personality Psychology

from Part I - Before You Dive In

Published online by Cambridge University Press:  12 December 2024

By
Klaus Fiedler  and
Florian Ermark
Edited by
Harry T. Reis ,
Tessa West  and
Charles M. Judd
Harry T. Reis
Affiliation:
University of Rochester, New York
Tessa West
Affiliation:
New York University
Charles M. Judd
Affiliation:
University of Colorado Boulder
Get access

Summary

Without doubt the validity of scientific theories and their usability for solving societal, economic, ecological, and health-related problems are contingent on the existence of robust and replicable empirical findings. However, a review of the recently published replication literature portrays a rather pessimistic picture of the replicability of even very prominent empirical results, as is evident both from large-scale meta-analytic replication projects and from distinct attempts to replicate selected examples of well-established key findings of personality and social psychology. The present chapter offers a twofold explanation for this undesirable state of affairs. On one hand, the widespread evidence on replication failures reflects to a considerable extent the neglect of logical and methodological standards in replication studies, which sorely ignore such essential issues as manipulation checks, reliability control, regressive shrinkage, and intricacies of multi-causality. On the other hand, however, the community of behavioral scientists must blame themselves for an intrinsic weakness of their corporate identity and their incentive and publication system, namely the tendency to mistake sexy and unexpected findings for original insights and neglect of the assets of cumulative science and theoretical constraints.

Keywords

theoretical constraintsregression to the meanreliabilitymanipulation checksreplicandumfacial feedbackego depletionconstrual-level theory
Type
Chapter
Information
Handbook of Research Methods in Social and Personality Psychology , pp. 45 - 67
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Baumeister, R. F. (2002). Ego depletion and self-control failure: An energy model of the self’s executive function. Self and Identity, 1(2), 129–136.CrossRefGoogle Scholar
Baumeister, R. F., Bratslavsky, E., Muraven, M., and Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology, 74(5), 1252–1265.CrossRefGoogle Scholar
Baumeister, R. F., Vohs, K. D., and Tice, D. M. (2007). The strength model of self-control. Current Directions in Psychological Science, 16(6), 351–355.CrossRefGoogle Scholar
Bhatia, S., and Walasek, L. (2016). Event construal and temporal distance in natural language. Cognition, 152, 1–8.CrossRefGoogle ScholarPubMed
Becker, D. V., and Srinivasan, N. (2014). The vividness of the happy face. Current Directions in Psychological Science, 23(3), 189–194.CrossRefGoogle Scholar
Bertsch, S., Pesta, B. J., Wiscott, R., and McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & Cognition, 35(2), 201–210.CrossRefGoogle ScholarPubMed
Blázquez, D., Botella, J., and Suero, M. (2017). The debate on the ego-depletion effect: Evidence from meta-analysis with the p-uniform method. Frontiers in Psychology, 8, 197, https://doi.org/10.3389/fpsyg.2017.00197.CrossRefGoogle ScholarPubMed
Bullard, S. E., Fein, D., Gleeson, M. K., Tischer, N., Mapou, R. L., and Kaplan, E. (2004). The Biber cognitive estimation test. Archives of Clinical Neuropsychology, 19(6), 835–846.CrossRefGoogle ScholarPubMed
Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., Kirchler, M., Nave, G., Nosek, B. A., Pfeiffer, T., Altmejd, A., Buttrick, N., Chan, T., Chen, Y., Forsell, E., Gampa, A., Heikensten, E., Hummer, L., Imai, T., … Wu, H. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour, 2(9), 637–644, with supplementary material at https://doi.org/10.1038/s41562-018-0399-z.CrossRefGoogle Scholar
Campbell, D. T., and Kenny, D. A. (1999). A Primer on Regression Artifacts. Guilford Press.Google Scholar
Cialdini, R. B., Vincent, J. E., Lewis, S. K., Catalan, J., Wheeler, D., and Darby, B. L. (1975). Reciprocal concessions procedure for inducing compliance: The door-in-the-face technique. Journal of Personality and Social Psychology, 31(2), 206–215.CrossRefGoogle Scholar
Coles, N. A., Larsen, J. T., and Lench, H. C. (2019). A meta-analysis of the facial feedback literature: Effects of facial feedback on emotional experience are small and variable. Psychological Bulletin, 145(6), 610–651.CrossRefGoogle ScholarPubMed
Coles, N. A., March, D. S., Marmolejo-Ramos, F., Larsen, J., Arinze, N. C., Ndukaihe, I., Willis, M., Francesco, F., Reggev, N., Mokady, A., Forscher, P. S., Hunter, J., Gwenaël, K., Yuvruk, E., Kapucu, A., Nagy, T., Hajdu, N., Tejada, J., Freitag, R., … Marozzi, M. (2022). A multi-lab test of the facial feedback hypothesis by the Many Smiles Collaboration. PsyArXiv, https://doi.org/10.31234/osf.io/cvpuw.CrossRefGoogle Scholar
Combs, B., and Slovic, P. (1979). Newspaper coverage of causes of death. Journalism Quarterly, 56(4), 837–849.CrossRefGoogle Scholar
Costello, F., and Watts, P. (2019). The rationality of illusory correlation. Psychological Review, 126(3), 437–450.CrossRefGoogle ScholarPubMed
Crawford, H. J., Brown, A. M., and Moon, C. E. (1993). Sustained attentional and disattentional abilities: Differences between low and highly hypnotizable persons. Journal of Abnormal Psychology, 102(4), 534–543.CrossRefGoogle ScholarPubMed
Dang, J. (2018). An updated meta-analysis of the ego depletion effect. Psychological Research, 82(4), 645–651.CrossRefGoogle ScholarPubMed
Dang, J., Barker, P., Baumert, A., Bentvelzen, M., Berkman, E., Buchholz, N., Buczny, J., Chen, Z., De Cristofaro, V., de Vries, L., Dewitte, S., Giacomantonio, M., Gong, R., Homan, M., Imhoff, R., Ismail, I., Jia, L., Kubiak, T., Lange, F., … Zinkernagel, A. (2021). A multilab replication of the ego depletion effect. Social Psychological and Personality Science, 12(1), 14–24.CrossRefGoogle ScholarPubMed
Duhem, P. (1954). The Aim and Structure of Physical Theory (trans. P. Wiener). Atheneum (original work published 1914).CrossRefGoogle Scholar
Earp, B. D., and Trafimow, D. (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology, 6, 621, https://doi.org/10.3389/fpsyg.2015.00621.CrossRefGoogle ScholarPubMed
Feld, G. B., and Born, J. (2015). Exploiting sleep to modify bad attitudes. Science, 348(6238), 971–972.CrossRefGoogle ScholarPubMed
Festinger, L. (1957). A theory of cognitive dissonance, vol. 2. Stanford University Press.CrossRefGoogle Scholar
Fiedler, K. (1991). The tricky nature of skewed frequency tables: An information loss account of distinctiveness-based illusory correlations. Journal of Personality and Social Psychology, 60(1), 24–36.CrossRefGoogle Scholar
Fiedler, K., Jung, J., Wänke, M., Alexopoulos, T., and de Molière, L. (2015). Toward a deeper understanding of the ecological origins of distance construal. Journal of Experimental Social Psychology, 57, 78–86.CrossRefGoogle Scholar
Fiedler, K., and Krueger, J. I. (2012). More than an artifact: Regression as a theoretical construct. In Krueger, J. I. (ed.) Social Judgment and Decision Making. Psychology Press.Google Scholar
Fiedler, K., Kutzner, F., and Krueger, J. I. (2012). The long way from α-error control to validity proper: Problems with a short-sighted false-positive debate. Perspectives on Psychological Science, 7(6), 661–669.CrossRefGoogle Scholar
Fiedler, K., McCaughey, L., and Prager, J. (2021). Quo vadis, methodology? The key role of manipulation checks for validity control and quality of science. Perspectives on Psychological Science, 16(4), 816–826.CrossRefGoogle Scholar
Fiedler, K., and Prager, J. (2018). The regression trap and other pitfalls of replication science: Illustrated by the report of the Open Science Collaboration. Basic and Applied Social Psychology, 40(3), 115–124.CrossRefGoogle Scholar
Fiedler, K., Unkelbach, C., and Freytag, P. (2009). On splitting and merging categories: A regression account of subadditivity. Memory & Cognition, 37(4), 383–393.CrossRefGoogle ScholarPubMed
Fiedler, K., and Walther, E. (2004). Stereotyping as Inductive Hypothesis Testing. Psychology Press.CrossRefGoogle Scholar
Friese, M., Loschelder, D. D., Gieseler, K., Frankenbach, J., and Inzlicht, M. (2019). Is ego depletion real? An analysis of arguments. Personality and Social Psychology Review, 23(2), 107–131.CrossRefGoogle Scholar
Genschow, O., Westfal, M., Crusius, J., Bartosch, L., Feikes, K. I., Pallasch, N., and Wozniak, M. (2021). Does social psychology persist over half a century? A direct replication of Cialdini et al.’s (1975) classic door-in-the-face technique. Journal of Personality and Social Psychology, 120(2), e1–e7.CrossRefGoogle ScholarPubMed
Hagger, M. S., Chatzisarantis, N. L., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A multilab preregistered replication of the ego-depletion effect. Perspectives on Psychological Science, 11(4), 546–573.CrossRefGoogle ScholarPubMed
Hauser, D. J., Ellsworth, P. C., and Gonzalez, R. (2018). Are manipulation checks necessary? Frontiers in Psychology, 9, 998, https://doi.org/10.3389/fpsyg.2018.00998.CrossRefGoogle ScholarPubMed
Hilton, D. J., and Slugoski, B. R. (1986). Knowledge-based causal attribution: The abnormal conditions focus model. Psychological Review, 93, 75–88.CrossRefGoogle Scholar
Hofmann, W., De Houwer, J., Perugini, M., Baeyens, F., and Crombez, G. (2010). Evaluative conditioning in humans: A meta-analysis. Psychological Bulletin, 136(3), 390–421.CrossRefGoogle ScholarPubMed
Kassin, S. M. (2008). False confessions: Causes, consequences, and implications for reform. Current Directions in Psychological Science, 17(4), 249–253.CrossRefGoogle Scholar
Krüger, T., Fiedler, K., Koch, A. S., and Alves, H. (2014). Response category width as a psychophysical manifestation of construal level and distance. Personality and Social Psychology Bulletin, 40(4), 501–512.CrossRefGoogle ScholarPubMed
Kruger, J., and Evans, M. (2004). If you don’t want to be late, enumerate: Unpacking reduces the planning fallacy. Journal of Experimental Social Psychology, 40, 586–598.CrossRefGoogle Scholar
Kuhl, J., and Kazén, M. (1999). Volitional facilitation of difficult intentions: Joint activation of intention memory and positive affect removes Stroop interference. Journal of Experimental Psychology: General, 128(3), 382–399.CrossRefGoogle Scholar
Kvarven, A., Strømland, E., and Johannesson, M. (2020). Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature Human Behaviour, 4(4), 423–434.CrossRefGoogle ScholarPubMed
Lawrence, D. H., and Festinger, L. (1962). Deterrents and Reinforcement: The Psychology of Insufficient Reward. Stanford University Press.Google Scholar
Liberman, N., and Trope, Y. (1998). The role of feasibility and desirability considerations in near and distant future decisions: A test of temporal construal theory. Journal of Personality and Social Psychology, 75(1), 5–18.CrossRefGoogle Scholar
Loftus, E. F. (1993). The reality of repressed memories. American Psychologist, 48(5), 518–537.CrossRefGoogle ScholarPubMed
McShane, B. B., Tackett, J. L., Böckenholt, U., and Gelman, A. (2019). Large-scale replication projects in contemporary psychological research. American Statistician, 73(sup. 1), 99–105.CrossRefGoogle Scholar
Maglio, S. J., and Trope, Y. (2011). Scale and construal: How larger measurement units shrink length estimates and expand mental horizons. Psychonomic Bulletin & Review, 18(1), 165–170.CrossRefGoogle ScholarPubMed
Mongeau, P. A. (1993). The brainstorming myth. Paper presented to the Western States Communication Association, Albuquerque, NM.Google Scholar
Moore, D. A., and Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502–517.CrossRefGoogle ScholarPubMed
Morewedge, C. K., Gilbert, D. T., Keysar, B., Berkovits, M. J., and Wilson, T. D. (2007). Mispredicting the hedonic benefits of segregated gains. Journal of Experimental Psychology: General, 136, 700–709.CrossRefGoogle ScholarPubMed
Muraven, M., Tice, D. M., and Baumeister, R. F. (1998). Self-control as a limited resource: Regulatory depletion patterns. Journal of Personality and Social Psychology, 74(3), 774–789.CrossRefGoogle ScholarPubMed
Oberauer, K., and Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618.CrossRefGoogle ScholarPubMed
Collaboration, Open Science (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.CrossRefGoogle Scholar
Pfungst, O. (1911). Clever Hans (The Horse of Mr. Von Osten): A Contribution to Experimental Animal and Human Psychology. Holt, Rinehart andWinston.CrossRefGoogle Scholar
Polivy, J., and Doyle, C. (1980). Laboratory induction of mood states through the reading of self-referent mood statements: Affective changes or demand characteristics? Journal of Abnormal Psychology, 89(2), 286–290.CrossRefGoogle ScholarPubMed
Proctor, R. W., and Cho, Y. S. (2006). Polarity correspondence: A general principle for performance of speeded binary classification tasks. Psychological Bulletin, 132(3), 416–442.CrossRefGoogle ScholarPubMed
Quine, W. V. O. (1952). The Dogmas of Empiricism (reprinted from “A logical point of view”). Harvard University Press.Google Scholar
Rottenstreich, Y., and Tversky, A. (1997). Unpacking, repacking, and anchoring: Advances in support theory. Psychological Review, 104(2), 406–415.CrossRefGoogle ScholarPubMed
Sagristano, M. D., Trope, Y., and Liberman, N. (2002). Time-dependent gambling: Odds now, money later. Journal of Experimental Psychology: General, 131(3), 364–376.CrossRefGoogle ScholarPubMed
Sánchez, A. M., Coleman, C. W., and Ledgerwood, A. (2021). Does temporal distance influence abstraction? A large pre-registered experiment. Social Cognition, 39(3), 352–365.CrossRefGoogle Scholar
Savage, L. J. (1954). The Foundations of Statistics. John Wiley & Sons.Google Scholar
Schaller, M. (2014). When and how disgust is and is not implicated in the behavioral immune system. Evolutionary Behavioral Sciences, 8(4), 251–256.CrossRefGoogle Scholar
Simmons, J. P., Nelson, L. D., and Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366.CrossRefGoogle ScholarPubMed
Slamecka, N. J., and Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4(6), 592–604.Google Scholar
Soderberg, C. K., Callahan, S. P., Kochersberger, A. O., Amit, E., and Ledgerwood, A. (2015). The effects of psychological distance on abstraction: Two meta-analyses. Psychological Bulletin, 141(3), 525–548.CrossRefGoogle ScholarPubMed
Soto, C. J. (2019). How replicable are links between personality traits and consequential life outcomes? The life outcomes of personality replication project. Psychological Science, 30(5), 711–727.CrossRefGoogle ScholarPubMed
Stanley, D. J., and Spence, J. R. (2014). Expectations for replications: Are yours realistic? Perspectives on Psychological Science, 9, 305–318.CrossRefGoogle ScholarPubMed
Storbeck, J., and Clore, G. L. (2008). The affective regulation of cognitive priming. Emotion, 8(2), 208–215.CrossRefGoogle ScholarPubMed
Strack, F., Martin, L. L., and Stepper, S. (1988). Inhibiting and facilitating conditions of the human smile: A nonobtrusive test of the facial feedback hypothesis. Journal of Personality and Social Psychology, 54(5), 768–777.CrossRefGoogle ScholarPubMed
Surowiecki, J. (2004). The Wisdom of Crowds. New York: Random House.Google Scholar
Thorne, K. S. (2019). LIGO and gravitational waves, III: Nobel lecture, December 8, 2017. Annalen der Physik, 531(1), DOI:10.1002/andp.201800350.CrossRefGoogle Scholar
Trafimow, D. (2022). Generalizing across auxiliary, statistical, and inferential assumptions. Journal for the Theory of Social Behaviour, 52(1), 37–48.CrossRefGoogle Scholar
Trope, Y., and Liberman, N. (2010). Construal-level theory of psychological distance. Psychological Review, 117(2), 440.CrossRefGoogle ScholarPubMed
Tversky, A., and Koehler, D. J. (1994). Support theory: A nonextensional representation of subjective probability. Psychological Review, 101, 547–567.CrossRefGoogle Scholar
Unkelbach, C. (2006). The learned interpretation of cognitive fluency. Psychological Science, 17(4), 339–345.CrossRefGoogle ScholarPubMed
Vallacher, R. R., and Wegner, D. M. (1989). Levels of personal agency: Individual variation in action identification. Journal of Personality and Social Psychology, 57(4), 660–671.CrossRefGoogle Scholar
Vohs, K. D., Schmeichel, B. J., Lohmann, S., Gronau, Q. F., Finley, A. J., Ainsworth, S. E., Alquist, J. L., Baker, M. D., Brizi, A., Bunyi, A., Butschek, G. J., Campbell, C., Capaldi, J., Cau, C., Chambers, H., Chatzisarantis, N. L. D., Christensen, W. J., Clay, S. L., Curtis, J., … Albarracín, D. (2021). A multisite preregistered paradigmatic test of the ego-depletion effect. Psychological Science, 32(10), 1566–1581.CrossRefGoogle ScholarPubMed
Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R. B., Albohn, D. N., Allard, E. S., Benning, S. D., Blouin-Hudon, E.-M., Bulnes, L. C., Caldwell, T. L., Calin-Jageman, R. J., Capaldi, C. A., Carfagno, N. S., Chasten, K. T., Cleeremans, A., Connell, L., DeCicco, J. M., … Zwaan, R. A. (2016). Registered replication report: Strack, Martin, & Stepper (1988). Perspectives on Psychological Science, 11(6), 917–928.CrossRefGoogle Scholar
Williamson, D. A., and Blanchard, E. B. (1979). Heart rate and blood pressure biofeedback. Biofeedback and Self-Regulation, 4(1), 1–34.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×