Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-05T02:56:44.133Z Has data issue: false hasContentIssue false
  • English
  • Français

Putting replication in its place

Published online by Cambridge University Press:  27 July 2018

Evan Heit  and
Caren M. Rotello
Evan Heit
Affiliation:
E.H. Division of Research on Learning, Education and Human Resources Directorate, National Science Foundation, Alexandria, VA 22314. [email protected]
Caren M. Rotello
Affiliation:
CMR Department of Psychological and Brain Sciences, University of Massachusetts, Amherst, MA 01003. [email protected]://www.umass.edu/pbs/people/caren-rotello
Get access Rights & Permissions [Opens in a new window]

Abstract

Direct replication is valuable but should not be elevated over other worthwhile research practices, including conceptual replication and checking of statistical assumptions. As noted by Rotello et al. (2015), replicating studies without checking the statistical assumptions can lead to increased confidence in incorrect conclusions. Finally, successful replications should not be elevated over failed replications, given that both are informative.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 41 , 2018 , e133
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

1.

Parts of this commentary are a work of the U.S. Government and are not subject to copyright protection in the United States.

2.

This material includes work performed by Evan Heit while serving at the National Science Foundation. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

References

Dubé, C., Rotello, C. M. & Heit, E. (2010) Assessing the belief bias effect with ROCs: It's a response bias effect. Psychological Review 117:831–63.Google Scholar
Evans, J. St. B. T., Barston, J. L. & Pollard, P. (1983) On the conflict between logic and belief in syllogistic reasoning. Memory and Cognition 11:295–306.Google Scholar
Heit, E., Hahn, U. & Feeney, A. (2005) Defending diversity. In: Categorization inside and outside the laboratory: Essays in honor of Douglas L. Medin, ed. Ahn, W.-K., Goldstone, R. L., Love, B. C., Markman, A. B. & Wolff, P., pp. 87–99. American Psychological Association.Google Scholar
Heit, E. & Rotello, C. M. (2014) Traditional difference-score analyses of reasoning are flawed. Cognition 131:75–91.Google Scholar
Nuijten, M. B., Hartgerink, C. H. J., Van Assen, M. A. L. M., Epskamp, S. & Wicherts, J. M. (2016) The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods 48(4):1205–26. Available at: http://doi.org/10.3758/s13428-015-0664-2.Google Scholar
Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349(6251):aac4716. Available at: http://doi.org/10.1126/science.aac4716.Google Scholar
Rotello, C. M., Heit, E. & Dubé, C. (2015) When more data steer us wrong: Replications with the wrong dependent measure perpetuate erroneous conclusions. Psychonomic Bulletin and Review 22:944–54.Google Scholar
Rotello, C. M., Masson, M. E. J. & Verde, M. F. (2008) Type I error rates and power analyses for single-point sensitivity measures. Perception and Psychophysics 70:389–401.Google Scholar
Salmon, W. C. (1984) Scientific explanation and the causal structure of the world. Princeton University Press.Google Scholar