Hostname: page-component-cd9895bd7-fscjk Total loading time: 0 Render date: 2024-12-31T15:32:48.830Z Has data issue: false hasContentIssue false
  • English
  • Français

Science Deserves Better: The Imperative to Share Complete Replication Files

Published online by Cambridge University Press:  29 December 2013

Allan Dafoe
Allan Dafoe*
Affiliation:
Yale University
Get access Rights & Permissions [Opens in a new window]

Extract

In April 2013, a controversy arose when a working paper (Herndon, Ash, and Pollin 2013) claimed to show serious errors in a highly cited and influential economics paper by Carmen Reinhart and Kenneth Rogoff (2010). The Reinhart and Rogoff paper had come to serve as authoritative evidence in elite conversations (Krugman 2013) that high levels of debt, especially above the “90 percent [debt/GDP] threshold” (Reinhart and Rogoff 2010, 577), posed a risk to economic growth. Much of the coverage of this controversy focused on an error that was a “perfect made-for-TV mistake” (Stevenson and Wolfers 2013) involving a simple error in the formula used in their Excel calculations. The real story here, however, is that it took three years for this error and other issues to be discovered because replication files were not publicly available, nor were they provided to scholars when asked. If professional norms or the American Economic Review had required that authors publish replication files, this debate would be advanced by three years and discussions about austerity policies would have been based on a more clear-sighted appraisal of the evidence.

Type
Symposium: Openness in Political Science
Information
PS: Political Science & Politics , Volume 47 , Issue 1 , January 2014 , pp. 60 - 66
Copyright
Copyright © American Political Science Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

APSA. 2012. “A Guide to Professional Ethics in Political Science.” URL: http://www.apsanet.org/content86135.cfm.Google Scholar
Bhattacharjee, Yudhijit. 2013. “The Mind of a Con Man.” New York Times Magazine, April 26. URL: http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all&_r=1&.Google Scholar
Bowers, Jake. 2011. “Six Steps to a Better Relationship with Your Future Self.” Political Methodologist 18 (2): 9.Google Scholar
Bueno de Mesquita, Bruce, Gleditsch, Nils Petter, James, Patrick, King, Gary, Metelits, Claire, Ray, James Lee, Russett, Bruce, Strand, Håvard, and Valeriano, Brandon. 2003. “Symposium on Replication in International Studies Research.” International Studies Perspective 4 (1): 72–107. URL: http://onlinelibrary.wiley.com/doi/10.1111/1528-3577.04105/full.Google Scholar
Davenport, Christian, and Moore, Will H.. 2013. “Conflict Consortium Data Standards & Practices.” Manuscript. Google Scholar
Gandrud, C. 2013. Reproducible Research with R and Rstudio. Chapman & Hall/CRC. Google Scholar
Gerring, John. 2011. Social Science Methodology: A Unified Framework. New York: Cambridge University Press.Google Scholar
Gherghina, Sergiu, and Katsanidou, Alexia. 2013. “Data Availability in Political Science Journals.” European Political Science 12: 333–49.Google Scholar
Gleditsch, Nils Petter, Metelits, Claire, and Strand, Håvard. 2003. “Posting Your Data: Will You Be Scooped or Will You Be Famous?” International Studies Perspectives 4 (1): 9–95. URL: http://gking.harvard.edu/files/replvdc.pdf.Google Scholar
Herndon, Thomas, Ash, Michael, and Pollin, Robert. 2013. “Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff.” Political Economy Research Institute Working Paper Series (322). CrossRefGoogle Scholar
Hook, Les A., Vanna, Suresh K. Santhana, Beaty, Tammy W., Cook, Robert B., and Wilson, Bruce E.. 2010. “Best Practices for Preparing Environmental Data Sets to Share and Archive.” URL: daac.ornl.gov/PI/BestPractices-2010.pdf. An earlier version of this was published in: 2001. Bulletin of the Ecological Society of America 82 (2): 138–41.Google Scholar
King, Gary. 1995. “Replication, Replication.” PS: Political Science and Politics 28 (3): 444–52.Google Scholar
Krugman, Paul. 2013. “The Excel Depression.” New York Times . URL: http://www.nytimes.com/2013/04/19/opinion/krugman-the-excel-depression.html.Google Scholar
Moravcsik, Andrew. 2010. “Active Citation: A Precondition for Replicable Qualitative Research.” PS: Political Science and Politics 43 (1): 29–35.Google Scholar
Peng, Roger D. 2011. “Reproducible Research in Computational Science.” Science 334 (6060): 1226–27.CrossRefGoogle ScholarPubMed
Piwowar, Heather A, Day, Roger S., and Fridsma, Douglas B. 2007. “Sharing Detailed Research Data Is Associated with Increased Citation Rate.” PLOSOne 2 (3): e308. Google Scholar
Reinhart, Carmen M., and Rogoff, Kenneth S.. 2010. “Growth in a Time of Debt.” American Economic Review: Papers & Proceedings 100 (2): 573–78.Google Scholar
Rich, Timothy S. 2013. “Publishing as a Graduate Student: A Quick and (Hopefully) Painless Guide to Establishing Yourself as a Scholar.” PS: Political Science & Politics 46 (2): 376–79.Google Scholar
Savage, C. J., and Vickers, A. J.. 2009. “Empirical study of data sharing by authors publishing in PLoS journals.” PLoS One 4 (9): e7078. Google Scholar
Shalizi, Cosma Rohilla. 2013. “Advanced Data Analysis from an Elementary Point of View.” URL: http://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/.Google Scholar
Shea, Christopher. 2012. “The Data Vigilante.” The Atlantic, December. URL: http://www.theatlantic.com/magazine/archive/2012/12/the-data-vigilante/309172/.Google Scholar
Simonsohn, Uri. 2012. “Just Post It: The Lesson from Two Cases of Fabricated Data Detected by Statistics Alone.” Psychological Science 24: 1875–88.Google Scholar
Stevenson, Betsey, and Wolfers, Justin. 2013. “Six Ways to Separate Lies from Statistics.” Bloomberg. URL: http://www.bloomberg.com/news/2013-05-01/six-ways-to-separate-lies-from-statistics.html.Google Scholar
Wicherts, Jelte M., Borsboom, Denny, Katsand, Judith, and Molenaar, Dylan. 2006. “The Poor Availability of Psychological Research Data for Reanalysis.” American Psychologist 61 (7): 726–28.Google Scholar
Wicherts, Jelte M., Bakker, Marjan, and Molenaar, Dylan. 2011. “Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Statistical Results.” PLoS One 6 (11): e26828. Google Scholar
Supplementary material: PDF

Dafoe supplementary material

Appendix A

Download Dafoe supplementary material(PDF)
PDF 364.9 KB