Hostname: page-component-586b7cd67f-tf8b9 Total loading time: 0 Render date: 2024-11-28T00:22:07.701Z Has data issue: false hasContentIssue false
  • English
  • Français

Testing for Publication Bias in Political Science

Published online by Cambridge University Press:  04 January 2017

Alan S. Gerber ,
Donald P. Green  and
David Nickerson
Alan S. Gerber
Affiliation:
Department of Political Science, Yale University, New Haven, CT 06520-8301. e-mail: [email protected]
Donald P. Green
Affiliation:
Department of Political Science, Yale University, New Haven, CT 06520-8301. e-mail: [email protected]
David Nickerson
Affiliation:
Department of Political Science, Yale University, New Haven, CT 06520-8301. e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

If the publication decisions of journals are a function of the statistical significance of research findings, the published literature may suffer from “publication bias.” This paper describes a method for detecting publication bias. We point out that to achieve statistical significance, the effect size must be larger in small samples. If publications tend to be biased against statistically insignificant results, we should observe that the effect size diminishes as sample sizes increase. This proposition is tested and confirmed using the experimental literature on voter mobilization.

Type
Research Article
Information
Political Analysis , Volume 9 , Issue 4 , 2001 , pp. 385 - 392
Copyright
Copyright © 2001 by the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adams, William C., and Smith, Dennis J. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44 (Autumn): 389–395.Google Scholar
Begg, C. B. 1985. “A Measure to Aid in the Interpretation of Published Clinical Trials.” Statistics in Medicine 4: 1–9.Google Scholar
Begg, C. B. 1994. “Publication Bias.” In The Handbook of Research Synthesis, eds. Cooper, H. and Hedges, L. V. New York: Russell Sage Foundation.Google Scholar
Begg, C. B., and Berlin, J. A. 1988. “Publication Bias: A Problem in Interpreting Medical Data.” Journal of the Royal Statistical Society Series B 151: 419–463.CrossRefGoogle Scholar
Berlin, J. A., Begg, C. B., and Louis, T. A. 1989. “An Assessment of Publication Bias Using a Sample of Published Clinical Trials.” Journal of the American Statistical Association 84: 381–392.Google Scholar
Chalmers, I., Hetherington, J., Newdick, M., Mutch, L., Grant, A., Enkin, E., and Dickersin, K. 1986. “The Oxford Database of Perinatal Trials: Developing a Register of Published Reports of Controlled Trials.” Controlled Clinical Trials 7: 306–324.CrossRefGoogle ScholarPubMed
Coursol, A., and Wagner, E. 1986. “Effect of Positive Findings on Submission and Acceptance Rates: A Note on Meta-Analysis Bias.” Professional Psychology 17: 136–137.Google Scholar
DeLong, J. B., and Lang, K. 1992. “Are All Economic Hypotheses False?” Journal of Political Economy 100: 1257–1272.Google Scholar
Dickersin, K. 1990. “The Existence of Publication Bias and Risk Factors for Its Occurrence.” Journal of the American Medical Association 263: 1385–1389.Google Scholar
Eldersveld, Samuel J. 1956. “Experimental Propaganda Techniques and Voting Behavior.” American Political Science Review 50 (Mar.): 154–165.Google Scholar
Gerber, Alan S., and Green, Donald P. 2000. “The Effects of Canvassing, Direct Mail, and Telephone Contact on Voter Turnout: A Field Experiment.” American Political Science Review 94 (Sept.): 653–663.Google Scholar
Greene, William H. 1997. Econometric Analysis, 3rd ed. Upper Saddle River, NJ: Simon and Schuster.Google Scholar
Greenwald, A. G. 1975. “Consequences of Prejudice Against the Null Hypothesis.” Psychological Bulletin 82: 1–12.Google Scholar
Iyengar, S., and Greenhouse, J. B. 1988. “Selection Models and the File Drawer Problem.” Statistical Science 3: 9–135.Google Scholar
Light, R. J., and Pillemer, D. B. 1984. Summing Up: The Science of Reviewing Research. Cambridge, MA: Harvard University Press.Google Scholar
Mahoney, M. J. 1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy Research 1: 161–175.Google Scholar
Miller, Roy E., Bositis, David A., and Baer, Denise L. 1981. “Stimulating Voter Turnout in a Primary: Field Experiment with a Precinct Committeeman.” International Political Science Review 2(4): 445–460.Google Scholar
Sigelman, Lee. 1999. “Publication Bias Reconsidered.” Political Analysis 8: 201–210.Google Scholar
Simes, R. J. 1986. “Publication Bias: The Case for an International Registry of Clinical Trials.” Journal of Clinical Oncology 4: 1529–1541.Google Scholar