In response to our recent announcement of a single-blind review process at Political Analysis, we have heard considerable feedback and questions about our decision. This is a discussion that started within the Society for Political Methodology in 2011 and continues today, indicating widespread interest and concern about the review process at the journal. As the editorial team, we wanted to comment on this issue and foster open discussion.
The primary criticism rests on the assumption that overt or implicit prejudices regarding gender, rank and institutional affiliation will lead to biases in acceptances and rejections of manuscripts to the journal. Specifically, authors are concerned that a single-blind review process will result in larger proportions of men, senior faculty, and faculty from top ranked departments having their manuscripts accepted. In fact, the one prominent paper that examined this question for gender in political science found no significant differences between male and female acceptance/rejection rates after the move was made to a double blind from a single-blind review process (Ostby, Strand, and Nordas Reference Ostby, Strand and Nordas2013).
In addition, it is very likely that some of the reviewers in the process do know the identity of the author. This is likely for multiple reasons, including:
(1) The Internet provides a quick and easy way to often identify authors. Indeed, the American Economic Association used to have a double-blind system for its journals, but in 2011 switched to single blind because it was so easy to identify authors. And, an in-house analysis of PA at Caltech by graduate students suggested that they were able to identify almost all of the articles for which they were given only a title and abstract.
(2) Authors often deliver their papers at conferences prior to submitting for publication. These paper deliveries, especially in political methodology, are often made at smaller and more intimate conferences resulting in increased recognition of the author’s manuscript after submission.
(3) Authors often are connected to specific research questions and specific data sets and when these data sets appear in a manuscript they are clear cues as to whose research it is.
(4) Science is inherently a public enterprise and subfields are often fairly small and interactions between scholars within a subfield are very likely. These community interactions increase the likelihood that authors and reviewers will know one another.
(5) Identifying references left out of a paper often provide another heuristic for identifying the author(s).
Given these facts, a fairer process is provided when the correct information is provided to each reviewer. Furthermore, now that editors know that reviewers know the identity of the author(s), this will help evaluate any context for the reviews (e.g., previous debates or scholarly exchanges), which may be positive or negative.
While it is possible that implicit or explicit bias may lead to increased or decreased acceptance rates it is also possible that providing a real context for a real author enhances that author’s standing as a person, which may lead to higher acceptance rates. For example, reviewers may be more supportive of junior scholars, offering more R&Rs, and may want to provide more valuable advice for manuscript improvement when the author is a real as opposed to an anonymous person. These positive benefits must also be considered. For example, in a recent study of the Nature journals acceptance rates were found to be higher for those manuscripts that submitted to the single-blind review process instead of the double-blind review process (Enserink Reference Enserink2017). Desk rejection rates were 92% for those who requested double-blind review and 77% for those who requested single-blind review and acceptance rates were 44% for those who requested single-blind review and a much lower 25% for those who requested double-blind review.
Additional benefits of this system for the journal include the reduction in the cost of reviewing manuscripts for anonymity. The process takes substantial journal resources and sometimes the author slips through anyway. By moving to a single-blind review this step is no longer necessary, so costs to the journal are reduced. By doing away with them, processing time for the articles is also reduced, which benefits both the author(s) and the journal.
It is common for authors in political science to complain about the total length of the review process at major journals in the discipline. It is hardly uncommon to have a submission to acceptance cycle that lasts over a year. This is an issue that particularly affects our pretenured colleagues. So the process of checking anonymity of submissions is not just a cost issue alone. It adds over two weeks on average, and must be done again on resubmission after a revise-and-resubmit decision.
It is worth noting that most journals in the natural sciences rely on single-blind review. For example, the journal Science, perhaps the most prominent general US science journal, maintains only a single-blind review process. Yet there tends to be a vaguely worded preference for double blind in the hallways of our departments and conferences. In addition, actual submissions to a double-blind review process in the Nature journals were only one in eight. Thus, there appears to be a disconnect between attitudes and behavior.
Importantly, we note that other journals in both our discipline and other disciplines are flirting with different review processes. For example, State Politics and Policy Quarterly is moving to a triple-blind review process. We think different journals dabbling in different review methods makes a lot of sense and will help to sort out what claims are true and what claims are false and help to identify a set of best practices. It is consistent with research in psychology and other literatures that shifting to single blind may produce more helpful, constructive reviews with less hostile tones, irrespective of the ultimate decisions rendered (Freedland Reference Freedland2017). An extensive study in information science finds very modest if any effects, including against interdisciplinary work (Lee et al. Reference Lee, Sugimoto, Zhang and Cronin2013).
In addition, Political Science Research and Methods (PSRM) has been single blind for 2 years with no bias issues reported by the current editor, Vera Troeger, and the Quarterly Journal of Political Science has been single blind since its inception with no known bias issues observed by one of the current editors, Josh Clinton.
Finally, it is important to note that we have a very diverse editorial board in terms of gender, rank, and institutional affiliation. The members of the editorial team and editorial board are important checks upon this process and are committed to a strong and unbiased Political Analysis journal that publishes the best work in political methodology. This has been true since the inception of the journal under the leadership of Professor Neal Beck.
Equally importantly these are valid empirical questions that can be answered and we commit to examining the data, making it transparent, and adjusting our decision if we find deficiencies in our processes. After December 2018, we will have one year of single-blind submissions data to compare to the previous policy. Furthermore, we will know the sex, seniority, department ranking, citation count, country of employment, and subdiscipline of both reviewers and authors. The obvious outcomes are the recommendation by reviewers and the eventual fate of the manuscript. This will allow us to report to you any differences that result from the change of policy to single blind.
We look forward to the future of Political Analysis and to answers to these critical questions.
