How the scholarly media in political science are evaluated has long attracted the attention of political scientists. Political scientists are often interested in where they should submit book manuscripts and which scholarly presses are likely to generate the greatest impact for the books that they publish. The ranking of journals and scholarly presses is an integral part of how political scientists evaluate their own work and the scholarly productivity of their colleagues, graduate students, and departments. The evaluation of scholarly media can play an important role in personnel decisions, including faculty hiring and promotion and tenure decisions. Scholars have adopted various approaches for measuring the impact of scholarly publications, including citations (Giles and Garand Reference Giles and Garand2007; Masuoka, Grofman, and Feld Reference Masuoka, Grofman and Feld2007) and subjective evaluations of scholarly journals and book publishers based on surveys of political scientists (Giles and Wright Reference Giles and Wright1975; Giles, Mizell, and Patterson Reference Giles, Mizell and Patterson1989; Garand Reference Garand1990; Crewe and Norris Reference Crewe and Norris1991; Garand and Giles Reference Garand and Giles2003; Garand et al. Reference Garand, Giles, Blais and McLean2009; Goodson, Dillman, and Hira Reference Goodson, Dillman and Hira1999).
It is fair to say that most of the evaluation of scholarly media in political science has focused on scholarly journals. Previous research shows a fairly clear-cut pecking order of scholarly journals that has remained relatively constant over time (Garand and Giles Reference Garand and Giles2003; Garand et al. Reference Garand, Giles, Blais and McLean2009). Some notable efforts have been made to evaluate scholarly presses, but these studies have been relatively few and far between. Moore (Reference Moore2010) has reported the results of a 2008 survey of 1,086 American political theorists in which respondents were asked to identify publishers of material relating to political theory whose books they were reading. The resulting ranking is generally consistent with common perceptions of leading scholarly presses, with major university presses dominating the top 15 positions on the list.
Perhaps the most noteworthy general study of scholarly book publishers to date is Goodson, Dillman, and Hira's (Reference Goodson, Dillman and Hira1999) study, which collected survey data from 347 American political scientists to rank 65 scholarly presses. These researchers asked political scientists to evaluate scholarly publishers on a scale from 0 (poor) to 4 (excellent), as well as whether they were familiar with each of the 65 publishers on their list. Their findings are not particularly surprising. The major university presses (e.g., Cambridge University Press, Princeton University Press, Oxford University Press) dominate the top dozen scholarly presses on the list in terms of quality assessments, with major commercial presses (e.g., Brookings Institution Press, CQ Press, Basic Books) competing with lower-ranked university presses (e.g., Columbia University Press, University of North Carolina Press, Duke University Press) for slots in the second tier. Interestingly, the scholarly publishers with the greatest familiarity for the public comprise a mix of highly regarded university presses and commercial presses.
In this article, we build on previous research relating to the evaluation of scholarly publishers. We use survey data collected from a sample of 603 American political scientists in 2005 to establish a new, updated ranking of scholarly presses. Our approach differs somewhat from that adopted by Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999), but it is compatible with the approach adopted by Garand and Giles (Reference Garand and Giles2003) and Garand et al. (Reference Garand, Giles, Blais and McLean2009) in their work on journal rankings. We demonstrate that this approach yields updated journal rankings that are similar to those found in Goodson, Dillman, and Hira's study.
Data and Methodology
In 2005, we conducted an omnibus survey of political scientists in the United States. The survey instrument included a wide range of items relating to political scientists' research philosophies; perceptions of what is required for success in political science; and hiring, promotion, and tenure standards. Our original sample was drawn from the APSA membership list and limited to individuals with both a Ph.D. and a faculty rank at American colleges and universities. Our final sample of political scientists who received mailed surveys included 1,164 political scientists from Ph.D.-granting institutions and 572 political scientists from non-Ph.D.-granting institutions, for a total sample of 1,736 individuals. Our response rate was 36% for the Ph.D. subsample and 32% for the non-Ph.D. subsample, yielding a final sample size of 603 and a total response rate of 35%. We conducted a variety of tests to determine the representativeness of our sample and found that no significant differences exist between survey respondents and nonrespondents on most variables; the only exceptions are that women, comparativists, and minority-group members are slightly underrepresented, but even here, the difference between the sample respondents and nonrespondents is not substantively large. A detailed appendix describing the sample's construction is available on request.
We included in the survey two open-ended items designed to measure respondents' perceptions of the leading scholarly presses that publish books in political science. These items mirror items included by Garand and Giles (Reference Garand and Giles2003) and Garand et al. (Reference Garand, Giles, Blais and McLean2009) in their previous studies of journal rankings. First, we asked respondents the following question:
Assume that you have just completed what you consider to be a very strong book manuscript on a topic in your area of expertise. Indicate the first book publisher to which you would submit such a manuscript. Assuming that the book manuscript is rejected at your first choice, please indicate the second and third publishers to which you would submit the manuscript.
This question was designed to offer respondents an alternative way of thinking about evaluations of scholarly presses by eliciting information about how they prioritize the book publishers that they would like to publish their own research. We coded up to three responses to this question and ranked scholarly presses based on the number of first, second, and third responses for each book publisher mentioned. In ranking book publishers, we created a weighted total by assigning three points for the first preference, two points for the second preference, and one point for the third preference.
In addition, we asked respondents, “Which book publishers or presses publish books do you read regularly or otherwise rely on for the best research in your area of expertise? (list up to five book publishers).” For this question, we coded up to five responses, which were aggregated to rank scholarly presses. We created a weighted total by assigning five points for each first preference, four points for each second preference, three points for each third preference, two points for each fourth preference, and one point for each fifth preference.
Some trade-offs are associated with using both an open-ended measure and the list-based measure employed by Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) to evaluate book publishers, as well as by other scholars to evaluate scholarly journals. First, Goodson, Dillman, and Hira provided respondents with a list of book publishers and asked them to evaluate the quality of work that each scholarly press published. The information provided for listed publishers was very useful but, by necessity, limited. Our approach provided less detailed information about how respondents should evaluate each publisher, but the open-ended nature of our questions permitted respondents to offer information about publishers that might not be included in a researcher-produced list. For example, the University Press of Kansas did not meet the criteria for inclusion on the Goodson, Dillman, and Hira list, but our open-ended format permitted respondents to place this press on the list as one of their preferred book publishers.
Second, the Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) approach permitted respondents to evaluate all scholarly presses on a list, while our approach focused only on those book publishers deemed by American political scientists to be among the elite presses publishing political science work. As with rankings of scholarly journals that are based on citations, our study found a rapid drop-off in the relative scores when moving from the highest-ranked scholarly presses to those publishers that fall below the elite levels. Such a drop-off is not generally observed for quality measures that are based on surveys in which respondents are asked to evaluate a list of scholarly journals or publishers. This phenomenon is likely due to a process whereby political scientists perceive many scholarly outlets as reasonable places to publish their research but only a few presses or journals as the elite outlets for publishing the best work in their fields of study (Giles and Garand Reference Giles and Garand2007).
There is value in each of these two approaches, and a case can be made that the best strategy is to use them in tandem (cf. Garand and Giles Reference Garand and Giles2003; Garand et al. Reference Garand, Giles, Blais and McLean2009). In this article, we use data reported by Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) to draw explicit comparisons between rankings of scholarly presses based on the list-based and open-ended approaches.
Ranking Scholarly Presses
Publication Preferences
How do American political scientists compare various book publishers? Table 1 reports the first, second, and third preferences of political science scholars among the top 50 scholarly presses to which they would be willing to send a high-quality book manuscript; the table also reports the total number of times each press was mentioned, the weighted total (with first, second, and third preferences weighted differently), and the relative weighted total (created by dividing the weighted total for each scholarly press by the weighted total for the highest-ranked scholarly press).Footnote 1 Ninety-two book publishers received at least one mention, suggesting that political scientists vary considerably in how they prioritize the scholarly publishers to which they might send their best work. Moreover, respondents mentioned 51 different book publishers (35.2% of total mentions) as their first choice for submitting their best scholarly work, with 54 (37.2%) and 68 (46.9%) different publishers mentioned as their second and third choices, respectively. Many of these scholarly presses drew only one or two mentions, but it is clear that political scientists perceive a wide range of possible scholarly outlets for their best research.
Table 1 Respondent Preferences for Publisher Submission of High-Quality Book Manuscript, Top-50 Ranked Publishers
Note. List includes only those publishers ranked among the top 50. A complete list is available from lead author upon request. Entries represent the number of respondents who report the book publisher as their first, second, or third preference for submitting a high-quality book manuscript. Respondents were asked the following question: “Assume that you have just completed what you consider to be a very strong book manuscript on a topic in your area of expertise. Indicate the first book publisher to which you would submit such a manuscript. Assuming that the book manuscript is rejected at your first choice, please indicate the second and third publishers to which you would submit the manuscript.” We coded up to three responses.
[1]. Weighted total is based on three points for first preference, two points for second preference, and one point for third preference. Relative total is the weighted total divided by the highest weighted total (i.e., 613 for Cambridge University Press).
[2] The numerator upon which these percentages are based is 145, which is the total number of scholarly presses in our data receiving at least one mention and a publishing or reading preference.
As one can see, there is a clear-cut pecking order of scholarly presses. We were struck by the small number of book publishers that hold a dominant position relative to other possible outlets for respondents' best book manuscripts. Cambridge University Press is the clear leader, followed by Princeton University Press. These two scholarly presses outpace all others by a fairly wide margin; Princeton University Press earned 88% of the weighted total preference points of Cambridge University Press, but the next nearest competitors—Oxford University Press and the University of Chicago Press—received only about 41% of the rating points of Cambridge University Press. These four presses are followed, in descending order, by Harvard University Press (24.3% of Cambridge University Press' rating points), Cornell University Press (23.7%), University of Michigan Press (19.4%), Yale University Press (15.8%), University Press of Kansas (14.7%), and CQ Press (9.1%).
The dominance of the leading publishers is particularly noteworthy. Together, Cambridge University Press and Princeton University Press were the first choice of 45% (235 of 524) of responses regarding individuals' first preference for an outlet for a good scholarly book; furthermore, the top 10 publishers received 80% (418 of 524) of respondents' first preference votes for where to send a very good book manuscript. Clearly, most American political scientists perceive these top 10 scholarly presses to be the leading outlets for their book-length manuscripts. The rating points drop off at a very rapid rate; after the 10th ranking position, scholarly presses received less than 10% of Cambridge University Press' rating points.
The second tier of scholarly presses comprises a mix of university presses and commercial publishers. While only one commercial press is represented among the top 10 scholarly presses (CQ Press, ranked 10th), the second tier includes Brookings Institution Press (13th), Lynne Rienner Publishing (16th), Routledge (18th), and Rowman and Littlefield (19th). Good second-tier university presses—University of California Press (11th), Johns Hopkins University Press (12th), Georgetown University Press (14th), MIT Press (15th), Stanford University Press (17th), and Columbia University Press (20th)—round out the second 10 scholarly publishers.
Reading Preferences
In addition to asking respondents about which scholarly press they would prefer to publish a very good book manuscript that they had written, we also asked them to identify the book publishers whose books they “read regularly or otherwise rely on for the best research” in their areas of expertise. Table 2 reports the first, second, third, fourth, and fifth preferences on this item, as well as the total mentions, the weighted total (based on five points for a first mention, four points for the second, and so on), and the relative weighted total.Footnote 2 The responses to this question were very similar to those reported in table 1. First, respondents indicated collectively that they rely on a wide array of book publishers for the best work in their respective fields of study. Indeed, 131 book publishers earned at least one mention, with 58 publishers (40%) mentioned as a first preference; 60 mentioned (41.4%) as a second preference; and 60 (41.4%), 68 (46.9%), and 64 (44.1%) mentioned as third, fourth, and fifth preferences, respectively. American political scientists vary considerably in the scholarly presses on which they rely to read the best work in their field.
Table 2 Respondent Preferences for Publisher Whose Books They Read or Rely upon for the Best Research in Their Areas of Expertise, Top-50 Ranked Publishers
Note. List includes only those publishers ranked among the top 50. A complete list is available from lead author upon request. The entries represent the number of respondents who report the book publisher as their first, second, third, fourth, or fifth preference for publishers whose books they read regularly or otherwise rely on for the best research in their fields. Respondents were asked the following question: “Which book publishers or presses publish books that you read regularly or otherwise rely on for the best research in your area of expertise?” We coded up to five responses.
[1] Weighted total is based on five points for first preference, four points for second preference, three points for third preference, two points for fourth preference, and one point for fifth preference. Relative total is the weighted total divided by the highest weighted total (i.e., 1,215 for Cambridge University Press).
[2] The numerator upon which these percentages are based is 145, which is the total number of scholarly presses in our data receiving at least one mention and a publishing or reading preference.
Yet again, a clearly-defined set of tiers emerges among the rankings of scholarly presses. Cambridge University Press and Princeton University Press lead the way once more, with preference counts for Princeton University Press constituting slightly more than 80% of the preference counts for Cambridge University Press. After these two, a significant drop-off in rankings occurs, with Oxford University Press (47.8% of the Cambridge University Press weighted total) and University of Chicago Press (44.5%) comprising a second tier. Harvard University Press (29.3%), Cornell University Press (24.2%), University of Michigan Press (23.0%), Yale University Press (22.6%), CQ Press (19.6%), University Press of Kansas (16.9%), and Brookings Institution Press (12.1%) comprise a third group. The remaining scholarly presses on the list each received less than 10% of the preference points enjoyed by Cambridge University Press.
Once again, the dominant positions of Cambridge University Press and Princeton University Press are worth noting. Taken together, these two book publishers were the first-place choices of 43.4% (219 of 505) of first preference responses, and the top 10 publishers represent 76.4% (486 of 505) of respondents' first preferences for sources of the best research in their field of expertise. Although a wide range of scholarly presses are high-priority reading sources for small numbers of political scientists, political scientists seem to look to only a dozen or so publishers for the lions' share of the best book-length research published on topics within their area of interest.
Comparing Open-Ended and List-Based Approaches
Do list-based and open-ended approaches yield similar rankings of scholarly publishers among political scientists? As noted, some differences do distinguish the list-based approach for rating scholarly publishers used by Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) and the open-ended approach used here. However, some evidence does show that these two approaches for evaluating scholarly media in political science generate similar ranking patterns. In their studies of journal ratings, Garand and Giles (Reference Garand and Giles2003) and Garand et al. (Reference Garand, Giles, Blais and McLean2009) have found that measures of journal quality and impact based on these two approaches produce similar results. Does this pattern of evaluation also extend to scholarly presses?
As a starting point, we used data from Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) to create an impact score for book publishers that mirrors the scores developed by Garand (Reference Garand1990) and used by Garand and Giles (Reference Garand and Giles2003) and Garand et al. (Reference Garand, Giles, Blais and McLean2009). Goodson, Dillman, and Hira reported two measures that can be used to measure the scholarly impact of book publishers: (1) publisher quality, based on respondents' perception of the overall quality of each of 65 book publishers; and (2) publisher familiarity, based on whether respondents indicated that they were familiar with a given publisher by offering an evaluation. Following Garand (Reference Garand1990), we measured publisher impact by weighting publisher quality by the proportion of individuals who reported being familiar with a given publisher:
This measure yields an impact score that is roughly equally correlated with both quality and familiarity and conveys respondents' joint perceptions of the degree to which a given publisher produces scholarly books that are both high quality and highly visible.
How is publisher impact related to our two open-ended measures of publisher quality? Our publication preference variable is highly skewed to the right, since there are only a few very highly ranked publishers; to compensate for the skewness of this variable, we took the log of this measure and created a scatter plot representing the relationship between the publisher impact measure derived from Goodson, Dillman, and Hira's (Reference Goodson, Dillman and Hira1999) data and our publication preference measure. Goodson, Dillman, and Hira included only 65 scholarly presses on the list they presented to their survey respondents, and some publishers included therein were not cited by our survey respondents. Ultimately, we were left with 48 publishers that could be used in our analysis.
Figure 1 presents the scatter plot of the relationship between publisher impact and our logged publication score, and a strong positive relationship between these two variables is readily apparent. The correlation between these two variables is r = 0.789, and a simple bivariate regression results in a strong, highly significant coefficient for the independent variable (b = 1.06, t = 8.71). Clearly, scholarly book publishers that are highly ranked in terms of list-based impact scores are also highly ranked in terms of open-ended scores.
Figure 1 Scatter Plot of Relationship between Logged Publication Score and Publisher Impact
A similar pattern emerges for our reading preferences variable. The scatter plot for our 55 book publishers that overlap with the Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999) dataset is presented in figure 2, and here again, there is a very strong relationship between these two variables (r = 0.786, b = 1.06, t = 9.26). The list-based publisher impact measure and the open-ended reading preference variable both generate rankings of book publishers that are positively and significantly related. The scholarly presses on which American political scientists rely for the best research in their area of expertise are also, unsurprisingly, the scholarly presses that are most highly evaluated and familiar to political scientists.
Figure 2 Scatter Plot of Relationship between Logged Reading Score and Publisher Impact
Note. Because some values on the reading score are equal to 0 and the log of 0 is undefined, we use the log +1 transformation by adding 1 to each score and taking the log of the +1 value.
Comparing Publishing and Reading Preferences across Subfield Specialties
Although there is general agreement among political scientists about the leading journals and scholarly presses, there may be some differences in the ratings given to scholarly media for political scientists in different subfields. Garand (Reference Garand2005) has found some differences across subfields in the ordering of scholarly journals at the high end of journal rankings, but taken as a whole, he has found that American political scientists tend to evaluate political science journals similarly in the subfields of American politics, comparative politics, international relations, and political theory. Do these findings for political science journals extend to scholarly publishers?
Table 3 reports the publication and reading preferences for the top five scholarly publishers, aggregated separately for the subfields of American politics, comparative politics, international relations, and political theory.Footnote 3 At the highest end, political scientists from different subfields reveal considerable similarity in how they evaluate scholarly publishers. Cambridge University Press and Princeton University Press were the first or second choices of scholars in all subfields, with Cambridge University Press ranked first for the subfields of American politics, comparative politics, and political theory, and Princeton University Press ranked first for the subfield of international relations. In addition, Oxford University Press appears among the top five presses for each of the subfields, although its exact position in the rankings varies across subfields. The remaining slots are filled by a variety of highly regarded scholarly (mostly university) presses, although variation does occur across subfields. For instance, Cornell University Press is highly ranked among comparative politics and international relations scholars but is not listed among the top five scholarly presses for American politics and political theory scholars; on the other hand, the University of Chicago Press is highly ranked in the American politics and political theory subfields but does not appear among the top five presses for the comparative politics and international relations subfields.
Table 3 Top Five Preferences for Publisher Submission of High-Quality Book Manuscripts and Publisher Whose Books They Read or Rely upon for the Best Research, by Respondent Subfield Specialty
Note. For publishing preferences, the weighted total is based on three points for first preference, two points for second preference, and one point for third preference. For reading preferences, the weighted total is based on five points for first preference, four points for second preference, three points for third preference, two points for fourth preference, and one point for fifth preference.
Although rankings of the top five presses vary across subfield, taken as a whole, there is a strong relationship among the publishing and reading preferences across subfields. The correlations between pairs of subfields are all high, ranging from a low of r = 0.705 (for reading preferences for American politics and international relations) to a high of r = 0.988 (for reading and publishing preferences for international relations); indeed, the mean correlation across all subfield pairs is r = 0.85. To illustrate this point further, we conducted a series of principal components analyses and found that the publishing and reading preference variables for each subfield load on a single factor for publishing preferences (eigenvalue = 3.573, variance explained = 0.893), reading preferences (eigenvalue = 3.429, variance explained = 0.857), and publishing and reading preferences combined (eigenvalue = 6.975, variance explained = 0.872). Clearly, how political scientists in one subfield evaluate scholarly presses is similar to how political scientists in other subfields evaluate those same scholarly presses.
Conclusion
Political scientists report the results of their research in a variety of scholarly media, including books, journal articles, book chapters, convention papers, online papers, and websites. Among these outlets, books and journal articles play a prominent role in scholars' efforts to disseminate their findings in the most visible and rigorous way. Many political scientists—particularly graduate students and junior faculty—are eager to know the best places to submit their work in order to position it to have the biggest impact and maximize their chances of obtaining academic employment, promotion, or tenure. Thus far, we know quite a bit about how political scientists evaluate scholarly journals, but less systematic understanding exists regarding how political scientists evaluate and rank the scholarly presses that publish research in political science.
In this article, we use survey data on American political scientists' publication and reading preferences to develop a rank ordering of scholarly presses in political science. We find that there are many book publishers that are highly regarded in terms of publication or reading among at least some political scientists, but the bulk of political scientists' publication and reading preferences reside in a dozen or so scholarly publishers. Cambridge University Press and Princeton University Press stand out as the leading scholarly presses, with Oxford University Press and the University of Chicago Press constituting a very strong second tier. A group of well-regarded university presses comprise most of the remainder of the top 10 publishers, and the rest of the top-20 publishers comprise a mix of university and commercial presses. Furthermore, political scientists from different subfields exhibit only modest differences in their top-five scholarly presses for publication and reading, and there are very high correlations within press rankings among scholars specializing in American politics, comparative politics, international relations, and political theory. Finally, our measures of publication and reading preferences are highly related to a publisher impact score derived from data collected by Goodson, Dillman, and Hira (Reference Goodson, Dillman and Hira1999). The finding of a strong relationship among these variables indicates that list-based and open-ended surveys generate similar rankings of scholarly presses. The bottom line is that our study produces a ranking of scholarly publishers that can be used to guide political scientists as they decide where to send their best book-length work.
