Recent attempts to understand the state of international relations (IR) have sought to debunk myths propagated by earlier impressionistic reviews through the systematic use of data (Waever Reference Waever1998). The recent efforts of the Teaching, Research, and International Policy (TRIP) program mark the most complete achievement of this goal to date.Footnote 1 Most notably, in addition to an ambitious series of surveys, the TRIP project has illuminated trends in the field based on rigorous coding of 2,807 IR articles published in top journals between 1980 and 2007 (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011).Footnote 2 However, we maintain that systematically capturing the nature of IR as a field necessitates an analysis of books. Books are a vital means by which IR scholars disseminate knowledge, accrue promotion and tenure, and influence audiences both within academia and among policy makers. Thus, this article's main contention is that to know IR we must know IR books. Then, we can better answer questions about methodological and theoretical trends in the field, as well as the relationship with the policy world.
In the first-ever systematic exercise, to our knowledge,Footnote 3 we coded 500 books from five leading book presses against the same 28 variables used in the TRIP journal coding system. These variables include methodology, theoretical paradigm, epistemology, policy application, substantive focus, level of analysis, and geographic and temporal scope. These data are supplemented with interview material from meetings with the editors of the same five presses (Cambridge University Press, Cornell University Press, Oxford University Press, Princeton University Press, and Routledge).Footnote 4 We compare the results of our coding against those of TRIP's journal coding, as well as the result of the 2011 TRIP survey of more than 3,700 IR scholars in 20 countries (Maliniak, Peterson, and Tierney Reference Maliniak, Peterson and Tierney2012). The resulting data reveal several telling differences between what IR scholars say they are doing, and what is being published in books versus journal articles. Here, we present only three standout trends and preliminary interpretation of these results in anticipation of inciting a larger debate. These key trends are:
1. Books are far less quantitative in their methodological orientation than journal articles and more closely reflect the self-reported methods of IR scholars.
2. Books are closer to the self-reported paradigmatic orientations of IR scholars than articles, with fewer liberals and more constructivists, but the same surprisingly low proportion of realists.
3. Books are more engaged with policy than articles, but less than would be expected from survey results.
We discover that books more closely reflect what IR scholars say they are doing in surveys than do journal articles. This trend is not true for all variables, and at times books and journals converge with each other while diverging collectively from survey results. Overall, however, our analysis reinforces the intuition that it is critical to include an analysis of books in any effort to accurately characterize the state of IR. While prevailing perceptions about what the field looks like may drive scholars' strategic behavior about how they conduct their research, where they try to publish, and how departments and universities evaluate scholars' research records for promotion and tenure decisions, this corrective may have important implications on the practice and trajectory of IR.
WHY BOOKS?
In his 1998 article analyzing “the sociology of a not-so-international discipline,” Waever cites prominent philosopher of science Robert Merton to the effect that we know a discipline by its journals (Reference Waever1998). In justifying their decision to exclude analysis of books, Maliniak et al. provide the most explicit statement on this subject. They assert, but do not provide evidence for, the claims that the theses of university press books are usually published as articles first, that articles provide a larger number of observations, and that the peer-review process is more rigorous for leading journals than for books (Reference Maliniak, Oakes, Peterson and Tierney2011, 438).
In fact, little evidence supports these claims or the belief that we can understand international relations in the same way that natural scientists understand trends in their field by studying what appears in their journals and ignoring books. Quite the contrary, a study of articles from International Organization, International Studies Quarterly, and World Politics shows that they are significantly more likely to cite books (48.2%) than articles (38.4%) (Zhang Reference Zhang2008). Leading scholars like Waltz, Keohane, and Wendt are cited more for their books than for their articles, even when the former are published years later.Footnote 5 Moreover, contrary to the claims that “books are no longer important,” there is a strong expectation in the academy that to gain tenure, IR scholars publish a book. This expectation is reflected in the 2011 TRIP survey, which reveals that 86% of IR scholars worldwide, averaged across all institutions and ranks, believe that single-authored, university press books are very important for advancement in their professional career (see table 1). This statistic strongly suggests that the importance of books has been seriously underemphasized in prevailing studies of IR, and that generalizations derived from analyses of journals may be misleading.
Table 1 Global Survey Results
Survey Question: “Rank the three kinds of research outputs that are most important for you to publish in order to advance your academic career.”
So, if books are important in our discipline, are books fundamentally different in content than journal articles? Might the portrait of the field painted by coding books more accurately correspond with that revealed in survey data than the picture we derive by coding articles in journals? Our results suggest positive answers on both counts, which bolsters the claim that we must pay attention to books, in addition to journals, to understand the nature and trends within IR.
METHOD
Just as there are high-impact journals that disproportionately shape IR through their prestige and wide readership, there are high-impact presses. Lacking the kind of sophisticated bibliometric analysis of citation data available for articles, we relied on reputation as measured by the TRIP survey for selecting presses. Question 44 of the 2008 TRIP survey asked respondents to list the four book presses that have the greatest influence on the way IR scholars think about international relations. The top eight results in descending order are Cambridge University Press (listed by 79% of respondents worldwide), Princeton University Press (58%), Cornell University Press (51%), Oxford University Press (51%), Routledge (21%), Columbia University Press (15%), Palgrave (11%), and Harvard University Press (10%) (Jordan et al. Reference Jordan, Maliniak, Oakes, Peterson and Tierney2009, 52). Based on these results, we limited our study to the top five book presses: Cambridge, Princeton, Cornell, Oxford, and Routledge.Footnote 6
To identify the universe of IR books, we compiled a list of more than 3,600 IR titles released by the five presses from 1990 to 2010, using Amazon.com's categorization system. We reviewed the abstracts for all books that we could not clearly identify as IR based on the title or author and excluded edited volumes and textbooks. At the end of the process, 1,834 eligible books remained for 1990 to 2010. Then, we drew a random sample of 500 books published between 2000 and 2010. The coding procedure for the books is almost identical to that used for the TRIP article coding and used the same web-based coding interface (see Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011). However, we only discuss the results of a few questions here. All books were independently coded twice, with outstanding differences reconciled by a senior coder (one of the two authors).
THREE KEY FINDINGS
In sections below we compare evidence from our booking coding with survey and journal evidence in presenting our main findings on method, paradigm, and policy engagement. In each case, examining books helps to shed light on puzzling disconnects between what IR scholars say they are doing versus what they publish in journals.
IR: Not as Quantitative as Journals Indicate
In the 1990s, the perestroika movement within the US academy was partly a reaction to the perception that political science (and IR as one of its subfields) was increasingly dominated by quantitative methods and hostile to qualitative work. The TRIP analysis of journal articles verifies this trend. Maliniak et al. note that “The rise of quantitative methods has been swift and dramatic,” and that by 2006, 53% of articles published in the 12 top journals were quantitative (Reference Maliniak, Oakes, Peterson and Tierney2011, 451). Yet these statistics are dramatically at odds with survey results. As Jordan et al. (Reference Jordan, Maliniak, Oakes, Peterson and Tierney2009, 5) report, “the percentage of articles using quantitative methods is vastly disproportionate to the actual number of scholars who identify statistical techniques as their primary methodology.” In the 2008 10-country TRIP survey data, for example, 72% of all respondents reported that they used qualitative methods as their primary methodology, versus only 17% who reported using quantitative methods as their primary approach (in the United States alone the figures were 68% and 23%) (Jordan et al. Reference Jordan, Maliniak, Oakes, Peterson and Tierney2009, 38).
This puzzling result may be partly reconciled by evaluating the content of books. Overall, 50.3% of books use qualitative methods, while only 8.2% use quantitative methods. Furthermore, the coding for methodology was not mutually exclusive. Of the 8.2% of books that used quantitative methods, many also included case studies (following the procedure for articles, books were coded as using quantitative measures if they used statistics to link independent and dependent variables with a hypothesis). The prevalence of qualitative methods is much closer to the survey results. In fact, books are noticeably less inclined toward quantitative methods than we might assume from survey data. Other notable differences are seen in other methods employed. For example, the use of formal modeling in journal articles showed a marked increase, averaging 13% of all articles from 1980 to 2007 (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 451). By contrast, formal models are only used in 3% of books and are reported as the primary methodology only by 1% of the survey respondents. Conversely, while descriptive methods are rarely used in journal articles (4.3% on average between 2000 and 2007),Footnote 7 in books, descriptive methods are the second most prominent, showing up in 38.1% of those coded.
Without further research, the reasons for these marked methodological differences between books and journal articles are difficult to discern. These differences may be the result of authors' self-selection for a myriad of reasons, including perceptions of fit or ability to survive peer-review processes. Of course, books provide more pages for richly detailed case studies that are not available within the strict word limits for articles. The bias toward qualitative methods may also reflect publisher preferences and market pressures. Our interviews with the leading press editors reveal a slight, but discernible, preference for qualitative over quantitative work. This preference is partly due to commercial concerns; quantitative work that involves charts and graphs is more expensive to reproduce and tends to sell fewer copies due to a smaller readership base. Often editors indicated, however, that the dearth of quantitative methods in books is a result of a general sense that work relying heavily on quantitative analysis or formal modeling often does not “lend itself” well to a book project. Overall, the clear differences between the methods used in books versus journal articles, especially in the context of survey results, indicate that we should thoroughly examine claims that IR is now dominated by quantitative work. See table 2.
Table 2 Methods in IR
* The 2011 Survey asked two questions related to methodology: Q28, “In your research, what methods do you primarily employ?” and Q29, “In your research, what other methods do you employ, not including your primary method? Check all that apply.” The survey did not offer the response option of “descriptive approach” or “analytical/non-formal approach,” but instead offered “pure theory” (3% of responses) and “legal or ethical analysis” (4% of responses).
Paradigms in IR
Another striking disconnect in the findings of the journal coding exercise versus the 2011 survey centered on the theoretical or paradigmatic persona of IR work. Specifically, when compared against survey results liberalism appears to be significantly overrepresented in leading journals, while constructivism in particular is very underrepresented. We suspected that some of these differences might be explained by what is published in books, especially if paradigms underrepresented in articles compared with survey results are more prominently represented in books. As indicated by table 3, these suspicions were only partly borne out. On one hand, constructivists are more likely to be published in leading book presses than in leading journals, but still not in proportion to their numbers. Scholars working in a liberal vein are still overrepresented in books, although less than articles, while Marxists are still underrepresented.
Table 3 Paradigms in IR
On the other hand, a clear consistency is seen between journal articles and books with respect to the declining presence of the realist paradigm. The 2011 survey revealed that 16% of scholars think of themselves as realists, but that, on average, IR scholars believe that realism still accounts for 33% of current IR literature (Maliniak, Peterson, and Tierney Reference Maliniak, Peterson and Tierney2012, 47). However, as in journal articles, realism is not a dominant paradigm in books from the last decade. An average of 8% of books and articles use realism as the chosen paradigm, and approximately 15% take realism seriously as an alternative paradigm.
The Myth of Policy Engagement?
The notion of a growing disconnect between academic IR work and the policy world has become an increasing source of angst in the field.Footnote 8 On average, only 2.9% of journal articles use policy analysis and 12% contained policy recommendations (with a higher proportion in security journals like International Security and fewer in international political economy articles). Yet in the 2011 survey, 33% of scholars worldwide (35% in the United States) said their research was primarily motivated by policy relevance or current events, as opposed to issue area, methodology, or paradigm. Moreover, when asked whether their research was more basic (research for the sake of knowledge) or applied (conducted with a specific policy application in mind), 26% of respondents reported that their research was either primarily applied or more applied than basic. Thus, significant evidence shows a gap between what IR scholars are saying regarding the place of policy in our intellectual inquiry versus what is appearing in the top journals. Is the “missing” policy-oriented research showing up in books?
In our sample, we found that books were more likely than journal articles to engage policy. On average, 10% of books engage in policy analysis, and 19.6% offered policy recommendations. More policy-relevant work may be published in books for several reasons. First, there is a greater proportion of nonuniversity-based book authors (current and former policy makers, journalists, and those in think tanks) relative to article authors, and these book authors outside academe may well be both more inclined and professionally motivated to ask and answer policy questions. Second, there is simply more space (word length) in books in which to explore policy prescriptions. Scholars writing journal articles may be deterred from offering policy lessons because there is no room in an 8,000 to 14,000 word article to properly present and defend these prescriptions. Third, books with a policy angle may generate more sales.
However, while we find that books come closer to reflecting what IR scholars are reporting about the discipline, significant gaps still exist between book coding and survey results that merit further investigation. For example, if IR scholars are indeed doing as much policy-relevant work as they report, where are they publishing it? Is this work published in journals not captured in the TRIP's list of twelve or our top five presses? Is it actually showing up in policy-oriented consultancy papers or nontraditional outlets, such as blogs?
CONCLUSIONS
The systematic study of international relations marks a welcome advance on earlier impressionistic reviews, which tended to give rise to myths, like the dominance of realism. Yet such studies are fundamentally flawed as long as they rest on unsubstantiated assertions that books are not important in IR, or at least are no different from articles. As we demonstrate, the second claim is demonstrably false. In important aspects the work published in top book presses is very different from that published in top journals. In books, work presented is far more qualitative, more reflective of the paradigmatic composition of the field, and more engaged with policy. Additional questions arise from the book coding results.
First, if the books published with leading presses are more representative of the field (at least as shown by survey data) than in journals in regard to methodology, paradigmatic orientation, and policy engagement, why is this? Could this be a product of self-selection, whereby certain kinds of scholars deliberately steer toward books and away from articles? Second, what are the implications for professional incentives in the field? Younger scholars have led the charge toward quantitative methods (Maliniak et al. Reference Maliniak, Oakes, Peterson and Tierney2011, 454), perhaps because they believe they need to be published in quantitatively inclined top journals to secure jobs. Are these scholars simultaneously reducing their chances of being published with top book presses and perhaps their chances of tenure? Finally, is the subdiscipline of IR representative of political science in terms of the divide between books and articles? Like IR, what we think we know about political science is based on articles; we may be ensnared by myths that provide a highly misleading impression of the basic features of the discipline.
