In an age of diminished resources for higher education, the ranking of programs takes on special significance, particularly for programs that rate poorly in the eyes of their peers. Before cost-conscious administrators use the apparent precision of the National Research Council's 1995 ratings to justify rewarding highly rated programs, and penalizing those that fared less well, an analysis of the factors that contribute to ratings success—particularly department size—draws attention to the importance of factors that lie outside the range of departmental control.

Our motivation is only partly ‘academic’. As members of a faculty that rated poorly in this exercise (UB political science ranked 72nd out of the 98 programs surveyed on faculty quality and tied for 48th on “effectiveness”), we have a special interest in reminding our colleagues, and especially our administrators, that our less-than-stellar performance reflects a variety of considerations, only some of which are our “fault.”

Rating academic departments as the NRC has done by relying on peer perceptions (nearly 8,000 graduate-faculty members participated in the ratings survey) involves tapping into a complex set of social processes that are not easily measured. However, we contend that the outcome of this process can be modeled successfully and parsimoniously with a six-variable model of the “determinants of ratings success.” We estimate the parameters of this six-variable model in this article, and show that the most statistically robust and important predictor of ratings success—department size—reflects a characteristic of departments that is beyond their control, and has no direct bearing on the quality of the members of the departments concerned, while a second important indicator—the proportion of full professors—is at least in part similarly beyond departmental control. Our confidence that this model taps central features in the ratings game is enhanced by the consistency of its performance in explaining the ratings of other social science departments.

Not surprisingly, the ratings of doctoral programs in the United States by the National Research Council (NRC 1995) are controversial. Some interpret the ratings as an indicator of relative program quality; others view them as little more than a gauge of the size and age of graduate programs, and still others deem them to be simply a popularity barometer.

Employing data assembled in the NRC report, we examine whether the political science program ratings reflect two general sets of characteristics—the size and the productivity of faculty.

All other things equal, program quality should vary directly with faculty size, and indeed size is emphasized as a key explanatory factor in the NRC report. The logic is straightforward. Very small departments lack the means to field a range of graduate courses of sufficient breadth to form the basis of a serious program. Larger departments enjoy the increased resources that allow for greater program depth and breadth. In addition, holding quality concerns constant, larger programs should on average receive higher ratings simply because they include more faculty.

The publication of the latest rating of doctoral programs by the National Research Council (NRC 1995) has sparked great interest among university faculties. For some, the results vindicated years of focused efforts to improve their department's capacity, performance, and image. For others, it generated frustration because of the small change in scores from previous NRC rankings. And for still others, the NRC report occasioned soul-searching or defensive rationalization, including complaints that the reputational rankings did not reflect the “true” quality of the faculty (Magner 1995).

The NRC study conducted surveys of faculty in each of 41 scientific fields to learn about each doctoral-granting faculty's reputation for scholarly quality (93Q) and for the effectiveness of their doctoral program (93E). In addition, the NRC gathered information about the performance of departments (number of publications, citations to the publications of faculty, and external funding) as well as some other characteristics of the faculty (number of faculty, number of full professors) and doctoral student enrollments (number of students enrolled, number of Ph.D.'s granted in recent years). With these data, it is possible to explore the extent to which reputations are responsive to the performance of the faculty as well as to other department characteristics.

Can philosophically inclined political scientists, as well as quantifying behavioralists, use diagrams? Of course … and have at least since Plato depicted Socrates drawing lines in the sand below the walls of Athens. Nothing will more quickly or comprehensively convey the relationships between the elements within a complex idea than a carefully constructed, skillfully rendered schematic diagram.

A few years back, I published a book on American politics (Roelofs 1992b). Some readers found it interesting, but others seemed unable to credit—much less to critique— its central claim: to have presented a detailed, integrated, and comprehensive account of the American political system as a conceptual whole. Apparently, for these readers, my argument was so extended, the elements to be connected within it so numerous (and sometimes individually so strange), they simply lost sight of my overall intent.

In a phenomenological perspective, the American political system, viewed comprehensively, is an Idea, extraordinarily complex, and filled with ingenious devices as well as major contradictions and paradoxes. Yet it is still a single system, as the diagram I have now developed to illustrate the argument of my book (Figure 1) attempts to show.