The study of bureaucracies and their relationship to political actors is central to understanding the policy process in the United States. Studying this aspect of American politics is difficult because theories of agency behavior, effectiveness, and control often require measures of administrative agencies' policy preferences, and appropriate measures are hard to find for a broad spectrum of agencies. We propose a method for measuring agency preferences based upon an expert survey of agency preferences for 82 executive agencies in existence between 1988 and 2005. We use a multirater item response model to provide a principled structure for combining subjective ratings based on scholarly and journalistic expertise with objective data on agency characteristics. We compare the resulting agency preference estimates and standard errors to existing alternative measures, discussing both the advantages and limitations of the method.

We present a simple method for estimating regressions based on recursive extensive-form games. Our procedure, which can be implemented in most standard statistical packages, involves sequentially estimating standard logits (or probits) in a manner analogous to backwards induction. We demonstrate that the technique produces consistent parameter estimates and show how to calculate consistent standard errors. To illustrate the method, we replicate Leblang's (2003) study of speculative attacks by financial markets and government responses to these attacks.

Ecological inference is a statistical problem where aggregate-level data are used to make inferences about individual-level behavior. In this article, we conduct a theoretical and empirical study of Bayesian and likelihood inference for 2 × 2 ecological tables by applying the general statistical framework of incomplete data. We first show that the ecological inference problem can be decomposed into three factors: distributional effects, which address the possible misspecification of parametric modeling assumptions about the unknown distribution of missing data; contextual effects, which represent the possible correlation between missing data and observed variables; and aggregation effects, which are directly related to the loss of information caused by data aggregation. We then examine how these three factors affect inference and offer new statistical methods to address each of them. To deal with distributional effects, we propose a nonparametric Bayesian model based on a Dirichlet process prior, which relaxes common parametric assumptions. We also identify the statistical adjustments necessary to account for contextual effects. Finally, although little can be done to cope with aggregation effects, we offer a method to quantify the magnitude of such effects in order to formally assess its severity. We use simulated and real data sets to empirically investigate the consequences of these three factors and to evaluate the performance of our proposed methods. C code, along with an easy-to-use R interface, is publicly available for implementing our proposed methods (Imai, Lu, and Strauss, forthcoming).

Models of ecological inference (EI) have to rely on crucial assumptions about the individual-level data-generating process, which cannot be tested because of the unavailability of these data. However, these assumptions may be violated by the unknown data and this may lead to serious bias of estimates and predictions. The amount of bias, however, cannot be assessed without information that is unavailable in typical applications of EI. We therefore construct a model that at least approximately accounts for the additional, nonsampling error that may result from possible bias incurred by an EI procedure, a model that builds on the Principle of Maximum Entropy. By means of a systematic simulation experiment, we examine the performance of prediction intervals based on this second-stage Maximum Entropy model. The results of this simulation study suggest that these prediction intervals are at least approximately correct if all possible configurations of the unknown data are taken into account. Finally, we apply our method to a real-world example, where we actually know the true values and are able to assess the performance of our method: the prediction of district-level percentages of split-ticket voting in the 1996 General Election of New Zealand. It turns out that in 95.5% of the New Zealand voting districts, the actual percentage of split-ticket votes lies inside the 95% prediction intervals constructed by our method.

In a recent article in the American Political Science Review, Laver, Benoit, and Garry (2003, “Extracting policy positions from political texts using words as data,” 97:311—331) propose a new method for conducting content analysis. Their Wordscores approach, by automating text-coding procedures, represents an advance in content analysis that will potentially have a large long-term impact on research across the discipline. To allow substantive interpretation, the scores produced by the Wordscores procedure require transformation. In this note, we address several shortcomings in the transformation procedure introduced in the original program. We demonstrate that the original transformation distorts the metric on which content scores are placed—hindering the ability of scholars to make meaningful comparisons across texts—and that it is very sensitive to the texts that are scored—opening up the possibility that researchers may generate, inadvertently or not, results that depend on the texts they choose to include in their analyses. We propose a transformation procedure that solves these problems.

In “A Robust Transformation Procedure,” Martin and Vanberg (2007, hereafter MV) propose a new method for rescaling the raw virgin text scores produced by the “Wordscores” procedure of Laver, Benoit, and Garry (2003, hereafter LBG). Their alternative method addresses two deficiencies they argue exist with the transformation of virgin text scores proposed by LBG: First, that the LBG transformation is sensitive to the selection of virgin texts, and second, that it distorts the reference metric by failing to recover the original reference scores when reference texts are scored and transformed as if they were virgin texts. Their proposed alternative is “robust” in the sense that it avoids both shortcomings. Not only is MV's transformation a welcome contribution to the Wordscores project but also the critical analysis on which it is based brings to light a number of assumptions and choices that face the analyst seeking to estimate actors' policy positions using statistical analyses of the texts they generate. When first describing the possibility of rescaling the raw virgin text estimates, we emphasized that our

We appreciate the positive reception of our transformation by Benoit and Laver (hereafter, BL), and we are grateful that they have incorporated it into the Wordscores package. Because their comment highlights a fundamental difference between the Martin-Vanberg (MV) and Laver-Benoit-Garry (LBG) approaches that is critical to the choice among transformations, we offer some brief comments that will allow users to make an informed decision regarding the appropriate use of the transformations. The central issue concerns comparisons between reference and virgin texts. As BL point out, researchers will often be interested in making such comparisons, and the LBG and MV transformations can yield substantially different results. In light of these differences, BL's primary suggestion is to focus analysis on the raw scores, which can be obtained for reference as well as virgin texts. We wholeheartedly agree with this prescription. In fact, it is precisely a concern for faithfully reporting the raw score information, while making it more intuitive, that motivates the MV transformation. As we show below, the MV transformation accurately reflects all and nothing but the information contained in raw scores. Therefore, “users [who] get eye strain” by looking at raw scores can safely substitute MV scores and be confident that the information provided is equivalent. The same will typically not be true of LBG scores.