Empirical analyses in social science frequently confront quantitative data that are clustered or grouped. To account for group-level variation and improve model fit, researchers will commonly specify either a fixed- or random-effects model. But current advice on which approach should be preferred, and under what conditions, remains vague and sometimes contradictory. This study performs a series of Monte Carlo simulations to evaluate the total error due to bias and variance in the inferences of each model, for typical sizes and types of datasets encountered in applied research. The results offer a typology of dataset characteristics to help researchers choose a preferred model.

This article examines online recruitment via Facebook, Mechanical Turk (MTurk), and Qualtrics panels in India and the United States. It compares over 7300 respondents—1000 or more from each source and country—to nationally representative benchmarks in terms of demographics, political attitudes and knowledge, cooperation, and experimental replication. In the United States, MTurk offers the cheapest and fastest recruitment, Qualtrics is most demographically and politically representative, and Facebook facilitates targeted sampling. The India samples look much less like the population, though Facebook offers broad geographical coverage. We find online convenience samples often provide valid inferences into how partisanship moderates treatment effects. Yet they are typically unrepresentative on such political variables, which has implications for the external validity of sample average treatment effects.

Amazon's Mechanical Turk is widely used for data collection; however, data quality may be declining due to the use of virtual private servers to fraudulently gain access to studies. Unfortunately, we know little about the scale and consequence of this fraud, and tools for social scientists to detect and prevent this fraud are underdeveloped. We first analyze 38 studies and show that this fraud is not new, but has increased recently. We then show that these fraudulent respondents provide particularly low-quality data and can weaken treatment effects. Finally, we provide two solutions: an easy-to-use application for identifying fraud in the existing datasets and a method for blocking fraudulent respondents in Qualtrics surveys.

To what extent do survey experimental treatment effect estimates generalize to other populations and contexts? Survey experiments conducted on convenience samples have often been criticized on the grounds that subjects are sufficiently different from the public at large to render the results of such experiments uninformative more broadly. In the presence of moderate treatment effect heterogeneity, however, such concerns may be allayed. I provide evidence from a series of 15 replication experiments that results derived from convenience samples like Amazon’s Mechanical Turk are similar to those obtained from national samples. Either the treatments deployed in these experiments cause similar responses for many subject types or convenience and national samples do not differ much with respect to treatment effect moderators. Using evidence of limited within-experiment heterogeneity, I show that the former is likely to be the case. Despite a wide diversity of background characteristics across samples, the effects uncovered in these experiments appear to be relatively homogeneous.

Increasing numbers of comparative survey datasets span multiple waves. Moving beyond purely cross-sectional analyses, multilevel longitudinal analyses of such datasets should generate substantively important insights into the political, social and economic correlates of many individual-level outcomes of interest (attitudes, behaviors, etc.). This article describes two simple techniques for extracting such insights, which allow change over time in y to be a function of change over time in x and/or of a time-invariant x. The article presents results from simulation studies that assess the techniques in the presence of complications that are likely to arise with real-world data, and concludes with applications to the issues of generalized social trust and postmaterialist values, using data from World/European Values Surveys.

Fixed effects estimators are frequently used to limit selection bias. For example, it is well known that with panel data, fixed effects models eliminate time-invariant confounding, estimating an independent variable’s effect using only within-unit variation. When researchers interpret the results of fixed effects models, they should therefore consider hypothetical changes in the independent variable (counterfactuals) that could plausibly occur within units to avoid overstating the substantive importance of the variable’s effect. In this article, we replicate several recent studies which used fixed effects estimators to show how descriptions of the substantive significance of results can be improved by precisely characterizing the variation being studied and presenting plausible counterfactuals. We provide a checklist for the interpretation of fixed effects regression results to help avoid these interpretative pitfalls.

Lagged dependent variables (LDVs) have been used in regression analysis to provide robust estimates of the effects of independent variables, but some research argues that using LDVs in regressions produces negatively biased coefficient estimates, even if the LDV is part of the data-generating process. I demonstrate that these concerns are easily resolved by specifying a regression model that accounts for autocorrelation in the error term. This actually implies that more LDV and lagged independent variables should be included in the specification, not fewer. Including the additional lags yields more accurate parameter estimates, which I demonstrate using the same data-generating process scholars had previously used to argue against including LDVs. I use Monte Carlo simulations to show that this specification returns much more accurate coefficient estimates for independent variables (across a wide range of parameter values) than alternatives considered in earlier research. The simulation results also indicate that improper exclusion of LDVs can lead to severe bias in coefficient estimates. While no panacea, scholars should continue to confidently include LDVs as part of a robust estimation strategy.

The mobilization of culturally rooted issues has altered political competition throughout Western Europe. This article analyzes to what extent the mobilization of immigration issues has affected how people identify with politics. Specifically, it analyzes whether voters’ left/right self-identifications over the past 30 years increasingly correspond to cultural rather than economic attitudes. This study uses longitudinal data from the Netherlands between 1980 and 2006 to demonstrate that as time progresses, voters’ left/right self-placements are indeed more strongly determined by anti-immigrant attitudes than by attitudes towards redistribution. These findings show that the issue basis of left/right identification is dynamic in nature and responsive to changes in the political environment.

The science of human rights requires valid comparisons of repression levels across time and space. Though extensive data collection efforts have made such comparisons possible in principle, statistical measures based on simple additive scales made them rare in practice. This article uses a dynamic measurement model that contrasts with current approaches by (1) accounting for the fact that human rights indicators vary in the level of information they provide about the latent level of repression, (2) allowing realistic descriptions of measurement uncertainty in the form of credible intervals and (3) providing a theoretical motivation for modeling temporal dependence in human rights levels. It presents several techniques, which demonstrate that the dynamic ordinal item-response theory model outperforms its static counterpart.

A common causal identification strategy in political science is selection on observables. This strategy assumes one observes a set of covariates that is, after statistical adjustment, sufficient to make treatment status as-if random. Under adjustment methods such as matching or inverse probability weighting, coefficients for control variables are treated as nuisance parameters and are not directly estimated. This is in direct contrast to regression approaches where estimated parameters are obtained for all covariates. Analysts often find it tempting to give a causal interpretation to all the parameters in such regression models—indeed, such interpretations are often central to the proposed research design. In this paper, we ask when we can justify interpreting two or more coefficients in a regression model as causal parameters. We demonstrate that analysts must appeal to causal identification assumptions to give estimates causal interpretations. Under selection on observables, this task is complicated by the fact that more than one causal effect might be identified. We show how causal graphs provide a framework for clearly delineating which effects are presumed to be identified and thus merit a causal interpretation, and which are not. We conclude with a set of recommendations for how researchers should interpret estimates from regression models when causal inference is the goal.

Does globalization affect the demand-side of politics, and if so, how? This paper builds on new developments in trade theory to argue that globalization matters, but that its effects on individuals’ perceptions of labor market risk and policy preferences are more heterogenous than previous research has acknowledged. Globalization exposure increases risk perceptions and demands for social protection among low-skilled individuals, but decreases them among high-skilled individuals. This conditional effect is observationally distinct from classic trade models as well as arguments that deindustrialization or ideology predominantly drive such perceptions and preferences. Analyzing cross-national survey data from 16 European countries and focusing both on trade and offshoring, the empirical analyses support the prediction that exposure to globalization affects high- and low-skilled individuals differently, leading to variation in labor market risk perceptions and policy preferences.

Coalition parties have to reconcile two competing logics: they need to demonstrate unity to govern together, but also have to emphasize their own profile to succeed in elections. We argue that the electoral cycle explains whether unity or differentiation prevails. While differentiation dominates at the beginning and the end of the legislative term in close proximity to elections, compromise dominates the middle of the term when coalition governments focus on enacting a common policy agenda. To test our theoretical claims, we draw on an innovative quantitative text analysis of more than 21,000 press releases published by coalition parties from 2000 until 2010.

This study uses roll-call voting data from 16 legislatures to investigate how the institutional context of politics—such as whether a country is a parliamentary or presidential regime, or has a single-party, coalition or minority government—shapes coalition formation and voting behavior in parliaments. It uses a geometric scaling metric to estimate the “revealed space” in each of these legislatures and a vote-by-vote statistical analysis to identify how much of this space can be explained by government-opposition dynamics as opposed to parties’ (left-right) policy positions. Government-opposition interests, rather than parties’ policy positions, are found to be the main drivers of voting behavior in most institutional contexts. In contrast, issue-by-issue coalition building along a single policy dimension is only found under certain restrictive institutional constraints: presidential regimes with coalition governments or parliamentary systems with minority governments. Put another way, voting in most legislatures is more like Westminster than Washington.

Corruption scandals have been found to have significant but mild electoral effects in the comparative literature (Golden 2006). However, most studies have assumed that voters punish all kinds of illegal practices. This article challenges this assumption by distinguishing between two types of corruption, according to the type of welfare consequences they have for the constituency. This hypothesis is tested using data from the 2011 Spanish local elections. We exploit the abundance of corruption allegations associated with the Spanish housing boom, which generated income gains for a wide segment of the electorate in the short term. We find that voters ignore corruption when there are side benefits to it, and that punishment is only administered in those cases in which they do not receive compensation.

This article challenges Fixed Effects (FE) modeling as the ‘default’ for time-series-cross-sectional and panel data. Understanding different within and between effects is crucial when choosing modeling strategies. The downside of Random Effects (RE) modeling—correlated lower-level covariates and higher-level residuals—is omitted-variable bias, solvable with Mundlak's (1978a) formulation. Consequently, RE can provide everything that FE promises and more, as confirmed by Monte-Carlo simulations, which additionally show problems with Plümper and Troeger's FE Vector Decomposition method when data are unbalanced. As well as incorporating time-invariant variables, RE models are readily extendable, with random coefficients, cross-level interactions and complex variance functions. We argue not simply for technical solutions to endogeneity, but for the substantive importance of context/heterogeneity, modeled using RE. The implications extend beyond political science to all multilevel datasets. However, omitted variables could still bias estimated higher-level variable effects; as with any model, care is required in interpretation.

Recent years have seen a renaissance of conjoint survey designs within social science. To date, however, researchers have lacked guidance on how many attributes they can include within conjoint profiles before survey satisficing leads to unacceptable declines in response quality. This paper addresses that question using pre-registered, two-stage experiments examining choices among hypothetical candidates for US Senate or hotel rooms. In each experiment, we use the first stage to identify attributes which are perceived to be uncorrelated with the attribute of interest, so that their effects are not masked by those of the core attributes. In the second stage, we randomly assign respondents to conjoint designs with varying numbers of those filler attributes. We report the results of these experiments implemented via Amazon's Mechanical Turk and Survey Sampling International. They demonstrate that our core quantities of interest are generally stable, with relatively modest increases in survey satisficing when respondents face large numbers of attributes.

At least two theories have been offered that explain the rise of affective polarization. Some scholars, relying on social identity theory, argue that as the relevance of party identification increased, Americans became more likely to see their in-party in positive terms and the out-party in negative terms. Other scholars argue that affective polarization is a reaction to increasingly extreme political actors. This study seeks to arbitrate between these two theories of affective polarization through a survey experiment which asks respondents to rate candidates whose party (or lack thereof) and ideology (or lack thereof) is randomly assigned. In line with the policy-oriented view of affective polarization, respondents reacted far more strongly to ideology than party, especially if it was the ideology of the member of the out-party.

This article analyzes the impact of economic voting in federal elections for the German parliament. It combines theories of coalition politics and cabinet decision making—like prime ministerial government, collective cabinet decision making and ministerial discretion—with theoretical approaches on voting behavior to test which cabinet actor voters reward for improved economic conditions. The empirical results, which are based on data from German national election studies from 1987–2009, show that the party of the chancellor (and, thus, the strongest coalition party) benefits most from a positive evaluation of economic policy outcomes. There is, however, no consistent empirical evidence that the coalition parties collectively benefit from perceived positive economic performance. Therefore the findings demonstrate that economic voting occurs in German parliamentary elections, but is targeted specifically toward the chancellor's party.

Cluster-robust standard errors (as implemented by the eponymous cluster option in Stata) can produce misleading inferences when the number of clusters G is small, even if the model is consistent and there are many observations in each cluster. Nevertheless, political scientists commonly employ this method in data sets with few clusters. The contributions of this paper are: (a) developing new and easy-to-use Stata and R packages that implement alternative uncertainty measures robust to small G, and (b) explaining and providing evidence for the advantages of these alternatives, especially cluster-adjusted t-statistics based on Ibragimov and Müller. To illustrate these advantages, we reanalyze recent work where results are based on cluster-robust standard errors.

A large literature argues that public opinion is vulnerable to various types of framing and cue effects. However, we lack evidence on whether existing findings, which are typically based on lab experiments involving low-salience issues, travel to salient and contentious political issues in real-world voting situations. We examine the relative importance of issue frames, partisan cues, and their interaction for opinion formation using a survey experiment conducted around a highly politicized referendum on immigration policy in Switzerland. We find that voters responded to frames and cues, regardless of their direction, by increasing support for the position that is in line with their pre-existing partisan attachment. This reinforcement effect was most visible among low knowledgeable voters that identified with the party that owned the issue. These results support some of the previous findings in the political communication literature, but at the same time also point toward possible limits to framing effects in the context of salient and contested policy issues.