We show that, in some ranked ballot elections, it may be possible to violate the secret vote. There are so many ways to rank even a handful of candidates that many possible rankings might not be cast by any voter. So, a vote buyer could pay someone to rank the candidates a certain way and then use the announced election results to verify that the voter followed through. We examine the feasibility of this attack both theoretically and empirically, focusing on instant runoff voting (IRV). Although many IRV elections have few enough candidates that this scheme is not feasible, we use data from San Francisco and a proposed election rule change in Oakland to show that some important IRV elections can have large numbers of unused rankings. There is no evidence that this vote-buying scheme has ever been used. However, its existence has implications for the administration and security of IRV elections. This scheme is more feasible when more candidates can be ranked in the election and when the election results report all the ways that candidates were ranked.

Supervised machine learning is an increasingly popular tool for analyzing large political text corpora. The main disadvantage of supervised machine learning is the need for thousands of manually annotated training data points. This issue is particularly important in the social sciences where most new research questions require new training data for a new task tailored to the specific research question. This paper analyses how deep transfer learning can help address this challenge by accumulating “prior knowledge” in language models. Models like BERT can learn statistical language patterns through pre-training (“language knowledge”), and reliance on task-specific data can be reduced by training on universal tasks like natural language inference (NLI; “task knowledge”). We demonstrate the benefits of transfer learning on a wide range of eight tasks. Across these eight tasks, our BERT-NLI model fine-tuned on 100 to 2,500 texts performs on average 10.7 to 18.3 percentage points better than classical models without transfer learning. Our study indicates that BERT-NLI fine-tuned on 500 texts achieves similar performance as classical models trained on around 5,000 texts. Moreover, we show that transfer learning works particularly well on imbalanced data. We conclude by discussing limitations of transfer learning and by outlining new opportunities for political science research.

Political scientists commonly use Grambsch and Therneau’s (1994, Biometrika 81, 515–526) ubiquitous Schoenfeld-based test to diagnose proportional hazard violations in Cox duration models. However, some statistical packages have changed how they implement the test’s calculation. The traditional implementation makes a simplifying assumption about the test’s variance–covariance matrix, while the newer implementation does not. Recent work suggests the test’s performance differs, depending on its implementation. I use Monte Carlo simulations to more thoroughly investigate whether the test’s implementation affects its performance. Surprisingly, I find the newer implementation performs very poorly with correlated covariates, with a false positive rate far above 5%. By contrast, the traditional implementation has no such issues in the same situations. This shocking finding raises new, complex questions for researchers moving forward. It appears to suggest, for now, researchers should favor the traditional implementation in situations where its simplifying assumption is likely met, but researchers must also be mindful that this implementation’s false positive rate can be high in misspecified models.

A recent paper by Häffner et al. (2023, Political Analysis 31, 481–499) introduces an interpretable deep learning approach for domain-specific dictionary creation, where it is claimed that the dictionary-based approach outperforms finetuned language models in predictive accuracy while retaining interpretability. We show that the dictionary-based approach’s reported superiority over large language models, BERT specifically, is due to the fact that most of the parameters in the language models are excluded from finetuning. In this letter, we first discuss the architecture of BERT models, then explain the limitations of finetuning only the top classification layer, and lastly we report results where finetuned language models outperform the newly proposed dictionary-based approach by 27% in terms of $R^2$ and 46% in terms of mean squared error once we allow these parameters to learn during finetuning. Researchers interested in large language models, text classification, and text regression should find our results useful. Our code and data are publicly available.

We provide closed-form solutions for measuring electoral closeness of candidates in proportional representation (PR) systems. In contrast to plurality systems, closeness in PR systems cannot be directly inferred from votes. Our measure captures electoral closeness for both open- and closed-list systems and for both main families of seat allocation mechanisms. This unified measure quantifies the vote surplus (shortfall) for elected (nonelected) candidates. It can serve as an assignment variable in regression discontinuity designs or as a measure of electoral competitiveness. For illustration, we estimate the incumbency advantage for the parliaments in Switzerland, Honduras, and Norway.

Theoretical expectations regarding communication patterns between legislators and outside agents, such as lobbyists, agency officials, or policy experts, often depend on the relationship between legislators’ and agents’ preferences. However, legislators and nonelected outside agents evaluate the merits of policies using distinct criteria and considerations. We develop a measurement method that flexibly estimates the policy preferences for a class of outside agents—witnesses in committee hearings—separate from that of legislators’ and compute their preference distance across the two dimensions. In our application to Medicare hearings, we find that legislators in the U.S. Congress heavily condition their questioning of witnesses on preference distance, showing that legislators tend to seek policy information from like-minded experts in committee hearings. We do not find this result using a conventional measurement placing both actors on one dimension. The contrast in results lends support for the construct validity of our proposed preference measures.

Who joins extremist movements? Answering this question is beset by methodological challenges as survey techniques are infeasible and selective samples provide no counterfactual. Recruits can be assigned to contextual units, but this is vulnerable to problems of ecological inference. In this article, we elaborate a technique that combines survey and ecological approaches. The Bayesian hierarchical case–control design that we propose allows us to identify individual-level and contextual factors patterning the incidence of recruitment to extremism, while accounting for spatial autocorrelation, rare events, and contamination. We empirically validate our approach by matching a sample of Islamic State (ISIS) fighters from nine MENA countries with representative population surveys enumerated shortly before recruits joined the movement. High-status individuals in their early twenties with college education were more likely to join ISIS. There is more mixed evidence for relative deprivation. The accompanying extremeR package provides functionality for applied researchers to implement our approach.

Used by politicians, journalists, and citizens, Twitter has been the most important social media platform to investigate political phenomena such as hate speech, polarization, or terrorism for over a decade. A high proportion of Twitter studies of emotionally charged or controversial content limit their ability to replicate findings due to incomplete Twitter-related replication data and the inability to recrawl their datasets entirely. This paper shows that these Twitter studies and their findings are considerably affected by nonrandom tweet mortality and data access restrictions imposed by the platform. While sensitive datasets suffer a notably higher removal rate than nonsensitive datasets, attempting to replicate key findings of Kim’s (2023, Political Science Research and Methods 11, 673–695) influential study on the content of violent tweets leads to significantly different results. The results highlight that access to complete replication data is particularly important in light of dynamically changing social media research conditions. Thus, the study raises concerns and potential solutions about the broader implications of nonrandom tweet mortality for future social media research on Twitter and similar platforms.

In the early weeks of the 2020 coronavirus (COVID-19) pandemic, the Fox News Channel advanced a skeptical narrative that downplayed the risks posed by the virus. We find that this narrative had significant consequences: in localities with higher Fox News viewership—exogenous due to random variation in channel positioning—people were less likely to adopt behaviors geared toward social distancing (e.g., staying at home) and consumed fewer goods in preparation (e.g., cleaning products, hand sanitizers, and masks). Using original survey data, we find that the effect of Fox News came not merely from its long-standing distrustful stance toward science, but also due to program-specific content that minimized the COVID-19 threat. Taken together, our results demonstrate the significant impact that misinformation in media coverage can exert on viewers’ beliefs and behavior, even in high-stakes situations.

Social scientists have become increasingly interested in how narratives—the stories in fiction, politics, and life—shape beliefs, behavior, and government policies. This paper provides an unsupervised method to quantify latent narrative structures in text documents. Our new software package relatio identifies coherent entity groups and maps explicit relations between them in the text. We provide an application to the U.S. Congressional Record to analyze political and economic narratives in recent decades. Our analysis highlights the dynamics, sentiment, polarization, and interconnectedness of narratives in political discourse.

Survey experiments on probability samples are a popular method for investigating population-level causal questions due to their strong internal validity. However, lower survey response rates and an increased reliance on online convenience samples raise questions about the generalizability of survey experiments. We examine this concern using data from a collection of 50 survey experiments which represent a wide range of social science studies. Recruitment for these studies employed a unique double sampling strategy that first obtains a sample of “eager” respondents and then employs much more aggressive recruitment methods with the goal of adding “reluctant” respondents to the sample in a second sampling wave. This approach substantially increases the number of reluctant respondents who participate and also allows for straightforward categorization of eager and reluctant survey respondents within each sample. We find no evidence that treatment effects for eager and reluctant respondents differ substantially. Within demographic categories often used for weighting surveys, there is also little evidence of response heterogeneity between eager and reluctant respondents. Our results suggest that social science findings based on survey experiments, even in the modern era of very low response rates, provide reasonable estimates of population average treatment effects among a deeper pool of survey respondents in a wide range of settings.

A standard text-as-data workflow in the social sciences involves identifying a set of documents to be labeled, selecting a random sample of them to label using research assistants, training a supervised learner to label the remaining documents, and validating that model’s performance using standard accuracy metrics. The most resource-intensive component of this is the hand-labeling: carefully reading documents, training research assistants, and paying human coders to label documents in duplicate or more. We show that hand-coding an algorithmically selected rather than a simple-random sample can improve model performance above baseline by as much as 50%, or reduce hand-coding costs by up to two-thirds, in applications predicting (1) U.S. executive-order significance and (2) financial sentiment on social media. We accompany this manuscript with open-source software to implement these tools, which we hope can make supervised learning cheaper and more accessible to researchers.

Survey experiments are an important tool to measure policy preferences. Researchers often rely on the random assignment of policy attribute levels to estimate different types of average marginal effects. Yet, researchers are often interested in how respondents trade-off different policy dimensions. We use a conjoint experiment administered to more than 10,000 respondents in Germany, to study preferences over personal freedoms and public welfare during the COVID-19 crisis. Using a pre-registered structural model, we estimate policy ideal points and indifference curves to assess the conditions under which citizens are willing to sacrifice freedoms in the interest of public well-being. We document broad willingness to accept restrictions on rights alongside sharp heterogeneity with respect to vaccination status. The majority of citizens are vaccinated and strongly support limitations on freedoms in response to extreme conditions—especially, when they vaccinated themselves are exempted from these limitations. The unvaccinated minority prefers no restrictions on freedoms regardless of the severity of the pandemic. These policy packages also matter for reported trust in government, in opposite ways for vaccinated and unvaccinated citizens.

We tackle the problem of simulating seat- and vote-shares for a party system of a given size. We show how these shares can be generated using unordered and ordered Dirichlet distributions. We show that a distribution with a mean vector given by the rule described in Taagepera and Allik (2006, Electoral Studies 25, 696–713) fits real-world data almost as well as a saturated model where there is a parameter for each rank/system size combination.

Instrumental variable (IV) strategies are widely used in political science to establish causal relationships, but the identifying assumptions required by an IV design are demanding, and assessing their validity remains challenging. In this paper, we replicate 67 articles published in three top political science journals from 2010 to 2022 and identify several concerning patterns. First, researchers often overestimate the strength of their instruments due to non-i.i.d. error structures such as clustering. Second, IV estimates are often highly uncertain, and the commonly used t-test for two-stage-least-squares (2SLS) estimates frequently underestimate the uncertainties. Third, in most replicated studies, 2SLS estimates are significantly larger in magnitude than ordinary-least-squares estimates, and their absolute ratio is inversely related to the strength of the instrument in observational studies—a pattern not observed in experimental ones—suggesting potential violations of unconfoundedness or the exclusion restriction in the former. We provide a checklist and software to help researchers avoid these pitfalls and improve their practice.

Many studies exploit close elections in a regression discontinuity framework to identify partisan effects, that is, the effect of having a given party in office on some outcome. We argue that, when conducted on single-member districts, such design may identify a compound effect: the partisan effect, plus the majority status effect, that is, the effect of being represented by a member of the legislative majority. We provide a simple strategy to disentangle the two, and test it with simulations. Finally, we show the empirical relevance of this issue using real data.