The ability of news media to report on events and opinions that are critical of the executive branch of government is central to media freedom and a marker of meaningful democratization. Existing indices use scoring criteria or expert surveys to develop country year measures of media criticism. In this article, we introduce a computationally inexpensive and fully open-source method for estimating media criticism from news articles using à la carte (ALC) word embeddings. We validate our approach using Arabic-language news media published during the Arab Spring. An applied example demonstrates how our technique generates credible estimates of changes in media criticism after a democratic transition is ended by a military coup. Experiments demonstrate the method works even with sparse data. Analyses of synthetic news media demonstrate that the method extends to multiple languages. Our approach points to new possibilities in the monitoring of media freedom within authoritarian and democratizing settings.

Moralising language is a powerful rhetorical tool for signaling political identity, persuading audiences, and mobilising voters. The valid and reliable classification of moral language is therefore a critical objective for political scientists. Recent advances in automated text analysis have introduced myriad new strategies for measuring morality in language, but have often produced conflicting, inconclusive findings. We investigate whether this diversity of moral content analyses might partially explain inconclusive findings, using a large corpus of political manifestos in four different languages (N=810 manifestos). Our results show that, despite starting from the same framework of Moral Foundations Theory (MFT), different instruments and underlying methodologies lead to remarkably different results for extracting moral foundations. Reproducing a previous study on political parties’ ideology and their use of moral foundations, we find that different measurements can lead to opposite effect directions. We discuss the relevance of our findings for research at the intersection of politics and moral rhetoric using automated text analysis.

Codebooks—documents that operationalize concepts and outline annotation procedures—are used almost universally by social scientists when coding political texts. To code these texts automatically, researchers are increasingly turning to generative large language models (LLMs). However, there is limited empirical evidence on whether “off-the-shelf” LLMs faithfully follow real-world codebook operationalizations and measure complex political constructs with sufficient accuracy. To address this, we gather and curate three real-world political science codebooks—covering protest events, political violence, and manifestos—along with their unstructured texts and human-coded labels. We also propose a five-stage framework for codebook-LLM measurement: Preparing a codebook for both humans and LLMs, testing LLMs’ basic capabilities on a codebook, evaluating zero-shot measurement accuracy (i.e., off-the-shelf performance), analyzing errors, and further (parameter-efficient) supervised training of LLMs. We provide an empirical demonstration of this framework using our three codebook datasets and several pre-trained 7–12 billion open-weight LLMs. We find current open-weight LLMs have limitations in following codebooks zero-shot, but that supervised instruction-tuning can substantially improve performance. Rather than suggesting the “best” LLM, our contribution lies in our codebook datasets, evaluation framework, and guidance for applied researchers who wish to implement their own codebook-LLM measurement projects.

Pretrained text embeddings are a fast and scalable method for determining whether two texts have similar meaning, capturing not only lexical similarity, but semantic similarity as well. In this article, I show how to incorporate these measures into a probabilistic record linkage procedure that yields considerable improvements in both precision and recall over existing methods. The procedure even allows researchers to link datasets across different languages. I validate the approach with a series of political science applications, and provide open-source statistical software for researchers to efficiently implement the proposed method.

Understanding network influence and its determinants are key challenges in political science and network analysis. Traditional latent variable models position actors within a social space based on network dependencies but often do not elucidate the underlying factors driving these interactions. To overcome this limitation, we propose the social influence regression (SIR) model, an extension of vector autoregression tailored for relational data that incorporates exogenous covariates into the estimation of influence patterns. The SIR model captures influence dynamics via a pair of $n \times n$ matrices that quantify how the actions of one actor affect the future actions of another. This framework not only provides a statistical mechanism for explaining actor influence based on observable traits but also improves computational efficiency through an iterative block coordinate descent method. We showcase the SIR model’s capabilities by applying it to monthly conflict events between countries, using data from the Integrated Crisis Early Warning System (ICEWS). Our findings demonstrate the SIR model’s ability to elucidate complex influence patterns within networks by linking them to specific covariates. This paper’s main contributions are: (1) introducing a model that explains third-order dependencies through exogenous covariates and (2) offering an efficient estimation approach that scales effectively with large, complex networks.

Spatial voting models are widely used in political science to analyze legislators’ preferences and voting behavior. Traditional models assume that legislators’ ideal points are static across different types of votes. This article extends the Bayesian spatial voting model to incorporate hierarchical Bayesian methods, allowing for the identification of covariates that explain differences in legislators’ ideal points across voting domains. We apply this model to procedural and final passage votes in the U.S. House of Representatives from the 93rd through 113th Congresses. Our findings indicate that legislators in the minority party and those representing moderate constituencies are more likely to exhibit different ideal points between procedural and final passage votes. This research advances the methodology of ideal point estimation by simultaneously scaling ideal points and explaining variation in these points, providing a more nuanced understanding of legislative voting behavior.

This article develops the first dynamic method for systematically estimating the ideologies and other traits of nearly the entire federal judiciary. The Jurist-Derived Judicial Ideology Scores (JuDJIS) method derives from computational text analysis of over 20,000 written evaluations by a representative sample of tens of thousands of jurists as part of an ongoing, systematic survey initiative begun in 1985. The resulting data constitute not only the first such comprehensive federal-court measure that is dynamic, but also the only such measure that is based on judging, and the only such measure that is potentially multi-dimensional. The results of empirical validity tests reflect these advantages. Validation on a set of several-thousand appellate decisions indicates that the ideology estimates predict outcomes significantly more accurately than the existing appellate measures, such as the Judicial Common Space. In addition to informing theoretical debates about the nature of judicial ideology and decision-making, the JuDJIS initiative might lead courts scholars to revisit some of the lower-court research findings of the last two decades, which are generally based on static, non-judicial models. Perhaps most importantly, this method could foster breakthroughs in courts research that, until now, were impossible due to data limitations.

People regularly get information about the political world in visual form, such as graphs of past economic growth, nonverbal cues from politicians, or projections of future climate change. Visual characteristics affect people’s preferences, but it is difficult to measure the extent of this effect precisely and concisely in surveys. We present a new adaptive design that measures the impact of visual characteristics on people’s preferences: The plot staircase. We apply it to graphs of time series data, identifying the effect of the slope of a sequence on evaluations of the sequence. The plot staircase replicates the existing finding that people have a strong preference for increasing trends. Using fewer survey questions than past approaches, it measures at the individual level how much overall welfare a survey respondent is willing to sacrifice for an increasing trend. We demonstrate the flexibility of the plot staircase across domains (economic growth, jobs creation, and the COVID-19 vaccine rollout) and across sequence characteristics. Survey measurement is more difficult for concepts that cannot be represented textually or numerically; our method enables researchers to measure preferences for graphical properties not reducible to the individual pieces of information.