Public opinion surveys are vital for informing democratic decision-making, but responding to rapidly changing information environments and measuring beliefs within hard-to-reach communities can be challenging for traditional survey methods. This paper introduces a crowdsourced adaptive survey methodology (CSAS) that unites advances in natural language processing and adaptive algorithms to produce surveys that evolve with participant input. The CSAS method converts open-ended text provided by participants into survey items and applies a multi-armed bandit algorithm to determine which questions should be prioritized in the survey. The method’s adaptive nature allows new survey questions to be explored and imposes minimal costs in survey length. Applications in the domains of misinformation, issue salience, and local politics showcase CSAS’s ability to identify topics that might otherwise escape the notice of survey researchers. I conclude by highlighting CSAS’s potential to bridge conceptual gaps between researchers and participants in survey research.

Most political science studies are, at root, about how people make decisions—how voters choose whether and for whom to vote, how prejudice influences political choices, and the effects of emotions and morals on political choice. However, what people are thinking during these decisions remains obscure; currently utilized methods leave us with a “black box” of decision making. Eye tracking offers a deeper insight into these processes by capturing respondents’ attention, salience, emotion, and understanding. But how applicable is this method to political science questions, and how does one go about using it? Here, we explain what eye tracking allows researchers to measure, how these measures are relevant to political science questions, and how political scientists without expertise in the method can nonetheless use it effectively. In particular, we clarify how researchers can understand the choices made in preset software in order to arrive at correct inferences from their data and discuss new developments in eye tracking methodology, including webcam eye tracking. We additionally provide templates for preregistering eye tracking studies in political science, as well as starter code for processing and analyzing eye tracking data.

Scholars often face a choice when designing conjoint experiments: to allow for or to exclude “odd” combinations of attribute levels in the randomized conjoint profiles shown to respondents (such as a profile of a Democratic candidate who does not support abortion rights or an individual who is a medical doctor but does not have a graduate degree). While previous work has studied the statistical and theoretical implications of this decision, there has been little effort to analyze how it impacts the behavior of survey respondents. Utilizing eye-tracking, this study considers how respondents’ attention, information search behavior, and choice patterns respond to odd combinations of attributes included in conjoint profiles. We find that the impact of odd attribute-level combinations is minimal. They do not impact attention, search, or choice behavior substantially or consistently. Our conclusion is that scholars should prioritize other considerations—such as statistical, theoretical, and substantive considerations—when designing conjoint experiments.

Social scientists often use ranking questions to study people’s opinions and preferences. However, little is understood about the general nature of measurement errors in such questions, let alone their statistical consequences and what researchers can do about them. We introduce a statistical framework to improve ranking data analysis by addressing measurement errors in ranking questions. First, we characterize measurement errors from random responses—arbitrary and meaningless responses based on a wide range of random patterns. We then quantify bias due to random responses, show that the bias may change our conclusion in any direction, and clarify why item order randomization alone does not solve the statistical issue. Next, we introduce our methodology based on two key design-based considerations: item order randomization and the addition of an “anchor” ranking question with known correct answers. They allow researchers to (1) learn about the direction of the bias and (2) estimate the proportion of random responses, enabling our bias-corrected estimators. We illustrate our methods by studying the relative importance of people’s partisan identity compared to their racial, gender, and religious identities in American politics. We find that about 30% of respondents offered random responses and that these responses may affect our substantive conclusions.

Conditioning on variables affected by treatment can induce post-treatment bias when estimating causal effects. Although this suggests that researchers should measure potential moderators before administering the treatment in an experiment, doing so may also bias causal effect estimation if the covariate measurement primes respondents to react differently to the treatment. This paper formally analyzes this trade-off between post-treatment and priming biases in three experimental designs that vary when moderators are measured: pre-treatment, post-treatment, or a randomized choice between the two. We derive nonparametric bounds for interactions between the treatment and the moderator under each design and show how to use substantive assumptions to narrow these bounds. These bounds allow researchers to assess the sensitivity of their empirical findings to priming and post-treatment bias. We then apply the proposed methodology to a survey experiment on electoral messaging.

We investigate a bracketing property that purports to yield upper- and lower bounds on the treatment effects obtained from a fixed effects (FE) and lagged dependent variable (LDV) model. Referencing both analytical results and a Monte Carlo simulation, we explore the conditions under which the bracketing property holds, confirming this to be the case when the data generating process (DGP) is characterized by either unobserved heterogeneity or feedback effects from a lagged dependent variable. However, when the DGP is characterized by both features simultaneously, we find that bracketing of the treatment effect only holds under certain conditions—but not in general. Practitioners can nevertheless obtain the lower bound estimate by referencing a model that includes both FE and an LDV. While the Nickell bias in the coefficient of the LDV is known to be of order $1/T$, we show that the Nickell-type bias in the estimator of the treatment effect is of order $1/T^2$.

Many influential political science articles use close elections to study how important outcomes vary after a certain type of candidate wins, such as a Democrat or a Republican. This politician characteristic regression discontinuity (PCRD) design offers opportunities for inferential leverage but also the potential for confusion. In this article, we clarify what causal claims the PCRD licenses, offering a rigorous causal analysis that points to three principal lessons. First, PCRDs do nothing to isolate the effect of the politician characteristic of interest as apart from other politician characteristics. Second, selection processes (regarding both “who runs” and “which elections are close”) can generate and exacerbate such confounding, as noted in Marshall (2024). Third and more fortunately, this approach does make it possible to estimate the average effect of electing a leader of type “A” vs. “B” in the context of close elections, treating the units as districts, not leaders. We also suggest a set of tools that can aid in falsifying key assumptions, avoiding unwarranted claims, and surfacing mechanisms of interest. We illustrate these issues and tools through a reanalysis of an influential study about what happens when extremists win primaries (Hall 2015).

Building on the availability of geospatial data, improvements in mapping software, and innovations in spatial statistics, political scientists are increasingly taking geography seriously. As we adopt the tools of geographers, we must also consider the methodological challenges they have identified. We focus on the modifiable areal unit problem (MAUP)—the idea that the size of aggregate spatial units and the location of their borders affect the empirical results we obtain. We first describe the logic of the MAUP, and then demonstrate the MAUP through simulations, showing MAUP-related inconsistency in regression results in randomly generated and real-world data. We identify MAUP concerns, and best practices, in top journals in political science. We conclude by suggesting how scholars may respond in theoretical and empirical terms to concerns about validity and reliability that arise from the MAUP.

While researchers often study message features like moral content in text, such as party manifestos and social media posts, their quantification remains a challenge. Conventional human coding struggles with scalability and intercoder reliability. While dictionary-based methods are cost-effective and computationally efficient, they often lack contextual sensitivity and are limited by the vocabularies developed for the original applications. In this paper, we present an approach to construct “vec-tionaries” that boost validated dictionaries with word embeddings through nonlinear optimization. By harnessing semantic relationships encoded by embeddings, vec-tionaries improve the measurement of message features from text, especially those in short format, by expanding the applicability of original vocabularies to other contexts. Importantly, a vec-tionary can produce additional metrics to capture the valence and ambivalence of a message feature beyond its strength in texts. Using moral content in tweets as a case study, we illustrate the steps to construct the moral foundations vec-tionary, showcasing its ability to process texts missed by conventional dictionaries and to produce measurements better aligned with crowdsourced human assessments. Furthermore, additional metrics from the vec-tionary unveiled unique insights that facilitated predicting downstream outcomes such as message retransmission.

If instant runoff voting (IRV) mitigates strategic voting, is that because the rules of the system mechanically reduce strategic opportunities, or because of other more indirect effects? In single-vote plurality elections, a voter can be pivotal if adding one vote to a candidate would cause that candidate to win. In IRV, it is more complicated to identify when one voter can be pivotal. This letter derives all the ways that a single ballot can change the result of an IRV election, for any number of candidates and voters. I obtain an expression for the probability of casting a pivotal vote in IRV by phrasing the probability that any pivotal event occurs as a function of all the rankings cast by other voters. This expression facilitates modeling vote choice in IRV, and enables the estimation of voters’ strategic opportunities in IRV contests between any number of candidates. I present some illustrative simulations estimating pivotal probabilities in both IRV and Single-Member District Plurality for stylized electorates with identical preference structures. These simulations produce similar estimated pivotal probabilities in the two systems, suggesting that these systems may provide similar opportunities to strategically cast a decisive vote.

We perform the first mapping of the ideological positions of European parties using generative Artificial Intelligence (AI) as a “zero-shot” learner. We ask OpenAI’s Generative Pre-trained Transformer 3.5 (GPT-3.5) to identify the more “right-wing” option across all possible duplets of European parties at a given point in time, solely based on their names and country of origin, and combine this information via a Bradley–Terry model to create an ideological ranking. A cross-validation employing widely-used expert-, manifesto- and poll-based estimates reveals that the ideological scores produced by Large Language Models (LLMs) closely map those obtained through the expert-based evaluation, i.e., CHES. Given the high cost of scaling parties via trained coders, and the scarcity of expert data before the 1990s, our finding that generative AI produces estimates of comparable quality to CHES supports its usage in political science on the grounds of replicability, agility, and affordability.