Many recent papers in political science and economics use rainfall as a strategy to facilitate causal inference. Rainfall shocks are as-if randomly assigned, but the assignment of rainfall by county is highly correlated across space. Since clustered assignment does not occur within well-defined boundaries, it is challenging to estimate the variance of the effect of rainfall on political outcomes. I propose using randomization inference with historical weather patterns from 73 years as potential randomizations. I replicate the influential work on rainfall and voter turnout in presidential elections in the United States by Gomez, Hansford, and Krause (2007) and compare the estimated average treatment effect (ATE) to a sampling distribution of estimates under the sharp null hypothesis of no effect. The alternate randomizations are random draws from national rainfall patterns on election and would-be election days, which preserve the clustering in treatment assignment and eliminate the need to simulate weather patterns or make assumptions about unit boundaries for clustering. I find that the effect of rainfall on turnout is subject to greater sampling variability than previously estimated using conventional standard errors.

We introduce a Bayesian method, LASSOplus, that unifies recent contributions in the sparse modeling literatures, while substantially extending pre-existing estimators in terms of both performance and flexibility. Unlike existing Bayesian variable selection methods, LASSOplus both selects and estimates effects while returning estimated confidence intervals for discovered effects. Furthermore, we show how LASSOplus easily extends to modeling repeated observations and permits a simple Bonferroni correction to control coverage on confidence intervals among discovered effects. We situate LASSOplus in the literature on how to estimate subgroup effects, a topic that often leads to a proliferation of estimation parameters. We also offer a simple preprocessing step that draws on recent theoretical work to estimate higher-order effects that can be interpreted independently of their lower-order terms. A simulation study illustrates the method’s performance relative to several existing variable selection methods. In addition, we apply LASSOplus to an existing study on public support for climate treaties to illustrate the method’s ability to discover substantive and relevant effects. Software implementing the method is publicly available in the R package sparsereg.

The large majority of inferences drawn in empirical political research follow from model-based associations (e.g., regression). Here, we articulate the benefits of predictive modeling as a complement to this approach. Predictive models aim to specify a probabilistic model that provides a good fit to testing data that were not used to estimate the model’s parameters. Our goals are threefold. First, we review the central benefits of this under-utilized approach from a perspective uncommon in the existing literature: we focus on how predictive modeling can be used to complement and augment standard associational analyses. Second, we advance the state of the literature by laying out a simple set of benchmark predictive criteria. Third, we illustrate our approach through a detailed application to the prediction of interstate conflict.

I develop a novel method to detect election fraud from irregular patterns in the distribution of vote-shares. I build on a widely discussed observation that in some elections where fraud allegations abound, suspiciously many polling stations return coarse vote-shares (e.g., 0.50, 0.60, 0.75) for the ruling party, which seems highly implausible in large electorates. Using analytical results and simulations, I show that sheer frequency of such coarse vote-shares is entirely plausible due to simple numeric laws and does not by itself constitute evidence of fraud. To avoid false positive errors in fraud detection, I propose a resampled kernel density method (RKD) to measure whether the coarse vote-shares occur too frequently to raise a statistically qualified suspicion of fraud. I illustrate the method on election data from Russia and Canada as well as simulated data. A software package is provided for an easy implementation of the method.

Mixed-methods designs, especially those where cases selected for small-N analysis (SNA) are nested within a large-N analysis (LNA), have become increasingly popular. Yet, since the LNA in this approach assumes that units are independently distributed, such designs are unable to account for spatial dependence, and dependence becomes a threat to inference, rather than an issue for empirical or theoretical investigation. This is unfortunate, since research in political science has recently drawn attention to diffusion and interconnectedness more broadly. In this paper we develop a framework for mixed-methods research with spatially dependent data—a framework we label “geo-nested analysis”—where insights gleaned at each step of the research process set the agenda for the next phase and where case selection for SNA is based on diagnostics of a spatial-econometric analysis. We illustrate our framework using data from a seminal study of homicides in the United States.

Randomized experiments are increasingly used to study political phenomena because they can credibly estimate the average effect of a treatment on a population of interest. But political scientists are often interested in how effects vary across subpopulations—heterogeneous treatment effects—and how differences in the content of the treatment affects responses—the response to heterogeneous treatments. Several new methods have been introduced to estimate heterogeneous effects, but it is difficult to know if a method will perform well for a particular data set. Rather than using only one method, we show how an ensemble of methods—weighted averages of estimates from individual models increasingly used in machine learning—accurately measure heterogeneous effects. Building on a large literature on ensemble methods, we show how the weighting of methods can contribute to accurate estimation of heterogeneous treatment effects and demonstrate how pooling models lead to superior performance to individual methods across diverse problems. We apply the ensemble method to two experiments, illuminating how the ensemble method for heterogeneous treatment effects facilitates exploratory analysis of treatment effects.

Over the past decade, a number of new measures have been developed that attempt to capture the political orientation of both incumbent and nonincumbent candidates for Congress, as well as other offices, on the same scale. These measures pose the possibility of being able to answer a host of fundamental questions about political accountability and representation. In this paper, we examine the properties of six recent measures of candidates’ political orientations in different domains. While these measures are commonly viewed as proxies for ideology, each involves very different choices, incentives, and contexts. Indeed, we show that there is only a weak relationship between these measures within party. This suggests that these measures are capturing domain-specific factors rather than just candidates’ ideology. Moreover, these measures do poorly at distinguishing between moderate and extreme roll call voting records within each party. As a result, they fall short when it comes to facilitating empirical analysis of theories of accountability and representation in Congress. Overall, our findings suggest that future research should leverage the conceptual and empirical variation across these measures and avoid assuming they are synonymous with candidates’ ideology.

There is increasing interest in experiments where outcomes are measured by surveys and treatments are delivered by a separate mechanism in the real world, such as by mailers, door-to-door canvasses, phone calls, or online ads. However, common designs for such experiments are often prohibitively expensive, vulnerable to bias, and raise ethical concerns. We show how four methodological practices currently uncommon in such experiments have previously undocumented complementarities that can dramatically relax these constraints when at least two are used in combination: (1) online surveys recruited from a defined sampling frame (2) with at least one baseline wave prior to treatment (3) with multiple items combined into an index to measure outcomes and, (4) when possible, a placebo control. We provide a general and extensible framework that allows researchers to determine the most efficient mix of these practices in diverse applications. Two studies then examine how these practices perform empirically. First, we examine the representativeness of online panel respondents recruited from a defined sampling frame and find that their representativeness compares favorably to phone panel respondents. Second, an original experiment successfully implements all four practices in the context of a door-to-door canvassing experiment. We conclude discussing potential extensions.

Missing outcome data plague many randomized experiments. Common solutions rely on ignorability assumptions that may not be credible in all applications. We propose a method for confronting missing outcome data that makes fairly weak assumptions but can still yield informative bounds on the average treatment effect. Our approach is based on a combination of the double sampling design and nonparametric worst-case bounds. We derive a worst-case bounds estimator under double sampling and provide analytic expressions for variance estimators and confidence intervals. We also propose a method for covariate adjustment using poststratification and a sensitivity analysis for nonignorable missingness. Finally, we illustrate the utility of our approach using Monte Carlo simulations and a placebo-controlled randomized field experiment on the effects of persuasion on social attitudes with survey-based outcome measures.

How do individuals’ influence in a large social network change? Social scientists have difficulty answering this question because measuring influence requires frequent observations of a population of individuals’ connections to each other, while sampling that social network removes information in a way that can bias inferences. This paper introduces a method to measure influence over time accurately from sampled network data. Ranking individuals by the sum of their connections’ connections—neighbor cumulative indegree centrality—preserves the rank influence ordering that would be achieved in the presence of complete network data, lowering the barrier to measuring influence accurately. The paper then shows how to measure that variable changes each day, making it possible to analyze when and why an individual’s influence in a network changes. This method is demonstrated and validated on 21 Twitter accounts in Bahrain and Egypt from early 2011. The paper then discusses how to use the method in domains such as voter mobilization and marketing.

Difference-in-differences (DID) is commonly used for causal inference in time-series cross-sectional data. It requires the assumption that the average outcomes of treated and control units would have followed parallel paths in the absence of treatment. In this paper, we propose a method that not only relaxes this often-violated assumption, but also unifies the synthetic control method (Abadie, Diamond, and Hainmueller 2010) with linear fixed effects models under a simple framework, of which DID is a special case. It imputes counterfactuals for each treated unit using control group information based on a linear interactive fixed effects model that incorporates unit-specific intercepts interacted with time-varying coefficients. This method has several advantages. First, it allows the treatment to be correlated with unobserved unit and time heterogeneities under reasonable modeling assumptions. Second, it generalizes the synthetic control method to the case of multiple treated units and variable treatment periods, and improves efficiency and interpretability. Third, with a built-in cross-validation procedure, it avoids specification searches and thus is easy to implement. An empirical example of Election Day Registration and voter turnout in the United States is provided.

This article presents a new method and open source R package that uses syntactic information to automatically extract source–subject–predicate clauses. This improves on frequency-based text analysis methods by dividing text into predicates with an identified subject and optional source, extracting the statements and actions of (political) actors as mentioned in the text. The content of these predicates can be analyzed using existing frequency-based methods, allowing for the analysis of actions, issue positions and framing by different actors within a single text. We show that a small set of syntactic patterns can extract clauses and identify quotes with good accuracy, significantly outperforming a baseline system based on word order. Taking the 2008–2009 Gaza war as an example, we further show how corpus comparison and semantic network analysis applied to the results of the clause analysis can show differences in citation and framing patterns between U.S. and English-language Chinese coverage of this war.

This study analyzes the political agenda of the European Parliament (EP) plenary, how it has evolved over time, and the manner in which Members of the European Parliament (MEPs) have reacted to external and internal stimuli when making plenary speeches. To unveil the plenary agenda and detect latent themes in legislative speeches over time, MEP speech content is analyzed using a new dynamic topic modeling method based on two layers of Non-negative Matrix Factorization (NMF). This method is applied to a new corpus of all English language legislative speeches in the EP plenary from the period 1999 to 2014. Our findings suggest that two-layer NMF is a valuable alternative to existing dynamic topic modeling approaches found in the literature, and can unveil niche topics and associated vocabularies not captured by existing methods. Substantively, our findings suggest that the political agenda of the EP evolves significantly over time and reacts to exogenous events such as EU Treaty referenda and the emergence of the Euro Crisis. MEP contributions to the plenary agenda are also found to be impacted upon by voting behavior and the committee structure of the Parliament.

We discuss a statistical procedure to carry out empirical research that combines recent insights about preanalysis plans (PAPs) and replication. Researchers send their datasets to an independent third party who randomly generates training and testing samples. Researchers perform their analysis on the training sample and are able to incorporate feedback from both colleagues, editors, and referees. Once the paper is accepted for publication the method is applied to the testing sample and it is those results that are published. Simulations indicate that, under empirically relevant settings, the proposed method delivers more power than a PAP. The effect mostly operates through a lower likelihood that relevant hypotheses are left untested. The method appears better suited for exploratory analyses where there is significant uncertainty about the outcomes of interest. We do not recommend using the method in situations where the treatment are very costly and thus the available sample size is limited. An interpretation of the method is that it allows researchers to perform direct replication of their work. We also discuss a number of practical issues about the method’s feasibility and implementation.

Parties and social movements play an important role in many theories of political science. Yet, the study of intraparty politics remains underdeveloped as random samples are difficult to conduct among political activists. This paper proposes a novel procedure to sample different parties over time and space by utilizing the advertising option of the social media webpage Facebook. As this method allows for quotas and the collection of large samples at relatively low cost, it becomes possible to improve the representativeness through poststratification and subsample robustness checks. Three examples illustrate these advantages of Facebook sampling: First, a Facebook sample approximated intraparty decisions and the outcome of a leadership contest of the Alternative for Germany. Second, a weighted Facebook sample achieved similar estimates as a representative local leader survey of the Social Democratic Party of Germany. Third, by evaluating subgroups of key demographics for parties with unknown population parameters, two Facebook samples show that the color-coded conflict in Thailand was driven by different concepts of regime type, but not by a left–right divide on economic policy-making. Facebook sampling appears to be the best and cheapest method to conduct time-series cross-sectional studies for political activists.

Due to the crucial role of political knowledge in democratic participation, the measurement of political knowledge has been a major concern in the discipline of political science. Common formats used for political knowledge questions include multiple-choice items and open-ended identification questions. The conventional wisdom holds that multiple-choice items induce guessing behavior, which leads to underestimated item-difficulty parameters and biased estimates of political knowledge. This article examines guessing behavior in multiple-choice items and argues that a successful guess requires certain levels of knowledge conditional on the difficulties of items. To deal with this issue, we propose a Bayesian IRT guessing model that accommodates the guessing components of item responses. The proposed model is applied to analyzing survey data in Taiwan, and the results show that the proposed model appropriately describes the guessing components based on respondents’ levels of political knowledge and item characteristics. That is, in general, partially informed respondents are more likely to have a successful guess because well-informed respondents do not need to guess and barely informed ones are highly seducible by the attractive distractors. We also examine the gender gap in political knowledge and find that, even when the guessing effect is accounted for, men are more knowledgeable than women about political affairs, which is consistent with the literature.

Methodologists and substantive scholars alike agree that one of process tracing’s foremost contributions to qualitative research is its capacity to adjudicate among competing explanations of a phenomenon. Existing approaches, however, only provide explicit guidance on dealing with mutually exclusive explanations, which are exceedingly rare in social science research. I develop a tripartite solution to this problem. The Relationships among Rivals (RAR) framework (1) introduces a typology of relationships between alternative hypotheses, (2) develops specific guidelines for identifying which relationship is present between two hypotheses, and (3) maps out the varied implications for evidence collection and inference. I then integrate the RAR framework into each of the main process-tracing approaches and demonstrate how it affects the inferential process. Finally, I illustrate the purchase of the RAR framework by reanalyzing a seminal example of process-tracing research: Schultz’s (2001) analysis of the Fashoda Crisis. I show that the same evidence can yield new and sometimes contradictory inferences once scholars approach comparative hypothesis testing with this more nuanced framework.

Media-based event data—i.e., data comprised from reporting by media outlets—are widely used in political science research. However, events of interest (e.g., strikes, protests, conflict) are often underreported by these primary and secondary sources, producing incomplete data that risks inconsistency and bias in subsequent analysis. While general strategies exist to help ameliorate this bias, these methods do not make full use of the information often available to researchers. Specifically, much of the event data used in the social sciences is drawn from multiple, overlapping news sources (e.g., Agence France-Presse, Reuters). Therefore, we propose a novel maximum likelihood estimator that corrects for misclassification in data arising from multiple sources. In the most general formulation of our estimator, researchers can specify separate sets of predictors for the true-event model and each of the misclassification models characterizing whether a source fails to report on an event. As such, researchers are able to accurately test theories on both the causes of and reporting on an event of interest. Simulations evidence that our technique regularly outperforms current strategies that either neglect misclassification, the unique features of the data-generating process, or both. We also illustrate the utility of this method with a model of repression using the Social Conflict in Africa Database.

Scholars of representation are increasingly interested in mass–elite congruence—the degree to which the preferences of elected elites mirror those of voters. Yet existing measures of congruence can be misleading because they ignore information in the data, require arbitrary decisions about quantization, and limit researchers to comparing masses and elites on a single dimension. We introduce a new measure of congruence—borrowed from computer science—that addresses all of these problems: the Earth Mover’s Distance (EMD). We demonstrate its conceptual advantages and apply it to two debates in research on mass–elite congruence: ideological congruence in majoritarian and proportional systems and the determinants of congruence across countries in Latin America. We find that improving measurement using the EMD has important implications for inferences regarding both empirical debates. Even beyond studies of congruence, the EMD is a useful and reliable way for political scientists to compare distributions.