The increasing multimodality (e.g., images, videos, links) of social media data presents opportunities and challenges. But text-as-data methods continue to dominate as modes of classification, as multimodal social media data are costly to collect and label. Researchers who face a budget constraint may need to make informed decisions regarding whether to collect and label only the textual content of social media data or their full multimodal content. In this article, we develop five measures and an experimental framework to assist with these decisions. We propose five performance metrics to measure the costs and benefits of multimodal labeling: average time per post, average time per valid response, valid response rate, intercoder agreement, and classifier’s predictive power. To estimate these measures, we introduce an experimental framework to evaluate coders’ performance under text-only and multimodal labeling conditions. We illustrate the method with a tweet labeling experiment.

This paper introduces a simple approach for assessing which survey questions are more likely to elicit political identity-influenced responses. We use daily data from Gallup to test which survey self-reports exhibit more or less susceptibility to politicization, finding the highest likelihood of politicization for societal-level questions. Conversely, we show that self-reported assessments of personal finances are less sensitive to partisan motivated responding. We also show how egotropic economic evaluations are influenced by the presence of other items on the same survey. Taken together, our results uncover scope conditions for how to interpret self-reported views of the economy, and we argue that measures of public opinion which have not yet been strongly politicized are better proxies for capturing the underlying welfare of the public.

Twitter has been a prominent forum for academics communicating online, both among themselves and with policy makers and the broader public. Elon Musk’s takeover of the company brought sweeping changes to many aspects of the platform, including public access to its data; Twitter’s approach to censorship and mis/disinformation; and tweaks to the affordances of the platform. This article addresses a narrower empirical question: What did Elon Musk’s takeover of the platform mean for this academic ecosystem? Using a snowball sample of more than 15,700 academic accounts from the fields of economics, political science, sociology, and psychology, we show that academics in these fields reduced their “engagement” with the platform, measured by either the number of active accounts (i.e., those registering any behavior on a given day) or the number of tweets written (including original tweets, replies, retweets, and quote tweets). We further tested whether this decrease in engagement differed by account type; we found that verified users were significantly more likely to reduce their production of content (i.e., writing new tweets and quoting others’ tweets) but not their engagement with the platform writ large (i.e., retweeting and replying to others’ content).

Large language models (LLMs) offer new research possibilities for social scientists, but their potential as “synthetic data” is still largely unknown. In this paper, we investigate how accurately the popular LLM ChatGPT can recover public opinion, prompting the LLM to adopt different “personas” and then provide feeling thermometer scores for 11 sociopolitical groups. The average scores generated by ChatGPT correspond closely to the averages in our baseline survey, the 2016–2020 American National Election Study (ANES). Nevertheless, sampling by ChatGPT is not reliable for statistical inference: there is less variation in responses than in the real surveys, and regression coefficients often differ significantly from equivalent estimates obtained using ANES data. We also document how the distribution of synthetic responses varies with minor changes in prompt wording, and we show how the same prompt yields significantly different results over a 3-month period. Altogether, our findings raise serious concerns about the quality, reliability, and reproducibility of synthetic survey data generated by LLMs.

We present a method for estimating the ideology of political YouTube videos. The subfield of estimating ideology as a latent variable has often focused on traditional actors such as legislators, while more recent work has used social media data to estimate the ideology of ordinary users, political elites, and media sources. We build on this work to estimate the ideology of a political YouTube video. First, we start with a matrix of political Reddit posts linking to YouTube videos and apply correspondence analysis to place those videos in an ideological space. Second, we train a language model with those estimated ideologies as training labels, enabling us to estimate the ideologies of videos not posted on Reddit. These predicted ideologies are then validated against human labels. We demonstrate the utility of this method by applying it to the watch histories of survey respondents to evaluate the prevalence of echo chambers on YouTube in addition to the association between video ideology and viewer engagement. Our approach gives video-level scores based only on supplied text metadata, is scalable, and can be easily adjusted to account for changes in the ideological landscape.

The link between objective facts and politically relevant beliefs is an essential mechanism for democratic accountability. Yet the bulk of empirical work on this topic measures objective facts at whatever geographic units are readily available. We investigate the implications of these largely arbitrary choices for predicting individual-level opinions. We show that varying the geographic resolution—namely aggregating economic data to different geographic units—influences the strength of the relationship between economic evaluations and local economic conditions. Finding that unemployment claims are the best predictor of economic evaluations, especially when aggregated at the commuting zone or media market level, we underscore the importance of the modifiable areal unit problem. Our methods provide an example of how applied scholars might investigate the importance of geography in their own research going forward.

The relationship between anxiety and investor behavior is well known enough to warrant its own aphorism: a “flight to safety.” We posit that anxiety alters the intensity of voters’ preference for the status quo, inducing a political flight to safety toward establishment candidates. Leveraging the outbreak of the novel coronavirus during the Democratic primary election of 2020, we identify a causal effect of the outbreak on voting, with Biden benefiting between 7 and 15 percentage points at Sanders’s expense. A survey experiment in which participants exposed to an anxiety-inducing prompt choose the less disruptive hypothetical candidate provides further evidence of our theorized flight to safety among US-based respondents. Evidence from 2020 French municipal and US House primary elections suggests a COVID-induced flight to safety generalizes to benefit mainstream candidates across a variety of settings. Our findings suggest an as-yet underappreciated preference for “safe” candidates in times of anxiety.

Knowledge creation is a social enterprise, especially in political science. Sharing new findings widely and quickly is essential for progress. Scholars can now use Twitter to rapidly disseminate ideas, and many do. What are the implications of this new tool? Who uses it, how do they use it, and what are the implications for exacerbating or ameliorating existing inequalities in terms of research dissemination and attention? We construct a novel dataset of all 1,236 political science professors at PhD-granting institutions in the United States who have a Twitter account to answer these questions. We find that female scholars and those on the tenure track are more likely to use Twitter, especially for the dissemination of research. However, we consistently find that research by men shared on Twitter is more likely to be passed along further by men than research by women.

Multilevel regression and post-stratification (MRP) is the current gold standard for extrapolating opinion data from nationally representative surveys to smaller geographic units. However, innovations in nonparametric regularization methods can further improve the researcher’s ability to extrapolate opinion data to a geographic unit of interest. I test an ensemble of regularization algorithms and find that there is room for substantial improvement on the multilevel model via more flexible methods of regularization. I propose a modified version of MRP that replaces the multilevel model with a nonparametric approach called Bayesian additive regression trees (BART or, when combined with post-stratification, BARP). I compare both methods across a number of data contexts, demonstrating the benefits of applying more powerful regularization methods to extrapolate opinion data to target geographical units. I provide an R package that implements the BARP method.

Precise international metrics and assessments may induce governments to alter policies in pursuit of more favorable assessments according to these metrics. In this paper, we explore a secondary effect of global performance indicators (GPIs). Insofar as governments have finite resources and make trade-offs in public goods investments, a GPI that precisely targets the provision of a particular public good may cause governments to substitute away from the provision of other, related, public goods. We argue that both the main effect of the GPI (on the measured public good) and this substitution effect vary systematically based on the domestic political institutions and informational environments of targeted states. Specifically, we contend that both the main and substitution effects of GPIs should be largest for governments that are least accountable (opaque and nondemocratic) and should be smallest for those that are most accountable. We illustrate the logic of these arguments using a formal model and test these claims using data on primary and secondary enrollment rates across 114 countries. We find that countries substitute toward primary education enrollment rates (which is targeted by the Millennium Development Goals) and away from secondary (which is not), and that these effects are mitigated as accountability rises.

To answer questions about the origins and outcomes of collective action, political scientists increasingly turn to datasets with social network information culled from online sources. However, a fundamental question of external validity remains untested: are the relationships measured between a person and her online peers informative of the kind of offline, “real-world” relationships to which network theories typically speak? This article offers the first direct comparison of the nature and consequences of online and offline social ties, using data collected via a novel network elicitation technique in an experimental setting. We document strong, robust similarity between online and offline relationships. This parity is not driven by shared identity of online and offline ties, but a shared nature of relationships in both domains. Our results affirm that online social tie data offer great promise for testing long-standing theories in the social sciences about the role of social networks.