1 Introduction and Summary
Foundation models (FMs) are rapidly transforming political science research, offering unprecedented capabilities for analyzing political content (Heseltine and Clemm von Hohenberg Reference Heseltine and von Hohenberg2024; Törnberg Reference Törnberg2023; Waight et al. Reference Waight2025; Weidmann, Faulborn, and García Reference Weidmann, Faulborn and García2025). Given their widespread adoption, understanding how they process and create political information is crucial.
Extensive research documents that FMs perpetuate gender (e.g., Hofmann and Mõttus Reference Hofmann and Mõttus2025; Rudinger et al. Reference Rudinger, Naradowsky, Leonard and Van Durme2018; Zhao et al. Reference Zhao, Wang, Yatskar, Ordonez and Chang2018) and group-based stereotypes (e.g., Abid, Farooqi, and Zou Reference Abid, Farooqi and Zou2021; Lee, Montgomery, and Lai Reference Lee, Montgomery and Lai2024; Lucy and Bamman Reference Lucy and Bamman2021).Footnote 1 These systems also exhibit liberal leanings when evaluated through standardized political orientation tests (e.g., Feng et al. Reference Feng, Park, Liu and Tsvetkov2023; Rozado Reference Rozado2023; Rozado Reference Rozado2024). However, these two streams of research have developed largely in isolation, and a critical gap remains: we do not understand whether or how demographic characteristics influence the way FMs attribute political characteristics. This gap has significant implications for political science research. For example, when using FMs for labeling tasks, demographic cues could systematically influence political coding in ways that researchers do not anticipate or desire, potentially introducing confounders into measures (Knox, Lucas, and Cho Reference Knox, Lucas and Cho2022).
Unlike text-only models, vision-language models (VLMs) can process visual cues of race and gender in naturalistic stimuli and enable researchers to test political inferences of these stimuli in realistic contexts. By presenting identical political content while varying only the demographic features of individuals in images, we can isolate the effect of visual demographic information on political attributions.
Human perception research provides clear expectations for how demographic cues might influence political judgments. Decades of research demonstrate that people use demographic characteristics as heuristics for inferring political ideology: Black Americans are often assumed to be liberal, White Americans conservative, and women more left-leaning than men (Dolan Reference Dolan2014; Lerman and Sadin Reference Lerman and Sadin2016). These associations significantly influence how political candidates are perceived and evaluated, even when their actual policy positions are identical (Crowder-Meyer et al. Reference Crowder-Meyer, Gadarian, Trounstine and Vue2020; McDermott Reference McDermott1997; Sanbonmatsu Reference Sanbonmatsu2002). Research on prototypicality, the stereotypical representation of features of their social group, suggests that more prototypical features strengthen category-based judgments (Burge, Wamble, and Cuomo Reference Burge, Wamble and Cuomo2020; Lemi Reference Lemi2021; Livingston and Brewer Reference Livingston and Brewer2002; Ma, Correll, and Wittenbrink Reference Ma, Correll and Wittenbrink2018; Maddox and Gray Reference Maddox and Gray2002). If FMs learn from human-generated data, we expect similar demographic-based ideology attributions.
To test whether and how VLMs link demographic characteristics to political ideology, we conducted detailed experiments with GPT-4o-mini and validated key findings across GPT-4o and LLaVA (Liu et al. Reference Liu, Li, Qingyang and Lee2023).Footnote 2 Using identical campaign advertisements that varied only in the race and gender of the featured candidate, we found that demographic-political associations were both systematic and model-dependent. In our primary analysis with GPT-4o-mini, Black candidates and women were consistently attributed more liberal ideologies than White candidates and men—patterns that persisted across temperature settings and prompt variations, and exceeded both random allocation baselines and real-world political identification patterns from nationally representative survey data. Validation tests revealed an important distinction: while the gender effect replicated across all three models, racial associations varied, weakening in GPT-4o and disappearing in LLaVA. Racial prototypicality showed no significant effect in any model, suggesting that when racial associations occur, models may process racial categories as discrete rather than continuous features.
2 Methods
2.1 Main Study Design
Figure 1 shows a visualization of the study design. The complete set of faces we used in this study is shown in Appendix I of the Supplementary Material. We conducted our primary analysis using GPT-4o-mini, with systematic replication in GPT-4o and LLaVA.Footnote 3 For all models, we presented identical campaign advertisements that varied only in the race and gender of the featured candidate. Each advertisement consisted of a facial image overlaid on a standardized Facebook campaign template containing neutral text (“Please endorse me for my election campaign, and I will send you a free window sticker. You can vote today!”). We prompted each model to generate captions or answer questions about the candidates’ political views, specifically asking it to categorize each candidate as very conservative, conservative, moderate, liberal, or very liberal.Footnote 4
Visualization of the study design.
To do this, we used 40 images from the GAN Face Database (GANFD; Marsden et al. Reference Marsden, Jaurique, McDonald and Burke2024) representing four intersectional groups: Black women, Black men, White women, and White men. Within each racial group, we selected images with both high and low racial prototypicality ratings to examine whether the strength of racial features influenced political attributions. The GAN Face Database was generated using the StyleGAN2 model (Karras et al. Reference Karras, Laine, Aittala, Hellsten, Lehtinen and Aila2020), which was trained on the Flickr-Faces-HQ (FFHQ) dataset (Karras, Laine, and Aila Reference Karras, Laine and Aila2019). A necessary limitation of this study is its reliance on synthetic images, which, despite allowing experimental control and overcoming important ethical and licensing challenges, may not fully capture the complex visual cues that AI models use to attribute ideology to real individuals.Footnote 5 We generated 50 completions for each image/prompt combination. This resulted in 8,000 total responses (50 iterations, 40 images, and 4 prompts).
After generating responses, we used GPT-4o to code political ideology from the generated text, achieving 100% accuracy compared to human coding in our pilot study. We assigned each response to a five-point ideological scale ranging from
$-2$
(most conservative) to 2 (most liberal), excluding responses labeled as “refused to answer” (refusal rates varied by model; see Appendix C.3 of the Supplementary Material for details).
2.2 Baselines
To provide context to our results, we compared model outputs to five benchmarks. None represents a “true” or correct baseline. Instead, they offer different reference points to help interpret the magnitude and direction of any demographic-ideology associations in model outputs (additional details are in Appendix D of the Supplementary Material). Appendix J of the Supplementary Material briefly considers a sixth baseline of human labels.
-
1. No-Picture Baseline: The distribution of attributions assigned when no images were included to isolate the contribution of visual demographic cues.
-
2. Equal Distribution: Testing whether outputs deviate from uniform random assignment.
-
3. Normal Distribution: Testing whether outputs deviate from Gaussian random assignment centered at the moderate category.
-
4. U.S. Population Data: Overall political identification from the Weidenbaum Center Survey (WCS), a nationally representative sample collected with YouGov.
-
5. U.S. Subgroup Data: Race- and gender-specific distributions from the WCS, which included a large oversample of Black Americans.
2.3 Statistical Approach
We compared the distribution of attributed ideologies across the five categories (very conservative to very liberal) to our baselines using both descriptive statistics and
$\chi ^2$
goodness-of-fit tests. These tests evaluated whether observed ideological distributions differed significantly from expected baseline distributions, both for the overall sample and for specific demographic subgroups. We also examined differences-in-differences, comparing how the gap between demographic groups in model outputs differed from the corresponding gaps in baseline data. (Note that expected group differences are by construction zero except in the WCS benchmark; we do not expect different ideology by race or gender for the other benchmarks.)
Second, we fit linear models with random effects for each image. These models examined how race, gender, and their interaction influenced political ideology attributions on average while accounting for random variation between individual images. Again, we compare these reported differences to the WCS baseline, noting that the expected coefficients for the other baselines are zero.
2.4 Model Comparison and Robustness
We replicated our main analysis using GPT-4o-2024-11-20 (henceforth GPT-4o) and the open-source LLaVA model to examine whether demographic-political associations generalize across architectures (accessed July 2025). For our primary model (GPT-4o-mini), we also conducted additional ablation experiments: a Debias condition with explicit instructions to avoid demographic assumptions, an Order condition randomizing ideology label sequence, and Temperature variations (0, 0.3, 0.6). Additional details are provided in Appendix H of the Supplementary Material.Footnote 6
3 Results
3.1 Main Analysis
Figure 2 presents the difference between GPT-4o-mini’s ideological attributions and each benchmark distribution. Positive values (shaded green) indicate that the model assigned more candidates to that ideological category than expected under the baseline, while negative values (shaded red) indicate fewer assignments than expected.
$\chi ^2$
statistics comparing the model outputs to the baseline distributions are shown on the right.
Percentage point differences between GPT-4o-mini ideological attributions and various benchmarks. Green indicates the model assigns more candidates to that ideological category relative to the benchmark; red indicates fewer.
$\chi ^2$
tests compare overall distributions between model outputs and each benchmark. An asterisk denotes statistical significance at
$p < 0.05$
.
The pattern is clear and consistent across all benchmarks: GPT-4o-mini systematically over-attributes moderate, liberal, and very liberal ideologies while under-attributing conservative and very conservative ideologies. This pattern appears regardless of baseline comparison and is always statistically significant. Figure E.1 in the Supplementary Material shows that while the model shifts all demographic groups leftward relative to benchmarks, it does so most dramatically for women and Black candidates, effectively eliminating conservative attributions for these groups.
Figure 3 presents “difference-in-differences” analyses comparing demographic gaps in model attributions versus U.S. survey data. Note again that for other benchmarks, these differences are exactly zero. Panel (a) shows that while WCS data reveal modest gender differences (women are slightly more likely to identify as liberal), the model dramatically amplifies these patterns. Panel (b) reveals even more complicated racial patterns. While Black Americans do identify as more liberal than White Americans in WCS data, the model exaggerates these differences. The model fails to capture Black respondents’ lower likelihood of identifying as conservative only because it assigns conservative labels so rarely to any group. Additionally, while Black WCS respondents are more likely than Whites to identify as moderate, the model reverses this pattern substantially.
Difference-in-differences analysis comparing demographic gaps in GPT-4o-mini outputs versus WCS survey data. Positive values indicate women (panel a) or Black candidates (panel b) receive higher percentages in that ideological category than men or White candidates, respectively. The model amplifies demographic differences relative to baselines, particularly in liberal/very liberal categories, while creating new gaps (e.g., gender differences in moderate attributions) absent from survey data.
Table 1 compares demographic effects in GPT-4o-mini outputs versus WCS benchmarks using linear models. For gender, the model’s effect (
$b = 0.41$
,
$SE = 0.09$
, and
$p < 0.05$
) is nearly three times larger than the WCS benchmark (
$b = 0.15$
,
$SE = 0.07$
, and
$p < 0.05$
), indicating substantial relative amplification of gender-based political associations. The racial effect shows interesting convergence: while both model (
$b = 0.44$
,
$SE = 0.08$
, and
$p < 0.05$
) and WCS (
$b = 0.47$
,
$SE = 0.06$
, and
$p < 0.05$
) data show similar magnitudes, this masks important differences in how these effects manifest across the ideological spectrum (as shown in Figure 3). When examining intersectional effects, neither model nor WCS data show significant race-by-gender interactions, suggesting these demographic effects operate additively rather than multiplicatively. Racial prototypicality showed no significant effects in any specification (see Appendix F of the Supplementary Material).
Effect of predictors on ideology in GPT-4o-mini and WCS data.
Note: Standard errors in parentheses. An asterisk denotes statistical significance at
$p < 0.05$
. We used image random effects for GPT and robust standard errors for WCS models.
3.2 Robustness Analyses
Our replication across GPT-4o and LLaVA revealed both consistencies and important differences in how models link demographics to political ideology. All three models showed systematic liberal overattribution relative to baselines and consistent gender effects: women received more liberal attributions than men across all architectures (see Table 1 and Appendix G of the Supplementary Material). However, racial associations varied by model. While GPT-4o-mini showed strong racial effects (Black candidates received 0.44 points more liberal on our scale), these effects attenuated in GPT-4o (0.18 points) and disappeared in LLaVA, where racial differences failed to reach statistical significance. This model-dependent pattern suggests that while gender-ideology associations appear robust across vision-language architectures, racial associations are contingent on specific model implementations.
Table 2 presents ablation studies examining the robustness of demographic-ideology associations in GPT-4o-mini across various conditions. In the Debias condition, we instructed the model to disregard demographic characteristics when inferring political ideology. While this instruction reduced the association, it did not eliminate it; women and Black candidates were still significantly more likely to be labeled as liberal. Similarly, in the Order condition, we reversed the sequence of ideological options presented in the prompt. This change, aimed at testing for ordering effects, also attenuated the associations but failed to remove them. We also tested the model’s consistency across various temperature settings (0, 0.3, and 0.6), which control output randomness. The demographic associations remained remarkably stable.
Liberal attribution effects across GPT-4o-mini ablation conditions.
Note: All results are significant at
$p < 0.05$
. We used robust standard errors for the WCS survey data. Standard errors in parentheses. Effects show an increase in liberal attribution for women vs. men and Black vs. White. Main = original results; WCS = survey benchmark; Debias = prompt to ignore demographics; Order = reversed options; Temp = temperature settings. Full model results are shown in Appendix H of the Supplementary Material.
4 Discussion and Implications
VLMs (sometimes) link demographics to political ideologies, even when analyzing ideologically neutral content. Persistent gender associations exceeded real-world benchmarks, whereas racial effects were model-dependent: strong in GPT-4o-mini, attenuated in GPT-4o, and insignificant in LLaVA. For GPT-4o-mini, where effects were most pronounced, demographic associations persisted even with explicit debiasing instructions, suggesting these patterns can be resistant to simple prompt-based interventions (although more successful debiasing variations surely exist).
Limitations include the use of 40 images, three models, and one political concept (ideology). Although we collected 8,000 responses for each model, we may not have sufficient power to detect small interaction effects between race and prototypicality. Perhaps more concerning, the variation we observed across models—with racial associations strong in GPT-4o-mini, attenuated in GPT-4o, and absent in LLaVA—reveals how demographic-political associations can vary unpredictably across architectures. This instability means researchers cannot know ex ante whether their chosen model will exhibit such associations, particularly for proprietary models where training data and architectural choices remain opaque. Future work should expand to more images, additional models, and other political outputs. As we used only English prompts and American framing of ideology, these results would also benefit from replications across cultures.
Despite limitations, our findings have important implications. Using VLMs for political content analysis may unknowingly introduce model-specific demographic confounds. To avoid this, we recommend excluding images from inputs unless appearance is central to the research question. If images are necessary, researchers should benchmark their chosen model’s performance beforehand. The open-source LLaVA model displayed the weakest racial bias, which suggests it may work as a reasonable default. However, this finding must be interpreted cautiously given its notably high refusal rate, which suggests a potential trade-off between reduced bias and model usability. As VLM architectures evolve, ongoing validation and monitoring of demographic effects will be essential.
More broadly, as VLMs integrate into our political information infrastructure, these associations risk reshaping political communication in subtle but consequential ways. Search engines using AI-generated summaries might systematically mischaracterize candidates’ ideologies based on appearance alone, while automated journalism tools could inject these patterns into political coverage. As AI becomes more fully embedded in how we create, distribute, and consume political information, researchers and practitioners must carefully attend to how these systems encode associations between visual demographic characteristics and political attributes.
Supplementary Material
For supplementary material accompanying this paper, please visit https://doi.org/10.1017/pan.2026.10038.
Data Availability Statement
Replication code for this article has been published in the Harvard Dataverse at https://doi.org/10.7910/DVN/2JMT9J (Jeon et al. Reference Jeon, Lee, Montgomery and Lai2026).
Acknowledgements
The authors thank three anonymous reviewers and Michael Strawbridge for helpful comments. The authors used AI tools (ChatGPT, Claude, and Gemini) during writing, revision, and coding tasks. These tools were accessed in 2024 and 2025 and used with/without modification. The authors reviewed and verified all outputs. All errors are our own.
Funding Statement
The authors declare that no specific funding has been received for this article.
Competing Interests
The authors declare none.
Open access
