Laws change behaviors in part because they change incentives, but those incentives may extend beyond the risk of criminal prosecution. Legal scholars have theorized that many laws serve an “expressive function” with the goal of changing social norms (Sunstein Reference Sunstein1996, Reference Sunstein2019). For example, countries that ban certain behaviors (such as adultery or marijuana use), but do not prosecute offenders, may be attempting to influence what society views as acceptable. A similar argument could be made for countries that mandate certain behaviors (such as voting or vaccination), but do not punish violators (Funk Reference Funk2007). In these cases, violating the social norm, rather than criminal prosecution, becomes the punishment.
The extension of new rights to marginalized groups may also serve an expressive function. Anti-discrimination laws—such as the Civil Rights Act, the Equal Rights Amendment, or marriage equality—may not have been intended solely to extend equal rights. The authors of these laws may have also wished to change norms surrounding discrimination on the basis of sex, race, and sexual orientation, even in domains not affected by the law, such as social life and public discourse.
This paper analyzes how the extension of voting rights to women affected one aspect of culture: public discourse in American newspapers. I ask the following questions: How did language in American newspapers change after women were granted the right to vote? Did articles about suffrage become more positive toward the concept of women’s suffrage or was there a wave of anti-suffrage backlash? Did suffrage change how newspapers discussed women and their position in society?
To answer these questions, I estimate pro- and anti-suffrage sentiment for newspaper pages published from 1880 to 1922 in the Chronicling America newspaper archive. I start by creating a training dataset of books, pamphlets, and essays that explicitly argued for or against women’s suffrage. I then use LASSO logistic regression to identify which words appearing close to the word “suffrage” predict pro- and anti-suffrage sentiment. Words such as “enfranchise” and the names of famous suffragists predict pro-suffrage sentiment, whereas words like “anti-suffrage,” “feminist,” and “socialist” predict anti-suffrage sentiment. I then use the LASSO model to predict sentiment for over half a million newspaper pages.
In addition to measuring sentiment, I use topic modeling—a tool from computational linguistics—to group newspaper pages about women into sets of topics. In this context, a topic can be thought of as a group of pages that share a vocabulary. The model estimates topics that correspond to women in politics, women in the workplace, and participation in associations or clubs.
I then use staggered difference-in-differences to estimate the effects of women’s suffrage on newspaper sentiment and topics. I find that pages about suffrage increase in the year women are granted the right to vote and subsequently decrease below baseline. This is consistent with interest in suffrage peaking when women won the vote and waning as it became settled law. I refer to this as the coverage effect. While the coverage effect holds true for both pro- and anti-suffrage pages, pro-suffrage pages decrease at a faster rate than anti-suffrage pages, implying that the average page becomes more negative. This is consistent with newspaper language moving in opposition to the status quo, which could result from either women’s suffrage increasing anti-suffrage protests, newspapers viewing their role as challenging the status quo, or readers demanding coverage that challenges the status quo. I refer to this finding as the opposition effect. Lastly, I find that the proportion of pages about women in politics increased for several years after suffrage. I call this finding the women-in-politics effect. Taken together, these results imply that suffrage changed the media’s narrative regarding women’s suffrage and, at least in the short run, women’s role in the political domain. Other areas of discussion were relatively unaffected.
Understanding the determinants of newspaper language is important for three reasons. First, media language is likely influenced by consumer demand and thus a reflection of public opinion (Gentzkow and Shapiro Reference Gentzkow and Shapiro2010). This contribution is especially important given that there were not widespread public opinion polls at the time.Footnote 1 Second, journalists frequently cover events of public interest; thus, language will also be a reflection of public behavior, such as anti-suffrage protests. Third, when media language does not reflect public opinion or behavior, it likely influences public opinion. Research on the causal effects of language shows that language choices matter in determining beliefs.Footnote 2
This paper contributes to multiple bodies of literature. First, I contribute to the literature on how laws affect cultural norms and public sentiment. Previous research has identified judicial backlash in response to abortion (Chen, Levonyan, and Yeh Reference Chen, Levonyan and Yeh2020), vaccine mandates (Brehm and Saavedra 2025), and cases regarding specific groups (such as Black Americans) (Ash, Chen, and Galletta Reference Ash, Chen and Galletta2022). Similarly, Wheaton (Reference Wheaton2020) shows that various U.S. policies result in backlash, with public opinion shifting in opposition to the status quo. His paper focuses on the Equal Rights Amendment, but he also shows some evidence for this effect with respect to other policies as well. This finding is similar to the opposition effect documented in this paper.Footnote 3
Second, this paper contributes to our understanding of the consequences of women’s suffrage. Several papers in economics have found that granting women the vote increased government spending in general (Lott and Kenny Reference Lott and Kenny1999), infant health (Miller Reference Miller2008), educational spending (Carruthers and Wanamaker Reference Carruthers and Wanamaker2015), and human capital development (Kose, Kuka, and Shenhav Reference Kose, Kuka and Shenhav2021). See Moehling and Thomasson (Reference Moehling and Thomasson2020) for a review of this literature.
Third, this paper contributes to the literature on understanding the relationship between the media and political behavior. Access to certain media—particularly newspapers (Cagé and Rueda Reference Cagé and Rueda2016; Drago, Nannicini, and Sobbrio Reference Drago, Nannicini and Sobbrio2014) and radio (Wang 2021b; Strömberg Reference Strömberg2004)—has been shown to increase political participation and even empower marginalized groups. At the same time, media bias can also polarize voters and even spread extremism (DellaVigna and Kaplan Reference DellaVigna and Kaplan2007; Wang 2021a; Allcott and Gentzkow Reference Allcott and Gentzkow2017). Fewer papers have examined how changes in the political system affect the media. Gentzkow et al. (Reference Gentzkow, Petek, Shapiro and Sinkinson2015) find that newspapers do not respond to changes in the state’s political party. My paper contributes to this literature by showing that the media not only affects electoral behavior and beliefs, but also that the composition of the electorate affects media language.Footnote 4 Other papers have examined the effects of exogenous events on newspaper language, including the Great Migration on racist language (Fouka, Mazumder, and Tabellini Reference Fouka, Mazumder and Tabellini2022) and WWI on anti-German sentiment (Ferrara and Fishback Reference Ferrara and Fishback2022). See Gentzkow, Kelly, and Taddy (Reference Gentzkow, Kelly and Taddy2019) and Ash and Hansen (Reference Ash and Hansen2023) for reviews of the text-as-data literature in economics.
Fourth, this paper contributes to the text-as-data literature that studies stereotypes and biased language. There is a large literature on gender stereotypes in the context of the media (Ash et al. Reference Ash, Durante, Grebenschikova and Schwarz2022), judges (Ash, Chen, and Ornaghi Reference Ash, Chen and Ornaghi2021), and education (Alan, Ertac, and Mumcu Reference Alan, Ertac and Mumcu2018), but that literature has not identified how stereotypes change in response to the extension of new rights. This paper shows that empowering a marginalized group can change the way the media discusses that group, at least in the short run.
A BRIEF HISTORY OF WOMEN’S SUFFRAGE
In the United States, voting rights were initially determined by the states. These rights were often limited to white adult males who owned property, though there were exceptions. For instance, until 1807, New Jersey permitted women who owned property to vote (Moehling and Thomasson Reference Moehling and Thomasson2020). Non-white individuals, regardless of their status as free or enslaved, women, children, and those who did not own land were commonly denied voting rights. Children remain disenfranchised to this day.
The first women’s suffrage convention took place in Seneca Falls, New York, led by Elizabeth Cady Stanton and Lucretia Mott. Participants of the Seneca Falls Convention drafted the Declaration of Sentiments, a document advocating for equal rights for women and inspired by the Declaration of Independence. Although the suffrage movement predates the convention, many point to it as the beginning of the suffrage movement, perhaps because it is prominently featured in the writings of Susan B. Anthony and Elizabeth Cady Stanton.
The early women’s suffrage movement often intersected with the anti-slavery and Black suffrage movements, but these interactions were not always characterized by aligned interests. The Fifteenth Amendment prevented states from formally denying the vote on the basis of race, but not on sex.Footnote 5 Support for the Fifteenth Amendment divided the women’s suffrage movement. The National Woman Suffrage Association (NWSA), led by Susan B. Anthony and Elizabeth Cady Stanton, opposed any amendment that extended Black male suffrage without also granting women’s suffrage. The competing American Woman Suffrage Association (AWSA), led by Lucy Stone and Henry Blackwell, supported extending voting rights to Black males. The two associations eventually merged in 1890 to form the National American Woman Suffrage Association (NAWSA).
The arguments for and against enfranchising women varied. Some argued that granting women the right to vote would shift their focus from domestic responsibilities to public life: if women governed, who would tend to children and manage the household? Conversely, others posited that if women could vote, policy would be more attuned to the needs of women and children, with a stronger focus on education and public health. There were also notable interactions between the suffrage and temperance movements. A common argument in favor of prohibition was that it would curb domestic violence against women and children (García-Jimeno, Iglesias, and Yildirim Reference García-Jimeno, Iglesias and Yildirim2022).
Several theories exist for why an enfranchised group would extend voting rights to a disenfranchised group. Elected officials may grant women the vote if they believe those votes are more likely to keep them in power (Teele Reference Teele2018). Similarly, one already-enfranchised group may support women’s suffrage if they believe women are likely to align with them on other issues (McConnaughy Reference McConnaughy2013). Alternatively, male support for women’s suffrage may result from intra-household bargaining (Bertocchi Reference Bertocchi2011). Another theory is that granting suffrage could have been used as a tool to attract women to frontier states with imbalanced sex ratios (Braun and Kvasnicka Reference Braun and Kvasnicka2013). See Hanlon (Reference Hanlon, Jeffery and Rubin2022) and Doepke, Tertilt, and Voena (Reference Doepke, Tertilt and Voena2012) for reviews of this literature.
Women did not win the right to vote all at once. Many states gave women either full or partial suffrage before the Nineteenth Amendment’s ratification. Partial suffrage included allowing women to vote only in primary, presidential, or local elections. Some states gave women the right to vote in school board elections earlier because it was thought women knew what was best for children. The passage of state-level women’s suffrage came in waves. Wyoming gave women the vote in 1869, as did Utah the following year.Footnote 6 Two decades passed until Colorado enfranchised women in 1893, followed by Idaho in 1896. Then, 14 years passed with little progress. Most states that gave women the vote before the Nineteenth Amendment did so during the 1910s when 25 states first passed either full or partial suffrage for women (Lott and Kenny Reference Lott and Kenny1999). See Figure 1 for a map of when women were enfranchised in each state.Footnote 7 The map displays the first year women had either full, presidential, or primary suffrage. Many of the states that granted early suffrage were located in the western half of the United States.
MAP OF EARLY SUFFRAGE STATES
Note: The blank states did not receive full or partial suffrage until the Nineteenth Amendment was ratified in 1920.
Source: Data come from Lott and Kenny (Reference Lott and Kenny1999) and Miller (Reference Miller2008).
Several studies have examined the consequences of women’s suffrage. Typically, these studies use state-level variation in the timing of women’s enfranchisement and use a difference-in-differences identification strategy.Footnote 8 Other studies use variation in the number of women enfranchised after the Nineteenth Amendment (Carruthers and Wanamaker Reference Carruthers and Wanamaker2015). Lott and Kenny (Reference Lott and Kenny1999) find that enfranchising women increased the size of state government expenditures. Miller (Reference Miller2008) finds that public health spending increased and childhood mortality decreased. Similarly, studies from European countries found an increase in social welfare spending following women’s suffrage (Aidt and Dallal Reference Aidt and Dallal2008; Abrams and Settle Reference Abrams and Settle1999).
Suffrage-induced increases in spending may have had lasting consequences. Carruthers and Wanamaker (Reference Carruthers and Wanamaker2015) use data from three southern states and find that the Nineteenth Amendment increased municipal spending on public education, with most (but not all) of the benefits directed toward white schools. Kose, Kuka, and Shenhav (Reference Kose, Kuka and Shenhav2021) find that children who grew up in states where women were enfranchised achieved higher levels of education.
Not all effects of suffrage were persistent. Moehling and Thomasson (Reference Moehling and Thomasson2012) found that only states where women were recently enfranchised were more likely to participate in the Sheppard-Towner program, which provided federally matched funds for states to invest in childhood and maternal health. This suggests that politicians may have catered to women’s issues in the years following suffrage but abandoned such efforts when it became clear that women had heterogeneous preferences.
Concurrently with the suffrage movement, women played a growing role in the production and consumption of newspapers during the late nineteenth and early twentieth centuries. Newspapers, in a bid to widen readership, introduced “the woman’s page.” This page featured content catered to women’s interests, including housework, beauty tips, and fashion (Wills Reference Wills2022). The woman’s page, which sometimes became its own section, would at times cover political issues, women’s suffrage, or news about clubs or association. As newspapers began to publish this content, they also hired more female journalists. However, journalism would remain a male-dominated profession for the first half of the twentieth century, with women often covering “soft news” rather than “hard news,” such as politics or foreign affairs.Footnote 9
For a more comprehensive review of the history of the movement toward women’s suffrage, see Moehling and Thomasson (Reference Moehling and Thomasson2020); for a review of women’s political participation after the Nineteenth Amendment, see Cascio and Shenhav (Reference Cascio and Shenhav2020) and Goldin (Reference Goldin2023).
DATA
The Chronicling America Newspaper Archive
The main data come from newspaper pages in the Chronicling America newspaper archive.Footnote 10 This archive was published by the U.S. Library of Congress, and at the time that the data were downloaded, it contained almost 20 million newspaper pages. The archive contains images, as well as OCRed text files of each newspaper page. Also available is the date and page number of every page and basic information about each newspaper, such as the name, location of publication, and sometimes information on the subject matter.Footnote 11
The digitization of newspapers is funded by state-level grants from the National Endowment for the Humanities (NEH). Recipients are typically flagship state universities or state historical societies. At the time of download, only two states have yet to receive the grants, Massachusetts and New Hampshire, which are excluded from the analysis.Footnote 12 Because the data come from state-level grants, the set of newspaper pages in the archive is more uniformly distributed across states than the population. For example, in 1900, New York had approximately 79 people for each person in Wyoming (7.3 million to 92,531), but for the same year, Chronicling America contains only 9.4 newspaper pages from New York for every page in Wyoming.Footnote 13
The NEH criteria to include a newspaper in the archive are that the newspaper should cover statewide issues, must “reflect the political, economic, and cultural history of the state,” and be a “paper of record” (National Endowment of the Humanities 2021). Additionally, the grants favor the digitization of papers that are available for long periods of time. These features, along with the oversampling of less populated states, make Chronicling America an ideal archive for constructing a state-year panel of sentiment in American newspapers.
Although the archive covers nearly two centuries of American history, it is relatively dense between 1880 to 1922. There were fewer newspapers before 1880, as the population was smaller and literacy rates were lower. Older newspapers are also less likely to survive to the present day for digitization. The number of pages declines after 1923, as Chronicling America has focused on newspaper pages in the public domain. The difference-in-differences analysis that I employ will only use pre-1920 pages, as all states are treated in 1920 with the passage of the Nineteenth Amendment. However, I leave 1920–1922 pages in the sample to provide some descriptive evidence of how newspaper narratives changed around the passage of the amendment, and I provide a brief analysis of the Nineteenth Amendment using already-treated observations as controls.
To construct the main sample, I scraped text that appears within 10 words of the word “suffrage” for all newspaper pages between 1880 and 1922.Footnote 14 Each observation is a newspaper page, not an article, as the OCR text does not clarify when an article begins and ends. Thus, the sample includes any mentions of the word suffrage, regardless of if it appears in a news article, editorial piece, or advertisement. Even with human inspection, it is sometimes difficult to distinguish these three categories as product placements within news articles were common. Another implication of this is that two sentences about suffrage on the same page, but in different articles, will effectively be concatenated. The 10-word restriction makes it likely that any words in the string refer to women’s suffrage.
This process results in a data set of 516,979 newspaper pages that make at least one reference to suffrage. To construct a sample of newspaper language about women, I follow a similar process, scraping all words that appear within 10 words of the word “women.” I do not include the variant “woman,” as such articles may focus on a particular woman, rather than women more generally. As “women” is a far more common word than suffrage, this results in a considerably larger data set of 6,096,885 newspaper pages.
Predicting Suffrage Sentiment
The goal of this section is to develop a methodology for assigning pro- and anti-suffrage sentiment to newspaper pages with unknown sentiment. The general strategy is to take a corpus of publications with known sentiment, determine the words that best predict sentiment, and then use a machine learning model to assign a predicted sentiment to half a million newspaper pages.
The training data set contains 18 publications (books, pamphlets, and essays) that clearly advocate for or against women’s suffrage. These publications were selected for two reasons. Firstly, the titles of the publications unambiguously convey the sentiment. Examples of anti-suffrage publications include The Fundamental Error of Woman Suffrage, The Unexpuraged Case Against Woman’s Suffrage, and Women’s Suffrage: The Reform Against Nature. Similarly, titles of pro-suffrage publications include The Citizenship of Women: A Plea for Women’s Suffrage, Why Men Should Work for Women’s Suffrage, and The Case for Women’s Suffrage. The exception is The History of Woman Suffrage (volume IV), which was written by notable suffragists Susan B. Anthony and Ida Husted Harper. These publications are sometimes authored by individuals and other times by associations explicitly advocating for or against women’s suffrage, such as the Illinois Association Opposed to the Extension of Suffrage to Women or the National American Woman Suffrage Association. The second selection criterion is the availability of a full and high-enough quality PDF to OCR the publication. The complete list of publications in the training data can be found in Online Appendix Table A.1.
To identify words that predict sentiment, I begin by following the strategy first employed in Gentzkow and Shapiro (Reference Gentzkow and Shapiro2010). I start by finding a set of candidate words that are likely to predict sentiment. To do so, I generate frequency counts of stemmed words in the corpus. I remove words that are rare (appear fewer than 50 times), as these are unlikely to appear in the training data set enough to predict suffrage sentiment. I also remove stop words (e.g., the, of, and that), which typically do not contain any meaningful sentiment. Words are then stemmed using the Porter stemmer so that words like vote, votes, and voting are coded as one word. However, the Porter stemmer does not always combine similar words. For instance, vote and voter have different stems, as do suffrage and suffragist.Footnote 15
After applying these sample restrictions, we are left with a set of stemmed words. Let 
be the frequency of word w in the anti-suffrage corpus, and define 
similarly. Moreover, let 
and 
be the frequency of all other words that are not word w in the anti- and pro-suffrage corpora, respectively (after implementing the sample restrictions). To test the null hypothesis that word w is used with the same frequency in both the anti- and pro-suffrage corpora, I compute the Pearson’s χ 2 statistic:
The words with larger χ 2 statistics are more likely to be suitable candidates for predicting sentiment compared to those with lower χ 2 statistics. Following the general strategy of Gentzkow and Shapiro (Reference Gentzkow and Shapiro2010), I retain words with the highest 500 χ 2 statistics for inclusion in the machine learning model.
The next step is to map a vector of the candidate words to sentiment predictions. To accomplish this, I consider each page of a publication with a known sentiment as an observation, resulting in a total of 3,779 pages. I then represent the language of each page as a bag-of-words vector. To generate this vector, I take each page and count the number of times that the 500 candidate words appear on that page. Thus, the bag-of-words vector is a 500-length vector in which each element is the count of a particular word divided by the total count of any candidate word appearances on that page. Formally, I index the words so that 
. Then I represent the w th element of the bag-of-words vector for page i as: 
, where 
is the frequency of word w on page i.
To predict suffrage sentiment while preventing overfitting of the data, I adopt a LASSO regression approach with separate training and hold-out samples. First, I randomly pick 80 percent of the pages to train a LASSO model. I use LASSO logit regression with the dependent variable being an indicator of whether a page came from the pro-suffrage corpus of the training data. The elements of the bag-of-words vector are normalized to have a mean of zero and a standard deviation of one, forming the set of independent variables. The LASSO model introduces a penalty parameter that forces the coefficients of many explanatory variables to zero. This step prevents overfitting the model and results in words that do not predict sentiment being dropped from the model. The penalty parameter is selected using 10-fold cross-validation to maximize out-of-sample fit. The fitted values then serve as predictions for sentiment and are guaranteed to be between 0 and 1. Because the training data set was somewhat arbitrary, I weight each page so that both pro- and anti-suffrage pages have equal total weight in the training data. The 20 percent held-out sample will then be used to evaluate goodness of fit. Brehm and Saavedra (2025) use a similar LASSO approach to predict anti-vaccine sentiment.
The top 100 pro- and anti-suffrage words appear in Online Appendix Table A.2. Common pro-suffrage words include stems of enfranchise, disenfranchise, and democracy. The names of well-known suffragists are also represented, as the stems of Anthony, Brown, Clay, and Stone appear as pro-suffrage. There are also references to protests, with the stems of the words imprison, prison, and jail. Conversely, anti-suffrage words include the stems of (perhaps unpopular) ideologies and policies associated with the suffrage movement, including feminist and socialist. Religious language appears more prominently in the anti-suffrage corpus with the stems of Christian, scripture, and evil. Titles from the training data set are common as well, as these often appear as running titles on the top of pages, and the OCR does not distinguish them from the main text. For example, the stem of antisuffrage predicts anti-suffrage text, but so does the stem of essays (Anti-Suffrage Essays is the title of one book in the training data set).
After estimating this model, I generate predicted sentiment for the Chronicling America newspaper pages. I start by taking all of the text within 10 words of the word suffrage. Using this text, I then generate the bag-of-words vector for that set of words. Because the denominator of each element is the total number of candidate words close to the word suffrage, it will be unaffected by the fact that these texts are often shorter than a full page. However, it should be noted that if the word suffrage appears repeatedly, the scraped string may encompass the majority of a newspaper page. Finally, I use the LASSO model to generate sentiment predictions for each newspaper page. Because the newspaper data set is disjoint from the training data, the training of the language model will not introduce a violation of the Stable Unit Treatment Value Assumption (SUTVA), alleviating concerns for making causal inferences in text analysis raised in Egami et al. (Reference Egami, Fong, Grimmer, Roberts and Stewart2022). Examples of newspaper articles that were labeled as highly pro- or anti-suffrage appear in Online Appendix Table A.3.
The accuracy of these predictions is assessed in Online Appendix B. The confusion matrix and calibration curve show that the LASSO model accurately predicts suffrage sentiment in a held-out sample. Additionally, I validate the measure in the newspaper data by examining suffrage sentiment in newspapers that explicitly advocated for women’s rights. This exercise shows that women’s rights newspapers had considerably higher pro-suffrage sentiment than the average newspaper. See Online Appendix B for more details.
Online Appendix Figure A.1 illustrates the trends in suffrage sentiment by the political party of the newspaper. Newspaper party data come from the 1905 N.W. Ayer and Son’s American Newspaper Annual and Directory. I group all newspapers with a political affiliation that contain the word “Republican” as a single category (thus, this group also includes “Independent Republican”). I classify “Democratic” newspapers similarly. Affiliations that do not contain either word are classified as “other.” Until approximately 1908, all parties exhibited relatively steady support for suffrage. After this point, the percentage of pages that were pro-suffrage began to rise, a trend that continued to increase until the passage of the Nineteenth Amendment in 1920. To my knowledge, this figure is the first to document the rise of pro-suffrage sentiment in the media leading up to the Nineteenth Amendment. While the exact cause of the increase in pro-suffrage sentiment is unclear, I will show in the following section that a rise in articles about suffrage protests roughly co-occurred. Following the passage of the amendment, suffrage sentiment declined. Throughout the period, Republican newspapers contained more pro-suffrage language than Democratic newspapers, but the gap narrows during the push for the Nineteenth Amendment.
Measuring Topics
To model the topics in newspapers, I use Latent Dirichlet Allocation (LDA), which was first developed by Blei, Ng, and Jordan (Reference Blei, Ng and Jordan2003) and has recently been applied in a variety of social science settings (Grimmer, Roberts, and Stewart Reference Grimmer, Roberts and Stewart2022; Ash and Hansen Reference Ash and Hansen2023).Footnote 16 LDA is a generative Bayesian model used to estimate the topic structure of documents within a corpus. It assumes that each document is a mixture of topics and that each topic has a distribution over the entire vocabulary. Formally, let Dir(·) be the probability density function of a Dirichlet distribution, and α and η be the Dirichlet parameters for the topic-document and word-topic distributions, respectively. In other words, for each document d, the topic distribution is θ d∼Dir(α), and for each topic k, the word distribution is β k∼Dir(η). For each word in document d, the data-generating process is assumed to first pick a topic k according to θ d, and then pick a word according to β k. Then the total number of occurrences of a particular word w in a document of length M follows a multinomial distribution. LDA ignores word order.
The number of topics, K, must be pre-selected by the researcher. Setting K too small may yield topics that are overly broad and heterogeneous. Conversely, setting K too large risks splitting genuinely similar topics into distinct categories. While data-based metrics for deciding the number of topics do exist, many researchers recommend testing multiple K values and choosing the one that produces the most human-interpretable topics (Grimmer, Roberts, and Stewart Reference Grimmer, Roberts and Stewart2022). In this study, I estimate LDA models with K = 4 topics for the set of newspaper pages that contain the word suffrage. For the set of pages that contain the word women, I pick K = 15. This choice reflects the diverse discourse surrounding a common word like women compared to suffrage. To reduce computational burden and decrease OCR errors, I apply the Porter stemmer and remove stems that appear fewer than 500 times in the set of newspaper pages that contain the word suffrage and remove stems that appear fewer than 1,000 times in the set of newspaper pages that contain the word women. I randomly pick half of the newspaper pages to train the LDA model and use the other half to estimate the effect of suffrage on topics.
The LDA model estimates the proportion of each document that pertains to each topic, as well as the probability that a specific topic will use a certain word. To examine the meaning of each topic, researchers can either read documents that are primarily only on that topic or examine common words within the topic.
Online Appendix Table A.4 presents the four suffrage topics, the 20 words with the highest word probabilities in each topic, a label for the topic, and a subjective confidence in that label. Stemmed words common to the first suffrage topic include right, univers, equal, and negro. These words indicate that this topic might represent discussion of universal suffrage, the idea that each person should have one vote regardless of sex or race. The second topic contains the stems of amendment, constitution, house, senate, and legislature. This topic addresses legislative activity and possibly the Nineteenth Amendment specifically. The third topic contains stems of militant and police, as well as prison, jail, and burn (although these are outside of the top 20). It also includes words related to British suffragist Emmeline Pankhurst (London and England), who was well-known for controversial protest techniques such as arson and encouraging hunger strikes of imprisoned protesters (the name Pankhurst also appears in this topic but is outside the top 20). It also contains words for more typical protests, such as the stems of demonstration, march, and parade. I refer to this as the “protest topic.” The last topic has the stems of association, convention, and league. These words indicate discussion about suffrage associations, organizations, or meetings. I refer to this as the “associations topic.”
Online Appendix Table A.5 displays all 15 women topics, along with the top 20 words in each topic. The first topic contains the stems of words related to work (employ, work, and labor) as well as war production (American, country, and war). I refer to this topic as “work and war production.” The fifth topic contains stems for women participating in politics, with stems related to women’s suffrage (vote, politics, and elect), the Woman’s Christian Temperance Union, and general politics (law and president). The seventh topic includes other job ads, with stems such as work, want, apply, and hard. The 12th topic describes social organizations, with stems such as church, societi, meet, and member. I refer to this as the “women’s associations” topic. Other topics include fashion, clothing advertisements, health care, footwear advertisements, health remedies, romantic life, professional and urban life, education, crime and public safety, uncertain/OCR errors, and retail advertisements.Footnote 17 Some of these topics will be used as placebo tests.
Summary statistics for the topics appear in Table 1. There are 258,492 pages that contain the stem suffrag that were published between 1880 and 1922 in the 50 percent test sample. Universal suffrage is the largest topic (37 percent), followed by legislative activity (26 percent). Suffrage associations comprise 22 percent of the sample, and protests comprise 16 percent. The average publication year is 1907, and the average year in which the first women’s suffrage law passed was 1915. Panel B presents summary statistics for pages that mention the word women. There are over 3 million such pages during the study period in the test sample. The topics include politics (4 percent), work and war production (18 percent), and other job ads (3 percent). The final panel displays the percentage of all newspaper pages that contain the words suffrage or women. Because these outcomes are not dependent on the LDA representation, this panel is not divided into a training/test sample. Approximately 46 percent of pages contain the word women, whereas 4 percent contain the word suffrage.
SUMMARY STATISTICS
Notes: The suffrage topics are from words that appear within ten words of suffrage, and the women topics are found similarly. The topics come from an LDA with K=4 for suffrage and K=15 for women.
Source: Data come from newspaper pages from the Chronicling America newspaper archive from 1880–1922.
Time series plots of the trends in the suffrage topics appear in Figure 2. The universal suffrage topic was dominant until the early 1900s, at which point the protest topic became more widely discussed until the beginning of WWI. Similarly, the associations topic increased in frequency until the mid-1910s and fell around the time the United States entered WWI. The legislative topic increased dramatically in 1919 and 1920 during the push for the Nineteenth Amendment and fell dramatically in 1921. Figure 3 displays the trends for the women’s topics. Discussion of women in politics spikes with the passage of the Nineteenth Amendment in 1920. The women in work and war production topic shows a large spike during WWI, most likely in response to labor shortages. The women’s association topic increases during this period.
TIME SERIES OF SUFFRAGE TOPICS
Note: Displays the average share of pages that are in each topic for pages that contain suffrage.
Source: Data come from newspaper pages from the Chronicling America newspaper archive from 1880–1922.
TIME SERIES OF WOMEN TOPICS
Note: Displays the average share of pages that are in each topic for pages that contain women.
Source: Data come from newspaper pages from the Chronicling America newspaper archive from 1880–1922.
ECONOMETRIC MODEL
Let 
be the potential sentiment of newspaper page i in state s in year t in a world in which women are disenfranchised, and define 
similarly for a world in which women are enfranchised. Then the individual-level treatment effect is 
. The parameters I am interested in are how τ changes relative to the year women become enfranchised.
To estimate the effect of state-level suffrage on sentiment in the media, I adopt a staggered difference-in-differences design. Until recently, such event-study treatment effects would have been estimated using a two-way fixed-effects estimator. However, Goodman-Bacon (Reference Goodman-Bacon2021) demonstrates that this estimator often makes the unstated assumption of treatment effect homogeneity and, in the presence of treatment effect heterogeneity, some treatment effects receive negative weights. If the heterogeneity is substantial, the estimated average treatment effect may even have the wrong sign. I use the Callaway and Sant’Anna (Reference Callaway and Sant’Anna2021) estimator, which accommodates arbitrary heterogeneous treatment effects across states and years and addresses concerns associated with the two-way fixed effects estimator raised in Goodman-Bacon (Reference Goodman-Bacon2021). In this analysis, the sample is treated as a repeated cross-section of newspaper pages. To lessen the computational burden, the newspaper pages are aggregated to a state-year cell and weighted by the number of pages. Standard errors are clustered at the state level.
The sample for the main analysis consists of state-year cells from 1880 to 1919. All states are treated in 1920; thus, those years are omitted from the Callaway and Sant’Anna estimator. Additionally, the Callaway and Sant’Anna estimator omits states that are always treated, effectively dropping Wyoming and Utah.Footnote 18 Because some states passed women’s suffrage close to 1920, the panel is not balanced in event time. To address this, I will also present specifications in which I drop treated states that are not treated for at least eight years before the Nineteenth Amendment.
The identifying assumptions are (1) parallel trends in the potential outcomes and (2) that newspapers do not anticipate treatment. The parallel trends assumption means that, in the absence of the intervention, newspaper language would have followed a similar trend in both early suffrage and Nineteenth Amendment states. This might seem surprising at first because media sentiment could have potentially influenced votes for early suffrage referenda. However, I find no evidence of non-parallel pre-trends in the main specifications. The parallel trends assumption also effectively rules out other contemporaneous shocks. To address this possibility, I will present robustness checks controlling for the presence of alcohol prohibition laws as well as laws granting married women equal property rights. As an additional robustness check, in some specifications, I control for linear interpolations of the white share, the female share, occupational income score, and LIDO score (a proxy for income) (Saavedra and Twinam Reference Saavedra and Twinam2020).
In this context, the no-anticipation assumption essentially means that newspapers could not predict women gaining the right to vote a calendar year in advance. While this assumption may seem strong at first, it aligns well with the historical evidence. Votes for women’s suffrage were often narrowly decided or unsuccessful. For instance, Tennessee was the 36th state to ratify the Nineteenth Amendment and the last one necessary to change the U.S. Constitution. Several states had already rejected the amendment, and Tennessee was widely expected to follow suit. Tennessee General Assembly member Harry T. Burn was expected to vote against ratification but unexpectedly changed his vote at the last minute. As a result, Tennessee ratified the amendment by a margin of one vote.Footnote 19
Early suffrage referenda would also have been difficult to predict. For example, women’s suffrage was on the ballot in Ohio in 1912 and 1914, and it failed both times. State Representative James A. Reynolds then introduced a bill to give women presidential suffrage in 1917, which was later signed by Governor Cox. A few months later, there was a referendum on the bill, which the voters rejected, effectively repealing presidential suffrage before women could vote in a presidential election. Only two years later, Ohio granted women presidential suffrage and became the fifth state to ratify the Nineteenth Amendment. It seems unlikely that a newspaper in 1917, following the failed referendum, would have anticipated the change in policy just two years later.
A threat to identification would be if suffrage in one state affects neighboring states. There is no doubt that some newspapers covered both national and regional news. National news, such as a major convention and federal legislation, would be accounted for through the time trends. To further address this concern, I produce estimates with region-by-decade effects, region-by-year effects, and regional linear-time trends.
Although I focus on the Callaway and Sant’Anna estimator, I also run the following two-way fixed effects regression to make my results comparable to the extant literature:
where y st is the average sentiment or topic proportion of newspaper pages in state s in time t. The parameters α s and β t are state and year fixed effects, respectively. The variable 
is the year that women gained the vote in state s, π k measures placebo treatment effects from the pre-treatment period, and δ k measures dynamic treatment effects during the post-treatment period. I estimate treatment effects for eight years before and after women’s suffrage. The standard errors are clustered at the state level. The two-way fixed effects estimator requires the additional assumption of treatment-effect homogeneity, and thus I use Callaway and Sant’Anna as the preferred specification.
In addition to this analysis, I estimate heterogeneous effects along several dimensions. First, I estimate the effect for full-suffrage and partial-suffrage states (states in which women were only granted presidential or primary suffrage). Second, I estimate the effect when the sample is restricted to only newspapers with a partisan affiliation (Democratic or Republican). Next, I merge the newspaper county of publication with the female share of that county in the 1920 census (Ruggles et al. Reference Ruggles, Fitch, Ronald Goeken, Matt Nelson, Schouweiler and Sobek2021). I then estimate effects for newspapers in counties with below and above median female population shares.
RESULTS
Sentiment Results
Figure 4 displays the effect of women’s suffrage on newspaper coverage of suffrage.Footnote 20 Panel A presents the results for the percentage of newspaper pages that mention the word “suffrage.” The bars denote uniform 95 percent confidence intervals. Mentions of the word suffrage spiked by approximately 2 percentage points the year women’s suffrage passed.Footnote 21 After three years, the estimated treatment effect becomes negative and statistically significant, indicating a decline of 2 to 3 percentage points in suffrage pages from the pre-treatment period. This is consistent with interest peaking when the status of women’s suffrage changed in the immediate term, but waned as it became settled law within that state, leading to a decline in media coverage in the medium- and long-run. The immediate spike is consistent with news media covering the law that just passed (and, in some sense, may even be considered mechanical). I refer to this finding as the coverage effect. Panels B and C demonstrate the coverage effect for pro-suffrage and anti-suffrage pages. Both pro- and anti-suffrage pages increased in the year women’s suffrage was passed, subsequently followed by a decline below the baseline.
ESTIMATES OF THE EFFECT OF SUFFRAGE ON THE PROPORTION OF PAGES THAT MENTION SUFFRAGE
Notes: This figure demonstrates the coverage effect. Each observation is the percent of all newspaper pages in that state-year cell that contain either suffrage pages, pro-suffrage pages, or anti-suffrage pages, respectively. Each bar represents a uniform 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level. N=1,749 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
The long-run decline in suffrage discussion is less obvious and is perhaps surprising given that this only uses data from the pre-Nineteenth Amendment era. This is consistent with interest in suffrage (and perhaps the Nineteenth Amendment) decreasing (or at least growing more slowly) in states where women already had the right to vote. To the extent that women’s suffrage associations could influence media coverage by determining the locations of meetings, protests, and parades, this is also consistent with suffrage associations shifting their efforts away from states that enfranchised women toward states where women were still disenfranchised. I will provide evidence of this channel in the topic modeling section.
The pre-treatment effects are near zero and do not exhibit a systematic trend. This suggests that the interest of newspapers in suffrage was not increasing in the years leading up to state-level suffrage. This might be because the passage of suffrage laws, as well as the votes that enfranchised women, may have been difficult to predict. That pre-trends appear approximately parallel lends credibility to the assumption that trends would have remained parallel in the absence of early suffrage laws.
As depicted in Figure 5, pro-suffrage sentiment declined over the long run in response to women’s suffrage (which I refer to as the opposition effect). The figure shows the event study for the percentage of newspaper pages containing the word “suffrage” that are pro-suffrage. The first panel displays the baseline results. Panels 2 to 4 add controls for decade-by-region effects, year-by-region effects, and region-time trends, respectively. Adding these effects will account for the fact that there may be regional shocks, such as a neighboring state passing women’s suffrage. Relative to anti-suffrage pages, pro-suffrage pages increased the year women’s suffrage passed. However, the increase in sentiment was transitory and generally statistically insignificant. Six to eight years post-enactment, the percentage of suffrage pages that are pro-suffrage declined by 3 to 5 percentage points. In conjunction with the results from the previous figure, this indicates that although both pro- and anti-suffrage pages declined in the medium run, pro-suffrage pages declined at a faster rate, making the average suffrage page more likely to contain anti-suffrage language. This is consistent with the notion that movements opposing the status quo tend to be more energized; once women’s suffrage became law, the opposition had more reason to protest than the advocates did. There is no evidence of non-parallel pre-trends and no evidence that accounting for regional effects changes the results. Because some states are not observed for the full eight years post-suffrage, it is possible these results are driven by states that adopted women’s suffrage relatively early.
EVENT-STUDY ESTIMATES FOR THE PROPORTION OF SUFFRAGE PAGES THAT ARE PRO-SUFFRAGE
Notes: This figure demonstrates the opposition effect. Each observation is pro-suffrage pages as a percent of suffrage pages. Each bar represents a uniform 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level. For all four panels, y̅=0.596, and N=1,748 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
Online Appendix Figure A.3 displays the same figure for only treated states that can be observed for the full eight years post-suffrage (dropping states treated from 1912–1919 for which we have some attrition). For this sub-population, the opposition effect remains null in the first two years, becomes negative and significant around year three, and mostly remains negative thereafter. Both the main result and this robustness check are consistent with suffrage having no effect on sentiment in the short run. The longer-run decline is consistent with women’s suffrage organizations ceasing operations after women received the vote and the anti-suffrage movement remaining more energized relative to the pro-suffrage movement. Another possibility is that medium- and long-run declines only held for the early-adopting states during a period when women’s suffrage may have been more controversial.
Topic Model Results
This section presents the topic model results. Figure 6 presents the suffrage topic model results for newspaper pages in a given topic as a share of all newspaper pages. A newspaper page that is 30 percent about a topic is coded as 0.3 of a page for that topic, while a page that does not contain the word “suffrage” at all (and hence contains no suffrage topics) is coded as zero. Pages about universal suffrage, legislative activity, and women’s suffrage associations all show an increase in the year women’s suffrage is passed and a decline in the medium- and long-run after the passage of women’s suffrage. This is consistent with women’s suffrage associations shifting focus to other states or even disbanding and legislators moving on to other policies that were not settled law. Perhaps surprisingly, pages on protests or demonstrations are largely unaffected by early suffrage laws. This is most consistent with protests making national or international news, and those protests receiving coverage even in states where women already have the vote. The pre-trends are parallel for all topics.
EVENT-STUDY ESTIMATES FOR SUFFRAGE TOPICS AS A SHARE OF ALL PAGES
Notes: The topics come from an LDA model in which K=4. Each observation is the percent of all newspaper pages that contain a given topic. Each bar represents a 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level using the wild bootstrap. N=1,749 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
Figure 7 presents the topics as a percentage of suffrage pages (thus, pages without any mention of suffrage are dropped). The results suggest that early suffrage laws decreased the percentage of suffrage pages that were about associations or legislative activities and increased the percentage that were about universal suffrage or protests and demonstrations.
EVENT-STUDY ESTIMATES FOR SUFFRAGE TOPICS AS A SHARE OF SUFFRAGE PAGES
Notes: The topics come from an LDA model in which K=4. Each observation is the percent of suffrage pages that contain a given topic. Each bar represents a 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level. N=1,749 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
As shown in Figure 8, suffrage increased the discussion of women in politics but not in other areas of public life. The top-left panel shows results for women in politics. In the wake of suffrage laws, the discussion of women in political life increased for three years. After four years, the effect returned to baseline. I refer to this as the women-in-politics effect. It may be tempting to think this spike is driven solely by increased coverage of women’s suffrage, but suffrage articles only spike in the year in which suffrage was passed. Furthermore, three years is well beyond a typical news cycle, even in the era before cable news and the internet. The top-right panel displays the results for work and war production, the bottom-left shows the results for other job advertisements, and the last panel displays women participating in associations or clubs. None of these topics show a response to early suffrage laws. The pre-trends appear parallel for all topics.
EVENT-STUDY ESTIMATES FOR WOMEN TOPICS AS A SHARE OF WOMEN PAGES
Notes: The topics come from an LDA model in which K=15. Each bar represents a 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level. N=1,749 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
Women’s suffrage could have plausibly altered public views toward women in politics, the workplace, or public groups and societies. However, it seems less plausible that early suffrage laws affected topics related to fashion or advertising. Figure 9 presents the results for topics unlikely to be affected by women’s suffrage, which can be thought of as placebo topics. These topics include fashion, clothing advertisements, health care, and health remedies or patent medicines. The effects of all four topics are largely null. The clothing topics are noisy but negative; however, much of this appears to be driven by non-parallel pre-trends. Taken together, I find no evidence that these placebo topics are affected by early suffrage laws.
EVENT-STUDY ESTIMATES FOR PLACEBO WOMEN TOPICS
Notes: The topics come from an LDA model in which K=15. Each bar represents a 95 percent confidence interval. Estimates use the Callaway and Sant’Anna estimator and are clustered at the state level. N=1,749 state-year cells.
Source: Data are from the Chronicling America newspaper archive.
One possibility is that there was only an increase in women’s topics because of newspaper entries and exits. For example, if political newspapers targeted toward women voters opened after suffrage, it could be that the discussion of women in politics increased even if there was no change in women’s narratives in existing newspapers. This hypothesis does not appear to be the case. In an omitted analysis, I re-ran the event study with newspaper-level fixed effects. The results are largely unchanged, with the discussion of women in politics increasing for several years before returning to baseline.
Heterogeneity and Robustness
This subsection further explores the heterogeneity and robustness of the three main findings: (1) the coverage effect, in which newspapers discuss suffrage more frequently in the year it passes, with coverage declining in the long run as it becomes established law; (2) the opposition effect, where the average suffrage page becomes more anti-suffrage in the medium/long run following the passage of women’s suffrage, consistent with movements opposing the status quo receiving more coverage; and (3) the women-in-politics effect, in which there is an increase in discussion about women engaging in political activity in the years immediately following the implementation of suffrage.
The results are robust to estimating the event study using a two-way fixed effects estimator (see Online Appendix Figure A.4). The top panel depicts the coverage effect, where the coverage of suffrage increases by 2 percentage points the year suffrage is passed, followed by a long-run decline in suffrage articles by approximately 1 percentage point. The middle panel indicates that articles about suffrage are approximately 2 percentage points less likely to be pro-suffrage (and consequently 2 percentage points more likely to be anti-suffrage) in the long run; however, these estimates are not quite statistically significant. The bottom panel reveals an increase in articles about women in politics for approximately three years, with an increase of about half a percentage point. To contextualize this effect, only 4 percent of articles that mention women fall under the women-in-politics topic.
As demonstrated in Online Appendix Figure A.5, the results are robust to restricting the sample to either only Republican or only Democratic newspapers. The results reveal similar effects for both Republican and Democratic newspapers. The coverage effect is observed for both parties, with an increase in the year of women’s suffrage followed by a long-run decline. Both display a long-run decline in the percentage of suffrage articles that are pro-suffrage (the opposition effect). The women-in-politics effect appears noisier for Republican newspapers because the confidence intervals for some periods are quite large. However, a closer inspection of the first treatment periods shows a similar increase in the magnitude of discussions of women in politics shortly after suffrage. This increase is statistically significant and there are no significant pre-trends. A similar effect is observed for Democratic newspapers but only for the first year.
Women did not have equal voting rights in all early suffrage states. Some states granted women full suffrage (where women had the same voting rights as men), whereas others only granted women partial suffrage (presidential-only suffrage or primary-only suffrage). Online Appendix Figure A.6 presents the results for full-suffrage or partial-suffrage states only. The classification of states as having either full or partial suffrage comes from Lott and Kenny (Reference Lott and Kenny1999) and Miller (Reference Miller2008). The results are predominantly driven by states with full suffrage. These states exhibit the coverage effect, the opposition effect, and the women-in-politics effect. In contrast, the partial-suffrage states do not demonstrate the coverage effect, and the opposition effect is present only in one year. The women-in-politics effect is observed in both full- and partial-suffrage states for the initial year. However, the partial-suffrage states show a decline in the women-in-politics topic over the long term. This could be due to partial suffrage making it more challenging to transition to full suffrage later and potentially reducing the likelihood of achieving full suffrage compared to states that did not pass partial suffrage at all.
Online Appendix Figure A.7 displays heterogeneity based on the share of females in the county of publication according to the 1920 census. The results seem more pronounced for counties with a below-median female share, which often includes counties in western states where women were granted voting rights earlier. The findings reveal a distinct coverage effect and women-in-politics effect for these below-median counties. Similarly, the opposition effect is also present, though only marginally statistically significant. For the above-median counties, there is a long-term decline in coverage as well.
I consider two final robustness checks that account for potentially contemporaneous events. The first check introduces time-varying controls. I linearly interpolate the share of women, the white share, the occupational income score, and the LIDO score (a proxy for income) between census years at the county level (Saavedra and Twinam Reference Saavedra and Twinam2020). I then merge these controls with the newspaper publication county before aggregating to the state level. Results controlling for these four variables appear in Online Appendix Figure A.8. All three effects are present in this analysis, although some of the standard errors are slightly wider. Second, I control for the presence of two state-level laws that were often supported by suffragists: laws granting married women equal property rights and alcohol prohibition laws (see Online Appendix Figure A.9). The equal property rights law dates are from Geddes and Lueck (Reference Geddes and Lueck2002), and prohibition law dates are from Law and Marks (Reference Law and Marks2020). Again, there is no evidence that controlling for these laws overturns the coverage effect, the opposition effect, or the women-in-politics effect.
While unlikely, it is possible that women’s suffrage is correlated with the quality of the data or the number of OCR errors. To address this possibility, I consider all the words in the LOC data that appear near “women” before Porter stemming and dropping infrequent words, and then I determine what percentage of words is in the English dictionary. Of those words, 83 percent are in the English dictionary. This is approximately in line with other estimates of OCR error rates in Chronicling America as well as other historical newspaper archives (Jatowt et al. Reference Jatowt, Mickael Coustaty and Doucet2019). It should be noted that not all words not in the English dictionary are OCR errors. For example, many proper nouns are not English words. To test whether OCR errors are associated with treatment, the fraction of words that appears in the English dictionary becomes the dependent variable, and a null result would imply that the OCR quality is not associated with early suffrage laws. The results are in Online Appendix Figure A.10, which shows no evidence that women’s suffrage is associated with the quality of the data.
An Analysis of the Nineteenth Amendment
In this subsection, I analyze the effect of the Nineteenth Amendment on newspaper narratives. I restrict the sample to the years 1916 to 1922, after which the newspaper database becomes sparse.
The challenge of examining the federal amendment rather than state-level suffrage laws is the absence of an evident control group that remains untreated after 1920. As Goodman-Bacon (Reference Goodman-Bacon2021) shows, using previously treated observations as a control group can identify parameters that have the opposite sign of the average treatment effect on the treated if dynamic treatment effects exist. Thus, for this analysis, I use states in which women were granted suffrage no later than 1913 as the control group, dropping any states that granted women’s suffrage between 1914 and 1919. This imposes the assumption that any treatment effects after 1916 were constant. This is a stronger assumption than in the previous subsections, and these results should be interpreted cautiously.
Rather than repeating this exercise for the entire paper, I focus on average sentiment for newspaper pages that mention suffrage and the topics of newspaper pages that mention women. I estimate the following regression:
where y ist is the sentiment or topics on newspaper page i from state s in year t. The parameters α s and β t are state and year fixed effects, respectively.Footnote 22 The variable Nineteenths is an indicator variable equal to one if state s obtained women’s suffrage via the Nineteenth Amendment, and k < –1 represents pre-treatment effects while k ≥ 0 represents dynamic treatment effects. Standard errors are clustered at the state level.
The results for sentiment are in Figure 10. The point estimates are statistically insignificant, which is consistent with the previous results, as the opposition effect took several years to show up.
THE EFFECT OF THE NINETEENTH AMENDMENT ON SENTIMENT
Notes: Data are from 1916 to 1922. States that gave women the vote between 1914 and 1919 are dropped. Standard errors are clustered at the state level.
Source: Data come from newspaper pages from the Chronicling America newspaper archive from 1880–1922.
Figure 11 presents the results of the topic analysis. The top-left panel presents the results for the women-in-politics topic, the top-right panel presents results for work and war production, the bottom-left panel presents results for other job advertisements, and the bottom-right panel presents results for women’s associations. The results are consistent with the previous subsections. Following the Nineteenth Amendment, for at least the next three years, there is an increase in the discussion of women in politics in states where women were newly enfranchised. One pre-treatment period is also statistically significant. Among the other topics, only work and war production is marginally significant in one year.
THE EFFECTS OF THE NINETEENTH AMENDMENT ON WOMEN TOPICS
Notes: Data are from 1916 to 1922. States that gave women the vote between 1914 and 1919 are dropped. Standard errors are clustered at the state level. All models include state and year fixed effects. Topics come from an LDA model with K=15.
Source: Data come from newspaper pages from the Chronicling America newspaper archive from 1880–1922.
CONCLUSION AND DISCUSSION
There is a large literature on how women’s suffrage affected government spending, health, and education. This paper extends this literature by showing that women’s suffrage also impacted an important aspect of social norms and culture: public discourse in American newspapers.
I provide evidence that women’s suffrage changed how newspapers discussed voting rights. Pages mentioning suffrage increased in the year women obtained the vote and decreased below baseline in the medium and long term. This pattern is true for both pro- and anti-suffrage pages. However, the decline is faster for pro-suffrage pages, resulting in relatively more anti-suffrage language in American newspapers. This is consistent with newspaper language moving in opposition to the status quo. This may be because newspapers view it as their role in society to challenge the status quo, that protest movements that challenge the status quo are more energized, or that readers demand perspectives that challenges the status quo.
Topic modeling also suggests that early suffrage laws changed how newspapers discussed women, at least in the short run. Following the enactment of these laws, discussions about women in politics increased. It should be noted that not all articles on this topic are about suffrage per se. Many articles in the political topic include, for example, women signing petitions for policies unrelated to voting rights. Despite the narrative shift in newspapers regarding women’s place in politics, I find no evidence of an increase in discussions about women in the workplace or women participating in associations. Thus, early suffrage laws altered interest in suffrage, how newspapers discuss suffrage, and narratives about women in politics temporarily. However, narratives about women in newspapers did not change in a broader sense. As the effects are observed only for a few years, state-level suffrage laws cannot account for longer-run changes in how the media discusses women. It is perhaps surprising that narratives about women did not change for a longer period of time when one considers (1) other research has found a long-run effect of women’s suffrage on public spending and public health, and (2) narratives surrounding women in public discourse clearly evolved during the twentieth century.
Several caveats should be kept in mind. First, newspaper sentiment is not the same as public opinion. Unfortunately, I am unaware of any large-scale public opinion polls during this time (1880–1922). However, the political slant of newspapers has been shown to largely reflect consumer preferences, suggesting that newspaper sentiment may reflect public opinion (Gentzkow and Shapiro Reference Gentzkow and Shapiro2010). Secondly, how newspaper sentiment changes in response to new rights may depend on public opinion and how favorably the public views the group in question. It is plausible that newspapers would react differently to an expansion of women’s rights than, for instance, to an expansion of rights for individuals with felony convictions.
Another point of caution is inherent to almost all empirical work in natural language processing. Many variables that empirical microeconomists explore have a ground truth (e.g., wages or employment). Language, however, is complex, and a ground truth interpretation of language does not exist, at least not without controversy. This problem is not a result of machine learning methods. Two well-trained individuals reading the same text may interpret it differently. Thus, representations of language are not necessarily “true,” but they can be useful. A different machine learning model may yield a different representation of the data, which may lead to new interpretations. In such cases, however, it is likely that both interpretations of the data are useful, rather than one being wrong and the other being right.
This paper investigates how women’s suffrage affected media language, but other areas of the newspaper industry remain unexplored. This paper examines whether suffrage changed the language surrounding voting rights or women. However, it is also possible that the newly enlarged electorate changed the demand for political news more generally (e.g., foreign affairs or coverage of elections). Other questions not answered in this paper include: Did women’s suffrage affect the total number of newspapers, newspaper competition, the number of journalists, or the percentage of journalists who are female? One challenge in answering these questions is that census data on occupations is only available every ten years, although progress could possibly be made on these issues from other sources. These questions are beyond the scope of this paper and are left for future research.
Open access
