An Area of Agreement: Open-Ended Interviewing in Academic Opinion Research

The World War II era was a period of both collaboration and vigorous intellectual debate between commercial pollsters—including George Gallup, Elmo Roper, and Archibald Crossley—and academic researchers, who often worked either side by side or at cross-purposes within government-sponsored survey initiatives (Herbst Reference Herbst1993; Igo Reference Igo2006). In 1939, the Bureau of Agricultural Economics recruited Rensis Likert, a Columbia-trained psychologist, to run its Program Surveys Department. Likert is best known for developing psychometric attitude scales, but during his time at Program Surveys he became a staunch defender of using open-ended interviewing to understand people’s attitudes (J. M. Converse Reference Converse2017).

Although Likert had no fondness for open-ended interviewing before joining Program Surveys, he inherited an operation that relied heavily on these methods to study farmers’ thinking about the government’s agricultural policies (J. M. Converse Reference Converse2017). As Jean Converse writes, “[Program Surveys] interviewers … had no written questionnaires—just topics to bring up when it seemed natural in the course of conversation” (271). Likert eventually developed an appreciation for this kind of openness in interviews. As Program Surveys shifted its focus during the war to design surveys on military and civilian morale, he sought to standardize interviews while retaining the flexibility that allowed interviewees to articulate their thoughts freely, using their own words and without being restricted by predefined options (Skott Reference Skott1943, 289). As described by Angus Campbell (Reference Campbell1945, 340), a psychologist who worked with Likert on the Program Surveys team and later cofounded the SRC, the open-ended question “is sufficiently flexible to permit related attitudes to come into the interview even though they have not been specifically asked for.”

But the additional insights offered by open-ended questions introduced considerable time and cost. Interviewers required extensive training on how to probe responses and then code them using complicated procedures. In part for this reason, Campbell and Likert’s endorsement of open questions received a fierce critique from commercial pollsters Elmo and Roper, who consulted for another group of government survey researchers, the “Polls Division” (J. M. Converse Reference Converse2017). The pollsters had been using large-scale, close-ended questionnaires for national election forecasts since 1936, and they claimed to achieve “adequate validity” (J. M. Converse Reference Converse1984, 272) at a much lower cost than the academics’ surveys. In reality, as Sarah Igo (Reference Igo2006) documents, the pollsters frequently prioritized financial concerns over scientific rigor: Open-ended interviews were too costly and thus were abandoned.

By the end of the war, the “open versus closed” debate had spilled out of the narrow realm of bureaucratic disagreement and into the public sphere (J. M. Converse Reference Converse1984). In the wake of the Gallup poll’s failure to predict Truman’s win in 1948, Likert—now head of the newly founded SRC—took to the pages of Scientific American to argue that it had failed for two reasons: (1) Gallup still relied largely on quota rather than area sampling, and (2) it had not included any “open-questions” (Likert Reference Likert1948). Likert pointed to the “real” reason why commercial pollsters would not invest in these more rigorous methods: “The major disadvantage of these methods (open questioning and probing), as with area sampling, is that they are somewhat more expensive than the more conventional polling techniques. Here again, this disadvantage is outweighed by the greater accuracy” (11).

In the midst of these debates, Paul F. Lazarsfeld, a Columbia social scientist, offered an integratory approach between the “open” and “closed” camps. In 1944, he published an article in which he elaborated on the utility of “open interviews” (OIs, as Lazarsfeld referred to them), emphasizing their inductive potential and their capacity to illuminate the “motivational context” within which interviewees operate. As he wrote, “The trained OI field worker has the goal of his inquiry clearly imprinted in his mind, but he adapts his inquiry to the concrete situation between the interviewee and himself. If properly conducted, such an OI will result in a detailed document which covers the whole area under investigation, including the interviewer’s observations of the respondent’s reactions and background” (Lazarsfeld Reference Lazarsfeld1944a, 39–40).

Much like Likert and Campbell, Lazarsfeld agreed that the main drawback of this approach was its expense, and he proposed cost-effective solutions that included relying on open-ended questions for deviant case analysis and placing them at the beginning and end of interviews for better analytic leverage. In sum, although pollsters favored close-ended questions for practical reasons, academic researchers associated with the Michigan and Columbia Schools remained steadfast in their commitment to open-ended interview questions on methodological grounds.

Open-Ended Interviews and the Early Voter Studies

Reflecting their belief in the importance of open-ended interviews, both schools published early studies relying on mixed-methods designs—even if these were not the terms that the researchers used in the 1940s and 1950s—that incorporated open-ended interviewing. Notably, it was the open-ended questions that formed the basis of many of the theories of political behavior that remain most central to the field today.

Lazarsfeld, Bernard Berelson, and other colleagues from the Bureau at Columbia were the first to publish. Their first monograph, The People’s Choice, was based on data collected during the 1940 presidential election in Erie County, Ohio, and their second monograph, Voting, was based on data collected in Elmira, New York, during the 1948 presidential election. At the core of both the 1940 and 1948 studies were panel surveys. Consistent with Lazarsfeld’s ideas about how to combine open- and close-ended questions in one interview, the Columbia studies mixed detailed response options with open-ended questions about the issues dividing the parties and how respondents were thinking about their vote choice (see the appendix to Voting). Because these interviews were conducted in person, fieldworkers had the flexibility to follow up and probe responses, thereby leveraging the inductive potential of in-depth interviews.

The Columbia School’s approach proved fruitful. Although the authors rarely cited these data in Voting, such citations were much more common in The People’s Choice (see chap. xvi, 49, 84–86, 91–93, and 97–100). Moreover, there is evidence that the responses to open-ended questions informed several of their key conclusions, in particular the idea that influence from people’s social circles shapes their vote choice. For example, in a 1944 paper published in advance of The People’s Choice, Lazarsfeld cited an interviewee who explained how she made up her mind about her vote choice between October and November: “My husband persuaded me to vote for Wilkie. He was opposed to the third term…. The lady where I worked wanted me to vote—took me to the polls and they all voted Republican so I did, too” (Lazarsfeld Reference Lazarsfeld1944b, 326). Such pieces of qualitative evidence were an enduring hallmark of the Columbia School and a key paradigm within the discipline (e.g., Carlson Reference Carlson2019; Druckman, Levendusky, and McLain Reference Druckman, Levendusky and McLain2018).

Shortly after The People’s Choice was published, researchers at the SRC in Michigan turned their attention to American political behavior. As Jean Converse writes, this happened somewhat by happenstance: SRC scholars included a few voting questions in a 1948 survey on foreign policy, and they found that their results were better than most polls at predicting the election outcome (recall Likert’s critique of the pollsters). After receiving funding from the Carnegie Foundation in 1952 and from the Rockefeller Foundation thereafter, the SRC developed a biennial survey, which we now know as the American National Election Studies (ANES; J. M. Converse Reference Converse2017, chap. 10).

In 1954, the SRC team published The Voter Decides (Campbell, Gurin, and Miller Reference Campbell, Gurin and Miller1954) based on the 1948 and 1952 surveys, followed by the much more widely cited The American Voter (Campbell et al. Reference Campbell, Converse, Stokes and Miller1960), which included additional data from 1956. Both texts, as well as Converse’s classic work on the lack of ideological coherence in the mass public (which drew on the same data [P. E. Converse Reference Converse2006]), relied extensively on open-ended questions to develop the central theoretical contributions for which they are best known. For example, Campbell, Gurin, and Miller (Reference Campbell, Gurin and Miller1954, 91–92) quoted several of their interviewees’ reflections on their party affiliation that the researchers used to develop the tenets of their theory about partisan identity:

As was true for the Columbia scholars, Campbell and coworkers’ open-ended interviews seem to have been central to their theoretical innovation. It is difficult to imagine them coming up with such a counterintuitive model of political behavior—the idea that a quasi-inherited group membership could structure so much political behavior even when it lacked ideological content—without having collected and analyzed these kinds of data (see also Stokes Reference Stokes1966). Very much in line with the suggestions that Lazarsfeld made in the 1940s, the qualitative portions of the interview provided the data from which the researchers built their theory, and the quantification of those responses (the correlation between party ID and vote choice) provided a test of whether that theoretical relationship was statistically representative.

Perhaps it is because the National Election Studies (NES) team saw how useful open-ended questions were for developing new theories of political behavior that they remained committed to them for decades: In 1952, open-ended questions made up fully one-third of the NES questions (according to Jean Converse’s tabulation). Up until 1972, the surveys began with 10 to 15 open-ended questions about interviewees’ evaluations of the parties, the presidential candidates, the problems they saw in the country, and whether they wanted the government to intervene in these (Burns Reference Burns2006). Despite their merits, however, open-ended questions were eventually excluded from the ANES.

Areas of Disagreement: The Appropriate Methods for Developing Generalizable Claims

The second methodological debate during this period concerned sampling. Likert again played a central role. As his comments in Scientific American indicate, Likert criticized commercial pollsters for their reliance on quota rather than area sampling: In the former approach, participants were chosen at the interviewer’s discretion, rather than with a known probability. As Likert argued, this hampered researchers’ abilities to make accurate inferences to the population (J. M. Converse Reference Converse2017). But Likert’s argument also contained an implicit critique of the so-called community studies that were mainly associated with Robert and Helen Lynd’s Middletown studies, which served as intellectual precursors to the Columbia School’s work.

In two books published in 1929 and 1937, the Lynds conducted in-depth fieldwork in Muncie, Indiana, to understand social and cultural changes in a “typical” American community. These community studies had their roots in an earlier tradition of settlement house studies. Settlement houses were Progressive-era institutions that housed educated women and men in poor neighborhoods, where they carried out research and teaching to better understand the social problems of the poor (Katz and Sugrue Reference Katz, Sugrue and Michael1998a). Wealthy philanthropists funded the scholarship of pioneering social scientists, such as W.E.B. DuBois (DuBois Reference DuBois1899) and Jane Addams and her team (Carr Reference Carr1895), who combined participant observation and historical analysis with careful collection of social statistics to develop detailed portrayals of the social, political, and economic conditions shaping life at the margins of American society (Deegan Reference Deegan2017; Katz and Sugrue Reference Katz and Sugrue1998b; Morris Reference Morris2015).Footnote ⁸

The tradition of community studies associated with Robert and Helen Lynd at the end of the 1920s owed much to the settlement house scholars, although the overwhelmingly White and male academy largely failed to credit them (Deegan Reference Deegan2017; Morris Reference Morris2015).Footnote ⁹ However, in one major way, the Lynds departed from the settlement houses’ focus on the specific dynamics of community life in one neighborhood: The Lynds assumed that an “average” American city could be identified as an idealized microcosm (hence the pseudonym “Middletown”) in which social processes could be studied at the local level and then generalized to the broader American public. Rather than claiming to provide detailed studies of the social and political conditions of one group, one neighborhood, or one city, as had DuBois and the Hull House team, the Middletown studies claimed to achieve something akin to statistical representativeness.Footnote ¹⁰ This later became the source of survey researchers’ central critique: No study of a single community could achieve such representativeness (Igo Reference Igo2007).

By the time the researchers at the Columbia School began their work, they were well aware of this criticism. As a result, their approach straddled the qualitative logic of careful attention to social process, context, and inductive theory building and the quantitative logic of statistical representativeness. On the one hand, they understood the limitations of large-N surveys and took a quasi-ethnographic approach to their field sites, collecting a wide range of materials to address empirical puzzles in the service of theory development. In addition to the closed- and open-ended panel survey that formed the spine of the book, the research team also conducted interviews with party leaders and members of community organizations. One member of the research team lived in the field sites for several months and conducted ethnographic observations of organizational and public meetings. The team also collected “pertinent materials” to campaigning and other organizational activities, as well as local radio, newspaper, and magazine coverage of the campaign (Berelson, Lazarsfeld, and McPhee Reference Berelson, Lazarsfeld and McPhee1954; Lazarsfeld, Berelson, and Gaudet Reference Lazarsfeld, Berelson and Gaudet1948). The combination of interviews with organizational leaders and ethnographic observation informed several of their key findings about the extent to which political parties do (or do not) matter for residents’ political behavior (see Voting, Berelson, Lazarsfeld, and McPhee Reference Berelson, Lazarsfeld and McPhee1954, chaps. 3 and 8).

But on the other hand, the Columbia School took for granted the pursuit of statistical representativeness as the goal of research design. Because of this, they proactively addressed concerns about their research by attempting to distinguish themselves from the Lynds. They argued that, although their studies were focused on specific communities, they were “not anthropological or sociological investigations of a community… (in the Lynd tradition)” (Voting, Berelson, Lazarsfeld, and McPhee Reference Berelson, Lazarsfeld and McPhee1954, 4, emphasis in the original). And yet, they justified their case selection using the same logic as the Lynds. They chose to study Elmira, for example, because it was “an ordinary American town” that was moderately sized, not part of a “metropolitan district but not isolated,” with a “typical ethnic composition” (6). At the same time, they also recognized that “generalizations in the future will depend heavily on repetition and comparison at different times and places, and there is great benefit in pinning down each study to a particular—not necessarily ‘typical’—context” (4). In short, the Columbia School scholars vacillated between understanding Elmira as distinct and “typical.”Footnote ¹¹

But ultimately, they failed to defend the possibility of producing more general theory from qualitative case studies according to a criterion distinct from statistical representativeness. As such, the Columbia School left itself vulnerable to criticism from Michigan scholars on those grounds. For example, drawing on just the voting questions incorporated in their 1948 survey, Morris Janowitz and Warren Miller (Reference Janowitz and Miller1952) critiqued Columbia’s The People’s Choice from 1940, arguing that its findings were not generalizable because it focused just on Erie County, Ohio. Specifically, their contention was that Lazarsfeld’s “Index of Political Predisposition” was not particularly predictive of voting behavior in 1948, as it had been in Lazarsfeld’s case and times.Footnote ¹²

The Voter Decides (Reference Campbell, Gurin and Miller1954)—with Campbell, Gerald Gurin, and Warren Miller at the helm—reemphasized these earlier critiques, claiming that methodological failures produced substantive errors, with too much focus on sociological rather than psychological dimensions of political behavior. In contrast, they argued that the best way to understand the American voter was through “an approach at the level of attitudes, expectations, and group loyalties, the psychological variables which intervene between the external events of the voter’s world and his ultimate behavior” (85–86). The “sociological” approach of Lazarsfeld and colleagues (Lazarsfeld et al. Reference Lazarsfeld, Berelson and Gaudet1948, 27)—the idea that “a person thinks, politically, as he is socially”—was not borne out in the data from these more recent elections.

Somewhat ironically, amidst the Columbia School’s ambivalent pursuit of statistical representativeness, it was these Michigan scholars who clarified how the Columbia School’s fieldwork approach could coexist in a symbiotic relationship with nationally representative surveys of American political behavior. As Campbell, Gurin, and Miller (Reference Campbell, Gurin and Miller1954, 204) wrote in the appendices to The Voter Decides, their findings about “interpersonal forces” are only preliminary, because “a national sample survey is not the method of choice for a study of group influence processes.”Footnote ¹³

Michigan scholars clearly did not anticipate that the Columbia School’s fieldwork approach would become marginalized so quickly; nevertheless, it largely did. After Personal Influence (Katz and Lazarsfeld Reference Katz and Lazarsfeld1955), the Columbia School did not publish another book taking a case study approach. Moreover, Columbia scholars were not invested in the study of American political behavior per se. If Lazarsfeld’s driving ambition was the study of human action and motivation, the institution he built within the Bureau and the students he advised reflected that breadth (Logemann Reference Logemann2019). As Barton (Reference Barton2001, 264) writes, the Bureau was neither a technique-oriented center like the SRC nor a subject-matter center; rather, it was an unspecialized center that cobbled together funds to support the research interests of its founder. As such, when Lazarsfeld’s attention was drawn to different subjects from the mid-1950s on, so too was the Bureau’s.

The Decline of Qualitative Methods

The intellectual “victory” for nationally representative surveys had several consequences for the study of American political behavior. First, the Columbia School’s failure to articulate a justification for case studies meant that fieldwork disappeared almost entirely from the set of approaches most valued in the field—although it persisted to some extent in studies of Congress and other institutions (Fenno Reference Fenno1978). Second, the reliance on national surveys further undermined in-depth interviewing as a method: The Michigan School’s commitment to national probability sampling meant that the inclusion of open-ended interviewing was incredibly expensive, leading to its eventual abandonment. As Jean Converse (Reference Converse2017, 369) argues in her historical account of survey research, budgetary constraints were a major concern: “Campbell was aware of the change [the decline in open-ended questions]. When asked why this had happened, he said, ‘Why else?’—meaning money.” The expense of sending highly qualified national field personnel every two years to conduct open-ended interviews and then code them laboriously was no longer feasible, especially in nationally representative samples.Footnote ¹⁴

The result of these mounting intellectual critiques and practical barriers was that the field moved quite rapidly from widespread agreement on the importance of qualitative methods for understanding mass political behavior to near-total quantification. As Eleanor Singer (Reference Singer1987) declared in the introduction to the fiftieth anniversary of Public Opinion Quarterly in 1987, it was time to celebrate the “triumph of the quantitative.” This outcome seemed to become institutionalized as qualitative and quantitative research developed as different “cultures” (Goertz and Mahoney Reference Goertz and Mahoney2012) and subfield and disciplinary boundaries hardened (Pierson Reference Pierson2007). As a result, scholars of American political behavior had to pursue training outside their field to become practitioners of qualitative methods or to learn how to assess the rigor of qualitative work (on this point, see Small and Calarco Reference Small and Calarco2022).

Moreover, in a field where the default assumption for many decades has been a purely quantitative research design, the costs of deviating from that norm—and the costs to move the field off this self-reinforcing pathway—are perilously high (Pierson Reference Pierson2000). Just as funding exigencies shaped the trajectory of both Columbia and Michigan School scholarship, funders today often include recommendations for proposal writing that emphasize hypothesis testing, privileging a deductive logic to research design.Footnote ¹⁵ As Holmes and colleagues (Reference Holmes, Guliford, Mendoza-Davé and Jurkovich2023, 1) recently argued, this is because the quantitative victory we just described “is deeply linked with the application of empirical methods to data to yield causal arguments.” Although qualitative approaches frequently produce causal arguments, as we discuss later, they often do so through a combination of “thick description” of social contexts (Geertz Reference Geertz2008), “surprising” findings, and the resolution of empirical puzzles that emerge after research has begun. Such projects can be poorly suited to meet the requirements of funding proposals and the expectations of reviewers trained in the same deductive approaches.

Meanwhile, the same intellectual arguments lodged against “community studies” in the 1950s persist today: How representative is this case? Questions like this belie the logic of qualitative research, the goal of which is not to produce statistical representativeness but rather to develop novel theoretical and empirical insights that may extend (under certain conditions) to other contexts. Nevertheless, their persistence continues to challenge the face validity of qualitative research—not just in American political behavior but also across the social sciences (on this point, see Small Reference Small2009). In short, the methodological closure that took place at this crucial moment in the institutionalization of the field has persisted. And perhaps most importantly, it has had significant repercussions for the theories employed in the study of American political behavior, as forecasted by Campbell, Gurin, and Miller in 1954 (on this point, see Cohen Reference Cohen1999; Lee Reference Lee2002; Pierson Reference Pierson2007; Robison et al. Reference Robison, Stevenson, Druckman, Jackman, Katz and Vavreck2018; Rogers Reference Rogers and Mosley2013).

Theoretical Innovation through the Discovery of Surprising Findings

Interviews and ethnography provide valuable opportunities for theory testing (Brady, Collier, and Seawright Reference Brady and Collier2010; Gerring Reference Gerring2007), but their particular advantage lies in their capacity for theoretical innovation (Small Reference Small2009). Although there has been extensive discussion of the relationship between theory and empirical analysis among qualitative methodologists (Burawoy Reference Burawoy1998; Glaser and Strauss Reference Glaser and Strauss1967; Timmermans and Tavory Reference Timmermans and Tavory2012), most scholars agree that qualitative-analytic approaches are distinguished by their openness to surprise during research (Timmermans and Tavory Reference Timmermans and Tavory2012). As Fenno (Reference Fenno1978, 250) wrote of the “soaking and poking” method (or participant observation), researchers “fully expect that an open-minded exposure to events in the milieu and to the perspectives of those with whom they interact will produce ideas that might never have occurred to them otherwise.”

The openness that Fenno describes is key not only at the beginning of a qualitative study but also throughout the entire research process (Collier Reference Collier2011; Small Reference Small2009). Interviewers and ethnographers often begin their fieldwork with a relatively adaptable research question and preliminary expectations, allowing for the exploration of unforeseen developments as the study progresses. This strategic openness proves invaluable when researchers immerse themselves in a particular context and encounter unexpected findings that challenge conventional notions or commonly held ideas (George and Bennett Reference George and Bennett2005). The effectiveness of qualitative approaches lies in their ability to recognize these moments of surprise, pursue emerging questions, and use these insights to develop novel theories (Timmermans and Tavory Reference Timmermans and Tavory2012). These theories can be systematized through within-case theory testing and inference (Brady and Collier Reference Brady and Collier2010) or out-of-sample testing, such as experiments to isolate causal mechanisms or analyses of observational data to assess their generalizability and heterogeneity (Soss Reference Soss1999).

Katherine Cramer’s Reference Cramer2016 pathbreaking work, The Politics of Resentment: Rural Consciousness and the Rise of Scott Walker, highlights the critical importance of flexibility and openness during qualitative research. This study, much like the early investigations into American political behavior, takes vote choice as its dependent variable. Specifically, Cramer asks, What factors explained vote choice in the 2012 Wisconsin gubernatorial election? As she recounts, Cramer began her study anticipating that class dynamics would be central to Wisconsin politics. However, after attending 37 regular “kaffeeklatsches” in 27 communities throughout the state, she became attuned to an unexpected dimension of her participants’ political behavior: Cramer discerned a distinct political subjectivity, which she terms “rural consciousness,” that served as a central framework shaping her interviewees’ perceptions of their relationship with the government. Rather than primarily identifying as members of a socioeconomic class, they viewed themselves as rural inhabitants marginalized by governmental policies, often to the advantage of urban residents. This spatially grounded anti-statism, as Cramer argues, explains why numerous Wisconsin residents—hailing from a state that was a historical labor movement stronghold—rallied behind Republican Scott Walker for governor, despite his agenda aimed at dismantling unions.

Cramer’s interview guides (see appendix C) indicate the importance of open-ended questions for allowing such findings to emerge from conversation. For example, the methodological appendix includes questions such as “What do you think are the major issues facing people in [this town] these days? What do you think should be done about this? What has been overlooked?” (Cramer Reference Cramer2016, 233). Although Cramer ultimately probes the specifics of people’s opinions about immigration, healthcare, taxes, and the University of Wisconsin system, these broad, open-ended questions were what allowed her to happen upon a surprising insight— her interlocutors’ sentiment that everything is overlooked in their communities.

Cramer’s work serves as a vivid illustration of the theory-building and research-inspiring role that qualitative research can have. It has spawned a wide literature on how rural resentment informs contemporary election outcomes (Borwein and Lucas Reference Borwein and Lucas2023; Jacobs and Munis Reference Jacobs and Munis2023) and inspired several other studies that also take a qualitative, inductive approach to theory building (Cramer and Toff Reference Cramer and Toff2017; Ternullo Reference Ternullo2024; Weaver, Prowse, and Piston Reference Weaver, Prowse and Piston2019). And although Cramer’s work is an archetype of a surprising finding leading to an innovative new research agenda, all the empirical studies that we highlight rely, to some measure, on emergent findings.

Theoretical Innovation through Research Design

In addition to building theory from emergent empirical insights, qualitative scholars may also pursue theoretical innovation through research design. Although qualitative approaches will never approximate statistical representativeness, that is only one way to develop generalizable (or transportable) theory (Donmoyer Reference Donmoyer, Gomm, Hammersley and Peter2000). Qualitative work aims at a different kind of representation: not by looking outward from a case for its representation of the “typical” but by “casing”, delving ever further into the historical and contemporary details of a single case to assess how to abstract from particular findings to more general (although always conditional) conclusions (Amenta Reference Amenta2009).

In fact, qualitative work that aims to achieve statistical representativeness by pursuing a research design like that of the Columbia School (the “typical” case) often fails doubly—by not achieving what quantitative studies could along this metric and by failing to maximize the true potential of qualitative studies (Small Reference Small2009). Having long recognized this bind, qualitative scholars instead choose cases based on their potential for theory development (Griffin and Ragin Reference Griffin and Ragin1994). This process is sometimes referred to as “theoretical sampling” (Glaser and Strauss Reference Glaser and Strauss1967; but see Tavory and Timmermans Reference Tavory and Timmermans2014 for a critique). There are several approaches to case selection in qualitative research design (see Seawright and Gerring Reference Seawright and Gerring2008), but many draw on controlled comparisons to isolate unexplained variation or focus on one case that presents a “theoretical anomaly” (e.g. Rogowski 2004; Seawright Reference Seawright2016).Footnote ¹⁶ Such comparisons allow researchers to identify patterned differences and develop new theories about the mechanisms that produce the variation of interest (Lichterman and Reed Reference Lichterman and Reed2015; Tavory and Timmermans Reference Tavory and Timmermans2014).

Researchers often use their own quantitative analysis of survey or administrative data to identify interesting cases for comparison, such as deviant cases, empirical puzzles, or theoretically ambiguous cases. Scholars have referred to such mixed-methods designs as “quantitative data as a starting point for qualitative analysis” (Tarrow Reference Tarrow, Brady and Collier2004) or “nested analyses” (Lieberman Reference Lieberman2005). But whether researchers rely on their original analysis of quantitative data (in a mixed-methods study) or on existing literature (in a purely qualitative study) to select cases, they retain the flexibility to choose the case(s) that are best suited to generate new theoretical insights (George and Bennett Reference George and Bennett2005; Seawright and Gerring Reference Seawright and Gerring2008; Skocpol and Somers Reference Skocpol and Somers1980; Small Reference Small2009).

Stephanie Ternullo’s Reference Ternullo2024 book, How the Heartland Went Red: Why Local Forces Matter in an Age of Nationalized Politics, offers a recent example of theoretical sampling in research design. Ternullo chooses both “on line” and “off line” cases for comparison (Lieberman Reference Lieberman2005). Using variation in county-level voting trajectories as a starting point, she selects three White working-class counties that were integral to the New Deal coalition in the 1930s and 1940s but have diverged in political alignment since the racial realignment. One county shifted to Republican voting in the 1960s, another swung to the right in 2016 after being a “swing” county for several decades, and in a rare case, Democratic voting persisted in a third county, like only 4% of other New Deal counties. Within each county, Ternullo focused fieldwork in the postindustrial cities that dominated local economic and political outcomes.

After conducting four rounds of in-depth interviews with residents in each of these three towns, Ternullo reveals the mechanisms driving this variation. She shows that local organizational contexts play a pivotal role, shaping how residents understand their social problems and identities and link them to party politics. Specifically, she attributes the enduring Democratic partisanship in the deviant case to the local labor movement’s historic and ongoing political mobilization. These unions provide residents with coherent cultural frameworks for understanding postindustrial social problems as rooted in systemic economic declines that have disadvantaged them and their community and for understanding the Democratic Party as the party that represents society’s “have-nots” by bringing in the state to level the playing field. By focusing on the variation in the three counties’ voting trajectories, Ternullo derives broader insights into how local contexts have influenced the rise of right-wing populism in White postindustrial cities and, more generally, how place intersects with race, class, and religion to shape and sustain partisan attachments. Her study highlights how attention to research design is one of the key strengths of qualitative approaches to the study of political behavior: Researchers’ flexibility in designing a study with original data collection and without the aim of statistical representativeness allows them to focus on a single case or on multiple cases that will maximize theoretical innovation (see also Rogers Reference Rogers2006).Footnote ¹⁷

Identifying How Context Shapes Meaning-Making

Another cornerstone of qualitative research is its focus on meaning-making. Many qualitative approaches take as their starting point that people make sense, interpret, and evaluate their environments in different ways. This means that the link between the objective and subjective is unknowable a priori and, further, that these subjective meanings matter for social and political outcomes because they can shape the way people behave. Neighboring disciplines, such as sociology and anthropology, have developed rich traditions to chart these “webs of signification” (Geertz Reference Geertz1978) by analyzing styles (Lichterman and Eliasoph Reference Lichterman and Eliasoph2014), repertoires (Swidler Reference Swidler2001), narratives (Polletta Reference Polletta1998), and social boundary making (Lamont Reference Lamont1992), among others.

Careful descriptive work detailing the context is often a crucial first step (Geertz Reference Geertz2008; Holmes et al. Reference Holmes, Guliford, Mendoza-Davé and Jurkovich2023), because it allows researchers to capture the observable dimensions of social contexts and the multifaceted ways in which individuals experience and interpret them—how they perceive, explicate, and take action (Cohen Reference Cohen1999; Lamont and Swidler Reference Lamont and Swidler2014; Mettler and Soss Reference Mettler and Soss2004; Michener, SoRelle, and Thurston Reference Michener, SoRelle and Thurston2022; Pugh Reference Pugh2013). As such, the analytical purchase of interview and ethnographic methods lies in their avoidance of rigid assumptions about the meanings assigned to particular entities or the reason for certain relationships—rather than infer the meaning, qualitative scholars ask participants to “create” those meanings in front of them (Hochschild Reference Hochschild1981, 24). Individuals are not entirely unrestricted in their ascriptions of meaning but rather are intricately entwined in a preexisting world of meaning (Tavory and Timmermans Reference Tavory and Timmermans2014, 26). In-depth interviews can illuminate this tension between constraint and creativity, revealing how individuals draw on different aspects of their experience of context in the meaning-making process (Lamont and Swidler Reference Lamont and Swidler2014; Pugh Reference Pugh2013).

For these reasons, in-depth interviews and ethnographic observations are ideally situated to elucidate the links between objective social and economic contexts and political behavior by detailing the processes through which individuals attribute meaning and adjudicate among different interpretations of their context. As such, qualitative studies may stand alone in developing new ideas about how voters are interpreting the political world. Alternatively, scholars may use those findings to create better measurements of quantitative variables— whether captured in a survey or survey experiment or used to develop a new measure of contextual variables (Gest Reference Gest2016; Thachil Reference Thachil2018).

This is the approach executed so powerfully in Jamila Michener’s Reference Michener2018 book Fragmented Democracy: Medicaid, Federalism, and Unequal Politics, which relies on in-depth interviews to develop and measure new contextual variables with quantitative data. Her work follows a long-standing tradition of qualitative and mixed-methods research in studies of “policy feedbacks” (Soss Reference Soss1999; Reference Soss2000; Mettler Reference Mettler2002), which has been carried on in other recent work on the (de)mobilizing consequences of carceral contact, school closures, and the civil legal system (Michener, SoRelle, and Thurston, Reference Michener, SoRelle and Thurston2022; Nuamah Reference Nuamah2023; Walker Reference Walker2020; Weaver, Prowse, and Piston Reference Weaver, Prowse and Piston2019). These studies have produced a body of research that shows how a diverse range of experiences with the state shape Americans’ political participation.

Michener’s work reveals how the political experiences of Medicaid beneficiaries vary within the same social program—across states and even neighborhoods—because of federalism and localized policy implementation. This focus was not, as Michener notes at the outset, part of her original research question but rather a finding that emerged from her field research as she delved into the contextually embedded meaning-making of her respondents. For example, she found that spatial inequalities within cities can exacerbate the negative experiences of marginalized citizens with Medicaid: “Neighborhoods mattered most for those upon whom they conferred the most disadvantage: African Americans living in poverty. And they mattered enough that beneficiaries consistently referenced neighborhoods with no prompting on my part… [because] I did not fully grasp the relevance of neighborhoods” (Michener Reference Michener2018, 121). Michener herself notes that this finding was unexpected because “the city” is “a frontier that few scholars recognize as relevant to either federalism or Medicaid” (14). In short, one of the most innovative elements of Michener’s work—a focus on spatial inequalities operating on multiple geographic levels—stems from the inductive approach she took to the interviews.

Unlike contextual studies that rely on existing theory to decide which measures of context might shape political behavior, Michener’s approach allows participants themselves to explain what elements of their social worlds matter—even if they think of these elements as “apolitical.” Moreover, these approaches recognize that contexts themselves are the products of historic political processes, which create the social and political material within which members of the mass public make meaning (Gest Reference Gest2016; Mettler and Soss Reference Mettler and Soss2004).

Tracing Dynamic Processes of Change

In-depth interviews and ethnographic methods can also provide valuable new insights into change and the transformation in American mass politics by borrowing from perspectives on change that are already well developed in other fields. Historical case researchers, for example, have developed a particular approach to process tracing that involves a distinct research logic in terms of case selection, causality, and counterfactual reasoning (Goertz and Mahoney Reference Goertz and Mahoney2013; Mahoney Reference Mahoney2010; Pierson Reference Pierson2007; Tarrow Reference Tarrow, Brady and Collier2004). As with process tracing in historical studies, repeated interviews or ethnographic observations allow researchers to pay careful attention to within-case variation over time and the sequencing of different “building blocks” of observations that enable the researcher to infer causal relationships (Collier Reference Collier2011; George and Bennett Reference George and Bennett2005; Gerring Reference Gerring2012; Mahoney Reference Mahoney2010).

Such approaches provide information beyond what can be gained through the primary tools that behavior researchers use to study stability and change over time—longitudinal surveys, repeated cross sections, or longitudinal electoral returns—in part through the other modes of inquiry we have already described. Although longitudinal quantitative studies can document stability or change, repeated ethnographic observation or in-depth interviews can adapt to unravel surprising puzzles—for example, why there is stability where we might have expected change or why do we observe change among some groups but not others—by attending to the way that participants are making sense of external shocks and diving into the experience of their social contexts to explain heterogeneity. Researchers can observe what Tavory and Timmermans (Reference Tavory and Timmermans2014, 27) refer to as “spirals” of meaning-making as meanings change from one social situation to the next. As such, scholars may rely on longitudinal qualitative studies on their own or pair them with longitudinal surveys to help address emergent puzzles.

Sally Nuamah’s compelling 2021 book, Closed for Democracy: How Mass School Closure Undermines the Citizenship of Black Americans, offers an example of how longitudinal interview and ethnographic studies can document the way changes in meaning-making unfold over time (see also Zepeda-Millán Reference Zepeda-Millán2017). As with Michener’s work, Nuamah’s book provides a novel framework for studying a core concept in political behavior research: participation. Nuamah conducts interviews with parents and community leaders threatened by public school closures in Chicago and Philadelphia and observes community meetings where closure policies are publicly discussed. She uses longitudinal interviews to assess how the experience of fighting to keep their schools open, over time, shapes her interviewees’ views on their political efficacy and their willingness to reengage in the future. Although they are confident in the beginning, the process of interacting with the political administration—regardless of whether the particular school is closed or not—causes their attitudes toward political engagement to sour (Nuamah Reference Nuamah2021, 1124, 1125). Nuamah develops this argument through repeated interviews, which reveal “what citizens eventually learn through their repeated participation” (1124, 1125)—in short, that the notion of a self-invigorating democratic participation is a myth, especially if political authorities are unresponsive. Alongside Nuamah’s work, a set of recent qualitative studies show that long-term observation in field sites (Barnes Reference Barnes2020), triangulation among multiple sources of qualitative evidence (Zepeda-Millán Reference Zepeda-Millán2017), and longitudinal interviews (Ternullo Reference Ternullo2022) are well suited to describing how processes of meaning-making unfold on the ground, often in response to exogenous changes or long-term experiences that shape political learning.

The Limitations of Qualitative Approaches

Thus far, we have extolled the comparative advantages of qualitative research designs; of course, this also means that these methods have comparative weaknesses. One of the most obvious perhaps is the one we have already discussed: their inability to offer statistical representativeness. Although this is not a goal of qualitative research and thus not grounds for a critique of any qualitative study’s rigor, it remains a limitation. It underscores the importance of a methodologically pluralist field, in which some studies offer evidence of the effects of campaign arguments on national (or even subgroup) public opinion—for example, how images from recent severe catastrophes lead Democrats and Republicans to think differently about climate change solutions—whereas others pursue explanations rooted in single cases, which help elucidate the mechanisms behind these effects (e.g., to what extent do Republicans in climate-threatened communities think about their routine problems and concerns as partisan or ideological) (Levy Paluck Reference Levy Paluck2010; Gerring Reference Zepeda-Millán2004). The latter study offers a different kind of representation—of how people in a particular social position, context, or period are drawing political meaning from their experiences. Both research approaches are essential to understanding American political behavior.

Second, in a discipline that privileges causal explanations (Holmes et al. Reference Holmes, Guliford, Mendoza-Davé and Jurkovich2023), qualitative studies, as we have argued, have a lot to offer (e.g., Nuamah’s work explaining the participatory effects of school closures). But even as qualitative research can develop causal arguments (King, Keohane, and Verba Reference King, Keohane and Verba1994), it cannot identify causal effects (on this point, see Gerring Reference Zepeda-Millán2004).Footnote ¹⁸ This is a fairly obvious point: The estimation of unbiased causal effects is a quantitative task—and, as with achieving statistical representativeness, it is also an important one. Each is one goal, among many, that scholars of American political behavior should pursue. When their goals fall into one of the four modes of inquiry we have highlighted, our argument is that they should incorporate qualitative methods into their research design.