This chapter uses history of polling to explain how pollsters have dealt with challenges of nonresponse. It tells the tale of three polling paradigms: large-scale polling, quota sampling, and random sampling. The first two paradigms came crashing down after pollsters made poor predictions for presidential elections. The third paradigm remains vibrant intellectually, but is increasingly difficult to implement. We do not yet know if the bad polling predictions in 2016 and 2020 will push the field to a new paradigm, but certainly they raised doubts about the current state of the field.

This chapter focuses on next-generation selection models that allow us to expand on the Heckman model using copula and control function models that allow one to estimate selection models for a large range of other statistical distributions. This chapter also shows how to generate weights that account for nonignorable nonresponse; not only do these weights increase the weight on demographic groups that respond with lower probabilities, they also increase weights on people with opinions that may make them less inclined to respond. This chapter also shows how to modify a Heckman model to allow for estimation of a nonignorable nonresponse selection model when we have a response-related variable that is available only for people in the survey sample.

This is a brief conclusion arguing that the direction forward is clear, even if the path is not. The time for assuming away problems is past. We should begin with a paradigm that reflects all the ways that polling can go wrong and then identify, model, and measure all the sources of bias, not just the ones that are easy to fix. Much work remains to be done, though, as these new models and data sources will require much evaluation and development theoretically, empirically, and practically. The payoff will be that survey researchers will be able to remain true to their aspirations of using information about a small number of people to understand the realities about many people, even as it gets harder and hear from anyone, let alone the random samples that our previous theory relied on.

This chapter explores ways to diagnose the potential for nonignorable nonresponse to cause problems. Section 7.1 describes how to define the range of possible values of population values that are consistent with the observed data. These calculations require virtually no assumptions and are robust to nonignorable nonresponse; they are simple yet tend to be uninformative. Section 7.2 shows how to postulate possible levels of nonignorability and assess how results would change.

This chapter brings together the argument so far, showing how nonignorable nonresponse may manifest itself and how the various models perform across these contexts, including how they may fail. It also highlights the ideal response to potential nonignorable nonresponse, which involves (1) creating randomized instruments, (2) using the randomized instrument to diagnose nonignorable nonresponse, (3) moving to conventional weights if there is no evidence of nonignorable nonresponse, but (4) using selection models explained here when there is evidence of nonignorable nonresponse. Section 11.1 simulates and analyzes data across a range of scenarios using multiple methods. Section 11.2 discusses how to diagnose whether nonresponse is nonignorable. Section 11.3 integrates the approaches with a decision tree based on properties of the data. Section 11.4 discusses how selection models can fail.

This chapter describes contemporary practices of probabilistic and nonprobabilistic pollsters. First, even pollsters who aspire to random sampling are doing something quite foreign to the random sampling paradigm. Continuing to use the language of random sampling is therefore becoming increasingly untenable. Second, the energy and growth in polling is concentrated in nonprobabilistic polls that do not even pretend to adhere to the tenets of the random sampling paradigm. When we use, teach, and critique such polls, we need a new language for assessing them. Finally, one of the biggest vulnerabilities for both probabilistic and nonprobabilistic polling is nonignorable nonresponse, something largely ignored in the current state of the art. It is striking that despite the incredible diversity of techniques currently deployed, academic and commercial pollsters mostly continue to use models that assume away nonignorable nonresponse.

This chapter illustrates how to use randomized response treatments to assess possible nonresponse bias. It focuses on a 2019 survey and shows how nonignorable nonresponse may have deflated Trump support in the Midwest and among Democrats even as nonignorable nonresponse inflated Trump support among Republicans. We also show that Democrats who responded to the poll were much more liberal on race than Democrats who did not respond, a pattern that was particularly strong among White Democrats and absent among non-White Democrats. Section 12.1 describes a survey design with a randomized response instrument. Section 12.2 discusses nonignorable nonresponse bias for turnout questions. Section 12.3 looks at presidential support, revealing regional and partisan differences in nonignorable nonresponse. Section 12.4 looks at race, focusing on partisan and racial differences in nonignorable nonresponse. Section 12.5 assesses nonignorable nonresponse on climate, taxes, and tariffs.

This chapter presents the intuition behind why nonignorable nonresponse can be a problem and how it can arise in many contexts. With a foundation that explicitly centers this possibility, we can better reason through when the problem may be larger, how to diagnose it, and how to fix or at least ameliorate it. Section 5.1 describes qualitatively when nonignorable nonresponse may be likely. Section 5.2 works through the intuition about how and why nonignorable nonresponse undermines polling accuracy. Section 5.3 presents a framework for modeling nonignorable nonresponse and culminates by describing Meng’s (2018) model of sampling error. Section 5.4 raises the possibility that nonignorability varies across groups, over time, and even across questions.