This chapter explains weighting in a manner that allows us to appreciate both the power and vulnerability of the technique and, by extension, other techniques that rely on similar assumptions. Once we understand how weighting works, we will better understand when it works. This chapter opens by discussing weighting in general terms. The subsequent sections get more granular. Sections 3.2 and 3.3 cover widely used weighting techniques: cell-weighting and raking. Section 3.4 covers variable selection, a topic that may well be more important than weighting technique. Section 3.5 covers the effect of weighting on precision, a topic that frequently gets lost in polling reporting. This chapter mixes intuitive and somewhat technical descriptions of weighting. The technical details in Sections 3.2 and3.3 can be skimmed by readers focused on the big picture how weighting works.

This chapter introduces selection models in a way that highlights important intuition about how they work. Section 8.1 formalizes the model we’ve been working with already. Section 8.2 uses the model to highlight a bad news, good news story. The bad news is that statistical estimation of a two-equation model like this will be challenging. The good news is that the model helps us recognize the traces nonignorable nonresponse leaves in observable data. Section 8.3 introduces the Heckman selection model. Section 8.4 uses the Heckman model to highlight the starkly different way that selection and weighting approaches use information. The Heckman model is far from perfect, however, as Section 8.5 explains.

This chapter explores the challenges of polling in light caused by nonignorable nonresponse. Nonprobability polling approach comes off poorly for reasons that harken back to the Literary Digest fiasco. The random sampling is far from perfect, but here we rename it the random contact approach – because what is random is who they contact, not who responds once contacted – and show that using random contact shifts error from being proportional to the population size – which can be catastrophic – to being proportional to response rates – which is not great, but much better. Section 6.1 assesses the big data approach by introducing the idea of effective sample size, a concept that allows us to compare potentially large nonrandom samples to their random sampling equivalents. Section 6.2 assesses the random contact approach that has become the last refuge of those clinging to the random sampling paradigm. Section 6.3 decomposes sampling error into elements associated with the choosing whom to contact and elements associated with individual choices given that they are contacted. This section helps clarify where the biggest threats are throughout the survey process.

Polling has become very difficult. People do not respond, and pollsters use methods that are far removed from the random sampling tools that built the field. This chapter introduces the book by outlining the main challenges facing polling today, how conventional tools fail to fully meet these challenges and how a new paradigm and new methods can more directly take on the full spectrum of nonresponse bias given contemporary polling practices.

This chapter highlights the critical importance of having the right kind of data for selection models that address nonignorable nonresponse. In general, we need a variable that is included in our response model and excluded from our outcome model. The best approach is creating a randomized response instrument that affects whether someone responds, but does not affect the content of their response. In many polling contexts, it is easy to create randomized response instruments. The pollster simply needs to figure out some protocol that affects response rate and then randomize it. Section 10.1 makes it clear that knowing the correct functional form is not enough to save a selection model. Section 10.2 highlights the difficulty of using observational response instruments. Section 10.3 discusses how and why to create randomized response instruments. Section 10.4 shows how to use randomized response instruments in a simple test for diagnosing nonignorable nonresponse. Section 10.5 shows how randomized response instruments enable us to use the full suite of selection models even when we do not observe data for nonrespondents.