Skip to main content Accessibility help
×
×
Home

Looking Beyond Demographics: Panel Attrition in the ANES and GSS

  • Laura Lazarus Frankel (a1) and D. Sunshine Hillygus (a1)

Abstract

Longitudinal or panel surveys offer unique benefits for social science research, but they typically suffer from attrition, which reduces sample size and can result in biased inferences. Previous research tends to focus on the demographic predictors of attrition, conceptualizing attrition propensity as a stable, individual-level characteristic—some individuals (e.g., young, poor, residentially mobile) are more likely to drop out of a study than others. We argue that panel attrition reflects both the characteristics of the individual respondent as well as her survey experience, a factor shaped by the design and implementation features of the study. In this article, we examine and compare the predictors of panel attrition in the 2008–2009 American National Election Study, an online panel, and the 2006–2010 General Social Survey, a face-to-face panel. In both cases, survey experience variables are predictive of panel attrition above and beyond the standard demographic predictors, but the particular measures of relevance differ across the two surveys. The findings inform statistical corrections for panel attrition bias and provide study design insights for future panel data collections.

Copyright

Corresponding author

e-mail: hillygus@duke.edu (corresponding author)

Footnotes

Hide All

Authors' note: Laura Lazarus Frankel is a Ph.D. candidate in political science at Duke University. D. Sunshine Hillygus is an Associate Professor of Political Science at Duke University. An earlier version of this paper was presented at the Annual Meeting of the American Association of Public Opinion Research. Replication data are available on the Political Analysis Dataverse.

Footnotes

References

Hide All
Atrostic, B. K., Bates, Nancy, Burt, Geraldine, and Silberstein, Adriana. 2001. Nonresponse in US government household surveys: Consistent measures, recent trends, and new insights. Journal of Official Statistics, Stockholm 17: 209–26.
Bartels, Larry M. 1999. Panel effects in the American National Election Studies. Political Analysis 8: 120.
Bartels, Larry M. 2006. Three virtues of panel data for the analysis of campaign effects. In Capturing campaign effects, eds. Brady, Henry E. and Johnston, Richard, 134–63. Ann Arbor: University of Michigan Press.
Behr, Andreas, Bellgardt, Egon, and Rendtel, Ulrich. 2005. Extent and determinants of panel attrition in the European Community Household Panel. European Sociological Review 23: 8197.
Brehm, John. 1993. The phantom respondents: Opinion surveys and political representation. Ann Arbor: University of Michigan Press.
Burchell, Brendan, and Marsh, Catherine. 1992. The effect of questionnaire length on survey response. Quality & Quantity 26(3): 233–44.
Burden, Barry C. 2000. Voter turnout and the National Election Studies. Political Analysis 8(4): 389.
Campanelli, Pamela, and O'Muircheartaigh, Colm. 1999. Interviewers, interviewer continuity, and panel survey nonresponse. Quality & Quantity 33(1): 5976.
Clinton, Joshua D. 2001. Panel bias from attrition and conditioning: A case study of the knowledge networks panel. In AAPOR 55th Annual Conference.
Couper, Mick P., and Miller, Peter V. 2009. Web survey methods: Introduction. Public Opinion Quarterly 72: 831–5.
Couper, Mick P., and Groves, Robert M. 1992. The role of the interviewer in survey participation. Survey Methodology 18(2): 263–78.
Davis, Darren W. 1997. Nonrandom measurement error and race of interviewer effects among African Americans. Public Opinion Quarterly 61(1): 183207.
de Leeuw, Edith Desiree. 1992. Data quality in mail, telephone and face to face surveys. Amsterdam, The Netherlands: TT Publikaties.
Debell, Matthew, Krosnick, Jon A., and Lupia, Arthur. 2010. Methodology report and users guide for the 2008–2009 ANES Panel Study. Technical Report, Stanford University and University of Michigan.
Deng, Yiting, Sunshine Hillygus, D., Reiter, Jerome P., Si, Yajuan, and Zheng, Siyu. 2013. Handling attrition in longitudinal studies: The case for refreshment samples. Statistical Science 28(2): 238–56.
Dennis, J. Micahel, and Li, Rick. 2003. Effects of panel attrition on survey estimates. Paper presented at Annual Meeting of the American Association for Public Opinion Research, Nashville, TN.
Dillman, Don A., Phelps, Glenn, Tortora, Robert, Swift, Karen, Kohrell, Julie, Berck, Jodi, and Messer, Benajmin L. 2009. Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR), and the Internet. Social Science Research 38(1): 118.
Durrant, Gabriele B., Groves, Robert M., Staetsky, Laura, and Steele, Fiona. 2010. Effects of interviewer attitudes and behaviors on refusal in household surveys. Public Opinion Quarterly 74(1): 136.
Fitzgerald, John, Gottschalk, Peter, and Moffitt, Robert. 1998. An analysis of sample attrition in panel data: The Michigan Panel Study of Income Dynamics. Technical Report, National Bureau of Economic Research.
Frankel, Joanne. 1980. Measurement of respondent burden: Study design and early findings. Technical Report. Washington, DC: Bureau of Social Science Research.
Gray, Rebecca, Campanelli, Pamela, Deepchand, Kavita, and Prescott-Clarke, Patricia. 1996. Exploring survey non-response: The effect of attrition on a follow-up of the 1984–85 Health and Life Style Survey. Statistician 45(2): 163–83.
Groves, Robert M., Dillman, Don, Eltinge, John L., and Little, Roderick J.A. 2002. Survey nonresponse. New York: Wiley.
Groves, Robert M., Fowler, Floyd J., Couper, Mick P., Lepkowski, James M., and Singer, Eleanor. 2009. Survey methodology, vol. 561. Hoboken, NJ: John Wiley & Sons.
Groves, Robert M., and Couper, Mick P. 1998. Nonresponse in household interview surveys. New York: Wiley.
Groves, Robert M., Cialdini, Robert B., and Couper, Mick P. 1992. Understanding the decision to participate in a survey. Public Opinion Quarterly 56(4): 475.
Henderson, Michael, Sunshine Hillygus, D., and Tompson, Trevor. 2010. “Sour grapes” or rational voting? Voter decision making among thwarted primary voters in 2008. Public Opinion Quarterly 74(3): 499529.
Hill, Daniel H., and Willis, Robert J. 2001. Reducing panel attrition: A search for effective policy instruments. Journal of Human Resources 36(2): 416–38.
Honaker, James, and King, Gary. 2010. What to do about missing values in time-series cross-section data. American Journal of Political Science 54(2): 561–81.
Honaker, J., King, G., and Blackwell, M. 2011. Amelia II: A program for missing data. Journal of Statistical Software 45(7): 147.
Imai, Kosuke, King, Gary, and Lau, Olivia. 2006. Zelig: Everyone's statistical software. R package version, 27. http://gking.harvard.edu/zelig (accessed February 2013).
Kalton, Graham, Lepkowski, James, Montanari, G. E., and Maligalig, Dalisay. 1990. Characteristics of second-wave nonrespondents in a panel survey. In Proceedings of the Section on Survey Research Methods, American Statistical Association, 462–7.
Keeter, Scott, Kennedy, Courtney, Dimock, Michael, Best, Jonathon, and Craighill, Peyton. 2006. Gauging the impact of growing nonresponse on estimates from a National RDD Telephone Survey. Public Opinion Quarterly 70: 759–79.
Lepkowski, James, and Couper, Mick P. 2002. Nonresponse in the second wave of longitudinal household surveys. In Survey nonresponse, eds. Groves, Robert M., Dillman, Don, Eltinge, John L., and Little, Roderick J. A., 259–72. New York: Wiley.
Lipps, Oliver. 2007. Attrition in the Swiss Household Panel. Methoden-Daten-Analysen 1(1): 4568.
Lipps, Oliver. 2010. Effects of different incentives on attrition and fieldwork effort in telephone household panel surveys. Survey Research Methods 4(2): 8190.
Little, Roderick J.A., and Rubin, Donald B. 2002. Statistical analysis with missing data. Hoboken, NJ: Wiley-Interscience.
Loosveldt, Geert, and Carton, Ann. 1997. Evaluation of nonresponse in the Belgian Election Panel Study 91–95. In Proceedings of the Survey Research Methods Section, American Statistical Association.
Loosveldt, Geert, and Carton, Ann. 2001. An empirical test of a limited model for panel refusals. International Journal of Public Opinion Research 13(2): 173.
Loosveldt, Geert, Pickery, Jan, and Billiet, Jaak. 2002. Item nonresponse as a predictor of unit nonresponse in a panel survey. Journal of Official Statistics, Stockholm 18(4): 545–58.
Lynn, Peter, Buck, Nicholas, Burton, Jonathan, Jackle, Annette, and Laurie, Heather. 2005. A review of methodological research pertinent to longitudinal survey design and data collection. Working paper, Institute for Social and Economic Research, University of Essex.
Martin, Elizabeth, Abreu, Denise, and Winters, Franklin. 2001. Money and motive: Effects of incentives on panel attrition in the survey of income and program participation. Journal of Official Statistics, Stockholm 17(2): 267–84.
Olsen, Randall J. 2005. The problem of respondent attrition: Survey methodology is key. Monthly Labor Review 128: 6371.
Olson, Kristen, and Witt, Lindsey. 2011. Are we keeping the people who used to stay? Changes in correlates of panel survey attrition over time. Social Science Research 40(4): 1037–50.
Pasek, Josh, Tahk, Alexander, Lelkes, Yphtach, Krosnick, Jon A., Keith Payne, B., Akhtar, Omair, and Tompson, Trevor. 2009. Determinants of turnout and candidate choice in the 2008 US presidential election: Illuminating the impact of racial prejudice and other considerations. Public Opinion Quarterly 73: 943–94.
Peress, Michael. 2010. Correcting for survey nonresponse using variable response propensity. Journal of the American Statistical Association 105(492): 1418–30.
Sharp, Laurie M., and Frankel, Joanne. 1983. Respondent burden: A test of some common assumptions. Public Opinion Quarterly 47(1): 36.
Singer, Eleanor. 2002. The use of incentives to reduce nonresponse in household surveys. In Survey nonresponse, eds. Robert, M., Dillman, Don, Eltinge, John L., and Little, Roderick J. A., 163–77. New York: Wiley.
Smith, Tom W., and Son, Jaesok. 2010. An analysis of panel attrition and panel change on the 2006–2008 General Social Survey Panel. Technical Report, NORC, University of Chicago.
Smith, Tom W., and Davis, James A. 1992. The 1992 General Social Survey reinterview. GSS Methodological Report 75. Technical Report, NORC, University of Chicago.
Traugott, Santa, and Morchio, Giovanna. 1990. Assessment of bias due to attrition and sample selection in the ANES 1989 Pilot Study. ANES Technical Report Series, No. NES010154. Technical Report, Center for Political Studies.
Traugott, Santa, and Rosenstone, Steven J. 1994. Panel attrition among the 1990–1992 panel respondents. ANES Technical Report Series, No. NES010163. Technical Report, Center for Political Studies.
Uhrig, S. C. Noah. 2008. The nature and causes of attrition in the British Household Panel Survey. Technical Report, Institute for Social and Economic Research, University of Essex.
Vandecasteele, Leen, and Debels, Annelies. 2007. Attrition in panel data: The effectiveness of weighting. European Sociological Review 23(1): 8197.
Voogt, Robert J. J. 2005. An alternative approach to correcting response and nonresponse bias in election research. Acta Politica 40(1): 94116.
Voogt, Robert J.J., and Saris, Willem E. 2003. To participate or not to participate: The link between survey participation, electoral participation, and political interest. Political Analysis 11(2): 164–79.
Watson, Nicole, and Wooden, Mark. 2009. Identifying factors affecting longitudinal survey response. In Methodology of longitudinal surveys, 157–81. West Sussex, UK: John Wiley & Sons.
Wawro, Gregory. 2002. Estimating dynamic panel data models in political science. Political Analysis 10: 2548.
Zabel, Jeffrey E. 1998. An analysis of attrition in the panel study of income dynamics and the survey of income and program participation with an application to a model of labor market behavior. Journal of Human Resources 33(2): 479506.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed