Skip to main content Accessibility help

Telephone versus Online Survey Modes for Election Studies: Comparing Canadian Public Opinion and Vote Choice in the 2015 Federal Election

  • Charles Breton (a1), Fred Cutler (a1), Sarah Lachance (a1) and Alex Mierke-Zatwarnicki (a2)


Election studies must optimize on sample size, cost and data quality. The 2015 Canadian Election Study was the first CES to employ a full mixed-mode design, aiming to take advantage of the opportunities of each mode while preserving enough commonality to compare them. This paper examines the phone interviews conducted by ISR-York and the online questionnaires from panellists purchased from a sample provider. We compare data quality and representativeness. We conduct a comprehensive comparison of the distributions of responses across modes and a comparative analysis of inferences about voting. We find that the cost/power advantages of the online mode will likely make it the mode of choice for subsequent election studies.

Les études électorales doivent optimiser la taille des échantillons, leur coût et la qualité des données. L’Étude électorale canadienne de 2015 a été la première ÉÉC qui a adopté un plan à mode de collecte mixte, visant à tirer parti des possibilités de chacun des modes tout en préservant suffisamment d’éléments communs pour permettre la comparaison. Cet article examine les interviews téléphoniques menées par l'Institut de recherche sociale (IRS) de l'Université York et les questionnaires des répondants en ligne achetés auprès d'un fournisseur d’échantillons. Nous comparons la qualité des données et la représentativité. Nous effectuons une comparaison complète des distributions des réponses selon les modes et une analyse comparative des inférences au sujet du vote. Nous constatons que du point de vue des avantages coûts-efficacité, il est probable que le mode en ligne représentera le mode de choix des études électorales ultérieures.


Corresponding author

Department of Political Science, University of British Columbia, 1866 Main Mall, Vancouver BC, V6T1Z1, email: (corresponding author)
Department of Political Science, 1866 Main Mall, Vancouver BC, V6T1Z1, email: (corresponding author)
Department of Political Science, 1866 Main Mall, Vancouver BC, V6T1Z1, email:
Department of Government, Harvard University, 1737 Cambridge St., Cambridge MA 02138, email:


Hide All
Achen, Christopher. H. 2005. “Let's put garbage-can regressions and garbage-can probits where they belong.” Conflict Management and Peace Science 22: 327–39.
Ansolabehere, Stephen and Schaffner, Brian. 2014. “Does survey mode still matter? Findings from a 2010 multi-mode comparison.” Political Analysis 22: 285303.
Berinsky, A.J., Margolis, Michele F. and Sances, Michael W.. 2014. “Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self-Administered Surveys.” American Journal of Political Science 58: 739–53.
Biemer, Paul P. 2010. “Total survey error: Design, implementation, and Evaluation.” Public Opinion Quarterly 74: 817–48.
Bytzek, Evelyn and Bieber, Ina E.. 2016. “Does survey mode matter for studying electoral behaviour? Evidence from the 2009 German Longitudinal Election Study.” Electoral Studies 43: 4151.
Chang, Linchiat and Krosnick, Jon. A.. 2009. “National surveys via RDD telephone interviewing versus the internet comparing sample representativeness and response quality.” Public Opinion Quarterly 73: 641–78.
DeSimone Justin., A., Harms, P.D. and DeSimone, Alice J.. 2014. “Best practice recommendations for data screening.” Journal of Organizational Behavior 36: 171–81.
Dillman, Donald A., Smyth, Jolene D. and Christian, Leah M.. 2014. Internet, phone, mail, and mixed-mode surveys: The tailored design method. New York: John Wiley & Sons.
Fournier, Patrick, Cutler, Fred and Soroka, Stuart S.. 2012. “Who Responds to Election Campaigns? The Two-Moderator Model Revisited.” Paper presented at the conference Duty and Choice: Participation and Preferences in Democratic Elections, Université de Montréal, January 20.
Fournier, Patrick, Cutler, Fred, Soroka, Stuart S., Stolle, Dietlind and Bélanger, Éric. 2013. “Riding the orange wave: leadership, values, issues, and the 2011 Canadian election.” Canadian Journal of Political Science 46: 863–97.
Groves, Robert M. and Lyberg, Lars. 2010. “Total survey error: Past, present, and future.” Public Opinion Quarterly 74: 849–79.
Johnston, Richard G.C. and Brady, Henry E.. 2002. “The rolling cross-section design.” Electoral Studies 21: 283–95.
Kreuter, Frauke, Presser, Stanley and Tourangeau, Roger. 2009. “Social desirability bias in CATI, IVR, and Web surveys the effects of mode and question sensitivity.” Public Opinion Quarterly 72: 847–65.
Loewen, P. and Rubenson, D.. 2015. “The Local Parliament Survey.” June 1, 2017).
Pasek, Josh. 2016. “When will Nonprobability Surveys Mirror Probability Surveys? Considering Types of Inference and Weighting Strategies as Criteria for Correspondence.” International Journal of Public Opinion Research 28: 269–91.
Pickup, Mark and Johnston, Richard G.C.. 2008. “Campaign trial heats as election forecasts: Measurement error and bias in 2004 presidential campaign polls.” International Journal of Forecasting 24 (2): 272–84.
Sala, Emanuela and Lillini, Roberto. 2015. “Undercoverage Bias in Telephone Surveys in Europe: The Italian Case.” International Journal of Public Opinion Research 29 (1): 133–56.
Sanders, David, Clarke, Harold D., Stewart, Marianne C. and Whiteley, Paul. 2007. “Does mode matter for modeling political choice? Evidence from the 2005 British Election Study.” Political Analysis 15: 257–85.
Shin, Eunjung, Johnson, Timothy P. and Rao, Kumar. 2012. “Survey mode effects on data quality: Comparison of web and mail modes in a US national panel survey.” Social Science Computer Review 30: 212–28.
Stephenson, Laura B. and Crête, Jean. 2011. “Studying Political Behavior: A comparison of internet and telephone survey.” International Journal of Public Opinion Research 23 (1): 2455.
Vannieuwenhuyze, Jorre, Loosveldt, Geert and Molenberghs, Geert. 2010. “A method for evaluating mode effects in mixed-mode surveys.” Public Opinion Quarterly 74: 1027–45.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Canadian Journal of Political Science/Revue canadienne de science politique
  • ISSN: 0008-4239
  • EISSN: 1744-9324
  • URL: /core/journals/canadian-journal-of-political-science-revue-canadienne-de-science-politique
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
Type Description Title
Supplementary materials

Breton supplementary material
Figures A1-A4 and Tables A1-A2

 Word (879 KB)
879 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed