Hostname: page-component-6bf8c574d5-gr6zb Total loading time: 0 Render date: 2025-02-15T07:55:20.971Z Has data issue: false hasContentIssue false
  • English
  • Français

Quebec 2018: A Failure of the Polls?

Published online by Cambridge University Press:  06 December 2019

Claire Durand  and
André Blais
Claire Durand*
Affiliation:
Département de sociologie, Université de Montréal, C.P. 6128, succursale Centre-ville, Montréal, H3C 3J7
André Blais
Affiliation:
Département de science politique, Université de Montréal, C.P. 6128, succursale Centre-ville, Montréal, H3C 3J7
*
*Corresponding author. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The polls of the 2018 Quebec election forecast a close race between the two leading parties. The result, a clear victory of the Coalition Avenir Québec (CAQ) over the Parti libéral du Québec (PLQ), was clearly at odds with the polls. We argue that when the polls get it wrong, it is important to determine whether there was a polling miss, in which the discrepancy is due to changing voter behaviour, or a poll failure, in which the problem stems from polling methodology. Our post-election poll shows that changing voter behaviour—last-minute shifts and the vote of non-disclosers—explains most of the discrepancy. These movements varied by region. We conclude that the Quebec 2018 election was among the worst polling misses in history but not necessarily a major poll failure.

Résumé

Résumé

Les sondages de l’élection Québécoise de 2018 avaient annoncé une lutte serrée entre les deux principaux partis. Le résultat, une victoire décisive de la Coalition Avenir Québec (CAQ) aux dépens du Parti Liberal du Québec (PLQ), n’était clairement pas ce qui avait été anticipé. Nous soutenons que, lorsque les sondages se trompent, il est important de déterminer si l’écart entre les estimations des sondages et le vote est dû à un changement dans le comportement des électeurs ou à un échec des sondages eux-mêmes attribuable à la méthodologie utilisée. Notre sondage post-électoral montre que des changements dans le comportement des électeurs—changements de préférences de dernière minute et vote des discrets—expliquent la majeure partie des écarts dans cette élection. Ces mouvements varient toutefois selon les régions. Nous concluons que l’élection québécoise de 2018 se situe historiquement parmi les pires écarts entre les sondages et le vote mais ne peut pas être considérée comme un échec majeur des sondages eux-mêmes.

Keywords

poll failurepolling missQuebec 2018 electionlate campaign shiftéchec des sondageserreur des sondagesÉlection Québec 2018changement de fin de campagne
Type
Research Article/Étude originale
Information
Canadian Journal of Political Science/Revue canadienne de science politique , Volume 53 , Issue 1 , March 2020 , pp. 133 - 150
Copyright
Copyright © Canadian Political Science Association (l'Association canadienne de science politique) and/et la Société québécoise de science politique 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Arzheimer, Kai and Evans, Jocelyn. 2014. “A New Multinomial Accuracy Measure for Polling Bias.” Political Analysis 22 (1): 31–44.10.1093/pan/mpt012CrossRefGoogle Scholar
Baker, Reg, Michael Brick, J., Bates, Nancy A., Battaglia, Mike, Couper, Mick P., Dever, Jill A., Gile, Krista J. and Tourangeau, Roger. 2013. Report of the AAPOR Task Force on Non-Probability Sampling. https://www.aapor.org/Education-Resources/Reports/Non-Probability-Sampling.aspx (February 22, 2019).10.1093/jssam/smt008CrossRefGoogle Scholar
Bodor, Tamas. 2012. “Hungary's ‘Black Sunday’ of Public Opinion Research: The Anatomy of a Failed Election Forecast.” International Journal of Public Opinion Research 24 (4): 450–71.CrossRefGoogle Scholar
Callegaro, Mario and Gasperoni, Giancarlo. 2008. “Accuracy of Pre-Election Polls for the 2006 Italian Parliamentary Election: Too Close to Call.” International Journal of Public Opinion Research 20 (2): 148–70.CrossRefGoogle Scholar
Cleveland, William S. and Devlin, Susan J.. 1988. “Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting.” Journal of the American Statistical Association 83 (403): 596–610.10.1080/01621459.1988.10478639CrossRefGoogle Scholar
Curtice, John. 1997. “So How Well Did They Do? The Polls of the 1997 Election.” Journal of the Market Research Society 39 (3): 449–61.10.1177/147078539703900303CrossRefGoogle Scholar
Duncan, Pamela. 2016. “How the Pollsters Got It Wrong on the EU Referendum.” Guardian, June 24.Google Scholar
Dunford, Daniel and Kirk, Ashley. 2016. “How Right or Wrong Were the Polls about the EU Referendum?” Telegraph, June 20.Google Scholar
Durand, Claire. 2002. “Are Quebec Polls Biased against Liberals?” Policy Options / Options politiques (March 1): 51–56.Google Scholar
Durand, Claire. 2008a. “La Méthodologie des sondages électoraux de l'élection présidentielle française de 2007, chroniques d'un problème récurrent.” Bulletin de méthodologie sociologique 97 (1): 5–17.CrossRefGoogle Scholar
Durand, Claire. 2008b. “The Polls of the 2007 French Presidential Campaign: Were Lessons Learned from the 2002 Catastrophe?” International Journal of Public Opinion Research 20 (3): 275–98.10.1093/ijpor/edn029CrossRefGoogle Scholar
Durand, Claire. 2018. “Quebec 2018: A Tough Night for Pollsters.” Policy Options / Options politiques. https://policyoptions.irpp.org/magazines/october-2018/quebec-2018-a-tough-night-for-pollsters/.Google Scholar
Durand, Claire and Blais, André. 1999. “Why Did the Polls Go Wrong in the 1998 Quebec Election? The Answer from Post Election Polls.” Bulletin de méthodologie sociologique 62 (1): 43–48.CrossRefGoogle Scholar
Durand, Claire, Blais, André and Larochelle, Mylène. 2004. “The Polls in the 2002 French Presidential Election: An Autopsy.” Public Opinion Quarterly 68 (4): 602–22.10.1093/poq/nfh042CrossRefGoogle Scholar
Durand, Claire, Blais, André and Vachon, Sébastien. 2001. “A Late Campaign Swing or a Failure of the Polls? The Case of the 1998 Quebec Election.” Public Opinion Quarterly 65 (1): 108–23.10.1086/320041CrossRefGoogle ScholarPubMed
Durand, Claire, Blais, André and Vachon, Sébastien. 2002. “Accounting for Biases in Election Surveys: The Case of the 1998 Quebec Election.” Journal of Official Statistics 18 (1): 25–44.Google Scholar
Durand, Claire, Johnson, Timothy, Moreno, Alejandro and Traugott, Michael. 2018. Report of the WAPOR Committee Reviewing the Pre-election Polls in the 2017 Presidential Election in Chile. https://wapor.org/wp-content/uploads/Final-WAPOR-Report-on-Chile-2017-Election.pdf.Google Scholar
Jacoby, William G. 2000. “Loess: A Nonparametric, Graphical Tool for Depicting Relationships between Variables.” Electoral Studies 19 (4): 577–613.CrossRefGoogle Scholar
Jennings, Will and Wlezien, Christopher. 2018. “Election Polling Errors across Time and Space.” Nature Human Behaviour 2: 276–83.10.1038/s41562-018-0315-6CrossRefGoogle ScholarPubMed
Jowell, Roger, Hedges, Barry, Lynn, Peter, Farrant, Graham and Heath, Anthony. 1993. “The 1992 British Election: The Failure of the Polls.” Public Opinion Quarterly 57 (2): 238–63.10.1086/269369CrossRefGoogle Scholar
Kennedy, Courtney, Blumenthal, Mark, Clement, Scott, Clinton, Joshua D., Durand, Claire, Franklin, Charles, McGeeney, Kyley, Miringoff, Lee, Olson, Kristen, Rivers, Douglas, Saad, Lydia, Evans Witt, C. and Wlezien, Christopher. 2017. “An Evaluation of 2016 Election Polls in the U.S.” Report of the AAPOR Ad Hoc Committee on 2016 Election Polling. https://www.aapor.org/Education-Resources/Reports/An-Evaluation-of-2016-Election-Polls-in-the-U-S.aspx.10.1093/poq/nfx047CrossRefGoogle Scholar
Kennedy, Courtney, Blumenthal, Mark, Clement, Scott, Clinton, Joshua D., Durand, Claire, Franklin, Charles, McGeeney, Kyley, Miringoff, Lee, Olson, Kristen, Rivers, Douglas, Saad, Lydia, Evans Witt, C. and Wlezien, Christopher. 2018. “An Evaluation of 2016 Election Polls in the U.S.” Public Opinion Quarterly 82 (1): 1–33.10.1093/poq/nfx047CrossRefGoogle Scholar
Léger, Jean-Marc. 2018. “Que s'est-il vraiment passé?” Journal de Montréal, October 6.Google Scholar
Lewis, Peter. 2019. “As Pollsters, We Are Rightly in the Firing Line after the Australian Election. What Happened?” Guardian, May 21.Google Scholar
Loader, Clive. 1999. Local Regression and Likelihood. New York: Springer.Google Scholar
Martin, Elizabeth A., Traugott, Michael W. and Kennedy, Courtney. 2005. “A Review and Proposal for a New Measure of Poll Accuracy.” Public Opinion Quarterly 69 (3): 342–69.10.1093/poq/nfi044CrossRefGoogle Scholar
Mellon, Jonathan and Prosser, Christopher. 2017. “Missing Nonvoters and Misweighted Samples: Explaining the 2015 Great British Polling Miss.” Public Opinion Quarterly 81 (3): 661–87.10.1093/poq/nfx015CrossRefGoogle Scholar
Mitofsky, Warren J. 1998. “Was 1996 a Worse Year for Polls than 1948?” Public Opinion Quarterly 62 (2): 230–49.10.1086/297842CrossRefGoogle Scholar
Mosteller, Frederick. 1949. “Measuring the Error.” In The Pre-election Polls of 1948: Report to the Committee on Analysis and Pre-election Polls and Forecast, ed. Mosteller, Frederick, Hyman, Herbert, McCarthy, Philip J., Marks, Eli S. and Truman, David B.. New York: Social Science Research Council.Google Scholar
Prosser, Christopher and Mellon, Jonathan. 2018. “The Twilight of the Polls? A Review of Trends in Polling Accuracy and the Causes of Polling Misses.” Government and Opposition 53 (4): 757–90.10.1017/gov.2018.7CrossRefGoogle Scholar
Puleston, Jon. 2017. “Are We Getting Worse at Political Polling?” ESOMAR Publications Series Volume Congress 2017. https://www.esomar.org/uploads/public/events-and-awards/events/2017/congress/documents/ESOMAR-Congress-2017_185446_61_Puleston.pdf.Google Scholar
Sturgis, Patrick, Baker, Nick, Callegaro, Mario, Fisher, Stephen, Green, Jane, Jennings, Will, Kuha, Jouni, Lauderdale, Benjamin E. and Smith, Patten. 2016. Report of the Inquiry into the 2015 British General Election Opinion Polls. The British Polling Council and the Market Research Society, London. http://eprints.ncrm.ac.uk/3789/1/Report_final_revised.pdf.Google Scholar
Sturgis, Patrick, Kuha, Jouni, Baker, Nick, Callegaro, Mario, Fisher, Stephen, Green, Jane, Jennings, Will, Lauderdale, Benjamin E. and Smith, Patten. 2018. “An Assessment of the Causes of the Errors in the 2015 UK General Election Opinion Polls.” Journal of the Royal Statistical Society: Series A 181 (3): 757–81.CrossRefGoogle Scholar
Wright, Malcolm J., Farrar, David P. and Russell, Deborah F.. 2014. “Polling Accuracy in a Multiparty Election.” International Journal of Public Opinion Research 26 (1): 113–24.CrossRefGoogle Scholar