Canadian Polarization and Partisan and Ideological Bias

There is considerable debate in the United States about the nature of mass polarization. Scholars have identified conceptually distinct dimensions of this phenomenon that have become conflated in popular discourse. There is only minimal evidence that Americans are gravitating toward the ideological poles, known as ideological polarization (Fiorina and Abrams, Reference Fiorina and Abrams2008; Fiorina et al., Reference Fiorina, Abrams and Pope2008). Rather, partisans have developed increasingly dissimilar ideological beliefs. This is also known as partisan sorting or partisan polarization (Abramowitz, Reference Abramowitz2010; Baldassarri and Gelman, Reference Baldassarri and Gelman2008; Levendusky, Reference Levendusky2009b). In addition, there appears to have been considerable affective polarization where Democrats and Republicans increasingly dislike one another's parties and their supporters (Iyengar et al., Reference Iyengar, Sood and Lelkes2012; Druckman and Levendusky, Reference Druckman and Levendusky2019).

Historically there has been a tendency in literature on Canadian political behaviour to emphasize differences between Canadian and American public opinion, which are often thought to stem from each country's unique foundational circumstances and resulting institutions (Horowitz, Reference Horowitz1966). Americans, from a country born of revolution, are more receptive to values of individualism, populism and egalitarianism, while Canadians are more elitist and collectivist owing to their counter-revolutionary heritage (Lipset, Reference Lipset Seymour and M.1990). Early scholarship also drew attention to the less stable nature of Canadian partisanship and vote choice, which was seen as a contrast to the United States (Elkins, Reference Elkins1978; LeDuc et al., Reference LeDuc, Clarke, Jenson and Pammett1984). More recent scholarship shows that these differences are often overblown. Most importantly for my purposes here, partisanship is just as stable and influential in Canada as the United States after accounting for differences in measurement (Bélanger and Stephenson, Reference Bélanger, Stephenson, Turgeon, Papillon, Wallner and White2014; Blais et al., Reference Blais, Gidengil, Nadeau and Nevitte2001; Green et al., Reference Green, Palmquist and Schickler2002; Johnston, Reference Johnston1992, Reference Johnston2006; Merolla et al., Reference Merolla, Stephenson and Zechmeister2008, Reference Merolla, Stephenson, Zechmeister, Blais, Laslier and Van der Straeten2016).

Notwithstanding this shift in Canadian public opinion research, to date there has been only sporadic interest in studying mass polarization in the Canadian context and its causes and implications. The evidence accumulated so far indicates that Canadians are not becoming more ideologically extreme (Johnston, Reference Johnston, Hopkins and Sides2014; Merkley, Reference Merkley2020). There is some evidence that Canadians are sorting, such that New Democratic party (NDP) and Liberal partisans, on the one hand, and Conservative partisans, on the other, are becoming more ideologically distinct (Merkley, Reference Merkley2020). Kevins and Soroka (Reference Kevins and Soroka2018) find that public attitudes toward redistribution are becoming more strongly correlated with partisanship and vote choice over time.

At the same time, Liberal and NDP partisans increasingly dislike the Conservative party and vice versa—evidence of affective polarization (Cochrane, Reference Cochrane2015; Johnston, Reference Johnston2019). Cochrane (Reference Cochrane2015) observes that this form of polarization was occurring as the parties themselves were diverging ideologically, as evidenced by platform coding done by the Comparative Manifestos Project. Johnston (Reference Johnston2019) uses multilevel modelling to show that there is indeed an association between left/right manifesto scores and out-party affect in the Canadian Election Study cumulative file, though perhaps with weaker effects in Quebec. More recently, work by the Digital Democracy Project has revealed that affective polarization extends to how partisans view supporters of other parties and to how comfortable they feel with out-party supporters in their close social proximity—known as social distance (Owen et al., Reference Owen, Bridgman, Gorwa, Loewen, MacLellan, Merkley, Ruths and Zhilin2020). Canadian politics are more affectively charged now than in the past.

The rise of affectively charged politics and the increasingly intertwined nature of partisanship and ideology are bound to have important implications. My focus here is how they relate to the search and processing of political information. Partisan and affective polarization heightens directional motivation, making citizens more likely to choose and accept congenial political information. Canadian public opinion research has been relatively slow in picking up on the polarization of the Canadian mass public, so research on the consequences of polarized politics has been underemphasized.

Study 1: Selective Exposure

All three studies were included in wave nine of the Digital Democracy Project's study of the 2019 Canadian election, fielded from October 24 to November 4, 2019. This wave included a sample of 2,068 Canadian citizens of voting age collected through the online panel sample provider Qualtrics. It used national quotas for age, gender, region and language to ensure representativeness. The data was further weighted within region by age and gender based on population benchmarks provided by the 2016 Canadian census. The recruited sample provides a reasonable partisan balance. Thirty-four per cent of respondents who reported voting in the 2019 election reported voting for the Liberal party, compared to their received 33 per cent, while 32 per cent reported voting Conservative, compared to 34 per cent on election day.

I used a paired conjoint design to evaluate selective exposure in the Canadian public. Conjoint experiments are a technique used in market research to shed light on which attributes of a product appeal to consumers. In this case, I am assessing whether certain features of a news article lead to increases or decreases in its likelihood of selection.

Respondents were given a table with the attributes of two hypothetical news stories, including its source, headline, author and date. They were provided four pairs of news stories and were asked to choose one story from each pair. Every news article had its source randomized across 14 different outlets. They could be assigned stories from the CBC, CTV, Global, the Globe and Mail or the National Post.Footnote ¹ I call these sources national news. They could also receive news articles from what are identified as a “local newspaper,” “local radio station” or a “local television news station,” which I label local news. Finally, respondents could receive articles from PressProgress, National Observer or Rabble, which I treat as left-congenial news, or from Rebel Media, Quillette or True North News, which I treat as right-congenial news. The classification of outlets is based on research done by the Digital Democracy Project's social media team that evaluated whether certain sources were more likely to be selectively shared or followed by partisans on Twitter (see Owen et al., Reference Owen, Bridgman, Gorwa, Loewen, MacLellan, Merkley, Ruths and Zhilin2020, for more details).Footnote ² National news is used as the baseline for comparison. I also randomized the headline. Respondents were exposed to a series of headlines that were either left-congenial or right-congenial and either positive or negative in their tone and content.Footnote ³

I further randomized the author, so that some were male and others were female, and I randomized the date, so that some articles were closer to the fielding date than others. I have no substantive interest in these last two randomizations. They were included to enhance the realism of the experiment by giving respondents two additional characteristics of news articles that are first apparent when respondents read a news story in the real world. They also allowed me to mask the randomizations of substantive interest and prevent possible carryover effects precipitated by experimental demand. An example of the layout can be found in Figure A1 of the Appendix.

My design leads to the following three hypotheses:

I test these hypotheses using ordinary least squares (OLS) regression where I estimate the average marginal component effects (AMCEs) of local news (local), left-congenial sources (leftsource), right-congenial sources (rightsource; reference category = national news), negative headlines (negativeheadline; reference category = positive news), right-congenial headlines (rightheadline; reference category = left-congenial headline), author gender (male; reference category = female), and article date (October 7, September 30 and September 23; reference category = October 14) on the likelihood of article selection. I cluster standard errors by respondent. The AMCEs provide the average causal effect of a given attribute on story selection averaged across the joint distributions of the other remaining attributes. Estimating the AMCEs requires including every randomized attribute in the model, so author and date are included even though they are not of substantive interest here. Balance tests for all three studies can be found in the supplementary materials.

In this case, my theory necessitates the inclusion of an interaction between the source or headline attributes, on the one hand, and political predispositions, on the other (predisposition). I focus here on ideology and partisanship. Partisanship is a stable social identity (Green et al., Reference Green, Palmquist and Schickler2002), while ideology is an interconnected set of beliefs characterized by consistency and constraint (Converse, [1964] Reference Converse2006). Though there has been an increasing relationship between these concepts over time in Canada (Merkley, Reference Merkley2020), this relationship is not automatic. There is considerable scholarly disagreement about whether ideology causes partisanship or vice versa. I am agnostic on this point here, so I estimate two models using each as the moderator of interest.

I measure policy-based ideology using a 15-item policy battery in the Digital Democracy Project to construct a measure of ideological consistency ranging from −15 as most consistently left-wing to 15 as most consistently right-wing (mean = −1.71, standard deviation = 4.25). The distribution of ideology is unimodal with 90 per cent of the distribution falling between −9 and 5. Descriptive statistics for each of the moderators used in these three studies can be found in Table A1. Using the standard 0–10 ideological self-placement scale does not fundamentally change the result. For partisanship, I use a 7-point strength of partisanship scale from strong partisans of the left-leaning parties (Liberals, NDP and Greens) to strong Conservative partisans. Ideology and partisanship, as expected, are highly correlated (0.50). This leads to the estimation of the following equation:

(1)$$\eqalign{& pr( {selection} ) = \alpha + \beta _1local + \beta _2leftsource + \beta _3rightsource + \beta _4negativeheadline \cr & \quad + \beta _5rightheadline + \beta _6male + \beta _7oct7 + \beta _8sept30 + \beta _9sept23 \cr & \quad + \beta _{10}predisposition + \beta _{11}predisposition\;\ast\; leftsource \cr & \quad + \beta _{12}predisposition\;\ast \;rightsource + \beta _{13}predisposition\ast rightheadline + \varepsilon } $$

We should expect β₁₁ to be negative and significant in support of H2A and β₁₂ and β₁₃ to be positive and significant in support of H2B and H1, respectively. I present the interactive effects as the predicted marginal means in Figure 1. The full results, including estimates for the other attributes, can be found in Table A2 of the Appendix.

Figure 1 Predicted Likelihood of Story Selection by Congeniality of the Headline (top) or Source (bottom) across Levels of Partisanship (left) and Policy-based ideology (right)

Note: 95 per cent confidence intervals.

Study 2: Party Cues

Study 1 provides some evidence that Canadians choose information that is congenial with their political beliefs. Quite separately, people may be inclined to accept or reject information based on its alignment with their political views—known as partisan or ideological motivated reasoning. One observable implication of partisan bias in information processing is the tendency of people to follow the lead of their own party's elites (Cohen, Reference Cohen2003; Kam, Reference Kam2005; Mondak, Reference Mondak1993) or to be repelled by signals sent by the opposing party's officials (Merkley and Stecula, Reference Merkley and Stecula2020; Nicholson, Reference Nicholson2012). Some studies have found this process in Canada as well, though perhaps weaker in magnitude than in the United States (Merolla et al., Reference Merolla, Stephenson and Zechmeister2008, 2016).

However, responsiveness to party cues is not necessarily evidence of partisan bias. Many respondents may have limited information on a policy question and use party cues as a low-information shortcut to form an opinion that is generally in line with their partisan or ideological interests (Kam, Reference Kam2005; Mondak, Reference Mondak1993). It is not then much of a surprise that party cues are weaker when people are given policy information (Bullock, Reference Bullock2011). Further, party cues may activate source credibility assessments that are at least partially independent of political motivation (Tappin et al., Reference Tappin, Pennycook and Rand2020). Nonetheless, responsiveness to elite cues is a central observation in a wide range of public opinion research, and such responsiveness could very well be—at least in part—because of partisan-motivated reasoning.

Study 2 evaluates directionally motivated responsiveness to party elite cues using wave nine of the Digital Democracy Project. I follow a strategy used by Bolsen et al. (Reference Bolsen, Druckman and Cook2014) to experimentally manipulate not just party cues but motivation as well. The exogenous manipulation of motivation prevents the conflation of prior beliefs and directional motivation. This intervention is also orthogonal to the source credibility assessments that might be activated when respondents are exposed to party cues. Respondents who indicated they are supporters of the three major parties in Canada (that is, Liberals, Conservatives and NDP) were given a description of a bill passed in 2019:

They were randomly assigned into three groups: one with no party cues, one with in-party cues and one with out-party cues. For example, a Conservative partisan in the in-party condition would receive a cue signalling Conservative party opposition to the bill, while NDP and Liberal partisans would receive a cue signalling their own party's support in Parliament. For the out-party condition, I combine Liberal and NDP cues for Conservative respondents and use the Conservative party for both Liberal and NDP partisans because partisan bias in responsiveness in elite cues is unlikely to manifest when Liberals observe NDP cues or vice versa. I make this choice because partisan sorting and affective polarization in Canada have been characterized by divergence between the NDP and Liberals, on the one hand, and Conservatives, on the other. There is an added benefit of preserving statistical power by limiting the number of treatment arms.

I crossed this randomization with a motivational prime. Respondents could either receive no motivational prime, an accuracy prime or a directional prime, leaving me with nine treatment conditions. Individuals can have a number of different motivational goals, with direction and accuracy drawing the most attention from scholars.

Accuracy refers to a motivation to reach correct, or at least relatively accurate, opinions or beliefs (Kunda, Reference Kunda1990). Individuals may achieve this by paying attention to pertinent information and ignoring less relevant pieces of information (Bolsen et al., Reference Bolsen, Druckman and Cook2014)—in this case, by focusing on policy details and ignoring party endorsements. Pursuing accuracy goals, however, requires more cognitive effort and the use of more complex decision rules (Kunda, Reference Kunda1990). Observing accuracy goals in practice is difficult, but social psychologists have been able to study accuracy motivation through experimental manipulation. Here, I use the following prime, following Bolsen et al. (Reference Bolsen, Druckman and Cook2014):

In contrast to accuracy, directional motivation refers to the desire to reach opinions and beliefs that are directionally aligned, such as with one's partisanship or ideology. People may feel a need to support positions of one's own party and to maximize differences with parties that they oppose, which I prime explicitly the following way, again following Bolsen et al. (Reference Bolsen, Druckman and Cook2014):

Respondents were asked their level of support for the bill (strongly support to strongly oppose, 5-point) after the vignette. I recoded the variable into a binary measure so that 1 was equal to a partisan-congenial response and 0 was otherwise. I have a clear expectation that party cues will induce partisan-congenial responses when directional motivation is primed.

In contrast, an accuracy prime should negate the influence of partisan cues by encouraging respondents to carefully consider the particulars of a policy. We should find no positive effect of party cues on the share of partisan-congenial responses among respondents given this prime. The prediction for respondents in the control condition, in contrast, is unclear. If the default mode of information processing involving party cues is toward direction, we should expect party cues to induce congenial responses at the similar magnitude as the condition with the directional prime. The control condition offers a useful baseline to situate the results from respondents given either the accuracy or directional primes (see Bolsen et al., Reference Bolsen, Druckman and Cook2014, for more discussion).

I use OLS to estimate the following model:

(2)$$pr( {congenial} ) = {\rm \;}\alpha + \beta _1cue + \beta _2accuracy + \beta _3direction{\rm \;} + \beta _4cue\; {\rm \ast }\; accuracy + \beta _5cue\; {\rm \ast }\; direction + \varepsilon {\rm \;}$$

My results here are robust to alternative modelling strategies as shown in Table S1 of the supplementary materials.

Study 3: Ideologically Motivated Numeracy

Study 2 is based on a party cue design that is specifically tailored to identify partisan bias in the acceptance of source cues. Another approach is to manipulate the congeniality of political information and observe its effect on policy attitudes. As noted, the problem with this design is that the provided information may affect reasoning due to one's prior beliefs, which may or may not be informed by one's political motivation (Tappin et al., Reference Tappin, Pennycook and Rand2020). Observing belief change (or not) in response to congenial (or uncongenial) information is not necessarily indicative of partisan or ideological motivated reasoning.

I use Kahan et al.'s (Reference Kahan, Peters, Dawson and Slovic2017) numeracy task to overcome these causal inference challenges. Respondents were given one of four versions of a problem that required them to provide the correct interpretation of data presented in a 2 × 2 table. Respondents in two conditions received a nonpolitical version of the table illustrating whether the application of a skin cream made rashes better or worse among a set of study subjects. They were then asked whether the data provided suggested people who used the skin cream were more likely to get better or worse than those who did not. One group received a version of the study suggesting the cream made the rash better on average. The other group saw the same numeracy task except with the numbers flipped, so that the skin cream appeared to make the rash worse. An example of the nonpolitical task can be found in Figure A2 of the Appendix.

The second pair of conditions included a study related to the effect of a handgun ban on crime rates. They included a table with the exact same numbers as the skin cream study, but in this case, it indicated how many cities that adopted (and failed to adopt) a handgun ban experienced increases (and decreases) in crime afterward. One of these conditions included numbers suggesting that cities that implemented the handgun ban were more likely to experience decreases in crime, while the other condition included numbers that implied the opposite—in the exact same manner as the nonpolitical version of the task. An example of the political task can be found in Figure A3 of the Appendix.

The main advantage to this design is that I am not asking respondents their opinion about gun control. I am asking about their interpretation of the data. This constitutes a direct observation of biased reasoning brought on by respondents' ideological worldviews. The numeracy task is tough. Most respondents will use a simple heuristic, such as comparing cells 1 and 2 without accounting for patients who did not receive the skin cream. Respondents who have strong quantitative aptitude should be more able to answer correctly than those who do not. In the absence of bias, this should be the case in both the political and nonpolitical conditions.

The expectation, however, is that the congeniality of the information will influence the propensity of respondents to get the correct answer among those given the political variant of the task:

I use a measure of policy-based ideology from study 1 as the primary moderating variable. I rescale it from 0 to 1 such that 0 represents respondents for whom the information in the numeracy task is maximally dissonant and 1 identifies those for whom the information is maximally congenial. I call this measure their ideological congeniality. So, for example, a person would score 1 if they were perfectly consistent in their policy responses as a left- or right-leaning individual and their stimulus suggested the skin rash or crime rate improved or worsened, respectively, as a result of the intervention. I estimate the following model with OLS where β₃ should be positive and significant in support of H4:

(3)$$pr( {correct} ) = \alpha + \beta _1political\;treatment + \beta _2congeniality + \beta _3political\;treatment \;\ast \; congeniality + \varepsilon \;$$

Conversely, I expect numerical aptitude to predict correct answers among those given the nonpolitical task but not for those given the political variant:

I operationalize numerical aptitude using a numeracy index validated in previous literature (Liberali et al., Reference Liberali, Reyna, Furlan, Stein and Pardo2012; Peters et al., Reference Peters, Västfjäll, Slovic, Mert, Maocco and Dickert2006). I rescale this index from 0 to 1, where 1 represents those who had a perfect score across all items and are thus are strong in their quantitative ability. I estimate the following model where β₂ should be positive and significant and β₃ should be negative and significant in support of H5:

(4)$$pr( {correct} ) = \alpha + \beta _1political\;treatment + \beta _2numeracy + \beta _3political\;treatment \; \ast \; numeracy + \varepsilon \;$$

Ideological bias will also affect people's evaluation of information quality. The prediction here is tricky with this design. Normally we would expect congenial information to generate more positive evaluations of evidence quality compared to uncongenial information. But in this case, not everyone draws the correct inference from the presented data. Here I am instead interested in perceived ideological congeniality:

After completion of the numeracy task, I ask respondents their perception of the strength of the evidence presented by the study (very strong to not strong at all, 4-point), which was rescaled from 0 to 1, where 1 represents respondents with the most positive evaluation of the evidence. Perceived ideological congeniality is based on the same congeniality measure as in equation 3 but with the scale reversed for respondents that arrived at the wrong answer. I estimate the following model where β₃ should be positive and significant in support of H6:

(5)$$evaluation = \alpha + \beta _1political{\rm \;}treatment + \beta _2perceived{\rm \;}congeniality + \beta _3political{\rm \;}treatment\; {\rm \ast } \; perceived{\rm \;}congeniality + \varepsilon $$

Study 3 results

I illustrate the results using linear predictions, but the estimates can be found in Table A4 of the Appendix. The left panel of Figure 2 shows a powerful relationship between ideological congeniality and correct responses among those given the political numeracy task. Eighty-three per cent of respondents were expected to get the correct answer when the treatment was the most aligned with their ideological outlook (for example, a perfectly consistent conservative given a variant in which gun control increases crime), while the same was true of only 13 per cent of respondents when the treatment was maximally dissonant with their worldviews. In sharp contrast, there was no relationship between ideological congeniality and correct responses among those respondents who were given the otherwise identical nonpolitical variant of the numeracy task. The interaction is significant (p < .001), providing strong support for H4.

Figure 2 Predicted Share of Respondents Getting Correct Answer across Levels of Ideological Congeniality (left) and Numerical Ability (right) for Political and Nonpolitical Conditions

Note: 95 per cent confidence intervals.

Ideological congeniality also crowds out numeracy in predicting correct answers in the politically charged conditions. The right panel of Figure 2 illustrates this effect. Among respondents who were given the nonpolitical task, those with the lowest score on the numeracy index were able to get the answer correct only 31 per cent of the time, compared to 66 per cent for those with the highest level of numeracy. Strikingly, there was no relationship between numeracy and correct responses among those given the political numeracy task. The interaction is again significant (p < .001), providing support for H5.

Ideological bias seeps into the numeracy task, leading many respondents astray when the result does not conform to their prior beliefs. It also plays a role in how they evaluate the quality of the evidence presented, as shown in Figure 3. Among respondents who were given the political numeracy task, those who perceived themselves to be most ideologically aligned with the study's findings were more impressed with the quality of the evidence (0.66) than those who perceived the information to be the most dissonant (0.48), a relationship that is significant (p < .001). There was no such relationship among those given the nonpolitical task. This interaction is significant (p < .05), providing support for H6. The relationship presented resembles confirmation bias, rather than disconfirmation bias. People who perceived the study results to be uncongenial with their preferences had similar evaluations of the quality of evidence as those who were given the nonpolitical task.

Figure 3 Predicted Evaluation of Evidence across Levels of Perceived Ideological Congeniality for Political and Nonpolitical Conditions

Note: 95 per cent confidence intervals.

The upshot is that ideology disables faculties we typically use to reason about and evaluate evidence—in this case, respondents’ quantitative aptitude. Respondents facing a task revealing uncongenial ideological information were less likely to get the answer correct than similar respondents in a nonpolitical task, and they were more likely to do so when the information was aligned with their beliefs. In sharp contrast, respondents’ level of numerical aptitude predicted correct responses in the nonpolitical task, but this relationship disappeared entirely for those in the political treatment conditions.