Hostname: page-component-586b7cd67f-t8hqh Total loading time: 0 Render date: 2024-11-28T17:18:46.472Z Has data issue: false hasContentIssue false

Motivated reasoning, fast and slow

Published online by Cambridge University Press:  01 December 2021

Camilla Strömbäck
Affiliation:
Department of Management and Engineering, Division of Economics, Linköping University, Linköping, Sweden
David Andersson
Affiliation:
Department of Management and Engineering, Division of Economics, Linköping University, Linköping, Sweden
Daniel Västfjäll
Affiliation:
Department of Behavioral Sciences and Learning, Division of Psychology, Linköping, Sweden Decision Research, Eugene, OR, USA
Gustav Tinghög*
Affiliation:
Department of Management and Engineering, Division of Economics, Linköping University, Linköping, Sweden Sweden National Center for Health Care Priority Setting, Division of Health Care Analysis, Linköping University, Linköping, Sweden
*
*Correspondence to: E-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Are people more likely to (mis)interpret information so that it aligns with their ideological identity when relying on feelings compared to when engaging in analytical thinking? Or is it the other way around: Does deliberation increase the propensity to (mis)interpret information to confirm existing political views? In a behavioral experiment, participants (n = 1207, Swedish sample) assessed numerical information concerning the effects of gender quotas and immigration either under time pressure or under no time pressure. To measure trait differences in cognitive sophistication, we also collected data on numeric ability. We found clear evidence of motivated reasoning when assessing both the effects of gender quotas on companies’ financial results and the effect of refugee intake on crime rates. Subjects who prioritized equality over liberty on the labor market were 13 percentage points less likely to correctly assess numerical information depicting that companies that used gender quotas when hiring made less profit. Subjects who classified themselves as ‘Swedes’ rather than ‘World citizens’ were 14 percentage points less likely to correctly assess numerical information depicting that crime rates decreased following immigration. Time pressure did not affect the likelihood to engage in motivated reasoning, while subjects with higher numeric ability were less likely to engage in motivated reasoning when analyzing information concerning refugee intake, but more likely to engage in motivated reasoning when analyzing information regarding the effect of gender quotas. Together these results indicate that motivated reasoning is primarily driven by individual differences in analytical thinking at the trait level and not by situational factors such as time pressure, and that whether motivated reasoning is primarily driven by analysis or feelings depends on the topic at hand.

Type
Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Introduction

Partisan consistent evaluation of information, where people assess information in a way to protect valued beliefs, rather than objectively consider the facts, is commonly referred to as motivated reasoning (Lord et al., Reference Lord, Ross and Lepper1979; Kunda, Reference Kunda1990; Taber & Lodge, Reference Taber and Lodge2006; Epley & Gilovich, Reference Epley and Gilovich2016). In the process of evaluating information or facts, motivated reasoning can be thought of as a tradeoff between desirability and accuracy, where people derive utility from maintaining beliefs, much like they derive utility from consumption and other types of behaviors in which they willingly engage (Grant et al., Reference Grant, Kajii and Polak1998; Bénabou, Reference Bénabou2015; Golman et al., Reference Golman, Loewenstein, Moene and Zarri2016; Hagmann & Loewenstein, Reference Hagmann and Loewenstein2018). In this study, we investigate if motivated reasoning is driven by analytic or intuitive factors, contrasting a motivated reasoning-as-analysis account with a motivated reasoning-as-feelings account.

The impact of motivated reasoning when processing information has been shown in many areas of public policy. In a US context, liberals have been found to be more likely to trust scientific reports on climate change than conservatives (Drummond & Fischhoff, Reference Drummond and Fischhoff2017), and liberals were more likely to correctly interpret numerical information showing that banning guns decreased crime rates, while conservatives were more likely to correctly interpret information showing that banning guns increased crime rates (Kahan et al., Reference Kahan, Peters, Dawson and Slovic2017). Similarly in a Swedish context, globally oriented people were more likely to correctly interpret information showing that immigration decreased crime rates, while nationally oriented people were more likely to assess the information correctly when it showed that immigration was linked to increased crime rates (Lind et al., Reference Lind, Erlandsson, Västfjäll and Tinghög2018). Other canonical policy areas where motivated reasoning been established include performance of the national economy (Bartels, Reference Bartels2002), beliefs about Iraq's possession of weapons of mass destruction (Bullock, Reference Bullock2009) climate change (Druckman & McGrath, Reference Druckman and McGrath2019; Nurse & Grant, Reference Nurse and Grant2020), same-sex marriage and healthcare reforms (Washburn & Skitka, Reference Washburn and Skitka2018). Thus, there is mounting evidence showing that motivated reasoning exists and impacts information processing in many areas of public policy.

The underlying mechanisms are, however, still not fully understood. Existing studies suggest somewhat contradictory ideas about the role of intuitive processing (i.e., System 1) and more deliberate processing (i.e., System 2) in motivated reasoning. Especially, the role of numeracy (the ability to use, understand, and manipulate numeric information – a System 2 individual difference factor) is debated. Kahan et al. (Reference Kahan, Peters, Dawson and Slovic2017) found that people with high numeric ability were more likely to engage in motivated reasoning in a numeric task and proposed that people with higher numeric ability are better equipped to reason their way around information that do not fit their ideological view, which suggests that people with high numeric ability are more likely to engage in motivated reasoning. However, this finding was not successfully replicated in a recent high-powered pre-registered replication (Persson et al., Reference Persson, Andersson, Koppel, Västfjäll and Tinghög2021). Further, Lind et al. (Reference Lind, Erlandsson, Västfjäll and Tinghög2018) found that numeric ability had a buffering effect against motivated reasoning using the same paradigm as Kahan et al. (Reference Kahan, Peters, Dawson and Slovic2017) but adapted to the effect of immigration on crime rates in a Swedish sample. Moreover, in a study by Tappin et al. (Reference Tappin, Pennycook and Rand2020), higher analytical ability, measured by performance on the Cognitive Reflection Task, was associated with belief updating closer to an unbiased Bayesian benchmark. Thus, existing empirical findings are mixed regarding whether motivated reasoning is most accurately viewed as a result of deliberation or lack of deliberation. Importantly, the vast majority of previous research has solely relied on individual differences in analytical thinking and to our knowledge no previous study has used state manipulation to inhibit analytical thinking. In the current study, we investigate the underlying mechanisms of motivated reasoning, by both measuring individual differences in System 2 thinking (i.e., numeracy) and use time pressure to exogenously manipulate inhibition of System 2 processing. This also allows us to explore if inhibition of analytical thinking through time pressure affect people with different levels of numeracy differently.

In the context of motivated reasoning and public policy, System 1 and System 2 processing have been conceptualized as motivated reasoning-as-analysis and motivated reasoning-as-feelings, representing opposing hypotheses concerning the driving mechanisms (Lind et al., Reference Lind, Erlandsson, Västfjäll and Tinghög2018). The rationale behind the motivated reasoning-as-analysis hypothesis is that information processing is primarily driven by analytical processes where people seek to maximize their own utility by falsifying information that threatens valued beliefs. This implies that motivated reasoning is an analytical, System 2, process where people with high numeracy would be better equipped to interpret information, especially numeric information, in a way that fits their own ideological view. Moreover, inhibiting analytical thinking through time pressure should decrease the opportunity to engage in motivated reasoning. In contrast, the motivated reasoning-as-feelings hypothesis suggests that information processing is instead primarily driven by intuitive responses and lack of reasoning, that is, a System 1 process. That is, when people evaluate new information, motivated reasoning happens automatically without deliberation as people rely on emotional cues to minimize the cognitive cost of forming an assessment based on deliberation. For example, people may rely on a ‘how do I feel about it’ affect heuristic when processing new information (Slovic et al., Reference Slovic, Finucane, Peters and MacGregor2007). Following the rationale of the motivated reasoning-as-feelings hypothesis, high analytical ability should serve as a protector against motivated reasoning and inhibiting analytical thinking through time pressure should increase the likelihood of engaging in motivated reasoning.

Method and data

Sample

Participants were recruited in collaboration with Origo GroupFootnote 1 and drawn from a sample of the general adult Swedish population previously included in their subject pool. Data were collected during March 2019. 1008 participants completed the online experiment and received a small monetary compensation. Additionally, as a pilot study, 203 undergraduate students at Linköping University participated in the experiment during November 2018. The purpose of the pilot study was to test the logistical procedures and administration. The pilot data collection was successful, and the observations were added to those from the main data collection. The total sample, thus, consisted of 1211 respondents who completed the full survey (50.3% women; age 18–79). In addition, 257 participants failed an attention check halfway through the experiment (see Supplementary Table S1 for sample characteristics).

Time pressure manipulation

Participants were randomly assigned to either a time pressure treatment or to a control treatment. Instructions where identical across treatments except that participants in the time pressure treatment were instructed to respond to the scenarios within 45 s while participants in the control group could use as much time as they wanted.Footnote 2 Previous studies have shown that time pressure inhibits individuals' analytical System 2 processing (De Dreu, Reference De Dreu2003; Tinghög et al., Reference Tinghög, Andersson, Bonn, Johannesson, Kirchler, Koppel and Västfjäll2016; Kirchler et al., Reference Kirchler, Andersson, Bonn, Johannesson, Sørensen, Stefan and Västfjäll2017). Manipulation checks at the end of the experiment showed that the participants in the time pressure treatment to a greater extent felt that they were in a hurry, t(1211) = −16.55, p < 0.001, felt stressed, t(1211) = −11.99, p < 0.001, and answered the experimental scenarios more intuitively t(1211) = −4.02, p < 0.001, than the participants in the control group.Footnote 3 Moreover, the median response time in the time pressure treatment was 32 s per scenario, while the median response time in the control group was 56 s.

Numerical tasks

The materials presented to the participants consisted of three scenarios that were identical across treatments, except for the down counting clock which was shown only in the time pressure treatment. Figure 1 shows the numerical information presented in the experimental scenarios which the participants were asked to interpret. All scenarios, which were presented in random order, involved fictitious studies. The participants read a short text describing the scenario and continued to the next page where the same text was presented together with the corresponding table, containing four values. The time constraint only affected the second page, so the participants could use as much time they wanted to read the scenario. The key to correctly solving the problems is to compare not the absolute numbers presented in the tables, but the ratios between them. This is the same kind of numerical task to assess motivated reasoning as in Kahan et al. (Reference Kahan, Peters, Dawson and Slovic2017) and Lind et al. (Reference Lind, Erlandsson, Västfjäll and Tinghög2018).

Figure 1. Experimental scenarios.

Out of the three scenarios, two were ideologically polarizing and concerned the effects of immigration and gender quotas, whereas the third scenario concerning skin lotion was designed not to be ideologically polarizing. Thus, the skin lotion scenario served as a control scenario. Based on the numbers presented in the scenarios, the participants were asked to indicate whether ‘[t]he communities that received refugees experienced increased crime rate compared to communities that did not receive refugees’ or ‘[t]he communities that received refugees experienced decreased crime rate compared to communities that did not receive refugees’ in the immigration scenario. In the gender quota scenario, the participants were asked to indicate whether ‘[c]ompanies that have implemented gender quotas performed worse than index compared to the companies that have not implemented gender quotas’ or ‘[c]ompanies that have implemented gender quotas performed better than index compared to the companies that have not implemented gender quotas’. In the skin lotion scenario, the participants were asked to indicate whether ‘[p]atients who use lotion A have increased skin problems compared to those who used lotion B’ or ‘[p]atients who use lotion A have decreased skin problems compared to those who used lotion B’. The response options were presented in random order. The correct answers to the scenarios were that crime rates decreased, companies performed worse, and the skin problems increased. Full texts to all scenarios can be found in the Supplementary Material.

In the pilot study, participants who failed to answer within the time limit were excluded from the survey, but to avoid selection bias we decided to change this for the main data collection. Such an exclusion could otherwise generate noncomparable treatment groups with respect to numeracy (Tinghög et al., Reference Tinghög, Andersson, Bonn, Böttiger, Josephson, Lundgren and Johannesson2013). Instead of being excluded, participants in the time pressure treatment who failed to answer within the time limit were told that they had failed and were then asked to keep within the time limitation for future questions. Still, the participants had to give an answer to proceed to the following questions.

Ideological views

To elicit polarizing ideological views, participants were asked to answer two ideologically dividing questions regarding worldview, and equality versus liberty on the labor market. The worldview question was designed to map against the immigration scenario and the labor market question was designed to map against the gender quota scenario. In the worldview question, the participants read the following text:

People sometimes talk about whether they view themselves as ‘World citizens’ or as ‘Swedes’. People who view themselves as ‘World citizens’ believe that all people should have equal rights and that we have a common responsibility to help everybody, no matter their nationality. People who view themselves as ‘Swedes’ believe that Swedish citizens should have some privileges, and that we have a greater responsibility to help other Swedes than people from other countries. If one uses the definition above, would you describe yourself as more of a ‘World citizen’ or as more of a ‘Swede’.

Thereafter, they were asked to indicate on a 6-point Likert scale ranging from 1 = ‘I'm much more of a world citizen’ to 6 = ‘I'm much more of a Swede’ how they would rate themselves. The second question about equality/liberty on the labor market followed the same pattern. The participants were asked to indicate on a 6-point Likert scale ranging from 1 = ‘I think that equality is much more important’ to 6 = ‘I think that free choice is much more important’ how they felt after reading the following text:

Sometimes people talk about whether they prioritize equality or free choice in the labor market. People who prioritize equality believe that it is more important to give everybody equal opportunities to a career, even if that implies a more regulated labor market. People who prioritize free choice believe that it is more important that companies can recruit the candidate they prefer, without any restrictions, even if that leads to a less equal labor market. If one uses the definition above, do you think that equality or free choice is relatively more important?

We predicted that participants who rated themselves as more of a Swede, would be worse than those rating themselves as more of a world citizen at assessing numerical statistics portraying that immigration decreased crime rates, and that egalitarians, that is, equality-oriented participants, would be worse at accurately assessing statistics that suggested that gender quotas had a negative effect on companies’ performance. We expected the groups to perform equally well in the skin lotion scenario. The distribution of the responses to the ideologically dividing questions can be found in the Supplementary Material.

Measuring individual differences in numeracy

To assess the participants’ numeric ability, we used a combination of the Berlin Numeracy Test (Cokely et al., Reference Cokely, Galesic, Schulz, Ghazal and Garcia-Retamero2012) and a numeracy test developed by Schwartz et al. (Reference Schwartz, Woloshin, Black and Welch1997). All participants were asked to solve all seven questions, independent of whether they had answered the previous questions correctly or not. The average score was 3.15 correct answers (range 0–7, SD = 1.68). Additionally, we collected data on the participants’ age, sex, and educational background. The educational variable consisted of four options (compulsory school, 9 years; high school 2–4 years; university or vocational education less than 3 years; university or vocational education, 3 years or more) and the participants were asked to indicate which option that best corresponded to their situation. All students in the pilot study were coded to have ‘university or vocational education less than 3 years’ as we lack data on how far into their education they were.

At the end of the survey participants responded to how much of in a hurry they felt when assessing the scenarios, how stressed they felt, and how intuitive and well thought out their answers were. Responses to these questions were given on a 5-point Likert scale.

Data analysis

The main analysis plan and the hypotheses were specified and pre-registeredFootnote 4 before the main data collection. As a first step, we divided the sample by how they answered to the ideologically dividing questions. Participants who answered 1–3 on the question concerning worldview were categorized as cosmopolitans (51.3%), while participants who answered 4–6 (48.7%) were categorized as nationally oriented. Furthermore, participants who answered 1–3 on the question concerning equality/liberty at the labor market were categorized as egalitarians (46.7%), whereas participants who answered 4–6 were categorized as libertarians (53.3%). Thereafter, χ 2-tests were used to investigate the presence of motivated reasoning.

As a second step, we investigated whether motivated reasoning was driven by analytical or feelings-oriented processing. Once again, we split the sample into nationally oriented and cosmopolitans, and egalitarians and libertarians, respectively, and investigated the effect of time pressure and numeracy on motivated reasoning. As a preliminary analysis, we graphically presented our results, followed by elaboration of the results in form of regression analyses. In the regression analyses, the ideologically dividing questions were treated as continuous variables, taking a value between one and six. The responses to the ideologically dividing question regarding gender quotas were inverted before the regression analyses were performed. Our primary analyses were intention-to-treat analyses, where all observations, independent on whether the participant in the time pressure treatment stayed within the time limit or not, were included. By using the entire sample in our analyses, we did not systematically exclude a specific type of participants, for example, people preferring to answer slowly, from our analyses (Bouwmeester et al., Reference Bouwmeester, Verkoeijen, Aczel, Barbosa, Bègue, Brañas-Garza and Wollbrant2017). To test the robustness of the results, we also ran compliance-only analyses, excluding those participants who failed to respond within the time limit.

The three scenarios were analyzed separately. The dependent variable in all regressions was a binary variable taking the value 1 if the scenario was correctly solved and 0 otherwise. To measure ideological view, we used the ideological dividing question that mapped to each scenario. Time pressure was a dummy variable that took the value 1 if the participant was in the time pressure treatment and 0 otherwise. To investigate the motivated reasoning-as-feelings hypothesis and the motivated reasoning-as-analysis hypothesis, we were interested in the effect of time pressure and numeracy on motivated reasoning. In the first specification, the effect of time pressure was studied:

$${\rm correct\;assesmen}{\rm t}_i = \beta _0 + \beta _1\,{\rm ideological\;vie}{\rm w}_i + \beta _2\,{\rm time\;pressur}{\rm e}_i + \beta _3\,{\rm ideological\;vie}{\rm w}_i\ast {\rm time\;pressur}{\rm e}_i + \beta _j^{\prime} X_i + u_i$$

In the second specification, numeracy was added as well:

$${\rm correct\;assesmen}{\rm t}_i = \beta _0 + \beta _1\,{\rm ideological\;vie}{\rm w}_i + \beta _2\,{\rm time\;pressur}{\rm e}_i + \beta _3\,{\rm numerac}{\rm y}_i + \beta _4\,{\rm ideological\;vie}{\rm w}_i\ast {\rm numerac}{\rm y}_i + \beta _j^{\prime} X_i + u_i$$

In both specifications, the interaction term was the variable of main interest and X was a vector including all control variables (age, educational background, and sex). As a last step, we divided the sample by the time pressure treatment and ran the second analysis again. This enabled us to investigate if the presence of time pressure affected how numeracy correlated with motivated reasoning.

Results

We found clear evidence of motivated reasoning in both the polarizing scenarios (Figure 2). Participants who rated themselves as nationally oriented were 14 percentage points less likely to answer the polarizing immigration scenario, where crime rates decreased following immigration, correctly. This difference corresponds to Cohen's d of 0.29, which, using common classification, is a small effect size. A χ 2-test showed that the difference between the groups was statistically significant, χ 2(1, n = 1211) = 25.57, p < 0.001. In the gender quota scenario, participants prioritizing equality over liberty on the labor market were 13 percentage points less likely to interpret numerical information showing that companies with gender quotas performed worse, correctly. The difference between equalitarians and libertarians was statistically significant, χ 2(1, n = 1211) = 18.62, p < 0.001, and the effect size corresponds to Cohen's d of 0.25. Importantly, in the nonpolarizing scenario responses were similar both for participants categorized as nationally oriented and cosmopolitans, χ 2(1, n = 1211) = 0.06, p = 0.815, and as egalitarians and libertarians, χ 2(1, n = 1211) = 0.40, p = 0.525.Footnote 5

Figure 2. Proportion of participants assessing the different scenarios correctly, separated by ideological views. 95% confidence interval is included for each bar.

There was a general effect of time pressure in both the polarizing scenarios (Figure 3), where participants were less likely to respond correctly when set under time pressure. According to the motivated reasoning-as-feelings hypothesis, time pressure should increase motivated reasoning, that is, the difference in the proportion of correct answers between nationally oriented and cosmopolitans, and between libertarians and egalitarians should be larger when responding under time pressure. However, as shown in Figure 3, this pattern of results was not found. If anything, the tendency was more in line with the motivated reasoning-as-analysis hypothesis, as the difference between the opposing ideological groups decrease slightly when responding under time pressure.

Figure 3. Percent of correct assessments by treatment, depending on the ideological view, separated by experimental condition. Note: The correct answer to the immigration scenario is that the crime rate decreases. The correct answer to the gender quota scenario is that the companies perform worse. N = 1211.

All plots in Figure 4 show an increasing pattern with numeric ability, indicating that numeracy was positively correlated with the probability of correctly assessing the scenarios. In the immigration scenario, the difference in probability of correct assessment between nationally oriented and cosmopolitans decreased as numeric ability increased, while it remained stable across different levels of numeracy when comparing egalitarians and libertarians. Thus, numeracy had a buffering effect against motivated reasoning when interpreting information about the effect of immigration on crime rates but had no visible effect among subjects when interpreting the effect of gender quotas on company profits.Footnote 6

Figure 4. Percent of correct assessments by numeracy, depending on an ideological view.

Results were corroborated in regression analyses (Table 1). In general, time pressure had a negative effect on the probability of assessing the scenarios correctly, while numeracy had a positive effect. To test the contradicting accounts of motivated reasoning-as-feelings and motivated reasoning-as-analysis, we included interaction variables – one between time pressure and ideological view (Models 2 and 7) and one between numeracy and ideological view (Models 4 and 9). Given that the coefficients of ideological view and the interaction variables are conditional in our regression models, they cannot be interpreted in the same way as in a regular additive model (Golder, Reference Golder2003; Brambor et al., Reference Brambor, Clark and Golder2006). Figure 5 is, therefore, a more appropriate way to depict the joint effect of time pressure and numeracy on motivated reasoning. As the confidence interval of the control group and the treatment group overlaps in both Figure 5A,B, our results indicate that there was no effect of time pressure on motivated reasoning in neither the immigration scenario nor the gender quota scenario. Turning to Figure 5C, globally oriented participants with higher numeracy were less likely to engage in motivated reasoning in the immigration scenario. These results are in line with the motivated reasoning-as-feelings hypothesis. In Figure 5D, the opposite results are shown. Thus, in the gender quota scenario, higher numeracy increased the likelihood to engage in motivated reasoning, which is in line with the motivated reasoning-as-analysis hypothesis.Footnote 7

Figure 5. The impact of ideological view on the likelihood to correctly assess respectively scenario conditioned on time pressure (A,B) and numeracy (C,D), respectively. Note: The error bars and dashed lines indicate 95% confidence intervals. (A,C) The results of the immigration scenario, where the correct answer is that more immigration decreases the crime rates. (B,D) The results of the gender quota scenario, where the correct answer is that companies that have implemented genre quotas perform worse.

Table 1. Linear probability regressions on the role of time pressure and numeracy.

Note: The dependent variable is the assessments of the experimental scenarios, 1 if the correct assessment was made and 0 otherwise. Sex, age, education, and a dummy variable which takes the value of 1 if the respondent was part of the pilot study conducted as a pilot are included in all models, but not shown. Time pressure is a dummy variable taking the value 1 if the participant was in the time pressure treatment and 0 otherwise. Robust standard errors given in parentheses.

***p < 0.01, **p < 0.05, *p < 0.1.

To further facilitate the interpretation of the effect of ideological view, and its interaction with numeracy, on the probability to correctly assess the scenarios, we turn our attention to Figure 6 (see also Supplementary Table S10). In Figure 6, we observe the same pattern as in Figure 5, consistent with motivated reasoning-as-feelings in the immigration scenario but motivated reasoning-as-analysis in the gender quota scenario, also when analyzing each experimental condition separately (time pressure or no time pressure). The interaction effect between ideological view and numeracy on correct responses is similar across the experimental groups, indicating that motivated reasoning is primarily driven by individual differences in analytical thinking at the trait level and not by situational factors such as time pressure.

Figure 6. The impact of ideological view on the likelihood to correctly assess respectively scenario, conditioned on numeracy. Note: The dashed lines indicate 95% confidence intervals. The sample is divided by treatment. (A,C) The results of the participants in the time pressure treatment, and (B,D) the results of the participants in the control group.

Discussion

This study set out to examine if motivated reasoning is primarily driven by System 1, feelings-oriented processes, or by System 2, analytical reasoning. Hence, we contrast the notions of motivated reasoning-as-feelings and motivated reasoning-as-analysis. To do this, we examined both individual differences in System 2 abilities (numeric abilities), as well as induced inhibition of System 2 thinking through time pressure. Our results showed that time pressure did not affect the likelihood to engage in motivated reasoning, while subjects with higher numeric ability were less likely to engage in motivated reasoning when analyzing information concerning the effects of refugee intake, but more likely to engage in motivated reasoning when analyzing information regarding the effect of gender quotas. At the trait level, our results thus indicate that whether motivated reasoning is primarily driven by analysis or feelings depends on the topic at hand.

Our findings are consistent with the empirical literature on motivated reasoning that have also found mixed effects regarding the role of analytical ability in motivated reasoning. The results showing that numeric ability can serve as a buffer in some cases, align with studies showing that greater analytical ability correlates with more appropriate responsiveness to new information (Tappin et al., Reference Tappin, Pennycook and Rand2020), greater accuracy when assessing the truthfulness of political news headlines (Pennycook & Rand, Reference Pennycook and Rand2019; Bago et al., Reference Bago, Rand and Pennycook2020), general epistemic rationality (Pennycook et al., Reference Pennycook, Fugelsang and Koehler2015) and being less susceptible to conjunction fallacies (Scherer et al., Reference Scherer, Yates, Baker and Valentine2017). It thus appears as if motivated reasoning-as-feelings sometimes drive motivated reasoning, and, in other instances, motivated reasoning-as-analysis may underlie motivated reasoning. An important task for future research to help reconcile these findings is to identify under which conditions (decision, situation, individual) these two mechanisms are likely to dominate information processing. At this stage, we can only speculate that findings are dependent on cultural difference between samples but also regarding the politically polarizing tasks used. This is one of the reasons for why we did not explore motivated reasoning in the context of gun control and climate change as these issues are arguably less polarizing in a Swedish or west European sample compared to a US sample (Lind et al., Reference Lind, Erlandsson, Västfjäll and Tinghög2018). Future research should therefore systematically investigate motivated reasoning for a range of numeric tasks that are politically polarizing, as well as additional System 1 versus Systems 2 individual differences and situational manipulations.

Perhaps it is in the complex interaction of individual, situational, and task characteristics that one can understand the information processing underlying motivated reasoning? This was one of the motivations behind this study as we included several numeric tasks, as well as simultaneous state manipulation (time pressure) and trait measures (numeracy). In addition to possible (additive) main effects of these variables, state and trait could interact in several ways: (1) time pressure could affect those low in numeracy more, in that low numerates tend to be affected by System 1 manipulations and irrelevant sources of information more than high numerates (Peters, Reference Peters2012) and (2) motivated numeracy may increase so that high numerates respondents use their superior numeric skills even more heuristically and thus amplifying motivated reasoning; see Peters et al. (Reference Peters, Västfjäll, Slovic, Mertz, Mazzocco and Dickert2006) for an example of how high numeracy can lead to more biased numeric decisions. However, in the current study, even though our time pressure manipulation was effective, we found no evidence for any of these types of interaction. There might be several reasons for this, but one possibility is that trait factors are more robustly related to motivated reasoning in these types of numeric tasks (Gärtner et al., Reference Gärtner, Tinghög and Västfjäll2019). However, more research simultaneously measuring trait and manipulating state factors is needed.

From a societal viewpoint, the problem with motivated reasoning is that it prevents people from objectively evaluate information. If people choose to evaluate information in favor of their preexisting beliefs, changing someone's mind will be difficult, even in cases where hard, objective facts exist. This can be taken advantage of by political parties, or other interest groups, and will most likely lead to an even more polarized society. Understanding the mechanisms that drives motivated reasoning is thus an important venture to minimize the societal effects of motivated reasoning. Here, it also becomes important to understand if it is individual trait differences or the situation that primarily causes people to engage in motivated reasoning. Moreover, the fact that our results point to that trait differences in numeracy may in some situations be a protection against, rather than a risk factor for (i.e., Kahan et al., Reference Kahan, Peters, Dawson and Slovic2017), motivated reasoning in politically polarized tasks, nuances the picture of who is susceptible to these biases. Consequently, both System 1 and System 2 debias techniques or interventions (such as teaching numeric skills) may be effective in reducing motivated reasoning.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/bpp.2021.34.

Funding

This research was funded by the Swedish Research Council [grant number: 2018.01755]. The funders had no role in study design, data collection, analysis, decision to publish, or preparation of the manuscript.

Footnotes

1 Origo Group (origogroup.com) is a Swedish research company that specializes in data collection for national and international surveys. We hired them to collect a representative sample (based on age and gender in the Swedish adult population) of participants for our online experiment.

2 Before the experiment was run, we pre-tested different time limits on colleagues and students. We interviewed people who pre-tested the experiment about their experience of the time pressure manipulation. The time limit was set so that most participants would have the ability to solve the task within the given time, but still feel that they had to hurry up to do so.

3 Since all manipulation checks were self-assessed, we have no objective measure of whether the participants in the time pressure treatment actually were more stressed or made more intuitive decisions, which can be seen as a limitation of the experimental design.

4 https://aspredicted.org/td837.pdf. Data on self-assessed intuition was collected, but never analyzed since we focus on objective measures of analytical thinking and inhibition of it.

5 As we found no evidence that either of the ideologically dividing questions affected the probability of assessing the skin lotion scenario correctly, all results concerning that scenario is reported in Supplementary Table S3.

6 As the number of participants with the best and worst results in the numeracy test is low, Figure 4 should be interpreted with care. In the immigration scenario, the number of participants in each group is 33, 85, 136, 126, 103, 60, 42, and 5 for nationally oriented and 17, 69, 132, 124, 117, 87, 50, and 25 for cosmopolitans. The corresponding number for the gender quota scenario is 25, 72, 141, 141, 116, 83, 55, and 12 for libertarians and 25, 82, 127, 109, 104, 64, 37, and 18 for egalitarians.

7 As robustness checks, we performed a compliant-only analysis where we excluded the participants who failed to answer the scenario within the time limit, excluded the student sample collected in the pilot study and excluded those relatively neutral (answered 3 or 4) in the ideologically dividing questions (see Supplementary Tables S4–S6 for details). The results remained similar.

References

Bago, B., Rand, D. G. and Pennycook, G. (2020), ‘Fake news, fast and slow: Deliberation reduces belief in false (but not true) news headlines’, Journal of Experimental Psychology: General, 149(8): 16081613.CrossRefGoogle ScholarPubMed
Bartels, L. M. (2002), ‘Beyond the running tally: Partisan bias in political perceptions’, Political Behavior, 24(2): 117150. doi:10.1023/A:1021226224601.CrossRefGoogle Scholar
Bénabou, R. (2015), ‘The economics of motivated beliefs’, Revue d'Économie Politique, 125(5): 665685.CrossRefGoogle Scholar
Bouwmeester, S., Verkoeijen, P. P. J. L., Aczel, B., Barbosa, F., Bègue, L., Brañas-Garza, P. and Wollbrant, C. E. (2017), ‘Registered Replication Report: Rand, Greene, and Nowak (2012)’, Perspectives on Psychological Science, 12(3): 527542. doi:10.1177/1745691617693624.CrossRefGoogle Scholar
Brambor, T., Clark, W. R. and Golder, M. (2006), ‘Understanding interaction models: Improving empirical analyses’, Political Analysis, 14(1): 6382.CrossRefGoogle Scholar
Bullock, J. G. (2009), ‘Partisan bias and the Bayesian ideal in the study of public opinion’, Journal of Politics, 71(3): 11091124. doi:10.1017/S0022381609090914.CrossRefGoogle Scholar
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S. and Garcia-Retamero, R. (2012), ‘Measuring risk literacy: The Berlin Numeracy Test’, Judgment and Decision Making, 7(1): 2547.CrossRefGoogle Scholar
De Dreu, C. K. (2003), ‘Time pressure and closing of the mind in negotiation’, Organizational Behavior and Human Decision Processes, 91(2): 280295.CrossRefGoogle Scholar
Druckman, J. N. and McGrath, M. C. (2019), ‘The evidence for motivated reasoning in climate change preference formation’, Nature Climate Change, 9(2): 111119.CrossRefGoogle Scholar
Drummond, C. and Fischhoff, B. (2017), ‘Individuals with greater science literacy and education have more polarized beliefs on controversial science topics’, Proceedings of the National Academy of Sciences, 114(36): 95879592.CrossRefGoogle ScholarPubMed
Epley, N. and Gilovich, T. (2016), ‘The mechanics of motivated reasoning’, Journal of Economic Perspectives, 30(3): 133140.CrossRefGoogle Scholar
Gärtner, M., Tinghög, G. and Västfjäll, D. (2019), Decision-making traits and states as determinants of risky choices. Rationality and Competition. Collaborative Research Center Transregio 190. Ludwig- Maximilians-Universität München und Humboldt-Universität zu Berlin. Berlin. Retrieved from: http://hdl.handle.net/10419/208095.Google Scholar
Golder, M. (2003), ‘Electoral institutions, unemployment and extreme right parties: A correction’, British Journal of Political Science, 33(3): 525534.CrossRefGoogle Scholar
Golman, R., Loewenstein, G., Moene, K. O. and Zarri, L. (2016), ‘The preference for belief consonance’, Journal of Economic Perspectives, 30(3): 165188.CrossRefGoogle Scholar
Grant, S., Kajii, A. and Polak, B. (1998), ‘Intrinsic preference for information’, Journal of Economic Theory, 83(2): 233259.CrossRefGoogle Scholar
Hagmann, D. and Loewenstein, G. (2018), Persuasion with Motivated Beliefs. Working paper.Google Scholar
Kahan, D. M., Peters, E., Dawson, E. C. and Slovic, P. (2017), ‘Motivated numeracy and enlightened self-government’, Behavioural Public Policy, 1(1): 5486.CrossRefGoogle Scholar
Kirchler, M., Andersson, D., Bonn, C., Johannesson, M., Sørensen, , Stefan, M. and Västfjäll, D. (2017), ‘The effect of fast and slow decisions on risk taking’, Journal of Risk and Uncertainty, 54(1): 3759.CrossRefGoogle ScholarPubMed
Kunda, Z. (1990), ‘The case for motivated reasoning’, Psychological Bulletin, 108(3): 480498.CrossRefGoogle ScholarPubMed
Lind, T., Erlandsson, A., Västfjäll, D. and Tinghög, G. (2018), ‘Motivated reasoning when assessing the effects of refugee intake’, Behavioural Public Policy, 124.Google Scholar
Lord, C. G., Ross, L. and Lepper, M. R. (1979), ‘Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence’, Journal of Personality and Social Psychology, 37(11): 20982109.CrossRefGoogle Scholar
Nurse, M. S. and Grant, W. J. (2020), ‘I'll see it when I believe it: Motivated numeracy in perceptions of climate change risk’, Environmental Communication, 14(2): 184201.CrossRefGoogle Scholar
Pennycook, G. and Rand, D. G. (2019), ‘Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning’, Cognition, 188: 3950. doi:10.1016/j.cognition.2018.06.011.CrossRefGoogle Scholar
Pennycook, G., Fugelsang, J. A. and Koehler, D. J. (2015), ‘Everyday consequences of analytic thinking’, Current Directions in Psychological Science, 24(6): 425432. doi:10.1177/0963721415604610.CrossRefGoogle Scholar
Persson, E., Andersson, D., Koppel, L., Västfjäll, D. and Tinghög, G. (2021), ‘A preregistered replication of motivated numeracy’, Cognition, 214: 104768.CrossRefGoogle ScholarPubMed
Peters, E. (2012), ‘Beyond comprehension: The role of numeracy in judgments and decisions’, Current Directions in Psychological Science, 21(1): 3135.CrossRefGoogle Scholar
Peters, E., Västfjäll, D., Slovic, P., Mertz, C., Mazzocco, K. and Dickert, S. (2006), ‘Numeracy and decision making’, Psychological Science, 17(5): 407413.CrossRefGoogle ScholarPubMed
Scherer, L. D., Yates, J. F., Baker, S. G. and Valentine, K. D. (2017), ‘The influence of effortful thought and cognitive proficiencies on the conjunction fallacy: Implications for dual-process theories of reasoning and judgment’, Personality and Social Psychology Bulletin, 43(6): 874887.CrossRefGoogle ScholarPubMed
Schwartz, L. M., Woloshin, S., Black, W. C. and Welch, H. G. (1997), ‘The role of numeracy in understanding the benefit of screening mammography’, Annals of Internal Medicine, 127(11): 966972.CrossRefGoogle ScholarPubMed
Slovic, P., Finucane, M. L., Peters, E. and MacGregor, D. G. (2007), ‘The affect heuristic’, European Journal of Operational Research, 177(3): 13331352.CrossRefGoogle Scholar
Taber, C. S. and Lodge, M. (2006), ‘Motivated skepticism in the evaluation of political beliefs’, American Journal of Political Science, 50(3): 755769.CrossRefGoogle Scholar
Tappin, B. M., Pennycook, G. and Rand, D. G. (2020), ‘Bayesian or biased? Analytic thinking and political belief updating’, Cognition, 204: 104375.CrossRefGoogle ScholarPubMed
Tinghög, G., Andersson, D., Bonn, C., Böttiger, H., Josephson, C., Lundgren, G. and Johannesson, M. (2013), ‘Intuition and cooperation reconsidered’, Nature, 498(7452): E1E2.CrossRefGoogle ScholarPubMed
Tinghög, G., Andersson, D., Bonn, C., Johannesson, M., Kirchler, M., Koppel, L. and Västfjäll, D. (2016), ‘Intuition and moral decision-making – The effect of time pressure and cognitive load on moral judgment and altruistic behavior’, PLoS ONE, 11(10): e0164012.CrossRefGoogle ScholarPubMed
Washburn, A. N. and Skitka, L. J. (2018), ‘Science denial across the political divide: Liberals and conservatives are similarly motivated to deny attitude-inconsistent science’, Social Psychological and Personality Science, 9(8): 972980.CrossRefGoogle Scholar
Figure 0

Figure 1. Experimental scenarios.

Figure 1

Figure 2. Proportion of participants assessing the different scenarios correctly, separated by ideological views. 95% confidence interval is included for each bar.

Figure 2

Figure 3. Percent of correct assessments by treatment, depending on the ideological view, separated by experimental condition. Note: The correct answer to the immigration scenario is that the crime rate decreases. The correct answer to the gender quota scenario is that the companies perform worse. N = 1211.

Figure 3

Figure 4. Percent of correct assessments by numeracy, depending on an ideological view.

Figure 4

Figure 5. The impact of ideological view on the likelihood to correctly assess respectively scenario conditioned on time pressure (A,B) and numeracy (C,D), respectively. Note: The error bars and dashed lines indicate 95% confidence intervals. (A,C) The results of the immigration scenario, where the correct answer is that more immigration decreases the crime rates. (B,D) The results of the gender quota scenario, where the correct answer is that companies that have implemented genre quotas perform worse.

Figure 5

Table 1. Linear probability regressions on the role of time pressure and numeracy.

Figure 6

Figure 6. The impact of ideological view on the likelihood to correctly assess respectively scenario, conditioned on numeracy. Note: The dashed lines indicate 95% confidence intervals. The sample is divided by treatment. (A,C) The results of the participants in the time pressure treatment, and (B,D) the results of the participants in the control group.

Supplementary material: File

Strömbäck et al. supplementary material

Strömbäck et al. supplementary material
Download Strömbäck et al. supplementary material(File)
File 115.3 KB