1 Introduction
Are analytical thinkers less likely to be conservative across the globe? To answer this question, a series of recent studies conducted in different cultures have examined whether there is a difference in cognitive style (analytic vs. intuitive) between left and right-wing ideologies (Reference Brandt, Evans and CrawfordBrandt, Evans & Crawford, 2015; Reference Deppe, Gonzalez, Neiman, Jacobs, Pahlke, Smith and HibbingDeppe et al., 2015; Reference Eidelman, Crandall, Goodman and BlancharEidelman, Crandall, Goodman, & Blanchar, 2012; Reference Iyer, Koleva, Graham, Ditto and HaidtIyer, Koleva, Graham, Ditto & Haidt, 2012; Reference Jost, Sterling, Stern, Kopetz and FishbachJost, Sterling & Stern, 2018; Reference KahanKahan, 2013; Reference LandyLandy, 2016; Reference Piazza and SousaPiazza & Sousa, 2014; Pennycook, Cheyne, Seli, Koehler & Fugelsang, 2012; Reference Pennycook, Cheyne, Barr, Koehler and FugelsangPennycook, Cheyne, Barr, Koehler & Fugelsang, 2014; Saribay & Yilmaz, 2017; Reference Sterling, Jost and PennycookSterling, Jost, & Pennycook, 2016; Reference Talhelm, Haidt, Oishi, Zhang, Miao and ChenTalhelm, et al., 2015; Reference TalhelmTalhelm, 2018; Reference Van Berkel, Crandall, Eidelman and BlancharVan Berkel, Crandall, Eidelman & Blanchar, 2015; Reference Yilmaz and SaribayYilmaz & Saribay, 2016, Reference Yilmaz and Saribay2017a, Reference Yilmaz and Saribay2017b, Reference Yilmaz and Saribay2017c, Reference Yilmaz and Saribay2018a).Footnote 1 While some studies found a negative relationship between a performance measure of analytical cognitive style (ACS) as represented by the Cognitive Reflection Test (CRT; Frederick, 2005) and a single item political orientation question (ranging from extremely liberal to extremely conservative; Reference Deppe, Gonzalez, Neiman, Jacobs, Pahlke, Smith and HibbingDeppe et al., 2015; Pennycook et al., 2012; Reference Iyer, Koleva, Graham, Ditto and HaidtIyer et al., 2012; Reference Talhelm, Haidt, Oishi, Zhang, Miao and ChenTalhelm et al., 2015; Reference Yilmaz and SaribayYilmaz & Saribay, 2016), others failed to find any relationship (Reference KahanKahan, 2013; Reference LandyLandy, 2016; Reference Piazza and SousaPiazza & Sousa, 2014; Reference Yilmaz and SaribayYilmaz & Saribay, 2017c). The mixed findings in the literature raise the question whether this relationship is stable across cultures. In particular, although the ACS and ideology link is generally stable in Western samples (Reference Jost, Sterling, Stern, Kopetz and FishbachJost et al., 2018), the findings regarding this relationship are not always consistent in non-Western cultures (e.g., Reference Bahçekapili and YilmazBahçekapili & Yilmaz, 2017; Reference Yilmaz and SaribayYilmaz & Saribay, 2018a). Consistent with these correlational studies, experimental research also produced mixed findings: Reference Eidelman, Crandall, Goodman and BlancharEidelman et al. (2012) — with very low sample sizes — showed that the activation of intuitive thinking leads to conservatism in three different experiments in WEIRD samples (i.e., the US). However, two of these studies could not be replicated in non-WEIRD samples (i.e., Turkey) in high-powered attempts (Reference Yilmaz and SaribayYilmaz & Saribay, 2016, Reference Yilmaz and Saribay2018b). We argue that the link between cognitive style and social conservatism may be cross-culturally variable, which would explain at least some of the mixed findings in the literature. Therefore, relying on data from 30 politically diverse societies, this study provides a test whether the link between ACS and ideology is a cross-culturally stable phenomenon.
1.1 Toward a Cross-Culturally Valid Measure of Political Orientation
Although the single item political orientation question was previously considered a reliable method for measuring ideology (Reference JostJost, 2006), Reference Iyer, Koleva, Graham, Ditto and HaidtIyer et al. (2012) emphasized the confounding role of libertarians. Liberterians place themselves on the conservative side of the left-right (or liberal-conservative) spectrum when one-item political orientation question is used. At the same time, the libertarians have higher ACS scores than liberals and conservatives. Therefore, the one-item political orientation question has important limitations, and cannot be reliably used in cross-cultural research to investigate the link between ACS and ideology.
In other studies, the two dimensions of conservative ideology, namely social and economic conservatism, were measured separately to overcome the limitations of the one-item measure. Reference Jost, Sterling, Stern, Kopetz and FishbachJost et al. (2018), for example, conducted a meta-analysis and found that ACS (as measured by CRT) is correlated with social conservatism (r = −.15) and that the correlation between ACS and economic conservatism is weaker (r = −.08). The same moderating effect was evidenced in subsequent studies using different measures of ACS and ideology on American participants (Reference Yilmaz and SaribayYilmaz & Saribay, 2017c).
However, it is not clear whether the categories of social and economic conservatism provide a valid and useful distinction outside of the United States. The conservatism-as-motivated-social-cognition approach provides a better alternative to measuring ideology based on culture-free characteristics of the ideology (Reference Jost, Glaser, Kruglanski and SullowayJost, Glaser, Kruglanski & Sulloway, 2003). This approach defines ideology as comprised of two relatively independent motives: Resistance to change and opposition to equality, corresponding to social and economic conservatism in the US political context. These motives behind political conservatism are thought to be a defensive reaction, and serve to reduce the tension created by uncertainty and existential threat. CRT showed a significant negative relationship with only one of those motives, resistance to change, whereas it did not show any significant relationship with opposition to equality (Reference Yilmaz and SaribayYilmaz & Saribay, 2018a). Therefore, measuring ideology by focusing on resistance to change rather than the one-item general or social conservatism questions can be a more insightful and cross-culturally appropriate approach for testing whether the relationship between ACS and ideological orientation is cross-culturally stable.
Reference Jost, Sterling, Stern, Kopetz and FishbachJost et al. (2003) argue that these two motives are universal. However, there are reasons to question whether ACS is related only to social conservatism (among other ideology measures), and whether this relationship holds cross-culturally. Although Yilmaz and Saribay (Reference Yilmaz and Saribay2017a, Reference Yilmaz and Saribay2017b) found a causal effect of ACS on ideology, this finding does not directly correspond to the distinction between social and economic conservatism. In Reference Yilmaz and SaribayYilmaz and Saribay’s (2017b) study conducted in Turkey, binding moral foundations, which is a measure of social conservatism, were not related to CRT, although they were found to be related in American samples (Reference LandyLandy, 2016; Reference Pennycook, Cheyne, Barr, Koehler and FugelsangPennycook et al., 2014). Besides, most research on this link has been conducted among American participants (see Reference Jost, Sterling, Stern, Kopetz and FishbachJost et al., 2018), although there are some exceptions (e.g., Reference Yilmaz and SaribayYilmaz & Saribay, 2016). Therefore, there is a need for further cross-cultural investigation of this issue.
1.2 The Present Research
We argue that these mixed findings might be partially explained by cross-cultural differences across the samples. In order to categorize countries in terms of cultural differences, Henrich, Heine and Norenzayan (2010) proposed the concept of WEIRD, which stands for Western, educated, industrialized, rich, and democratic societies. They claim that the samples commonly used in psychology studies are mainly from WEIRD cultures, which represent only 15% of the world population; and thus the samples in most psychology studies are biased and not representative of the majority of the world population which is predominantly non-WEIRD. Although Reference Gervais, van Elk, Xygalatas, McKay, Aveyard, Buchtel and Svedholm-HäkkinenGervais et al. (2018) did not directly test the moderating role of cultural WEIRDness, they recently found a cross-cultural variability on the relationship between ACS (as measured by CRT) and religious belief (as measured by one item religious belief question), a related concept to social conservatism. This suggests that the WEIRDness of the culture might have a moderating role in the relationship between ACS and social conservatism.
In this study, we evaluate whether the relationship between social conservatism and ACS is cross-culturally stable using the large cross-cultural dataset of the Many Labs 2 Project (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018), and we investigate the moderating role of the WEIRDness of the culture. We use the binding subscale of the Moral Foundations Questionnaire developed by Reference Graham, Nosek, Haidt, Iyer, Koleva and DittoGraham et al. (2011) in order to measure social conservatism. This questionnaire was previously validated in several countries including WEIRD (e.g., Reference Davies, Sibley and LiuDavies, Sibley & Liu, 2014; Reference Métayer and PahlavanMétayer & Pahlavan, 2014; Reference Nilsson and ErlandssonNilsson & Erlandsson, 2015) and non-WEIRD cultures (Reference Berniūnas, Dranseika and SousaBerniūnas, Dranseika & Sousa, 2016; Reference Yilmaz, Harma, Bahçekapili and CesurYilmaz, Harma, Bahçekapili & Cesur, 2016a; Reference Zhang and LiZhang & Li, 2015). Binding foundations are comprised of patriotism (loyalty), respect for traditions (authority), and respect for the sacred (sanctity), and thus it is actually a repackaging of the previous literature on social conservatism (Reference Federico, Weber, Ergun and HuntFederico, Weber, Ergun & Hunt, 2013; Reference JostJost, 2012; Reference Kugler, Jost and NoorbaloochiKugler, Jost & Noorbaloochi, 2014; Reference Milojev, Osborne, Greaves, Bulbulia, Wilson, Davies and SibleyMilojev et al., 2014; Reference Sinn and HayesSinn & Hayes, 2016). In other words, while the individualizing foundations (corresponding to care and fairness) negatively relates to opposition to equality, binding foundations correspond to resistance to change (i.e., social conservatism; Reference Sinn and HayesSinn & Hayes, 2016). In different studies conducted with American (Reference Kugler, Jost and NoorbaloochiKugler et al., 2014) and Swedish (Reference Nilsson and ErlandssonNilsson & Erlandsson, 2015) participants, the relationship between the binding foundations and right-wing political orientation is largely explained by variation in resistance to change (i.e., social conservatism). In addition, Reference Sinn and HayesSinn and Hayes (2016) demonstrated in a factor analysis study that binding foundations correspond to resistance to change. More importantly, although the concept of social conservatism can have different meanings in different cultures, binding foundations represent an overall adherence to traditional moral values, which is likely to be an important indicator of social conservatism across different cultures (Reference Graham, Nosek, Haidt, Iyer, Koleva and DittoGraham et al., 2011). Therefore, in order to operationalize social conservatism in a cross-cultural study, we used the binding foundations measures for 30 politically diverse cultures. These cultures include relatively non-WEIRD cultures such as Brazil, China, Turkey, and India, and relatively WEIRD cultures such as Canada, Switzerland, France, and Austria. Overall, we aim to determine the direction and the magnitude of the relationship between ACS and social conservatism across WEIRD and non-WEIRD cultures.
2 Method
2.1 Participants
We retrieved the data from Many Labs 2 Project (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018), a replication project with a total sample of 15,305 participants from 36 countries. The original research consisted of two groups of studies called “slates”. The cognitive Reflection Test (CRT) was administered in both slates but the Moral Foundations Questionnaire and the one-item ideology measure were administered only in Slate 1. We hence only analyzed the data of Slate 1, which included 7,263 participants from 30 countries.
2.2 Measures
Cognitive reflection test
The cognitive reflection test (CRT; Frederick, 2005) consists of three mathematical questions. There are intuitive but incorrect answers to these questions. If one can suppress the intuitive answer and give the correct answer, this is considered an indicator of analytic thinking (e.g., Stagnaro, Pennycook & Rand, 2018; Meyer, Zhou & Frederick, 2018). Many Labs 2 Project included three CRT questions (Finucane & Gullion, 2010; see Table 1). We first cleaned the data by deleting any non-text characters in participants’ responses. For example, some participants responded as “2 minutes” or “$5”; in such cases, we deleted “minutes” and “$” to make these variables available for statistical analyses. We also removed invalid or meaningless responses (e.g., “I don’t care”). Then we coded incorrect responses as 0 and correct responses as 1. Lastly, we created an index score by summing up scores for these three questions which resulted in a potential range from 0 to 3. Higher scores indicate more analytic thinking. The Cronbach’s alpha score for correct responses to these three items was .610.
Table 1: List of CRT questions used in the current research
Moral foundations questionnaire
We measured social conservatism based on scores on binding moral foundations. Moral foundations questionnaire (Reference Graham, Nosek, Haidt, Iyer, Koleva and DittoGraham et al., 2011) employed in Many Labs 2 included three items for each of the five moral foundations (care, fairness, loyalty, authority, and sanctity). Individualizing foundations consist of care and fairness whereas binding foundations consist of loyalty, authority, and sanctity. Both individualizing (α = .822) and binding foundations (α = .777) subscales had sufficient reliability. We report the results regarding both individualizing and binding foundations, but we did not have any a priori hypothesis on the relation between CRT and individualizing foundations.
WEIRDness
In Many Labs 2 (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018), the level of WEIRDness of each country was quantified by scoring each of the components of WEIRD, namely Western (Western countries were rated as 1 whereas the others were rated as 0), Educated (rated based on the Education Index retrieved from the United Nations), Industrialized (rated based on the Industrial Development Report of the United Nations), Rich (developed countries were rated as 1 whereas the emerging economies were rated as 0), Democratic (democratization scores were retrieved from the Global Democracy Ranking) (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018; more detailed information is available at https://osf.io/b7qrt/). Thus they created a continuous scale for the level of WEIRDness of each nation. They also split the countries into two categories: Those who had higher than average WEIRDness scores were labeled as WEIRD samples whereas those with lower than average scores were labeled as non-WEIRD (Table 2).
Table 2: List of Non-WEIRD and WEIRD countries included in the sample (in alphabetical order)
Note. These countries were included in Slate 1 of Many Labs 2 Project. The second question was revised for certain cultures by using different currencies and amounts of money, although the required type of calculation was essentially the same. The expected correct answers for each country are listed in the Appendix.
Ideology
Ideology was assessed using a single item “Please rate your political ideology on the following scale” (1 = strongly left-wing, 7 = strongly right-wing).
We also considered using other measures to tap into the level of WEIRDness of countries. Reference Muthukrishna, Bell, Henrich, Curtin, Gedranovich, McInerney and ThueMuthukrishna et al. (2018) recently developed a scale to measure countries’ cultural distance to the United States (WEIRD scale) and China (Sino scale). However, we decided to base our analyses on the WEIRDness measurement employed by Many Labs 2 for two reasons: First, both WEIRD (r = −.739, p < .001) and Sino scales (r = .717, p < .001) were very strongly correlated with the WEIRDness measure in Many Labs 2. We reasoned that these different scales are tapping into the same construct. Second, Reference Muthukrishna, Bell, Henrich, Curtin, Gedranovich, McInerney and ThueMuthukrishna et al.’s (2018) study does not provide scores for six countries included in the current research (Costa Rica, United Arab Emirates, Czech Republic, Belgium, Portugal, and Austria).
3 Results
We conducted meta-analyses to examine (1) the overall effect of CRT on ideology, binding, and individualizing foundations, (2) and whether the level of WEIRDness accounts for the differences between countries. We used the meta-analysis function in JASP software (JASP Team, 2018) which is based on metafor, a meta-analysis package for R (Viechtbaur, 2010). We have utilized a fixed-effects method with moderators for the meta-analyses by restricting our inferences regarding the results to the countries included in the dataset (see Viechtbauer, 2010, for detailed discussion on the different types of meta-analyses). However, the Appendix reports analyses with a random-effects method (for foundations), which permits generalization to the population of countries (and yields weaker results consistent with the conclusions reported in the main text), and a restricted maximum likelihood method (with fixed effects). These Appendix also includes bivariate correlation tables.
3.1 The Effect on Ideology
The combined meta-analytic effect was statistically significant (b = −.051, SE = .018, z = −2.813, p = .005, 95% CI [−.086, −.015]). When continuous WEIRDness score was added to the model as a covariate, neither the intercept (b = .002, SE = .053, z = .045, p = .964, 95% CI [−.101, .134]), nor WEIRDness as a covariate had a significant effect (b = −.079, SE = .074, z = −1.079, p = .281, 95% CI [−.224, .065]). So, CRT was negatively related to right-wing ideology and WEIRDness did not moderate this effect (Figure 1). However, when a restricted maximum likelihood method was used, the meta-analytic effect of CRT did not reach to statistical significance (see the Appendix).
Figure 1: The distribution of unstandardized regression coefficients predicting ideology from CRT. Whiskers represent 95% confidence intervals for the coefficients. Grey diamonds represent the predictions from the level of WEIRDness. Countries are ranked based on their mean WEIRDness scores with India being the least WEIRD and Switzerland being the most WEIRD nation.
Lastly, we investigated which component of WEIRDness has stronger moderating effects. In Many Labs 2 (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018), separate scores for each of the five components of WEIRDness (Western, educated, industrialized, rich, and democratic) were provided. Instead of the mean WEIRDness score, we added these covariates to the model. Only education had a significant moderation effect (Table 3). Accordingly, CRT had a stronger effect on having a right-wing ideology in countries with better education.
Table 3: Estimates for the Effect of Different Components of WEIRDness on Ideology
3.2 The Effect on Binding Moral Foundations
The combined effect was significant (b = −.102, SE = .010, z = −9.769, p < .001, 95% CI [−.123, -.082], and suggested that higher scores in CRT were associated with lower levels of endorsement of binding moral foundations. Then, we added continuous WEIRDness score as a covariate to the model. The coefficient of the intercept was significant (b = .096, SE = .035, z = 2.730, p = .006, 95% CI [.027, .165]) and WEIRDness significantly explained the variance in effects for different countries (b = −.279, SE = .047, z = -5.889, p < .001, 95% CI [−.371, −.186]). Accordingly, CRT had a stronger negative effect on binding moral foundations in more WEIRD cultures (see Figure 2). When a restricted maximum likelihood model was used, similar findings were obtained except for that the moderation effect of WEIRDness was marginally significant (p = .053; see the Appendix).
Figure 2: The distribution of unstandardized regression coefficients predicting binding moral foundations from CRT. Whiskers represent 95% confidence intervals for the coefficients. Grey diamonds represent the predictions from the level of WEIRDness. Countries are ranked based on their mean WEIRDness scores with India being the least WEIRD and Switzerland being the most WEIRD nation.
We also conducted separate analyses for non-WEIRD and WEIRD countries (see Table 1 for the list of countries in each group). The meta-analytic estimate of the overall effect size was statistically significant for WEIRD countries(b = −.136, SE = .012, z = −11.352, p < .001, 95% CI [−.159, −.112]), but not for the non-WEIRD ones (b = .006, SE = .021, z = .278, p = .781, 95% CI [−.036, .048]). So, the negative relationship between CRT and binding foundations was unique to more WEIRD contexts.
Lastly, we added five components of WEIRDness as covariate to the model, instead of mean WEIRDness score. Only education and democracy components had significant effects (Table 3). Accordingly, the relationship between CRT and binding foundations is more negative for countries with higher education level and lower democracy.
3.3 Exploratory Analyses on Individualizing Foundations
Although we did not have any a priori hypothesis regarding the relationship between CRT and individualizing foundations or whether such relationship would vary based on the level of WEIRDness, we conducted exploratory analyses to provide a more complete picture of the data.
The combined effect without any covariate was not significant (b = −.016, SE = .017, z = −.939, p = .348, 95% CI [−.048, .017]). When continuous measure of WEIRDness was added to the model as a covariate, the coefficient for the intercept was significant (b = .117, SE = .042, z = −2.799, p = .005, 95% CI [.035, .199]). WEIRDness as a covariate was also significant (b = −.196, SE = .059, z = −3.328, p < .001, 95% CI [−.312, −.081]). Accordingly, the relationship between CRT and individualizing foundations was more negative in more WEIRD cultures (see Figure 3). The same results were obtained when a restricted maximum likelihood method was used (see the Appendix).
Figure 3: The distribution of unstandardized regression coefficients predicting individualizing moral foundations from CRT. Whiskers represent 95% confidence intervals for the coefficients. Grey diamonds represent the predictions from the level of WEIRDness. Countries are ranked based on their mean WEIRDness scores with India being the least WEIRD and Switzerland being the most WEIRD nation.
Table 4: Estimates for the effect of different components of WEIRDness on binding foundations
When the groups of WEIRD and non-WEIRD countries were separately analyzed (Table 1), CRT had a positive effect on individualizing foundations for non-WEIRD countries (b = .052, SE = .022, z = 2.393, p = .017, 95% CI [.009, .095]), whereas it had a negative effect for WEIRD countries (b = −.032, SE = .012, z = −2.677, p = .007, 95% CI [−.055, −.009]). So, high scores in CRT predicted higher endorsement of individualizing foundations in non-WEIRD countries but lower endorsement in WEIRD countries. When five components of WEIRDness, instead of the mean WEIRDness score, were added as covariates, none of them was uniquely significant (Table 5).
Table 5: Estimates for the effect of different components of WEIRDness on individualizing foundations
3.4 Exploratory Analyses on the Difference between Binding and Individualizing Foundations
As CRT had an effect on individualizing foundations and that effect varied based on the level of WEIRDness, similarly to the binding foundations, we investigated whether CRT had an effect on the difference between two types of foundations. We subtracted the mean score of individualizing foundations from binding foundations and used those scores as the outcome measures. These scores represented relative endorsement of binding foundations as opposed to individualizing ones.
The combined effect was significant (b = −.089, SE = .011, z = −8.370, p < .001, 95% CI [−.109, −.068]). The negative relationship suggested that higher performance in CRT was related to relatively more endorsement of individualizing foundations as opposed to the binding ones. When WEIRDness was added to the model as a covariate, the intercept, (b = −.030, SE = .031, z = −0.946, p = .344, 95% CI [−.091, .032]), was nonsignificant whereas the WEIRDness as moderator hadnegative effect (b = −.087, SE = .044, z = −1.993, p = .046, 95% CI [−.091, .032]; Figure 4). Accordingly, for more WEIRD countries, the negative effect of CRT on binding-minus-individualizing foundations was stronger. However, when restricted maximum likelihood model was used, the moderation was not significant (see the Appendix).
Figure 4: The distribution of unstandardized regression coefficients predicting the difference between binding and individualizing foundations from CRT. Whiskers represent 95% confidence intervals for the coefficients. Grey diamonds represent the predictions from the level of WEIRDness. Countries are ranked based on their mean WEIRDness scores with India being the least WEIRD and Switzerland being the most WEIRD nation.
When we conducted separate analyses for the groups of non-WEIRD and WEIRD countries (Table 1), CRT had a significantly negative association with the difference score for both non-WEIRD (b = −.054, SE = .019, z = −2.825, p = .005, 95% CI [−.092, −.017])and WEIRD countries (b = −.104, SE = .013, z = −8.171, p < .001, 95% CI [−.129, −.079]), but the effect was relatively stronger in WEIRD countries. So, higher scores in CRT was related to relatively higher endorsement of individualizing foundations as opposed to the bindings one, and this effect was stronger in WEIRD countries. Lastly, we added five components of WEIRDness, instead of mean WEIRDness score, as covariates to the model. Education was negatively and democracy was positively related (see Table 6). Accordingly, the negative effect of CRT on binding-minus-individualizing foundations was relatively stronger in countries with better education and lower democracy.
Table 6: Estimates for the effect of different components of WEIRDness on difference between binding and individualizing foundations
4 Discussion
The current research provides the first empirical test of cross-cultural generalizability of the ACS-ideology link in 30 politically diverse societies. The results show that although ACS has a significant negative relationship with social conservatism (measured by both the level of endorsement of binding moral foundations and the difference between binding and individualizing foundations) in the overall sample, the magnitude of this relationship is weaker in non-WEIRD cultures as compared to the WEIRD ones. There was also a negative correlation between the commonly used single-item political orientation question and ACS, but WEIRDness did not moderate this effect and a different meta-analysis technique did not yield any effect. These findings are consistent with past findings, suggesting that ACS is associated with social attitudes in general (e.g., Pennycook, Fugelsang & Koehler, 2015), and social conservatism in particular (e.g., Reference Yilmaz and SaribayYilmaz & Saribay, 2018a). The finding that the degree of WEIRDness of a culture influences the magnitude of the relationship has also the potential to explain a number of mixed findings previously observed in the literature.
First of all, the majority of the findings in the literature come from WEIRD cultures. Similarly, a meta-analysis of Reference Jost, Sterling, Stern, Kopetz and FishbachJost et al. (2018) is substantially limited to studies with WEIRD cultures. An exception is data collected from Turkey (Bahçekapili & Yilmaz, 2017; Reference Yilmaz and SaribayYilmaz & Saribay, 2016, Reference Yilmaz and Saribay2018a). Although a significant negative relationship between ACS and social conservatism was generally found (Reference Yilmaz and SaribayYilmaz & Saribay, 2016, Reference Yilmaz and Saribay2018a), there is no consistent pattern of relationship across samples in Turkey (Reference Yilmaz and SaribayYilmaz & Saribay, 2017b), a non-WEIRD culture (Reference Klein, Vianello, Hasselman, Adams, Adams, Alper and NosekKlein et al., 2018). The effect sizes of the studies conducted in Turkey are generally smaller than similar research conducted in WEIRD cultures (Reference Yilmaz and SaribayYilmaz & Saribay, 2017c). Similarly, whereas both Reference Pennycook, Cheyne, Barr, Koehler and FugelsangPennycook et al. (2014) and Reference LandyLandy (2016) found a significant negative relationship between CRT and binding moral foundations (patriotism, respect for traditions, and bodily purity) in WEIRD cultures (U.S.), Reference Yilmaz and SaribayYilmaz and Saribay (2017b) found no significant relationship in a non-WEIRD culture (Turkey). Therefore, the findings of the current research explain the mixed findings of some previous studies by proposing a boundary condition for the cross-cultural generalizability of the link between ideology and ACS.
Findings related to single item political orientation were mixed. Although a fixed-effects method yielded a negative effect of ACS on ideology, the same result was not obtained with a restricted maximum likelihood method. This suggests that the results regarding self-reported ideology were somewhat less reliable as compared to social conservatism. This was generally consistent with the literature since although it was previously used as a reliable measure in the literature (Reference JostJost, 2006), Reference Iyer, Koleva, Graham, Ditto and HaidtIyer et al.’s (2012) findings highlight its limitation. The argument is that although libertarians are different from conservatives in terms of social attitudes (i.e., social conservatism), they place themselves on the conservative side of this single item political orientation question. Assuming there might be different groups with different labels in other cultures having similar social attitudes of the libertarians, these results might imply that this single item political orientation question is not a reliable measurement tool that can be used in a cross-cultural examination.
So why is there a difference between WEIRD and non-WEIRD cultures in the relationship between ACS and social conservatism? Although this study remains silent on this issue, we conjecture that ACS is more effective in influencing political attitudes in cultures where the left-right division is very clearly differentiated. Although there is no direct evidence for this argument yet, there is some indirect support. For example, if we compare Turkish (a non-WEIRD culture) and US (a WEIRD culture) political systems, we can see that Turkish political atmosphere is not clearly divided by left-right or liberal-conservative distinction as in the American counterpart (Reference MardinMardin, 1973; Reference ÖnişÖniş, 2007; Reference ÖzbudunÖzbudun, 2006). Reference Yılmaz, Saribay, Bahçekapılı and HarmaYılmaz, Saribay, Bahçekapılı and Harma (2016b) also showed in Turkey that the supporters of CHP (Republican People’s Party) — the major social democratic party — did not distinguish from the supporters of MHP (Nationalist Movement Party) and AKP (Justice and Development Party) — the two major conservative parties — on the importance they give to the binding foundations (i.e., social conservatism). Social conservatism is very high in Turkey, and this might be the case for other non-WEIRD cultures as well. Therefore, the fact that the variance related to social conservatism is much narrower in non-WEIRD cultures might explain why the relationship between social conservatism and CRT is so weak here.
4.1 Limitations and Future Questions
First, an important potential limitation is that some results were weak, with their clarity somewhat dependent on the method used (fixed effects meta-analysis vs. random effects, maximum likelihood vs. restricted maximum likelihood), especially the results concerned with ideology, where CRT had no effect on ideology with maximum liklihood, and the moderating effect of WEIRDness on the relationship between CRT and binding foundations was only marginally significant. However, we argue that the pattern in results suggest that culture matters in the relationship between ACS and political orientations, although the direction and magnitude of this relationship might not be very clear.
Another limitation of this research is that it uses only three numerical CRT questions to measure ACS. Although there are some recent findings indicating that CRT corresponds to a stable personality trait (Meyer et al., 2018; Stagnaro et al., 2018), there are also some others claiming that it sometimes does not measure relevant aspects of ACS (e.g., Reference Baron, Scott, Fincher and MetzBaron, Scott, Fincher & Metz, 2015). Although the results generally remained constant when other ACS measures were used (e.g., Reference Pennycook, Cheyne, Barr, Koehler and FugelsangPennycook et al., 2014), Reference Yilmaz and SaribayYilmaz and Saribay (2017c) found that the three different ACS measures (CRT-2, Base rate Conflict Problems, and Actively Open Minded-Thinking) were significantly (and negatively) related to social conservatism, and that CRT was not individually related to any of the ideology measures (including social conservatism). Therefore, future studies should use other ACS measures in addition to CRT in order to show the cross-cultural validity of the findings here.
Assuming that these correlations might give an idea about the potential cause-effect relation between ACS and social conservatism (i.e., causality flows from a more basic, cognitive variable to sociopolitical attitudes, which is partially supported by Reference Yilmaz and SaribayYilmaz & Saribay, 2017a), the same boundary condition has also the potential to explain the mixed findings regarding experimental work in the literature.Footnote 2 One of the first empirical demonstrations of the causal effect of intuition on conservatism was conducted by Reference Eidelman, Crandall, Goodman and BlancharEidelman et al. (2012) using WEIRD (i.e., US) samples but were not replicated in later high-powered studies using non-WEIRD samples (Reference Yilmaz and SaribayYilmaz & Saribay, 2016, Reference Yilmaz and Saribay2018b). Therefore, similar attempts to reconcile the mixed findings might consider experimentally controlling the degree of WEIRDness of culture in future studies.
The most important contribution of this study is to show the moderating role of culture by moving beyond WEIRD samples. However, although this study was based on 30 politically diverse societies that are not frequently used in psychology research, it still does not say anything about less-represented groups such as small-scale hunter-gatherers or older adults. More importantly, none of the samples were selected based on a probabilistic random sampling procedure, therefore any of them do not represent the characteristics of each country as a whole. Another potential limitation, and an alternative explanation for the lower correlations in non-WEIRD cultures is the possibility of noisy data (i.e., translation issues) and lower levels of survey expertise (i.e., less familiarity with the survey response scales). Therefore, in order to confidently conclude that there are real cultural differences, future studies should account for these two issues. Reference Pennycook and RandPennycook and Rand (2019) also showed that political participation is positively correlated with CRT scores, and the association between CRT and binding foundations might be driven by some politically disengaged moderates in either WEIRD or non-WEIRD cultures in the current study. However, this goes beyond the scope of the present research as the available set of measures did not include a measure of political participation.
4.2 Conclusion
The current study shows for the first time that the relationship between ACS and social conservatism is stable for only WEIRD cultures. Therefore, if an independent observer wants to predict one’s degree of social conservatism, she must look at both the level of ACS and where she lives, since the findings of the current study illustrates that there is an important cross-cultural variability on this relationship. Although ACS explains unique variance in social conservatism, the effect sizes are small even in WEIRD cultures, which indicate that cultural factors may be more important than differences in cognitive style. Overall, this study emphasizes the limitations of studying only WEIRD cultures for making inferences about human universals (especially on the moral domain).
Appendix
Table 7: Correct responses to item #2 of Cognitive Reflection Test. For the remaining countries, the correct response was 2.25
Table 8: Correlations of main variables with CRT for each country
Table 9: Descriptive statistics for main variables (SD in parentheses)
Table 10: Bivariate correlations between CRT, ideology, binding foundations, and individualizing foundations
Table 11: Bivariate correlations for countries categorized as WEIRD cultures
Table 12: Bivariate Correlations for countries categorized as non-WEIRD cultures
Results of Meta-Analyses of Foundations with Random Effects
The Effect on Binding Foundations
The combined meta−analytic effect was significant (b = −.061, SE = .020, z = −3.095, p = .002, 95% CI [−.101, −.023]). When continuous WEIRDness score was added to the model as a covariate, intercept was nonsignificant (b = .038, SE = .051, z = .736, p = .462, 95% CI [−.063, .138]) and WEIRDness as a covariate was significant, but barely (b = −.156, SE = .074, z = −2.103, p = .036, 95% CI [−.301, .012]).
The Effect on Individualizing Foundations
The combined meta-analytic effect was not statistically significant (b = −.016, SE = .016, z = −.957, p = .339, 95% CI [−.048, .016]). When continuous WEIRDness score was added to the model as a covariate, both intercept (b = .117, SE = .041, z = 2.882, p = .004, 95% CI [.038, .197]) and WEIRDness as a covariate were significant (b = −.194, SE = .057, z = −3.413, p < .001, 95% CI [−.305, −.082]).
The difference between binding and individualizing foundations as a covariate was not close to significant (as in other analyses).
Results of Meta-Analyses with Restricted Maximum Likelihood Method
The Effect on Ideology
The combined meta-analytic effect was not statistically significant (b = −.035, SE = .029, z = −1.234, p = .217, 95% CI [−.091, .021]). When continuous WEIRDness score was added to the model as a covariate, neither the intercept (b = −.041, SE = .083, z = −.488, p = .625, 95% CI [−.204, .123]) nor WEIRDness as a covariate had a significant effect (b = .009, SE = .122, z = .071, p = .943, 95% CI [−.230, .247]; see Figure 1).
The Effect on Binding Foundations
The combined meta−analytic effect was significant (b = −.061, SE = .020, z = −3.041, p = .002, 95% CI [−.101, −.022]). When continuous WEIRDness score was added to the model as a covariate, intercepts was nonsignificant (b = .034, SE = .053, z = .639, p = .523, 95% CI [−.703, .138]) and WEIRDness as a covariate was marginally significant (b = −.149, SE = .077, z = −1.936, p = .053, 95% CI [−.300, .002]; see Figure 2).
The Effect on Individualizing Foundations
The combined meta-analytic effect was not statistically significant (b = −.016, SE = .017, z = −.939, p = .348, 95% CI [−.048, .017]). When continuous WEIRDness score was added to the model as a covariate, both intercept (b = .117, SE = .042, z = 2.799, p = .005, 95% CI [.035, .199]) and WEIRDness as a covariate were significant (b = −.196, SE = .059, z = −3.328, p < .001, 95% CI [−.312, −.081]; see Figure 3).
The Effect on the Difference between Binding and Individualizing Foundations
The combined meta-analytic effect was statistically significant (b = −.050, SE = .0182, z = −2.818, p = .005, 95% CI [−.086, −.015]). When continuous WEIRDness score was added to the model as a covariate, neither intercept (b = −.074, SE = .051, z = −1.435, p = .151, 95% CI [−.174, .027]) nor WEIRDness as a covariate was significant (b = .037, SE = .075, z = 0.494, p = .621, 95% CI [−.110, .184]; see Figure 4).
Open access
