To study the effect of cognitive reflection on behavioral anomalies, we used the cognitive reflection test to measure cognitive reflection. The study was conducted on 395 Iranian university students and shows that subjects with lower cognitive reflection are significantly more likely to exhibit the conjunction fallacy, illusion of control, overconfidence, base rate fallacy, and conservatism. In addition, test scores are correlated with risk preferences. The results do not show any relationship between cognitive reflection and self-serving bias or status quo bias. We also find that gender is significantly related to illusion of control and self-serving bias.

The aim of the study was to test convergent/discriminant validity of two measures of cognitive reflection, cognitive reflection test (CRT) and belief bias syllogisms (BBS) and to investigate whether their distinctive characteristic of luring participants into giving wrong intuitive responses explains their relationships with various abilities and disposition measures. Our results show that the same traits largely account for performance on both non-lure task, the Berlin Numeracy Test (BNT), and CRT and explain their correlations with other variables. These results also imply that the predictive validity of CRT for wide range of outcomes does not stem from lures. Regarding the BBS, we found that its correlations with other measures were substantially diminished once we accounted for the effects of BNT. This also implies that the lures are not the reason for the correlation between BBS and these measure. We conclude that the lures are not the reason why cognitive reflection tasks correlate with different outcomes. Our results call into question an original definition of CRT as a measure of ability or disposition to resist reporting first response that comes to mind, as well as the validity of results of studies showing “incremental validity” of CRT over numeracy.

Assessing how managers discount and evaluate risks is crucial in designing effective managerial policies. In this work, we examine whether risk preferences (RP; both in the domains of gain and loss) and time preferences (TP) are related to managers’ cognitive reflection (CR). To achieve this, the current study focuses on the responses of 601 corporate decision-makers, such as CEO and CFO, of 200 non-financial firms listed at the Pakistan Stock Exchange. Using the three-item of Cognitive Reflection Test (CRT; Frederick, 2005) as a measure of CR, we observe that males perform better on this test than females. Correlation analysis reveals that individuals’ RP in the gain domain are positively associated with their TP, implying that risk-taking individuals are more patient. Our evidence further shows that higher CR is associated with a higher likelihood of increased patience and a lower likelihood of willingness to take risks in the domain of loss. Greater CR is also linked to a higher likelihood of risk-taking in the domain of gain. These findings have important implications regarding the ability of managers to make financial decisions that involve uncertainty and delayed rewards but maximize firm value.

Right-wing ideology and cognitive ability, including objective numeracy, have been found to relate negatively. Although objective and subjective numeracy correlate positively, it is unclear whether subjective numeracy relates to political ideology in the same way. Replicating and extending previous research, across two samples of American adults (ns= 455, 406), those who performed worse on objective numeracy tasks scored higher on right-wing authoritarianism (RWA) and social dominance orientation (SDO), and they self-identified as more conservative on general, social, and economic continua. Controlling for objective numeracy, subjective numeracy related positively to measures of right-wing ideologies. In other words, those who strongly (vs. weakly) endorsed right-wing ideologies believed they are good with numbers yet performed worse on numeracy tasks. We discuss implications for the opposing direction of associations between ideology with objective versus subjective numeracy and similarities with literature on overconfidence.

In the ultimatum game, one player proposes a split of money between him- or herself and another player, who can accept the offer (and both players keep the allocated money) or reject the offer (and both players get nothing). The present study examined predictors of accepting unfair ultimatum offers. In Study 1, 184 participants responded to an unfair ultimatum offer, completed a measure of cognitive reflection, and completed a self-report measure of rational and experiential thinking. Slightly more than half of the participants (54.3%) accepted the unfair offer, and cognitive reflection was positively correlated with accepting unfair offers. The rational and experiential thinking scales were not significantly correlated with ultimatum decisions. In Study 2, 306 participants responded to 20 ultimatum offers that varied in fairness and completed an expanded measure of cognitive reflection. Performance on the cognitive reflection measure predicted the number of ultimatum offers accepted. These results suggest that rejecting ultimatum offers is related to intuitive, heuristic-based thinking, whereas accepting offers is related to deliberate, analytic-based thinking.

Proportion responding (PR) is the preference for proportionally higher gains, such that the same absolute quantity is valued more as the reference group decreases. This research investigated this kind of proportion PR in decisions about saving lives (e.g., saving 10/10 lives is preferred to saving 10/100 lives). The results of two studies suggest that PR does not stem from an overall tendency to choose higher proportions, but rather from faulty deliberative reasoning. In particular, people who display PR are less likely to engage in deliberative reflection as measured by response time, the Process Dissociation Procedure, the Cognitive Reflection Test, a numeracy test, and a task assessing denominator neglect. This association between faulty deliberation and PR was observed only when choosing the highest proportion was non-normative because it came at the expense of absolute gains (e.g., saving 10/10 lives is preferred to saving 11/100 lives). These results help to make sense of discrepant findings in previous research, pertaining to how PR relates to biased reasoning and decision making.

Much research in cognitive psychology has focused on the tendency to conserve limited cognitive resources. The CRT is the predominant measure of such miserly information processing, and also predicts a number of frequently studied decision-making traits (such as belief bias and need for cognition). However, many subjects from common subject populations have already been exposed to the questions, which might add considerable noise to data. Moreover, the CRT has been shown to be confounded with numeracy. To increase the pool of available questions and to try to address numeracy confounds, we developed and tested the CRT-2. CRT-2 questions appear to rely less on numeracy than the original CRT but appear to measure closely related constructs in other respects. Crucially, substantially fewer subjects from Amazon’s Mechanical Turk have been previously exposed to CRT-2 questions. Though our primary purpose was investigating the CRT-2, we also found that belief bias questions appear suitable as an additional source of new items. Implications and remaining measurement challenges are discussed.

In social contexts, we refer to strategic sophistication as the ability to adapt our own behavior based on the possible actions of others. In the current study, we explore the role of other-oriented attention and cognitive reflection in explaining heterogeneity in strategic sophistication. In two eye-tracking experiments, we registered eye movements of participants while playing matrix games of increasing relational complexity (2x2 and 3x3 matrices), and we analyzed individual gaze patterns to reveal the ongoing mechanisms of integration of own and others’ incentives in the current game representation. Moreover, participants completed the Cognitive Reflection Test (CRT), in addition to alternative measures of cognitive ability. In both classes of games, higher cognitive reflection levels specifically predict the ability to incorporate the counterpart’s incentives in the current model of the game, as well as higher levels of strategic sophistication. Conversely, players exhibiting low cognitive reflection tend to pay less attention to relevant transitions between the counterpart’s payoffs, and such incomplete visual analysis leads to out-of-equilibrium choices. Gaze patterns appear to completely mediate the relationship between cognitive reflection and strategic choices. Our results shed new light on the cognitive factors driving heterogeneity in strategic thinking and on theories of bounded rationality.

We investigated the role of individual differences in cognitive reflection in decision making. We measured the performance of 157 participants in the cognitive reflection test (Frederick, 2005) and a number of decision-making tasks. We examined the relation of cognitive reflection with performance in tasks that assess correspondence (as distinct from coherence), such as predicting the ratings of chess players. We found significant correlations between cognitive reflection and all the estimation measures in correspondence tasks. Our results suggest that cognitive reflection is a thinking disposition that includes more characteristics than originally proposed by Frederick (2005). We proposed that cognitive reflection is related to the concept of actively open-minded thinking (Baron, 1985, 2008). We concluded that cognitive reflection is a thinking disposition that interacts with knowledge, domain-specific heuristics and characteristics of the environment and that it may play an essential role in the adaptation of the decision maker to different environments and situations.

Manipulations for activating reflective thinking, although regularly used in the literature, have not previously been systematically compared. There are growing concerns about the effectiveness of these methods as well as increasing demand for them. Here, we study five promising reflection manipulations using an objective performance measure — the Cognitive Reflection Test 2 (CRT-2). In our large-scale preregistered online experiment (N = 1,748), we compared a passive and an active control condition with time delay, memory recall, decision justification, debiasing training, and combination of debiasing training and decision justification. We found no evidence that online versions of the two regularly used reflection conditions — time delay and memory recall — improve cognitive performance. Instead, our study isolated two less familiar methods that can effectively and rapidly activate reflective thinking: (1) a brief debiasing training, designed to avoid common cognitive biases and increase reflection, and (2) simply asking participants to justify their decisions.