1 Introduction
Millions of people regularly play the lottery despite the fact that tickets typically have an expected value of less than half of their price (Clotfelter & Cook, Reference Clotfelter and Cook1990; Matheson, Reference Matheson2001). Traditionally, economic models explain such preferences by the shape of the utility curve. The utility curve is a hypothetical construct linking observable values of outcomes onto subjective, internal values. Because decision makers exhibit diminishing marginal utilities in most contexts, the average utility of a large and small outcome is hypothesized to be less than the utility of a middle-sized outcome. Consequently, decision makers should be risk-averse in these situations. Conversely, contexts in which marginal utilities are convex should promote risk-seeking.
The fact that lottery advertisements focus on the specific benefits of winning suggests that drawing attention to the possible jackpot can influence the decision to gamble (Forrest et al., Reference Forrest, Simmons and Chesters2002; Weatherly & Brandt, Reference Weatherly and Brandt2004). Indeed, several reports have suggested that the relative salience of different outcomes in a risky situation may influence the way the gamble as a whole is evaluated (Folkes, Reference Folkes1988; Tversky and Kahneman, Reference Tversky and Kahneman1973; Weatherly & Brandt, Reference Weatherly and Brandt2004). Outcomes that are more vivid, easier to remember, or more emotional are often seen as more likely — an idea known as the “availability heuristic” in economics (Corney & Cummings, Reference Corney and Cummings1985; Tversky & Kahneman, Reference Tversky and Kahneman1973). In a similar vein, several studies have suggested that feelings about possible outcomes can influence the appeal of the gamble (Isen et al., Reference Isen, Shalker, Clark and Karp1978; Loewenstein et al., Reference Loewenstein, Weber, Hsee and Welch2001; Rottenstreich & Hsee, Reference Rottenstreich and Hsee2001).
Consistent with these ideas, we hypothesized that the value placed on a risky option reflects, in part, the outcome of a competition between the possibilities of favorable and unfavorable outcomes, and that this competition can be biased towards the more salient possible outcome. We use the term salience to indicate the attentional weighting of this possibility in decision-making. However, this biasing may be stronger in the presence of uncertainty; when options are certain, biasing may be less likely to occur. When a decision-maker considers whether to gamble, he or she compares the value of the safe option to this biased valuation of the risky option.
We performed two behavioral experiments in rhesus monkeys to test the idea that risk sensitivity reflects outcome salience and outcome uncertainty itself, rather than just nonlinear weighting of reward outcomes. Both experiments employed variants of a gambling task developed in our lab in which monkeys are reliably risk-seeking (Hayden & Platt, Reference Hayden and Platt2007; McCoy & Platt, Reference McCoy and Platt2005). First, we compared monkeys’ preferences for risky and predictably alternating options that, over time, both offered an equal mix of large and small rewards (a procedure with some similarities to that used by Bateson & Kacelnik, Reference Bateson and Kacelnik1997). The alternating option provided identical sets of outcomes, and therefore identical utilities, to the monkeys, so any preference between these options cannot reflect utility weighting. We found that monkeys strongly preferred the risky option to the alternating one, demonstrating that the uncertainty of the risky option is part of its appeal.
We also hypothesized that the risk-seeking we observed reflects, at least in part, the salience of large rewards. If monkeys preferentially attend to the larger outcome, they should be more sensitive to small changes in its size. We therefore examined monkeys’ sensitivity to incremental changes in the sizes of large and small outcomes in a second experiment. As predicted, we found that monkeys were sensitive to variations in the size of the large reward, but not to equivalent changes in the size of the small reward.
2 Method
2.1 Behavioral techniques
Five male rhesus monkeys (Macaca mulatta) served as subjects. All animals were trained to make oculomotor responses for liquid rewards. Eye positions were sampled at 1000 Hz by an eye-monitoring camera system (SR Research, Osgoode, ON). Data was read by a computer running Matlab (Mathworks, Natick, MA) with Psychtoolbox and Eyelink Toolbox (Brainard, Reference Brainard1997; Cornelissen, Peters, & Palmer, Reference Cornelissen, Peters and Palmer2002). Visual stimuli were presented on a computer monitor directly in front of each animal and centered on his eyes, except as noted below. A standard solenoid valve controlled the duration of juice delivery. We calibrated the juice delivery system before, during, and after both experiments to ensure that reward volume was linearly proportional to valve open time. We found that the relationship between open time and volume was a linear function and did not vary on a day-to-day or a month-to-month basis.
2.2 Tasks
On every trial, a central cue appeared, which stayed on until the monkey fixated it within 1° (Figure 1). Following a brief delay, two eccentric targets appeared while the cue remained illuminated. Following another brief delay in which all three stimuli were illuminated (the decision period), the central target disappeared and the animal was required to quickly shift gaze to one of the two eccentric targets (15° to the left or right). Failure to shift gaze led to the immediate end of the trial with no reward and a timeout period. Following delivery of reward, all visual stimuli were extinguished from the screen and the monitor was left blank for a specified duration (inter-trial interval, ITI). ITI was 3 seconds.
Figure 1: Gambling task: After monkey fixated on a small central square, two eccentric response targets appeared. Following a 1 second delay, the central cue was extinguished, indicating that gaze must be shifted to either of the two targets. Following choice, all stimuli were extinguished and reward was delivered. In the Alternation Task, choices were either safe, risky, or alternating. In the Variance Task, one side offered a certain juice volume while the other side offered a risky volume.
In the Alternation Task, two of three possible targets appeared on each trial. The pair of targets used varied in blocks of 20 trials. The safe target, a gray rectangle 2° across and 6° tall, offered 200 µL juice. The alternating target, an orange rectangle of the same dimensions, offered either 67 µL or 333 µL of juice. The value of the alternating target changed each time it was chosen. The risky target was a blue and red rectangle of the same dimensions, and paid either 67 µL or 333 µL of juice, chosen randomly on each trial. Monkeys were well-trained in performing choice tasks, and familiar with these targets before the experimental sessions began.
In the Variance Task, the two targets looked identical on all trials (small yellow squares, 1°). The safe target offered 200 µL juice and the risky target offered one of two rewards, selected at random on each trial and not signaled to the animal. On each trial, the size of either the low or high target was sometimes modified by a small amount (35 µL). We chose this volume because it is close to the just noticeable difference in a choice task with deterministic rewards (McCoy et al., Reference McCoy, Crowley, Haghighian, Dean and Platt2003). In each block, we changed either the size of the large reward (1/3 of trials) or the size of the small reward (1/3 of trials), or neither (1/3 of trials), but never both. We never changed the size of the safe option. In practice, risky payoff pairs were selected at random from the following lists: [90,275], [125,275], [160,275], [125,240], [125,275], [125,310], [15,350], [50,350], [85,350], [50,315], [50,350], [50,385]. Units were microliters in all cases. Payoffs were varied in blocks of 20 or 40 trials and the location of the risky and certain targets were switched in blocks of 10 or 20 trials. Changes in blocks were not signaled to the subjects. The Alternation Task and Variability Task were run in separate behavioral sessions.
2.3 Statistics
Logistic regression and confidence intervals were computed in Matlab. In all cases, the dependent variable was choice frequency, while the independent variables were the change in the size of the large and small variable. P-values for the difference between these variables was obtained from performing a logistic regression on the difference between the large and small variables.
3 Results
3.1 Monkeys prefer risky options to alternating options
In the first experiment, we recorded choices made by three monkeys performing a variant of the gambling task (Hayden & Platt, Reference Hayden and Platt2007; McCoy & Platt, Reference McCoy and Platt2005) in which we manipulated outcome predictability while preserving outcome variability. Monkeys chose between pairs of options offering all three possible combinations of risky, alternating, and safe payoffs. The safe option offered 200 µL of juice. The risky option offered an unpredictable payoff of either 67 or 333 µL. The alternating option offered either 67 or 333 µL, but the value of this option alternated whenever this option was chosen, so that its value was predictable. The critical comparison was between alternating and risky choices. Two other choice types (risky vs safe, alternating vs safe) served as controls.
We recorded behavior in 7184 trials total (2827 in monkey N, 2133 in monkey B, 2224 trials in monkeys E). All three monkeys strongly preferred the risky option to the alternating option (79.71% in monkey N, 95.68% in monkey B, 87.45% in monkey E, p < 0.0001 in all cases, 2-tailed binomial test, Figure 2). Because the total number of large and small rewards for each option was stochastically identical, the expected values of the alternating option and the risky options were the same. These results indicate that risk preferences cannot be explained in terms of the non-linear weighting of utilities alone.
Figure 2: Monkeys preferred risky to alternating options with the same value. When monkeys chose between risky and alternating options with the same average expected value, they strongly preferred the risky option. The unpredictability of the risky option thus contributes to its appeal. Error bars indicate one standard error.
One possible concern is that the monkeys failed to recognize that the alternating option presented rewards in a predictable manner. However, to the extent that this occurred, the two options would be subjectively equivalent, and the monkeys would be indifferent to the choice between the risky and alternating options.
An alternative explanation for the greater appeal of the risky over the alternating option arises from temporal discounting. Monkeys and other animals, including humans, prefer rewards sooner rather than later, so they may be more likely to avoid the alternating option when it promises a small reward. However, this explanation is contradicted by the data. If discounting makes the alternating option less appealing, then, during strings in which the monkey did not choose the alternating option, the alternating option would be predicted to be set at the smaller reward. However, this was not the case. For monkey E, the average value of the alternating option on trials when it was not chosen was 226 µL, which is greater than the average overall value of 200 µL (binomial test, p < 0.0001). The average value for the alternating option when it was not chosen was greater for the other two monkeys as well (monkey N, value 242 µL, p < 0.0001, and monkey B, value 219 µL, p < 0.001). These data indicate that monkeys were more likely to avoid the alternating option when a larger reward was queued than when a smaller reward was queued. Although our data do not explain this behavior, they do indicate that preferences in the risky-alternating task are not strongly driven by temporal discounting.
We note that the alternating option alternated only when it was chosen, so, mathematically, monkeys necessarily chose it just as often when it was set to deliver a large and when it was set to deliver a small reward. This means that on half the trials when the alternating option was rejected, the monkey chose the risky option over a sure thing paying as much as the largest possible outcome of the gamble. These data therefore clearly demonstrate the importance of uncertainty in motivating choice behavior.
Two other choice types (risky vs safe, alternating vs safe) served as controls, and were presented in randomly interleaved blocks. We observed a clear preference for the risky over the safe option ( > 95% in all three subjects, p < 0.0001, binomial test), reproducing and confirming prior results showing that monkeys are risk-seeking in this task (Hayden & Platt, Reference Hayden and Platt2007; McCoy & Platt, Reference McCoy and Platt2005).
The results of the alternating vs safe comparison were mixed. Although monkey N and monkey E preferred the alternating option (79.06% and 95.35% preference respectively), monkey B preferred the safe option (36.92% preference). All preference levels were significantly different from chance (p < 0.0001, binomial test). Monkey N and Monkey E’s preference for the alternating option over the safe option may reflect a convex utility curve that contributes to risk-seeking, but that this effect was not sufficient to fully explain risk-seeking. However, this possibility is inconsistent with monkey B’s concurrent preference for safe over alternating options. In combination with the results of the first condition, therefore, these results demonstrate clearly that non-linear weighting for rewards is insufficient to fully explain the monkeys’ risk preferences.
3.2 Monkeys preferentially attend to the large reward
In the second experiment, we recorded choices made by four male rhesus macaques performing a variant of the gambling task (two of these were the same as were used in the previous study). Without providing any overt cues, we subtly manipulated the size of either the large or the small reward in blocks. In some blocks of trials, we offered either a 35 µL bonus or a 35 µL penalty for choosing the risky option. We obtained data in a total of 16007 trials (5054 in monkey N, 2445 in monkey B, 5443 in monkey O, 3065 in monkey D.
As expected, all monkeys were generally risk-seeking: all mean choice frequencies were above 0.5 (p < 0.001, binomial test, in all cases). Furthermore, all monkeys were sensitive to changes in the size of the large reward. Figure 3 shows the aggregate behavior of the population of monkeys. Adding a premium to the large reward increased risk-seeking, while removing the same amount from the large reward reduced risk-seeking (solid line). When the same premium and penalty were assigned to the small reward, preferences did not change (dashed line).
Figure 3: Monkeys attended to changes in the large reward in a gamble. Average likelihood of choosing the risky option increased when the value of the large reward increased and decreased when the value of the large reward decreased (solid line). Average likelihood of choosing the risky option was uncorrelated with the value of the small reward (dashed line). Note that vertical scale is magnified to emphasize the effect. Error bars indicate one standard error above and below the mean.
These effects are supported by the results of a logistic regression of risky choices on changes in the size of the large and small reward (shown in Table 1). All four monkeys showed regression coefficients for change in the large reward that were significantly greater than zero. None of the monkeys showed regression coefficients for the change in the small reward that were significantly different from zero. (We would expect them to be positive. The difference in small vs. large coefficients was even significant across subjects at p < .02.) The results of this experiment suggest that monkeys selectively attended to the large reward. Although these effects are small, they are significant. Indeed, the small size of the effects is a consequence of our task design: they reflect the small size of the manipulations we have made on reward size.
Table 1: Monkeys were more sensitive to changes in the size of the large reward than to equal changes in the size of the small reward. Rows correspond to the subjects, columns indicate regression parameters. First column indicates regression coefficient for changes in the size of the large reward (β large) and second column indicates p-vale for a comparison of β large to 0. Third column indicates regression coefficient for changes in the size of the small reward (β small) and fourth column indicates p-value for comparison of β small to 0. Fifth column indicates p-value for comparison of β large to β small
Notably, behavior was not determined solely by small changes in outcomes. As we have reported earlier (McCoy & Platt, Reference McCoy and Platt2005), we found that behavior most strongly reflects the outcome of the last trial. The regression coefficients for choice as a function of prior reward size in all four monkeys were positive, and significantly different from zero (monkey D, p = 0.0076, all others, p < 0.0001). We also found a weak influence of trial within block on risk-seeking behavior in two monkeys. In monkeys B and N, preference for the risky option grew slightly stronger (by about 3%) over the course of each block (regression coefficient 0.0072, p = 0.0426 in monkey B, and regression coefficient 0.0082, p = 0.0023 in monkey N). For the other two monkeys, a positive, non-significant trend was observed (regression coefficient 0.0044, p = 0.10 in monkey O, regression coefficient 0.0030, p = 0.378 in monkey D). These data indicate that, although behavior becomes more risk-seeking across a block, the effect is weak and inconsistent, suggesting that learning played a small role in determining behavior in this task.
4 Discussion
In our first study, we found that monkeys preferred risky options to alternating options offering the same distribution of rewards. These results demonstrate that risk sensitivity is not simply a consequence of a non-linear utility function. The results of our second study suggest an alternative explanation for risk sensitivity. We found that subtle manipulations in the size of the large payoff of a gamble have a greater influence on risk preferences than identical changes in the size of the small payoff. Monkeys’ greater sensitivity to changes in the large reward suggests that they attend more strongly to large rewards than to small rewards. The asymmetric salience of these outcomes may contribute to preferences for risk.
In contrast to salience-based biasing of risky options, standard explanations for risk sensitivity rely on the idea that utility reflects non-linear weighting of value (Friedman & Savage, Reference Friedman and Savage1948; Von Neumann & Morgenstern, Reference Von Neumann and Morgenstern1944). Simple mathematical principles show that concave utility curves promote risk-aversion, convex utility curves promote risk-seeking, and sigmoidal curves explain more complex behaviors, such as a gambler who purchases health insurance (Friedman and Savage, Reference Friedman and Savage1948). Despite its elegance, the expected utility model and its variants, including prospect theory, do not explain the full range of human and animal behavior under risk (Bateson, Reference Bateson2002; Bateson and Kacelnik, Reference Bateson and Kacelnik1997; Battalio et al., Reference Battalio, Kagel and Jiranyakul1990; Kacelnik & Bateson, Reference Kacelnik and Bateson1996; Lopes & Oden, Reference Lopes and Oden1999). In fact, utility curves are likely to be flat over most values used in laboratory experiments (Lopes, Reference Lopes1981). Additional challenges to expected utility come from studies identifying factors that strongly influence risky choices that have nothing to do with utility (e.g. Battalio et al., Reference Battalio, Kagel and Jiranyakul1990; Hayden & Platt, Reference Hayden and Platt2007; Hertwig et al., Reference Hertwig, Barron, Weber and Erev2004; Prelec & Loewenstein, Reference Prelec and Loewenstein1991). These results, and others, provide strong motivation for alternative explanations for risk preferences, such as the ones discussed here.
One other study has directly compared preferences for risky and alternating options (Bateson & Kacelnik, Reference Bateson and Kacelnik1997). In that study, reward sizes were identical, but delays to reward were either risky or alternating. Similar to our monkeys, the authors found that starlings (Sturnus vulgaris) preferred risky to alternating options, and preferred both to a safe option. These results provide additional evidence that uncertainty per se influences how risky options are evaluated.
The risk-seeking behavior observed in our task is somewhat unusual in studies of risk. In most studies, animals (Kacelnik & Bateson, Reference Kacelnik and Bateson1996; Battalio, et al., Reference Battalio, Kagel and MacDonald1985) and humans (Kahneman & Tversky, Reference Kahneman and Tversky1979) are found to be risk-averse. Nonetheless, risk-seeking has been observed in species ranging from rats (Rachlin, Reference Rachlin2000) to apes (Heilbronner et al., Reference Heilbronner, Rosati, Stevens, Hare and Hauser2008). Recent studies indicate that specific factors of task design may have strong effects on risk sensitivity. Factors that promote risk seeking include short intervals between choices (Hayden & Platt, Reference Hayden and Platt2007) and small reward sizes (Prelec & Loewenstein, Reference Prelec and Loewenstein1991), and the affective state of the decision-maker (Isen et al., Reference Isen, Shalker, Clark and Karp1978). Given the results presented here, we hypothesize that such factors may influence the relative salience of different gamble outcomes.
One factor that is particularly relevant is whether probabilities are learned through experience or provided explicitly. Much research has shown that risk sensitive behavior may differ when information about risk is learned through experience and feedback is given immediately compared with when it is learned via explicit description. Two types of effects are often reported (Erev & Barron, Reference Erev and Barron2005; Barron and Erev, Reference Barron and Erev2003; Hertwig et al., Reference Hertwig, Barron, Weber and Erev2004). Low probabilities are underweighted when information is learned through experience but over-weighted when information is provided verbally. Second, choices become more random as variability increases. Neither of these effects is likely to play a large role in our study, as all probabilities were 50/50 at all times. In general, it remains unclear what factors distinguish these two forms of risky decision-making, adding a caveat to generalizing from the results presented here.
It remains unclear why monkeys in our study would find the large reward more salient than the small reward. Psychological research into the availability heuristic suggests that possibilities that more emotional, easier to remember, or more unusual should be more available, and thus more salient (Folkes, Reference Folkes1988; Forrest et al., Reference Forrest, Simmons and Chesters2002; Tversky & Kahneman, Reference Tversky and Kahneman1973; Weatherly & Brandt, Reference Weatherly and Brandt2004). Future studies will be needed to identify the factors that promote salience of different outcomes in monkeys in humans. Our results also provide a behavioral framework suitable to identify the brain processes supporting the transformation of veridical values into decision weights, and then to choices (Sugrue et al., Reference Sugrue, Corrado and Newsome2005). The present results thus serve as a foundation from which to begin developing brain-based models of decision-making.
Open access
