2.1 General Implications

Now we transition to using the STP objects to design our own lotteries without either inducing or assuming probable winner preferences. Let us consider decision making under risk when choice alternatives are lotteries — i.e. probability distributions over a nonempty finite set of outcomes. A decision maker faces a set of choice alternatives that contains at least two distinct elements. Next, the DM chooses the choice alternative that yields a strictly greater, context independent, expected utility. Often, no such choice alternative is present, so she compares the attributes of the available options to recognize where her preference lies. This step is required to avoid arbitrary choice; a growing body of evidence shows preferences are often known imperfectly (inter alia, Reference Butler and LoomesButler & Loomes, 2007).

Descriptively, the consensus is that true intransitive preference cycles are vanishingly infrequent. Evidence once taken to indicate systematic intransitivity (Reference Tversky and RussoTversky, 1969; Reference Loomes, Starmer and SugdenLoomes, Starmer & Sugden, 1991) has since been either reinterpreted as not reflecting fundamental intransitivity (inter alia: Reference Starmer and SugdenStarmer & Sugden, 1993) or found by newer statistical methods to be compatible with noisy but transitive responses (Baillon et al, 2015; Reference Birnbaum and DiecidueBirnbaum & Diecidue, 2015). In his highly influential 1969 article ’Intransitivity of Preference’, Amos Tversky lamented "...in the absence of a model that guides the construction of the alternatives, one is unlikely to detect consistent violations of weak stochastic transitivity (WST)".

One reason why experiments to date have only rarely found evidence of intransitive behaviour is lack of guidance from theory to select suitable lottery parameters. This lack of guidance is probably a result of the assumption any utility function must apply to all lottery pairs. However, this presumes there is no ’black hole’ in parameter space from which question sets can trigger preferences of a different kind. Drawing on the STP as a heuristic, we may address the problem Tversky faced and bespoke design candidate lotteries.

Let each consequence represent a sum of money, in £, a very familiar, directly comparable outcome for which magnitudes are easily interpretable by our subjects. For simplicity and comparability, the probability of each outcome in our design is always 1/3, 2/3 or 1. We imposed some filters to guide our selection of triples. Since expected value differences between each choice object, within a given triple, reach a maximum of £4 ²/3 ± £1 ²/3 ± 35.7%, our first filter is to focus on sets with larger EV differences (see Figure 1 for the distribution). We avoid constructing triples informed by sets with equal or near equal EV’s to avoid tipping the balance towards a cycle simply through noise.

Figure 1: Experimental Flow.

Next, the second filter reduces cognitive load by requiring each of the three STP objects to reuse integers such that there are no more than two different amounts as consequences (e.g., 5, 2 and 2); we therefore exclude any triple with three different money consequences on any lottery. It is also important to keep the presentation of the number of attributes in each object equal rather than coalescing identical outcomes. This is because past experiments have found the contrast between coalesced and non-coalesced outcomes (also known as event-splitting; Reference Starmer and SugdenStarmer and Sugden, 1993) can be confused as evidence for intransitivity (Reference Birnbaum and SchmidtBirnbaum & Schmidt, 2008; Baillon et al, 2015). To control for this we display each consequence even when all three lead to the same sum. We then made a number of modifications that increased the prizes on offer and then allowed for risk aversion in our experiment (risk-aversion plays no role for probable winner preferences). We make no claim this step in the parameter selection process involves more than a mix of informed guesswork and personal judgment.

Finally, as a third filter, to mimic the preference reversal (PR) phenomenon problems (Reference Lichtenstein and SlovicLichtenstein & Slovic, 1971) we decided to focus on the STP triples which have expected values strictly in the following order: $ ≻ P ≻ CE. Here, $ is a dollar bet (lottery which yields a large outcome with low probability), P is a probability bet (lottery which gives a small outcome with large probability) and CE is certainty. If we have 3 lotteries: X, Y and Z, we will assume that Z stands for the dollar bet, X - for the probability bet, and Y for certainty (degenerate lottery).

An important implication of our design choices is that the direction of cycles may be subject to two opposing forces, in aggregate, because the direction of cycles for standard PR lotteries is opposite to the ’probable winner’ cycle. However, this should not stop systematic cycles appearing at the individual level, if some people exhibit one tendency more strongly. This in mind, we can now put forward our first testable hypothesis. Reference Birnbaum and SchmidtBirnbaum & Schmidt (2008) succinctly state the currently dominant view regarding the evidence for intransitive preferences: "...we think the burden of proof should shift to those who argue that intransitive models are descriptive of more than five percent of the population".

Hypothesis 1: Drawing on the STP ingredients, we can design sets of lotteries for which cycles will occur with significantly greater frequency than 5%.

We can see the lotteries we designed in the left hand columns (columns 1–6) of Table 1.

Table 1: Binary choices and intransitivity.

In the spirit of Allais’ famous example, consider one such ’bespoke’ lottery set, and assume your preferred lottery is incentivized. The three choice objects are statistically independent lotteries: each outcome is a monetary amount with a one-third probability attached. For each of the binary choice sets {X,Y}; {Y,Z}; {Z,X}, viewed separately, we ask the reader to consider her preference, ideally looking only at each decision in isolation. In combination, there are eight possible binary preference patterns, of which just two are intransitive. Consider X versus Y, where X provides £15 with probability 1/3, £15 with probability 1/3, or £3 with probability 1/3; and Y yields £10 for certain - i.e., £10 with probability 1/3, £10 with probability 1/3, or £10 with probability 1/3 (see Table 1). Suppose Y ≻ X. Now compare Y which gives £10 for certain and Z which provides £27 with probability 1/3, £5 with probability 1/3, or £5 with probability 1/3. Perhaps, here Z ≻ Y. Finally, compare Z which yields £27 with probability 1/3, £5 with probability 1/3, or £5 with probability 1/3 and X which provides £15 with probability 1/3, £15 with probability 1/3, or £3 with probability 1/3.Footnote ³

In this case, maybe you found X ≻ Z. If you prefer Y ≻ X, Z ≻ Y and X ≻ Z, you have exhibited the preference cycle X ≻ Z ≻ Y ≻ X, the modal preference pattern for our subjects. The opposite cycle here is X ≻ Y ≻ Z ≻ X; we found these two intransitive patterns together exceeded, by a small majority, the six transitive patterns combined. Referring back to our opening example, suppose the consequences on each of X, Y and Z refer to investment returns on three portfolios and the probabilities are the historical frequencies. A consumer’s binary preferences over those risk-return combinations might potentially cycle also with implications for the structure of portfolios in finance.

2.2 Design Implications from Models of Probabilistic Choice

Although choice is often stochastic, and an intransitive cycle may arise from transitive latent preference due to noise, distinguishing structurally intransitive latent preferences from stochastic transitivity in experiments is not straightforward. How frequently can intransitive cycles arise for individuals with transitive preferences, but who choose probabilistically? For example, individuals may have transitive core preferences but choice probabilities are determined by embedding these preferences into a model of random errors (e.g., Blavatskyy, 2014). Such a modelling approach can generate a statistically significant asymmetry between the two possible cycle directions, but it cannot generate a proportion of intransitive cycles above 25% of all observed choice patterns, for any triple.

A more promising model of probabilistic choice for rationalizing intransitive cycles is the random preference approach (e.g., Loomes & Sugden, 1995). As the extreme example, let us consider an individual who has three transitive preference orderings X ≻ Y ≻ Z, Z ≻ X ≻ Y and Y ≻ Z ≻ X with each ordering equally likely to be drawn when a choice is to be made. It is straightforward to see that in a direct binary choice between X and Y, this individual chooses X with probability 2/3. Likewise, in a direct binary choice between Y and Z, this individual chooses Y with probability 2/3. Finally, in a direct binary choice between X and Z, this individual chooses Z with probability 2/3, thereby violating weak stochastic transitivity. Thus, a model of random transitive preferences generates a maximum of (2³)/(3³) = 8/27 (29.6%) intransitive choice cycles. This limit involves a strong asymmetry between the two possible intransitive patterns; the maximum frequency of a particular cycle given random sampling is 1/4; see Reference Rubinstein and SegalRubinstein & Segal (2012) for proofs of these propositions.

However, a model of random transitive preferences has another testable implication so far overlooked by a literature focused on binary choice sets. When comparing binary choice data with the choice data from a ternary set, we derive a new set of constraints that any stochastic but exclusively transitive preferences must meet. In such models of stochastic choice, the probability of choosing X from the ternary set {X,Y,Z} is given by the probability that a decision maker draws a preference order in which X is preferred to Y and X is preferred to Z. In contrast, for a direct binary choice between X and Y, this decision maker chooses X with a probability that is equal to the probability that he or she draws a preference order in which X is preferred to Y (but X may or may not be preferred to Z). Similarly, for a direct binary choice between X and Z, this decision maker chooses X with a probability that is equal to the probability that he or she draws a preference order in which X is preferred to Z (but X may or may not be preferred to Y). Hence, any model of random transitive preferences must make the following testable hypotheses. If any one of the three hypotheses fails to hold, no model of stochastic transitive preferences can be consistent with the data.

Hypothesis 2: The probability of choosing X from the ternary set {X,Y,Z} cannot exceed

(1)

Hypothesis 3: The probability of choosing Y from the ternary set {X,Y,Z} cannot exceed

(2)

Hypothesis 4: The probability of choosing Z from the ternary set {X,Y,Z} cannot exceed

(3)

Since, by definition, the probabilities of choosing X, Y and Z from the ternary set {X,Y,Z} must sum up to one, we have the following implication of any model of stochastic but transitive preferences:

(4)

A decision maker who violates weak stochastic transitivity, such that P(X,Y)>0.5, P(Y,Z)>0.5 and P(Z,X)>0.5, must still satisfy the inequality

(5)

which can be simplified as a triangle inequality

(6)

The triangle inequalities (5) and (6), it is usually argued, produce a stronger test than WST to separate genuine intransitivity from stochastic transitivity. However, Birnbaum (2011) showed that the triangle inequalities could be satisfied even by underlying intransitive preferences. Furthermore, recent work by Müller-Trede et al. (2015) demonstrates how these inequalities may be violated even when underlying preferences are 100% transitive. Their experiment also shows clear violations of these inequalities.

In other words, the triangle inequalities for stochastic transitive preferences can be satisfied when preferences are intransitive and violated when preferences are transitive, raising a concern that they are not as useful for identifying true intransitive preference cycles as generally believed, though see Cavagnaro & Davis-Stober (2014) for an alternative view. For these reasons, among others, our experiment was not designed specifically to test the triangle inequalities, which ideally would require multiple repetitions of the same lottery pairs for every person. However, we can test H2-H4 below, for each triple violating WST. We also follow Reference Birnbaum and DiecidueBirnbaum & Diecidue (2015) and repeat each set of choices once after a distractor task, which facilitates additional methods of separating noise from true preferences.

Consider a decision maker who: a) makes a direct binary choice between choice alternatives X and Y; and b) ranks choice alternatives X and Y as part of a ternary choice set in terms of their desirability. Ignoring the ranking of Z, this decision maker can reveal four different preference patterns:

• • X ≻ Y and X is ranked more desirable than Y (revealed preferences i);

• • Y ≻ X and Y is ranked more desirable than X (revealed preferences ii);

• • X ≻ Y and Y is ranked more desirable than X (revealed preferences iii);

• • Y ≻ X and X is ranked more desirable than Y (revealed preferences iv).

Revealed preferences i and ii are both consistent with the independence of irrelevant alternatives axiom. Revealed preferences iii and iv are inconsistent with this axiom. Thus, if preferences of a decision maker satisfy contraction and expansion consistency, we should observe patterns i and ii and not patterns iii and iv. Our experiment investigates.

2.3 Experimental Design

In total 100 subjects (all undergraduate students at the University of Warwick) were invited to take part in the experiment. We programmed the experiment using the Qualtrics software and consisted of 100 questions divided into 5 parts (see Figure 1).

Earlier tests for preference cycles primarily used state-contingent consequences in matrix-style displays. Those displays facilitate between-act comparisons and enhance the possibility of, for instance, anticipated regret when consequences are state-contingent and thus the potential for cycles. Our design maintains statistical independence between the choice objects such that any observed preference cycles are more likely to be rooted in description-invariant, intransitive latent preferences. Finally, we include a ’standard PR’ control set (Triple 9) to compare to our ’new PR’ gambles that is the focus of our experiment.

In Part 1, we broke up the 11 triples into binary choices between individual lotteries and asked subjects to answer 33 questions (3 binary choice questions per each triple). Table 1 provides a detailed list of all triples. We present each binary choice in the format shown in Figure 2 with two options — Left and Right. Each option shows a lottery with 3 equiprobable outcomes.

Figure 2: Binary choice display.

All binary choice questions gave subjects four different options on a slider. The initial starting point for each slider was "No preference". However, subjects were not able to proceed by leaving the slider in the original position (i.e., the choice of "No preference" was not allowed). Subjects were able to move the slider to the right and opt for "Slightly prefer Right" or "Strongly prefer Right" or, alternatively, to move the slider to the left and choose "Slightly prefer Left" or "Strongly prefer Left". Irrespective of whether a subject indicated slight or strong preference, we used only revealed preferences for "Left" or "Right" in the payoff calculations and we do not report the strength of preference results here. We randomized all 33 questions for each individual separately. In Part three, we repeated all 33 binary questions again but presented them in a different random order to each subject.

In Part 2, we asked subjects to make two choices in each ternary set, for the most and next most preferred object for each of the 11 triples (see Figure 3).

Figure 3: Ternary Choice Display.

We sought to maximize the similarity to the binary choice task and so we incentivized the choice of best and next best in each ternary set to obtain the full ordering in the triple. We explained that we would draw two of the three lotteries at random and they would play out whichever of the two they had positioned higher, if the ternary set is selected for payment. We use the standard ’random lottery incentive system’ for the binary choices. Our aim was to keep to a minimum any extraneous ’choice versus ranking’ disparities, to allow as clean a test as possible of the expansion consistency property and the transitive random preference restrictions, H2–4.

We randomized the order in which the ternary sets appeared, as well as the order in which the lotteries appeared on each screen. To avoid lazy acceptance of the default ordering, it was not possible to accept the default ranking. If the default was by chance preferred, they first had to move away from the default then move back to it by deliberate choice.

In Part 3, subjects were offered a Distractor Task in order to create a break between the two repetitions of binary and ternary choice tasks. The distractor task consisted of 12 risky choices using a different display, the results of which are not reported here. In Part 4, all 11 ternary choice set problems were repeated in a different random order.

We asked subjects to complete all tasks in the experiment online in a 5-day window. All 100 subjects received an invitation to the laboratory to play out their decisions for real money. Each subject drew a question number (between 1 and 100) at random and received payment based on his/her choice in that question. In each question, we looked at the lottery option chosen by the subject and played out that lottery according to the description on the experimental display (e.g., Figure 2 and Figure 3). Immediately after the draw, subjects received their payoff in cash. All 100 subjects turned up to play out the lottery and receive their winnings. Finally, all subjects completed a detailed online survey covering questions such as domain-specific risk attitudes and a variety of demographic variables not reported on here.