The Challenge of Elite Decision-Making

There are vast challenges to reaping the potential rewards of the behavioral revolution for the field of international law. Most of the scholarship in this domain runs experiments on convenience samples of students or some segment of the public to draw conclusions about what people think. That is perfectly ok if the subject of inquiry concerns choices or preferences held by these populations: for instance, whether information on international law shapes public opinion on the appropriateness of subjecting prisoners to solitary confinement.Footnote ¹ If a theory applies to any person, then evaluating it on any population is defensible.

Yet in reality, it is elites—or groups of elites—that make most critical decisions around international law, whether its design, negotiation, adoption, implementation, interpretation, or enforcement. It is the government agents and officials, judicial decision-makers, leaders of organized interest groups and political parties, among other high-level actors that play central roles in determining the letter, process, and outcome of the law. These people occupy leading positions of authority in their domain of experience. And for that reason, they are not easy to study.

Elites have demanding schedules, are hard to recruit into academic studies, and are cautious about sharing sensitive information.Footnote ² They don't make decisions in a vacuum or in the isolation of a lab or survey exercise. It can be extremely time consuming to recruit them for study, and often impossible to do so in a representative way. Getting enough people to participate in a study is critical to drawing reliable conclusions, yet also a huge challenge. Many elites cannot accept the types of incentives that researchers usually offer to entice them, such as a stipends or gift cards. And there are also ethical considerations about how much time these kinds of exercises take away from a person's duties.Footnote ³

The challenge then becomes how to apply insights from the behavioral revolution to international legal decision-making driven in large part by elites, when the predominant behavioral research is still driven by experiments on convenience samples of students or other populations, many of whom do not have the relevant experience or skills to put themselves in the shoes of experts. Asking the average person on the street to imagine they are in a position to make some informed decision about a hypothetical scenario they don't fully understand might produce a response, but whether that person's response reveals anything about what a real decision-maker in that domain would do is questionable.Footnote ⁴

At the core of this challenge is the fact that experienced elites tend to think and process information differently than the people who have traditionally been the most common subjects of most experimental research on behavioral decision-making. For example, studies suggest that experienced elites tend to use different heuristics when making complex decisions. They are often less averse to losses, possibly because they are more trusting and prone to cooperate. They tend to be more strategic in their interactions, to think like repeat players, and to evaluate time in a different way, placing higher value on future outcomes. They are also prone to over-confidence. Precisely because they have more experience, they are often better at taking risks, have more information at their fingertips, and operate under different structural or institutional constraints than does the general public.Footnote ⁵

These differences pose real challenges to researchers trying to draw behavioral inferences about elite-driven behaviors from studies which have drawn from the most readily accessible people: non-elites.

Strategic Reasoning and Patience

One illustration of how elites and non-elites differ in their decision-making capacities concerns the cognitive ability to reason strategically, arguably central to many international legal and governance decisions. Sitting at a negotiating or collective decision-making table; interacting with others who may or may not have the same preferences, capacities, or powers; trying to strategize the right action and anticipate the likely counter-reaction: These tasks are often at the core of what happens in the decision-making process surrounding international law and politics.

A key insight from the behavioral revolution is that not every actor is able to respond equally strategically—that is, not every actor is able to fully understand the “game” they are playing and to accurately anticipate what others are going to do. And, those deviations from purely strategic responses won't be universal; they will vary both across and within groups of people.

In a recent study, my co-authors and I investigated a unique group of more than one hundred high ranking (U.S. based) policy and business elites with extensive practical experience conducting international diplomacy or policy strategy. We recruited the participants in the study through multiple requests to personal contacts, followed by appeals to introduce us to their peers. The recruitment process took almost two years. Included in the study were members of Congress and their senior staff, top U.S. trade negotiators, senior executives in firms, and high-ranking civil servants in federal and state government. We compared them to college students and administered a variant on a standard game from the behavioral economics literature, (the “p-beauty contest”), designed to assess a person's strategic reasoning, including their awareness of other people's reasoning skills. That game asks each person to pick a number from 0 to 100 and defines the winner as the person whose number is closest to M times the average of all players’ numbers. The idea here is to measure whether a person will act non-strategically and pick 50 or whether they will think through how others are best likely to respond—where K is the number of iterative best responses that a person considers. While the game is not a perfect representation of strategic thinking, it has been widely used to predict strategic behavior.Footnote ⁶

Figure 1 provides a visual illustration of the differences between elites and college students, and also among elites. It shows that in our study, elites were much more prone to strategic thinking. Comparatively few elites responded randomly (or without strategy) to the decision-making task they were given (shown as level K=0 in the Figure). More elites acted at least in some measure strategically, taking into account what other players might do (shown as level K=1 and level K=2 types, meaning that they did consider other people's strategies for at least 1 or 2 iterations. K levels above 2 are extremely rare). Figure 1 also shows that, while most elites are more sophisticated than students in their reasoning abilities, not all elites are equally strategic.

Figure 1. Population differences between elites and undergraduates on strategic reasoning. Bars are sample means/proportions. Vertical lines are bootstrapped Standard Errors.

One implication from our study on behavioral international law is that bringing college students or average people into a laboratory setting may not produce accurate conclusions about the kinds of choices elite actors might make around a real-world task that requires strategic thinking. You can't easily generalize a lesson from the population under study to the real world of decision-making when people's reasoning capacities differ.

We also measured another behavioral characteristic that is critical to much legal and political decisionmaking: patience, or how much people value the future (sometimes known as the “discount rate”).Footnote ⁷ A central purpose of international law and legal institutions is to lengthen the shadow of the future, in part by creating opportunities and incentives for reciprocity and lasting interactions.Footnote ⁸ For elites who face such decisions, their own disposition toward patience could affect how they perceive the shadow's length because patient decision-makers are generally more willing to wait for benefits rather than simply take immediate gains (akin to the famous Stanford Marshmallow Experiment on delayed gratification).Footnote ⁹

To measure patience, we adapted another standard game from behavioral economics, asking people to make twenty different choices between a US$100 prize that would be paid to them within thirty days and a variable, larger prize that would be paid within sixty days. For each person, our measure of patience is the number of sixty-day choices (with a higher number indicating more patience).Footnote ¹⁰

Figure 2 provides a visual illustration of the measured levels of patience, or forward-looking thinking. The vertical axis shows the count of people and the horizontal axis shows the number of sixty-day (or most patient) choices they made. It shows that elites in our sample were on average more patient (falling more on the right side of the vertical axis) than our sample of college students, but also that there is substantial variation in patience among both students and elites.

Figure 2. Population differences between elites and undergraduates on patience. A) College students and B) Elites.

If patience is a likely factor in how an elite will reason through a task to come to a decision or action, college students in the lab or other convenience or general samples may not be able to approximate that decision-making process in the same way. That makes applying the insights of the behavioral revolution to the field of international law challenging to the extent that elites are not readily available to study. A big risk is faulty inference about how people make key decisions over legal tasks.

These are two behavioral traits that vary across individuals and groups, possibly affecting how people think about and make decisions regarding international law. They stand out, in part, because new research has shown how they relate to how real elites think about decisions concerning international law. The next section illustrates with examples just how this might work.

How Decision-Making Traits Affect International Legal Decision-Making

A central set of debates pervading the study of international law revolve around the design of international treaties. When, for instance, are agreements open to large-scale multilateral bargaining and participation preferable to those that concentrate on a more restricted club of states or actors? And when are formalized enforcement procedures desirable or, instead, risk deterring cooperation?Footnote ¹¹

Common answers point to things like domestic politics, the types and stakes of the problems at hand, the bargaining environment, and the costs of participation.Footnote ¹² And an entire cottage industry of research looks to the role that accountability mechanisms play, some suggesting that enforcement dampens the will to cooperate, while others arguing just the opposite.Footnote ¹³ These answers rest on two assumptions. First, people facing the same situation will come to similar decisions because context, structure, and institutions determine how they think about cooperation. Second, people are rational and equipped with high levels of strategic reasoning.

The behavioral turn in law calls both of these assumptions into question. Different people in the same situation, working within similar institutions, cultures, and contexts, can and do make different decisions. And not everyone thinks very strategically or possesses the same behavioral traits. People differ in ways that affect how they prefer to make decisions around legal tasks.

Mixing a substantive survey focused on two decision tasks regarding an international trade agreement with the behavioral games described above, my colleagues and I have shown how both patience and strategic reasoning affect how real world elites think about key characteristics of international law (in this particular study, trade law).Footnote ¹⁴ The first decision concerned the negotiation process for a new trade agreement—in particular, how many states parties to invite to the table.Footnote ¹⁵ We asked our sample of elites to reason through a specific trade-off highlighted in the literature: more actors at the table would benefit their own country (the United States) by covering more of world trade yet could complicate negotiations and risk less cooperation. The second decision concerned whether to support ratification of the trade agreement, with and without a formal dispute settlement mechanism in place. We framed the decision as one of strategic complement, where a country would most benefit from the agreement if other states also join and comply—both unknowns.

Our findings illustrate that elites, who are more strategic and patient, are also more cooperative than college students. They chose to invite more actors to the negotiating table at the outset and are more keen to join treaties. And while the promise of built-in enforcement generally made the elites in our study more willing to cooperate, expressed by their willingness to join the agreement, even more important in explaining their willingness were their strategic reasoning skills: more strategic elites were much more keen on trade cooperation. That sheds light into some canonical approaches to explaining why states cooperate through legal institutions, especially those that focus on the state of the world, the balance of power, or domestic politics and that tend to assume that any actor in the same position would make the same decision whether to cooperate. That assumption is not necessarily accurate.

Remedies

This challenge is no different for international legal studies than for any field where critical decisions are made by busy, high-ranking elites that want—and sometimes are required—to maintain privacy in their domain. Studying how convenience samples of non-expert populations make decisions regarding tasks that in the real world are elite-driven may produce incorrect conclusions in a wide array of fields.

There are, however, remedies to help minimize the dangers. One is to focus inquiry on the types of decisions that the accessible public might generally make, ideally in a balanced and representative way. There is a lot to learn, for instance, regarding public opinion on international law and the behavioral revolution could surely provide more insight there.

Another is to identify the behavioral traits that underlie the decision-making model being supposed and match on those traits. For instance, if your theory supposes a highly strategic decision-maker, evaluate the strategic reasoning skills of the population available to you (as demonstrated above) and select those people who have the necessary qualities. While not all college students or members of the general public are highly strategic, some are and so could better approximate a strategic elite.

A third remedy is to study the actual decision-makers, be they judges, arbitrators, lawyers or politicians. While this is not possible in all cases, a growing number of studies are applying the insights of the behavioral revolution directly to elites, with fruitful implications for the study of international law.Footnote ¹⁶

Finally, others are making fruitful efforts to overcome the limitations of lab experiments by complementing them with other methods, such as observational data or historical case studies based on archival work.Footnote ¹⁷ While none of these approaches provide a magic bullet to resolving the challenges outlined here, all provide some inroad toward heightening the promises and minimizing the perils of the behavioral study of international law.