INTRODUCTION

The purpose of this paper is to survey recent developments in a literature that combines ideas from experimentation, learning, and strategic interactions. Because this literature is multifaceted, let us start by circumscribing our overview. First and foremost, all surveyed papers involve nontrivial dynamics. Second, we will restrict attention to models that deal with uncertainty. Models of pure moral hazard, in particular, will not be covered. Third, we exclude papers that focus on monetary transfers. Our goal is to understand incentives via other channels – information in particular, but also delegation. Fourth, we focus on strategic and agency problems, and so leave out papers whose scope is decision-theoretic. However, rules are there to be broken, and we will briefly discuss some papers that deal with one-player problems, to the extent that they are closely related to the issues at hand. Finally, we restrict attention to papers that are relatively recent (specifically, we have chosen to start with Bolton and Harris, 1999).

Our survey is divided as follows. First, we start with models of strategic experimentation. These are abstract models with few direct economic applications, but they develop ideas and techniques that percolate through the literature. In these models, players are (usually) symmetric and externalities are (mostly) informational.

Moving beyond the exploitation/exploration trade-off, we then turn to agency models that introduce a third dimension: motivation. Experimentation must be incentivized. The first way this can be done (Section 3) is via the information that is being disclosed to the agent performing the experimentation, by a principal who knows more or sees more. A second way this can be done is via control. The nascent literature on delegation in dynamic environments is the subject of Section 4.

Section 5 turns to models in which information disclosure is not simply about inducing experimentation, but manipulating the agent's action in broader contexts. To abstract from experimentation altogether, we assume that the principal knows all there is to know, so that only the agent faces uncertainty.

Finally, Section 6 discusses experimentation with more than two arms (Callander, 2011).

EQUILIBRIUM INTERACTIONS

Strategic Bandits

Strategic bandit models are game-theoretic versions of standard bandit models. While the standard “multi-armed bandit” describes a hypothetical experiment in which a player faces several slot machines (“one-armed bandits”) with potentially different expected payouts, a strategic bandit involves several players facing (usually, identical) copies of the same slot machine.