In this chapter we relax the assumption that agents can perfectly predict the consequences of their actions. Instead agents understand that the outcomes are generated probabilistically from their choice of action – certain actions increase or decrease the likelihood of particular outcomes. People know which actions are more or less likely to produce specific outcomes. Recall the example from the last chapter where A = {send in the troops, try negotiating, do nothing} and X = {win large concessions, win minor concessions, status quo}. The agent might believe that large concessions are more likely when the troops are deployed than when negotiation is initiated. Thus, in her decision, she balances this likelihood of generating a better outcome against the costs of each action. Deploying the troops would be rational if it is much more likely to lead to large concessions, if the additional concessions are valuable to the agent, or if the costs of deployment are low. These are the basic trade-offs underlying the classical theory of choice under uncertainty.

There are two key elements of this model of uncertainty. The first are beliefs that we model as probability distributions or lotteries over the outcomes associated with each action. The second are the payoffs associated with each outcome. These two elements combine to generate von Neumann-Morgenstern utility functions over actions. This naming convention honors two pioneers of classical decision theory.