The optimization procedures in the previous chapter were static: this means that decisions did not change over time, and the sexual displays of males looked the same mating season after season to the day they died.

Often, however, decisions are dynamic, which means that they change over time. For example, old individuals may face different trade-offs from young ones, because they find themselves in a different state: for example, they may be more experienced foragers and can, therefore, budget their time differently from young individuals. Or, environments may change over time: whether an insect or a copepod should develop directly into an adult, or first enter diapause, depends on whether summer is coming to a close. Very often, the optimal act is not one that maximizes immediate rewards – as any student doing long and difficult coursework can confirm. A copepod, for example, might gain some reproductive success sooner if it develops quickly, but if the reproductive season really is about to end, larger rewards await an individual who ignores this immediate gain and instead develops slowly, to mature at the beginning of the following season.

Dynamic optimization is needed when individuals have to make multiple decisions over time, and past decisions influence the options available in the future.