This chapter treats optimal control for the purpose of application to optimal guidance. Optimal control can be used to find, from among several candidates, the best trajectory to accomplish a mission.

Note that Chapter 8 provides the tools to perform optimization in finite dimensional spaces. In practice, trajectory optimization can be, through appropriate discretization, transformed into a finite dimensional optimization problem; see, for instance, Section 8.4. So, the reader may ask: Why study optimal control beyond finite dimensional optimization? There are at least two compelling pragmatic answers to this question. First, after discretization, evaluating the gradients of the objective function and the constraints is generally much more cumbersome than writing the differential equations stemming from optimal control. Second, optimal control reveals the special structure of the state - co-state dynamics, the Hamiltonian or symplectic structure, which can be exploited in both analysis and computation.

Section 9.1 introduces and formulates the optimal control problem. Section 9.2 gives examples of optimal guidance problems fitting the formulation. Section 9.3 extends the results of Section 8.4 to obtain necessary conditions for optimal control without control constraints. Section 9.4 treats the case of control constraints, yielding the maximum principle. Section 9.5 is devoted to dynamic programming, which provides sufficient conditions. Section 9.6 elucidates the relationship between the results on optimal control and dynamic programming. Sections 9.7, 9.8, and 9.9 present a summary of the key results in the chapter, bibliographic notes for further reading and homework problems, respectively.