Introduction

Optimization is the term often used for minimizing or maximizing a function. It is sufficient to consider the problem of minimization only; maximization of F(x) is achieved by simply minimizing – F(x). In engineering, optimization is closely related to design. The function F(x), called the merit function or objective function, is the quantity that we wish to keep as small as possible, such as cost or weight. The components of x, known as the design variables, are the quantities that we are free to adjust. Physical dimensions (lengths, areas, angles, etc.) are common examples of design variables.

Optimization is a large topic with many books dedicated to it. The best we can do in limited space is to introduce a few basic methods that are good enough for problems that are reasonably well behaved and don't involve too many design variables. By omitting the more sophisticated methods, we may actually not miss all that much. All optimization algorithms are unreliable to a degree—any one of them may work on one problem and fail on another. As a rule of thumb, by going up in sophistication we gain computational efficiency, but not necessarily reliability.

The algorithms for minimization are iterative procedures that require starting values of the design variables x. If F(x) has several local minima, the initial choice of x determines which of these will be computed.