The system concept was the very first one introduced in Chapter 1. We argued that using the word “design” instead of “system” allows us to make use of the extensive work done on studying systems. In this chapter, we take up the system discussion again and assert that every design is a system.What is the exact “system” is a matter of subjective perspective and focus of the design effort. Following the discussion in Section 1.2, our working concept of a system is that the system comprises elements, probably interacting with each other, that function together. A system can be partitioned into its elements (system partitioning), and its overall function is achieved through properly tracking how the elements function together (system coordination). The elements can be physical parts (object partitioning) or function disciplines (aspect partitioning).

System optimization acknowledges this partitioning and coordination character and follows the basic principle of decomposition: break the problem into simpler problem pieces (subproblems), solve each subproblem separately, and coordinate the subproblem solutions so that you obtain the solution to the original problem. This decomposition-based optimization strategy is useful if: (i) the effort for partitioning, subproblem solution, and coordination is less than the effort required to solve the original problem; or (ii) the original problem is simply not solvable without decomposition; and (iii) the decomposition-based solution is indeed the same as the solution to the original nonpartitioned problem. The first two cases are a practical matter for when to use decomposition strategies. The third one is both practical and theoretical; it requires a mathematical convergence proof that the decomposition strategy will yield solutions within the solution set of the original problem. This mathematical requirement turns out to be quite demanding, as many strategies developed for solving practical system design problems tend to be heuristic with no convergence proofs. Still, getting a good answer for a complex problem is better than no answer.

Designing is a complex human process that has resisted comprehensive description and understanding. All artifacts surrounding us are the results of designing. Creating these artifacts involves making a great many decisions, which suggests that designing can be viewed as a decision-making process. In the decision-making paradigm of the design process we examine the intended artifact in order to identify possible alternatives and select the most suitable one. An abstract description of the artifact using mathematical expressions of relevant natural laws, experience, collected data, and geometry is the mathematical model of the artifact. This mathematical model may contain many alternative designs, and so criteria for comparing these alternatives must be introduced in the model. Within the limitations of such a model, the best, or optimum, design can be identified with the aid of mathematical methods.

In this first chapter we define the design optimization problem and describe most of the properties and issues that occupy the rest of the book.We outline the limitations of our approach and caution that an “optimum” design should be perceived as such only within the scope of the mathematical model describing it and the inevitable subjective judgment of the modeler.

Mathematical Modeling

Although this book is concerned with design, almost all the concepts and results described can be generalized by replacing theword design by theword system. We will then start by discussing mathematical models for general systems.

The System Concept

A system may be defined as a collection of entities that perform a specified set of tasks. For example, an automobile is a system that transports passengers. It follows that a system performs a function, or process, which results in an output. It is implicit that a system operates under causality; that is, the specified set of tasks is performed because of some stimulation, or input.

It is almost three decades since this book was first written. Much has changed since then. Perhaps the change most relevant to our readers is the central role that design has taken in society's interests, and in education and research, but also in how it impacts our lives. Design of products and systems is recognized as an important element of a vibrant economy and an innovative society. More importantly, there is increased awareness that the many big problemswe face today, such as environmental sustainability, can be addressed through thoughtful design and up-front assessment of the trade-offs involved, rather than as remedial efforts made after the fact.

Understanding and quantifying such trade-offs to support our collective decision making means that design optimization is now more important than ever. Optimal design is the goal not only of engineering, but also of every other social effort to shape our world. Many of our problems usually grow from our inability to agree on what is “optimal.”

The book was born out of our own desire to address explicitly what we mean by “optimal” and to put the concept of optimal design on a firm, rigorous foundation. There is an intimate relationship between the mathematical model that describes a design and the solution methods that optimize it. A basic premise from the start was that a good model can make optimization almost trivial, whereas a bad one can make correct optimization difficult or impossible. Software tools today provide capabilities for intricate analysis of many difficult performance aspects of a system. These analysis models, often referred to also as simulations, can be coupled with numerical optimization software to generate better designs iteratively. This virtual prototyping ability has grown dramatically and is an important contributor to reducing product development time and increasing robustness of systems.

In designing, as in other endeavors, one learns by doing. In this sense, the present chapter, although at the end of the book, is the beginning of the action. The principles and techniques of the previous chapters will be summarized and organized into a problem-solving strategy that can provide guidance in practical design applications. Students in a design optimization course should fix these ideas by applying them to a term project. For the practicing designer, actual problems at the workplace can serve as first trials for this new knowledge, particularly if sufficient experience exists for verifying the first results.

The chapter begins with a review in Section 9.1 of some modeling implications derived from the discussion in previous chapters about how numerical algorithms work. Although the subject is quite extensive, our goal here is to highlight again the intimacy between modeling and computation that was explored first in Chapters 1 and 2. The reader should be convinced by now of the validity of this approach and experience a sense of closure on the subject.

Sections 9.2 and 9.3 deal with two extremely important practical issues: the computation of derivatives and model scaling. Local computation requires knowledge of derivatives. The accuracy by which derivatives are computed can have a profound influence on the performance of the algorithm. A closed-form computation would be best, and this has become dramatically easier with the advent of symbolic computation programs. When this is not possible, numerical approximation of the derivatives can be achieved using finite differences. The newer methods of automatic differentiation are a promising way to compute derivatives of functions defined implicitly by computer programs.

Scaling the model's functions and variables is probably the single most effective “trick” that one can do to tame an initially misbehaving problem. Although there are strong mathematical reasons why scaling works (and sometimes even why it should not work), effective scaling in practice requires a judicious trial and error approach.

Building the mathematical model is at least half the work toward realizing an optimum design. The importance of a good model cannot be overemphasized. But what constitutes a “good” model? The ideas presented in the first chapter indicate an important characteristic of a good optimal design model: the model must represent reality in the simplest meaningful manner. An optimization model is “meaningful” if it captures trade-offs that provide rigorous insights to whoever will make decisions in a particular context. One should start with the simplest such model and add complexity (more functions, variables, parameters) only as the need for studying more complicated or extensive trade-offs arises. Such a need is generated by a previous successful (and simpler) optimization study, new analysis models, or changing design requirements. Clearly the process is subjective and benefits from experience and intuition.

Sometimes an optimization study is undertaken after a sophisticated analysis or simulation model has already been constructed and validated. Optimization ideas are then brought in to convert an analysis capability to a design capability. Under these circumstances one should still start with the simplest model possible. One way to reduce complexity is to use metamodels: simpler analysis models extracted from the more sophisticated ones using a variety of data-handling techniques. The early optimization studies are then conducted using these metamodels.

In this chapter we provide some examples of constructing simple design optimization models, primarily to help the reader through an initial modeling effort. Before we get to these examples, we discuss some basic techniques on creating metamodels, from the more classical forms of curve fitting to the newer ideas of neural networks and kriging.

Modeling Data

Design models often need to incorporate experimental data, usually given in the form of tables or graphs. In this section we describe how data can be put into useful equation form. This is a very desirable modeling practice, whether for further analysis or for computation.

The optimization techniques in Chapters 4, 5, and 6 used local slope and curvature information of the model functions, namely the first and second derivative values at each point during iterations. The availability of derivatives allowed us to state necessary and sufficient conditions for optimality, and to derive algorithms that search for points satisfying these conditions. In many practical design situations, derivatives for the model functions may not exist or may not be readily available. One such situation is when the design variables take integer or discrete values and the function is defined by a series of points. Another situation is when the model functions have discontinuities in the function itself or in its slope, or when the functions are computed through simulations.

In such situations, it is more practical to stop relying on local slope information and to look for methods that do not use derivatives. There is a great variety of such methods, and we use the general term nongradient search to refer to them. The terms derivative-free, direct search, and black-box methods are also used somewhat interchangeably due to historical reasons, as modern method variants blur classification distinctions.

The general idea for nongradient methods is to use only function values. We saw an example of this in line search methods, like the Davies, Swann, and Campey method in Section 6.2. In a minimization problem, a point is better than another if it has a lower function value. The current best point is referred to as the incumbent. The trick is how to look for a good candidate point to challenge the incumbent. In some methods we do not need to know the actual function values at each point; it is sufficient to know how to rank the points we examine in order to identify the best. In some other methods we create many points all at once using some random process. In yet other methods we use information from the current points to generate new candidate points, for example by building surrogate models of the functions following the methods of Chapter 2.

In modeling an optimization problem, the easiest and most common mistake is to leave something out. This chapter shows how to reduce such omissions by systematically checking the model before trying to compute with it. Such a check can detect formulation errors, prevent wasteful computations, and avoid wrong answers. As a perhaps unexpected bonus, such a preliminary study may lead to a simpler and more clearly understandable model with fewer variables and constraints than the original one.

The methods of this chapter, informally referred to as boundedness analysis, should be regarded as a model reduction and verification process to be carried out routinely before attempting any numerical optimization procedure. At the same time, one should be cautious about the limitations of boundedness arguments because they are based on necessary conditions, namely mathematical truths that hold assuming an optimal solution exists. Such existence, derived from sufficient conditions, is not always easy to prove. The complete optimality theory in Chapters 4 and 5 provides important additional tools to those presented in this chapter.

One of the common themes in this book is to advocate “model reduction” whenever possible; that is, rigorously cutting down the number of constraint combinations and variables that could lead to the optimum – before too much numerical computation is done. This reduction has two motivations. The first is to seek a particular solution, that is, a numerical answer for a given set of parameter values. The second is to construct a specific parametric optimization procedure, which for any set of parameter values would directly generate the globally optimal solution with a minimum of iteration or searching.

The motivation to reduce models to construct optimized parametric design procedures comes from three applications. The first is to generate, without unnecessary iterative computation, the optimal design directly from a set of input parameter values. The second is to reoptimize specific equipment configurations in the face of changing parameter values. Designers call this “resizing.”