In designing, as in other endeavors, one learns by doing. In this sense the present chapter, although at the end of the book, is the beginning of the action. The principles and techniques of the previous chapters will be summarized and organized into a problem-solving strategy that can provide guidance in practical design applications. Students in a design optimization course should fix these ideas by applying them to a term project. For the practicing designer, actual problems at the workplace can serve as first trials for this new knowledge, particularly if sufficient experience exists for verifying the first results.

The chapter begins with a review of some modeling implications derived from the discussion in previous chapters about how numerical algorithms work. Although the subject is quite extensive, our goal here is to highlight again the intimacy between modeling and computation that was explored first in Chapters 1 and 2. The reader should be convinced by now of the validity of this approach and experience a sense of closure on the subject.

The next two sections deal with two extremely important practical issues: the computation of derivatives and model scaling. Local computation requires knowledge of derivatives. The accuracy by which derivatives are computed can have a profound influence on the performance of the algorithm. A closed-form computation would be best, and this has become dramatically easier with the advent of symbolic computation programs.

Model analysis by itself can lead to the optimum only in limited and rather opportune circumstances. Numerical iterative methods must be employed for problems of larger size and increased complexity. At the same time, the numerical methods available for solving NLP problems can fail for a variety of reasons. Some of the reasons are not well understood or are not easy to remedy without changes in the model. It is safe to say that no single method exists for solving the general NLP problem with complete reliability. This is why it is important to see the design optimization process as an interplay between analysis and computation. Identifying model characteristics such as monotonicity, redundancy, constraint criticality, and decomposition can assist the computational effort substantially and increase the likelihood of finding and verifying the correct optimum. The literature has many examples of wrong solutions found by overconfident numerical treatment.

Our goal in this chapter is to give an appreciation of what is involved in numerical optimization and to describe a small number of methods that are generally accepted as preferable within our present context. So many methods and variations have been proposed that describing them all would closely resemble an encyclopedia. Workers in the field tend to have their own preferences.

Building the mathematical model is at least half the work toward realizing an optimum design. The importance of a good model cannot be overemphasized. But what constitutes a “good” model? The ideas presented in the first chapter indicate an important characteristic of a good optimal design model: The model must represent reality in the simplest meaningful manner. An optimization model is “meaningful” if it captures trade-offs that provide rigorous insights to whoever will make decisions in a particular context. One should start with the simplest such model and add complexity (more functions, variables, parameters) only as the need for studying more complicated or extensive trade-offs arises. Such a need is generated by a previous successful (and simpler) optimization study, new analysis models, or changing design requirements. Clearly the process is subjective and benefits from experience and intuition.

Sometimes an optimization study is undertaken after a sophisticated analysis or simulation model has already been constructed and validated. Optimization ideas are then brought in to convert an analysis capability to a design capability. Under these circumstances one should still start with the simplest model possible. One way to reduce complexity is to use metamodels: simpler analysis models extracted from the more sophisticated ones using a variety of data-handling techniques.

A dozen years have passed since this book was first published, and computers are becoming ever more powerful, design engineers are tackling ever more complex systems, and the term “optimization” is routinely used to denote a desire for ever increasing speed and quality of the design process. This book was born out of our own desire to put the concept of “optimal design” on a firm, rigorous foundation and to demonstrate the intimate relationship between the mathematical model that describes a design and the solution methods that optimize it.

A basic premise of the first edition was that a good model can make optimization almost trivial, whereas a bad one can make correct optimization difficult or impossible. This is even more true today. New software tools for computer aided engineering (CAE) provide capabilities for intricate analysis of many difficult performance aspects of a system. These analysis models, often referred to also as simulations, can be coupled with numerical optimization software to generate better designs iteratively. Both the CAE and the optimization software tools have dramatically increased in sophistication, and design engineers are called to design highly complex problems, with few, if any, hardware prototypes.

The success of such attempts depends strongly on how well the design problem has been formulated for an optimization study, and on how familiar the designer is with the workings and pitfalls of iterative optimization techniques.

In modeling an optimization problem, the easiest and most common mistake is to leave something out. This chapter shows how to reduce such omissions by systematically checking the model before trying to compute with it. Such a check can detect formulation errors, prevent wasteful computations, and avoid wrong answers. As a perhaps unexpected bonus, such a preliminary study may lead to a simpler and more clearly understandable model with fewer variables and constraints than the original one.

The methods of this chapter, informally referred to as boundedness checking, should be regarded as a model reduction and verification process to be carried out routinely before attempting any numerical optimization procedure. At the same time, one should be cautious about the limitations of boundedness arguments because they are based on necessary conditions, namely mathematical truths that hold assuming an optimal solution exists. Such existence, derived from sufficient conditions, is not always easy to prove. The complete optimality theory in Chapters 4 and 5 provides important additional tools to those presented in this chapter.

The chapter begins with the fundamental definitions of bounds and optima, allowing a precise definition of a well-bounded model. Since poor model boundedness is often a result of extensive monotonicities in the model functions, the boundedness theory presented here has become known as Monotonicity Analysis.

In the previous two chapters we set out the theory necessary for locating local optima for unconstrained and constrained problems. From the optimality conditions we derived some basic search procedures that can be used to reach optima through numerical iterations. In Chapter 7 we will look at local search algorithms more closely. But before we do that we need to pick up the thread of inquiry for two earlier issues: the role of parameters and the presence of discrete variables.

One of the common themes in this book is to advocate “model reduction” whenever possible, that is, rigorously cutting down the number of constraint combinations and variables that could lead to the optimum – before too much numerical computation is done. This reduction has two motivations. The first is to seek a particular solution, that is, a numerical answer for a given set of parameter values. The second is to construct a specific parametric optimization procedure, which for any set of parameter values would directly generate the globally optimal solution with a minimum of iteration or searching.

If the variables are continuous, the most practical way to find a particular optimum for a single set of parameter values is simply to employ a numerical optimization code. When discrete variables are present, the resulting combinatorial problem can only be solved if we examine all possible solutions, directly or indirectly.

Introduction to Situation Assessment

An abridged version of the polymorphic estimator was presented in Chapter 2. The PME is a finite-dimensional algorithm for generating the Gt-conditional mean of the zygostate along with several moments that are interesting in their own right. The plant dynamics of the hybrid system are partitioned into a base-state, modal-state pair, and the observation has a compatible partition. In this chapter we will focus on the modal-state measurement subsystem in which the measurements are discrete.

The specific application to be considered is the study of situation assessment by human decision makers. Many geographically distributed systems include both human decision makers and a diverse collection of sophisticated hardware/software subsystems. With the unavoidable errors and distortions in data accumulation, transfer, and presentation, it is difficult to determine the appropriate human role, or even if the decision maker is properly filling the role given him. Athans referred to decision-making systems as “event driven” and observed that “the state variables…are both continuous and discrete” [Ath87]. In Athans's partitioning of the comprehensive state space, the discrete states represent global (or meta) states that modulate the local (or micro) aspects of the task environment. The decision maker's reaction to local phenomena tends to have a reflexive quality. It is in this reaction to macroevents that particularly human idiosyncrasies are manifest.

A primary task for a decision maker is to identify changing circumstances in an environment characterized by noise and clutter.

Who Should Read This Book

This book is intended for engineers and designers who seek to develop effective estimation and control algorithms for nonlinear systems. The reader is assumed to have some background in random processes and estimation (see, for example, [Pap91]) along with familiarity with concepts of feedback control phrased within the context of linear state space models (see, for example, [DB95] or [Wo194]). This background should include knowledge of

• random variables and processes,

• probability density and distribution functions for random variables, both continuous and discrete,

• moments and cross moments, including correlation functions,

• second-order properties of stationary processes, including power spectral densities,

• conditional expectations with respect to an observation process,

• fundamental properties of feedback systems, including stability and controllability of a system model.

Both time-continuous and time-discrete processes will be encountered. Some familiarity with mean-square estimation is useful. For example, in the development of the Kalman filter [BW92, Chapter 7], linear state space models are integrated with Gaussian white noise. The Kalman filter will form a basis of comparison for many of the estimators that follow.

Our treatment is applications oriented, but the reader will find that nonlinear systems require more detailed analysis than is necessary in the study of linear systems.

Introduction

Common problems in design require that an engineer devise a control or decision algorithm that converts measurements of system and environmental variables into signals that aid in system regulation. For example, a control node converts sensor outputs into an actuating signal that moves the system toward the desired operating point and keeps it there. At this foundational level, the engineer must formulate a mapping from the system observables into an action or report; for example, a feedback regulator converts the measured outputs of the system to be controlled (the plant) into an input that stabilizes the system.

Design is made difficult by disturbances internal to the system and by noise at its output. For example, there may be no sensors that measure those plant variables most useful for regulation, or, if measured, the variables may be masked by noise in the sensor-to-regulator link. Lacking omniscience, an engineer must process the available measurements to produce a good approximation to relevant but “hidden” variables. And this inference must be done on-line. The processing algorithm must not only be adapted to the incoming data stream, it must be of a form that can be implemented: An implementable estimation algorithm is an explicit mapping of the sensor output process (the measurements) into a (nearly) concurrent estimate of the required variables. In the applications studied here, the need for contemporaneous response limits consideration to finite-dimensional recursive algorithms; new observations are integrated into an estimate in an accretive manner.