Model analysis by itself can lead to the optimum only in limited and rather opportune circumstances. Numerical iterative methods must be employed for problems of larger size and increased complexity. At the same time, the numerical methods available for solving NLP problems can fail for a variety of reasons. Some of the reasons are not well understood or are not easy to remedy without changes in the model. It is safe to say that no single method exists for solving the general NLP problem with complete reliability. This is why it is important to see the design optimization process as an interplay between analysis and computation. Identifying model characteristics such as monotonicity, redundancy, constraint criticality, and decomposition can assist the computational effort substantially and increase the likelihood of finding and verifying the correct optimum. The literature has many examples of wrong solutions found by overconfident numerical treatment.

Our goal in this chapter is to give an appreciation of what is involved in numerical optimization and to describe a small number of methods that are generally accepted as preferable within our present context. So many methods and variations have been proposed that describing them all would closely resemble an encyclopedia. Workers in the field tend to have their own preferences.