The goal of optimization is simple: identify the best possible outcome. In mathematical terms, the “outcome” is the value of some function, and “best possible” often is the maximum of the function, which is the point with the highest possible value. The function itself is often called objective function and its arguments are called control variables. Parameters of the objective function are sometimes called value coefficients. If the function describes the yield of a batch process, then the maximum may describe the control settings under which the yield is greatest. In a minimization problem, one might look for the cheapest solution. Searching for the minimum might sound like the opposite problem to maximization, but it is accomplished with the same mathematical concepts. In particular, if one multiplies the objective function with –1, the former maximum becomes the minimum and vice versa. Thus, finding a maximum or a minimum is basically the same, and we talk generically about finding an extremum or optimum.

Optimization is a huge field, and general methods and specific techniques have been developed not only in mathematics, but also in engineering, operations research, and many branches of quantitative science. To provide some structured overview, one may classify optimization tasks according to a hierarchy of complexity (Vagners 1983). The simplest tasks require the direct algebraic computation of maxima or minima of functions, primarily with tools of differentiation.