In this chapter, we (a) introduce the Lagrange function of an inequality-constrained mathematical programming problem, (b) formulate and prove convex programming Lagrange Duality Theorem, and (c) establish the connection between Lagrange duality and saddle points of the Lagrange function.

In this chapter, we prove the Separation Theorem for convex sets and extract from it basic results on the geometry of closed convex sets, specifically (a) supporting hyperplanes, extreme points, and the (finite-dimensional) Krein--Milman Theorem, (b) recessive directions and recessive cone of a convex set, (c) the definition and basic properties of the dual cone, (d) the (finite-dimensional) Dubovitski--Milutin Lemma, (e) existence of bases and extreme rays of nontrivial closed pointed cones and their relation to extreme points of the cone’s base, (f)the (finite-dimensional) Krein--Milman Theorem in conic form, and (g) polarity.

>Mark in the following list the sets which are convex

Mark with Y those of the cases listed below where the linear form f⊤x separates the sets S and T

In this chapter, we demonstrate that (a) substituting the vector of eigenvalues of a symmetric n x n matrix into a convex permutation symmetric function of n real variables results in a convex function of the matrix, and (b) that if g is a convex function on the real axis, and G is the set of symmetric matrices of a given size with spectrum in the domain of g, then G is a convex set, and when X is a matrix from G, the trace of the matrix g(X), is a convex function of X; here g(X) is the matrix acting at a spectral subspace of X associated with eigenvalue v as multiplication by g(v); both these facts will be heavily used when speaking about cone-convexity is chapter 21.

In this chapter, we (a) outline the subject and the terminology of mathematical and convex programming, (b) introduce the Slater and relaxed Slater conditions and formulate the Convex Theorem on the Alternative -- the basis of Lagrange duality theory in convex programming, (c) introduce the notions of cone-convexity and of the convex programming problem in cone-constrained form, thus extending the standard mathematical programming setup of convex optimization, and (d) formulate and prove the Convex Theorem on the Alternative in cone-constrained form, justifying, as a byproduct, the standard Convex Theorem on the Alternative.

In this chapter, we derive the standard first- and second-order necessary/sufficient conditions for local optimality of a feasible solution to a (possibly nonconvex) mathematical programming problem. We conclude the chapter by illustrating these on the S-Lemma.

In this chapter, we (a) discuss the notion of lower semicontinuity of a function and demonstrate that functions with this property have closed epigraphs, (b) show that the pointwise supremum of a family of lower semicontinous functions is lower discontinuous, (c) demonstrate that a proper lower semiconscious convex function is the pointwise supremum of the affine minorants of the function, (d) introduce the notion of a subgradient and the subdifferential of a convex function at a point and demonstrate existence of subgradients at points from the relative interior of the function’s domain, (e) outline elementary rules of subdifferential calculus, and (f) establish basic properties of the directional derivatives of convex functions and the connection between directional derivatives and subdifferentials.

In this chapter, we extract from the results of Chapter 3 the basic theory of finite systems of linear inequalities - Farkas’ Lemmas, General Theorem on the Alternative, certificates for feasibility/infeasibility of polyhedral sets, and linear programming Duality Theorem.

In this chapter, we (a) introduce the notion of Legendre transformation of a proper convex function, (b) establish basic properties of the Legendre transform, in particular, demonstrate that the transform of a proper lower semicontinuous convex function is itself a proper lower semicontinous convex function and that its Legendre transformation is the original function, (c) demonstrate that the set of minimizers of a proper lower semicontinuous convex function is the subdifferential, taken at the origin, of the function’s Legendre transform, and (d) derive the Young, Holder, and moment inequalities and discuss dual (a.k.a. conjugate) norms.