Optimization Models

A vector is a collection of numbers, arranged in a column or a row, which can be thought of as the coordinates of a point in n-dimensional space. Equipping vectors with sum and scalar multiplication allows us to define notions such as independence, span, subspaces, and dimension. Further, the scalar product introduces a notion of the angle between two vectors, and induces the concept of length, or norm. Via the scalar product, we can also view a vector as a linear function. We can compute the projection of a vector onto a line defined by another vector, onto a plane, or more generally onto a subspace. Projections can be viewed as a first elementary optimization problem (finding the point in a given set at minimum distance from a given point), and they constitute a basic ingredient in many processing and visualization techniques for high-dimensional data.

2.1 Vector basics

2.1.1 Vectors as collections of numbers

Vectors are a way to represent and manipulate a single collection of numbers. A vector x can thus be defined as a collection of elements x1, x2, …, xn, arranged in a column or in a row. We usually write vectors in column format:

Element xi is said to be the i-th component (or the i-th element, or entry) of vector x, and the number n of components is usually referred to as the dimension of x.