Desire to improve drives many human activities. Optimization can be seen as a means for identifying better solutions by utilizing a scientific and mathematical approach. In addition to its widespread applications, optimization is an amazing subject with very strong connections to many other subjects and deep interactions with many aspects of computation and theory. The main goal of this textbook is to provide an attractive, modern, and accessible route to learning the fundamental ideas in optimization for a large group of students with varying backgrounds and abilities. The only background required for the textbook is a first-year linear algebra course (some readers may even be ready immediately after finishing high school). However, a course based on this book can serve as a header course for all optimization courses. As a result, an important goal is to ensure that the students who successfully complete the course are able to proceed to more advanced optimization courses.

Another goal of ours was to create a textbook that could be used by a large group of instructors, possibly under many different circumstances. To a degree, we tested this over a four-year period. Including the three of us, 12 instructors used the drafts of the book for two different courses. Students in various programs (majors), including accounting, business, software engineering, statistics, actuarial science, operations research, applied mathematics, pure mathematics, computational mathematics, computer science, combinatorics and optimization, have taken these courses. We believe that the book will be suitable for a wide range of students (mathematics, mathematical sciences including computer science, engineering including software engineering, and economics).

In Chapter 2, we have seen how to solve LPs using the simplex algorithm, an algorithm that is still widely used in practice. In Chapter 3, we discussed efficient algorithms to solve the special class of IPs describing the shortest path problem and the minimum cost perfect matching problem in bipartite graphs. In both these examples, it is sufficient to solve the LP relaxation of the problem.

Integer programming is widely believed to be a difficult problem (see Appendix A). Nonetheless, we will present algorithms that are guaranteed to solve IPs in finite time. The drawback of these algorithms is that the running time may be exponential in the worst case. However, they can be quite fast for many instances, and are capable of solving many large-scale, real-life problems.

These algorithms follow two general strategies. The first attempts to reduce IPs to LPs – this is known as the cutting plane approach and will be described in Section 6.2. The other strategy is a divide and conquer approach and is known as branch and bound and will be discussed in Section 6.3. In practice, both strategies are combined under the heading of branch and cut. This remains the preferred approach for all general purpose commercial codes.

In this chapter, in the interest of simplicity we will restrict our attention to pure IPs where all the variables are required to be integer. The theory developed here extends to mixed IPs where only some of the variables are required to be integer, but the material is beyond the scope of this book.

Possible outcomes

Consider an LP (P) with variables x1, …, xn. Recall that an assignment of values to each of x1, …, xn is a feasible solution if the constraints of (P) are satisfied. We can view a feasible solution to (P) as a vector x = (x1, …, xn)T. Given a vector x, by the value of x we mean the value of the objective function of (P) for x. Suppose (P) is a maximization problem. Then recall that we call a vector x an optimal solution if it is a feasible solution and no feasible solution has larger value. The value of the optimal solution is the optimal value. By definition, an LP has only one optimal value; however, it may have many optimal solutions. When solving an LP, we will be satisfied with finding any optimal solution. Suppose (P) is a minimization problem. Then a vector x is an optimal solution if it is a feasible solution and no feasible solution has smaller value.

If an LP (P) has a feasible solution, then it is said to be feasible, otherwise it is infeasible. Suppose (P) is a maximization problem and for every real number α there is a feasible solution to (P) which has value greater than α, then we say that (P) is unbounded. In other words, (P) is unbounded if we can find feasible solutions of arbitrarily high value. Suppose (P) is a minimization problem and for every real number α there is a feasible solution to (P) which has value smaller than α, then we say that (P) is unbounded.

In this chapter, we revisit the shortest path and minimum-cost matching problems. Both were first introduced in Chapter 1, where we discussed practical example applications. We further showed that these problems can be expressed as IPs. The focus in this chapter will be on solving instances of the shortest path and matching problems. Our starting point will be to use the IP formulation we introduced in Section 1.5. We will show that studying the two problems through the lens of linear programming duality will allow us to design efficient algorithms. We develop this theory further in Chapter 4.

The shortest path problem

Recall the shortest path problem from Section 1.4.1. We are given a graph G = (V, E), nonnegative lengths ce for all edges e ∈ E, and two distinct vertices s, t ∈ V. The length c(P) of a path P is the sum of the length of its edges, i.e. Σ(ce: e ∈ P). We wish to find among all possible st-paths one that is of minimum length.

Example 7 In the following figure, we show an instance of this problem. Each of the edges in the graph is labeled by its length. The thick black edges in the graph form an st-path P = sa, ac, cb, bt of total length 3 + 1 + 2 + 1 = 7. This st-path is of minimum length, hence is a solution to our problem.

An algorithm is a formal procedure that describes how to solve a problem. For instance, the simplex algorithm in Chapter 2 takes as input a linear program in standard equality form and either returns an optimal solution, or detects that the linear program is infeasible or unbounded. Another example is the shortest path algorithm in Chapter 3.1. It takes as input a graph with distinct vertices s, t and nonnegative integer edge lengths, and returns an st-path of shortest length (if one exists).

The two basic properties we require for an algorithm are: correctness and termination. By correctness, we mean that the algorithm is always accurate when it claims that we have a particular outcome. One way to ensure this is to require that the algorithm provides a certificate, i.e. a proof, to justify its answers. By termination, we mean that the algorithm will stop after a finite number of steps.

In Section A.1, we will define the running time of an algorithm; we will formalize the notions of slow and fast algorithms. Section A.2 reviews the algorithms presented in this book and discusses which ones are fast and which ones are slow. In Sections A.3 and A.4 we discuss the inherent complexity of various classes of optimization problems and discuss the possible existence of classes of problems for which it is unlikely that any fast algorithm exists. We explain how an understanding of computational complexity can guide us in the design of algorithms.

Broadly speaking, optimization is the problem of minimizing or maximizing a function subject to a number of constraints. Optimization problems are ubiquitous. Every chief executive officer (CEO) is faced with the problem of maximizing profit given limited resources. In general, this is too general a problem to be solved exactly; however, many aspects of decision making can be successfully tackled using optimization techniques. This includes, for instance, production, inventory, and machine-scheduling problems. Indeed, the overwhelming majority of Fortune 500 companies make use of optimization techniques. However, optimization problems are not limited to the corporate world. Every time you use your GPS, it solves an optimization problem, namely how to minimize the travel time between two different locations. Your hometown may wish to minimize the number of trucks it requires to pick up garbage by finding the most efficient route for each truck. City planners may need to decide where to build new fire stations in order to efficiently serve their citizens. Other examples include: how to construct a portfolio that maximizes its expected return while limiting volatility; how to build a resilient tele-communication network as cheaply as possible; how to schedule flights in a cost-effective way while meeting the demand for passengers; or how to schedule final exams using as few classrooms as possible.

Suppose that you are a consultant hired by the CEO of the WaterTech company to solve an optimization problem.

Using a single symmetric relay feedback test, a method is proposed to identify all the three parameters of a first order plus time delay (FOPTD) model. On identifying a higher order dynamics system by an FOPTD model, the conventional method identifies a negative time constant (Li et al., 1991) due to the error in neglecting higher order dynamics in the system output. In the present work, all the parameters of an FOPTD model are estimated with adequate accuracy. Four simulation examples are given. The estimated model parameters of an FOPTD model are compared with those obtained by Li et al. (1991) and also those with the exact model parameters of the system. The performance of the controller designed on the identified model is compared with that identified by Li et al. (1991) and with that of the actual process. The method gives results close to that of the actual system. Simulation results for stable and unstable systems are given.

Introduction

Identification of transfer function models from experimental data is essential for model based controller design. Often derivation of a rigorous mathematical model is difficult due to the complex nature of chemical processes. Hence, system identification is a valuable tool to identify low order models, based on the input-output data. The relay feedback is a single-shot experiment and the magnitude of oscillations can be varied. From the principal harmonics approximation, the ultimate gain (Ku) and ultimate frequency (ωu) are found.

Yu (1999) introduced the saturation relay feedback test for determining the ultimate gain and ultimate frequency of the system that are required for the design of PI/PID controllers. In the present chapter, two modifications are proposed to improve the efficiency of the saturation relay feedback test, especially for systems with a large dead time. In the first method, the safety factor is corrected based on the ideal relay feedback test. In the second method, higher order harmonics are included in the analysis of ideal relay. Simulation results are given to show the improvement of the proposed method over that of the Yu (1999) method for an FOPTD system and for a higher order system. The method is also compared with that proposed by Luyben (2001) for identifying the FOPTD model parameters.

Introduction

Li et al. (1991) pointed out that the error in estimation of ultimate gain ranges from -18% to +27% (with respect to the theoretical value) in ideal relay feedback test. To reduce the error in the estimation of ultimate gain, Yu (1999) suggested the saturation relay feedback test. In this method, the ultimate gain estimated from the ideal relay feedback test is multiplied with a safety factor to get slope of the saturation relay and the saturation relay feedback test is conducted to get better results for the ultimate gain and frequency.

The method proposed by Yu (1999) gave accurate results for small D/τ ratio (generally D/τ < 2).