Dynamic programming is a standard technique in algorithm design in which an optimal solution for a problem is built up from optimal solutions for a number of subproblems, normally stored in a table or multidimensional array. Approximation algorithms can be designed using dynamic programming in a variety of ways, many of which involve rounding the input data in some way.

For instance, sometimes weakly NP-hard problems have dynamic programming algorithms that run in time polynomial in the input size if the input is represented in unary rather than in binary (so, for example, the number 7 would be encoded as 1111111). If so, we say that the algorithm is pseudopolynomial. Then by rounding the input values so that the number of distinct values is polynomial in the input size and an error parameter ε > 0, this pseudopolynomial algorithm can be made to run in time polynomial in the size of the original instance. We can often show that the rounding does not sacrifice too much in the quality of the solution produced. We will use this technique in discussing the knapsack problem in Section 3.1.

For other problems, such as scheduling problems, we can often make distinctions between “large” and “small” parts of the input instance; for instance, in scheduling problems, we distinguish between jobs that have large and small processing times. We can then show that by rounding the sizes of the large inputs so that, again, the number of distinct, large input values is polynomial in the input size and an error parameter, we can use dynamic programming to find an optimal solution on just the large inputs.

In the introduction, we said that one of the principal theses of this book is the central role played by linear programming in the design and analysis of approximation algorithms. In the previous two chapters, we have not used linear programming at all, but starting with this chapter we will be using it extensively.

In this chapter, we will look at one of the most straightforward uses of linear programming. Given an integer programming formulation of a problem, we can relax it to a linear program. We solve the linear program to obtain a fractional solution, then round it to an integer solution via some process.

The easiest way to round the fractional solution to an integer solution in which all values are 0 or 1 is to take variables with relatively large values and round them up to 1, while rounding all other variables down to 0. We saw this technique in Section 1.3 applied to the set cover problem, in which we chose sets whose corresponding linear programming variables were sufficiently large. We will see another application of this technique when we introduce the prize-collecting Steiner tree problem in Section 4.4. We will revisit this problem several times in the course of the book. For this problem we give an integer programming relaxation in which there are 0-1 variables for both nodes and edges. We round up the node variables that are sufficiently large in order to decide which nodes should be spanned in a solution; we then find a tree spanning these nodes.

In Section 8.5, we introduced the idea of approximating one kind of metric with another one; namely, we looked at the idea of approximating a general metric with a tree metric. Here we will consider approximating a general metric with another metric more general than a tree metric, namely, an ℓ1-embeddable metric. We show that we can approximate any metric (V, d) with an ℓ1-embeddable metric with distortion O(log n), where n = |V|. The ℓ1-embeddable metrics have a particularly close connection to cuts; we show that any such metric is a convex combination of the cut semimetrics we discussed at the beginning of Chapter 8. We show that the low-distortion embeddings into ℓ1-embeddable metrics have applications to cut problems by giving an approximation algorithm for the sparsest cut problem.

In Section 15.2, we give an algorithm that finds a packing of trees called cut-trees into a graph; this packing allows us to solve a particular routing problem. In the subsequent section, we show that the cut-tree packing can be used in a way analogous to the probabilistic approximation of metrics by tree metrics in Section 8.5. In that section, we showed that given an algorithm to solve a problem on a tree metric, we could provide an approximate solution for the problem in a general metric with only an additional factor of O(log n) in the performance guarantee.

The Whats and Whys of Approximation Algorithms

Decisions, decisions. The difficulty of sifting through large amounts of data in order to make an informed choice is ubiquitous in today's society. One of the promises of the information technology era is that many decisions can now be made rapidly by computers, from deciding inventory levels, to routing vehicles, to organizing data for efficient retrieval. The study of how to make decisions of these sorts in order to achieve some best possible goal, or objective, has created the field of discrete optimization.

Unfortunately, most interesting discrete optimization problems are NP-hard. Thus, unless P = NP, there are no efficient algorithms to find optimal solutions to such problems, where we follow the convention that an efficient algorithm is one that runs in time bounded by a polynomial in its input size. This book concerns itself with the answer to the question “What should we do in this case?”

An old engineering slogan says, “Fast. Cheap. Reliable. Choose two.” Similarly, if P ≠ NP, we can't simultaneously have algorithms that (1) find optimal solutions (2) in polynomial time (3) for any instance. At least one of these requirements must be relaxed in any approach to dealing with an NP-hard optimization problem.

The design of approximation algorithms has reached a period of relative maturity as a discipline. We hope that the wealth of results presented herein makes a strong case that this is so. However, we also believe that there is much work remaining, with many more fundamental contributions yet to be discovered.

We will outline a few questions to highlight some of the research that we speculate might have the potential to surprise us with new directions for this area. Since “top 10” lists are the norm not just for year-end film critics and late-night show hosts, we will structure these thoughts in that format.

For many optimization problems, the ultimate result is a performance guarantee with a matching lower bound (based on a complexity-theoretic assumption, at least until the time that questions such as P vs. NP are resolved). For a significant fraction of this book, we have been concerned with designing α-approximation algorithms for some constant α – polynomial-time algorithms that find solutions of objective function value within a factor of α of optimal. Implicitly, we were seeking the best value α that is achievable for the given problem at hand.

One of the significant developments of the past decade is the introduction of the unique games conjecture, as discussed in Section 16.5. This conjecture provides a stronger complexity-theoretic hypothesis on which to base lower bounds for performance guarantees, and recent work has shown that tight bounds on performance guarantees follow for a wide swath of optimization problems.

In this chapter, we give two somewhat more sophisticated applications of the standard primal-dual algorithm and analysis introduced in Chapter 7. We revisit the prize-collecting Steiner tree problem discussed in Sections 4.4 and 5.7, and give a primal-dual 2-approximation algorithm for it. We also revisit the feedback vertex set problem in undirected graphs introduced in Section 7.2, and give a primal-dual 2-approximation algorithm for this problem as well. In this case, we must give an alternative integer programming formulation for the problem since the previous formulation has an integrality gap that implies that the O(log n)-approximation algorithm of Section 7.2 is the best possible (within constant factors) using the formulation of that section.

The Prize-Collecting Steiner Tree Problem

In this section, we revisit the prize-collecting Steiner tree problem introduced in Section 4.4 and further discussed in Section 5.7. We show that a primal-dual algorithm can be used to give a 2-approximation algorithm for the problem; this improves on the performance guarantee of the previously given algorithms, and the primal-dual algorithm does not require solving a linear programming relaxation. Recall that in this problem we are given an undirected graph G = (V, E), edge costs ce ≥ 0 for all e ∈ E, a selected root vertex r ∈ V, and penalties πi ≥ 0 for all i ∈ V. The goal is to find a tree T that contains the root vertex r so as to minimize the cost of the edges in T plus the penalties of the vertices not in T; that is, ∑e∈Tce + ∑i∈V – V(T) πi, where V(T) is the set of vertices in the tree.

We now turn to a new tool that gives substantially improved performance guarantees for some problems. So far we have used linear programming relaxations to design and analyze various approximation algorithms. In this section, we show how nonlinear programming relaxations can give us better algorithms than we know how to obtain via linear programming; in particular we use a type of nonlinear program called a semidefinite program. Part of the power of semidefinite programming is that semidefinite programs can be solved in polynomial time.

We begin with a brief overview of semidefinite programming. Throughout the chapter we assume some basic knowledge of vectors and linear algebra; see the notes at the end of the chapter for suggested references on these topics. We then give an application of semidefinite programming to approximating the maximum cut problem. The algorithm for this problem introduces a technique of rounding the semidefinite program by choosing a random hyperplane. We then explore other problems for which choosing a random hyperplane, or multiple random hyperplanes, is useful, including approximating quadratic programs, approximating clustering problems, and coloring 3-colorable graphs.

A Brief Introduction to Semidefinite Programming

Semidefinite programming uses symmetric, positive semidefinite matrices, so we briefly review a few properties of these matrices. In what follows, XT< is the transpose of the matrix X, and vectors v ∈ ℛn are assumed to be column vectors, so that vTv is the inner product of v with itself, while vvT is an n by n matrix.

This book is designed to be a textbook for graduate-level courses in approximation algorithms. After some experience teaching minicourses in the area in the mid-1990s, we sat down and wrote out an outline of the book. Then one of us (DPW), who was at the time an IBM Research Staff Member, taught several iterations of the course following the outline we had devised, in Columbia University's Department of Industrial Engineering and Operations Research in Spring 1998, in Cornell University's School of Operations Research and Industrial Engineering in Fall 1998, and at the Massachusetts Institute of Technology's Laboratory for Computer Science in Spring 2000. The lecture notes from these courses were made available, and we got enough positive feedback on them from students and from professors teaching such courses elsewhere that we felt we were on the right track. Since then, there have been many exciting developments in the area, and we have added many of them to the book; we taught additional iterations of the course at Cornell in Fall 2006 and Fall 2009 in order to field test some of the writing of the newer results.

The courses were developed for students who have already had a class, undergraduate or graduate, in algorithms, and who were comfortable with the idea of mathematical proofs about the correctness of algorithms. The book assumes this level of preparation. The book also assumes some basic knowledge of probability theory (for instance, how to compute the expected value of a discrete random variable).

We introduced the primal-dual method in Section 1.5, and showed how it gave an approximation algorithm for the set cover problem. Although there it did not give a better performance guarantee than various LP rounding algorithms, we observed that in practice the primal-dual method gives much faster algorithms than those that require solving a linear program.

In this chapter, we will cover primal-dual algorithms in more depth. We begin by reviewing the primal-dual algorithm for the set cover problem from Section 1.5. We then apply the primal-dual method to a number of problems, gradually developing a number of principles in deciding how to apply the technique so as to get good performance guarantees. In discussing the feedback vertex set problem in Section 7.2, we see that it is sometimes useful to focus on increasing particular dual variables that correspond to small or minimal constraints not satisfied by the current primal solution. In Section 7.3, we discuss the shortest s-t path problem, and see that to obtain a good performance guarantee it is sometimes necessary to remove unneeded elements in the primal solution returned by the algorithm. In Section 7.4 we introduce the generalized Steiner tree problem (also known as the Steiner forest problem), and show that to obtain a good performance guarantee it can be helpful to increase multiple dual variables at the same time. In Section 7.5, we see that it can be useful to consider alternative integer programming formulations in order to obtain improved performance guarantees.

In this chapter, we will consider two standard and related techniques for designing algorithms and heuristics, namely, greedy algorithms and local search algorithms. Both algorithms work by making a sequence of decisions that optimize some local choice, though these local choices might not lead to the best overall solution.

In a greedy algorithm, a solution is constructed step by step, and at each step of the algorithm the next part of the solution is constructed by making some decision that is locally the best possible. In Section 1.6, we gave an example of a greedy algorithm for the set cover problem that constructs a set cover by repeatedly choosing the set that minimizes the ratio of its weight to the number of currently uncovered elements it contains.

A local search algorithm starts with an arbitrary feasible solution to the problem, and then checks if some small, local change to the solution results in an improved objective function. If so, the change is made. When no further change can be made, we have a locally optimal solution, and it is sometimes possible to prove that such locally optimal solutions have value close to that of the optimal solution. Unlike other approximation algorithm design techniques, the most straightforward implementation of a local search algorithm typically does not run in polynomial time. The algorithm usually requires some restriction to the local changes allowed in order to ensure that enough progress is made during each improving step so that a locally optimal solution is found in polynomial time.