In this chapter, we think about problems involving metrics. A metric (V, d) on a set of vertices V gives a distance duv for each pair of vertices u, v ∈ V such that three properties are obeyed: (1) duv = 0 if and only if v = u; (2) duv = duv for all u, v ∈ V; and (3) duv ≤ duw + dwv for all u, v, w ∈ V. The final property is sometimes called the triangle inequality. We will sometimes simply refer to the metric d instead of (V, d) if the set of vertices V is clear from the context. A concept related to a metric is a semimetric (V, d), in which properties (2) and (3) are obeyed, but not necessarily (1), so that if duv = 0, then possibly u ≠ v (a semimetric maintains that duu = 0). We may sometimes ignore this distinction between metrics and semimetrics, and call them both metrics.

Metrics turn out to be a useful way of thinking about graph problems involving cuts. Many important problems in discrete optimization require finding cuts in graphs of various types. To see the connection between cuts and metrics, note that for any cut S ⊆ V, we can define d where duv = 1 if u ∈ S and v ∉ S, and duv = 0 otherwise. Note that (V, d) is then a semimetric; it is sometimes called the cut semimetric associated with S.

We introduced the use of semidefinite programming for approximation algorithms in Chapter 6. The algorithms of that chapter solve a vector programming relaxation, then choose a random hyperplane (or possibly many hyperplanes) to partition the vectors in some way. The central component of the analysis of these algorithms is Lemma 6.7, which says that the probability of two vectors being separated by a random hyperplane is proportional to the angle between them. In this chapter, we look at ways in which we can broaden both the analysis of algorithms using semidefinite programming, and the algorithms themselves.

To broaden our analytical techniques, we revisit two of the problems we discussed initially in Chapter 6. In particular, we consider the problem of approximating integer quadratic programs, which was introduced in Section 6.3, and the problem of coloring a 3-colorable graph, which was introduced in Section 6.5. In our algorithms in this chapter, we again solve vector programming relaxations of the problems, and choose a random hyperplane by drawing its components from the normal distribution. Here, however, our analysis of the algorithms will rely on several more properties of the normal distribution than we used in the previous chapter; in particular, it will be helpful for us to use bounds on the tail of the normal distribution.

In this appendix, we briefly review the concepts of NP-completeness and reductions. We will use the knapsack problem of Section 3.1 as a running example. Recall that in the knapsack problem, we are given a set of n items I = {1, …, n}, where each item i has a value vi and a size si. All sizes and values are positive integers. The knapsack has capacity B, where B is also a positive integer. The goal is to find a subset of items S ⊆ I that maximizes the value ∑i∈Svi of items in the knapsack subject to the constraint that the total size of these items is no more than the capacity; that is, ∑i∈Ssi ≤ B.

Recall the definition of a polynomial-time algorithm.

Definition B.1.An algorithm for a problem is said to run in polynomial time, or said to be a polynomial-time algorithm, with respect to a particular model of computer (such as a RAM) if the number of instructions executed by the algorithm can be bounded by a polynomial in the size of the input.

More formally, let x denote an instance of a given problem; for example, an instance of the knapsack problem is the number n of items, the numbers si and vi giving the sizes and values of the items, and the number B giving the size of the knapsack.

Sometimes it turns out to be useful to allow our algorithms to make random choices; that is, the algorithm can flip a coin, or flip a biased coin, or draw a value uniformly from a given interval. The performance guarantee of an approximation algorithm that makes random choices is then the expected value of the solution produced relative to the value of an optimal solution, where the expectation is taken over the random choices of the algorithm.

At first this might seem like a weaker class of algorithm. In what sense is there a performance guarantee if it holds only in expectation? However, in most cases we will be able to show that randomized approximation algorithms can be derandomized: that is, we can use a certain algorithmic technique known as the method of conditional expectations to produce a deterministic version of the algorithm that has the same performance guarantee as the randomized version. Of what use then is randomization? It turns out that it is often much simpler to state and analyze the randomized version of the algorithm than to state and analyze the deterministic version that results from derandomization. Thus, randomization gains us simplicity in our algorithm design and analysis, while derandomization ensures that the performance guarantee can be obtained deterministically.

For this penultimate chapter, we turn from techniques for designing good approximation algorithms to techniques for proving that problems are hard to approximate within certain factors. Our coverage of this topic will be relatively brief: indeed, another book could be written on this subject alone.

We will look at several ways in which these results are proven. First, we start with reductions from NP-complete problems. We have already seen a few examples of such reductions in this book; for example, in Theorem 2.4, we showed that there is no α-approximation algorithm for the k-center problem for α < 2 unless P = NP, and in Theorem 2.9 we argued that there is no O(2n)-approximation algorithm for the general case of the traveling salesman problem unless P = NP. Second, we will look at reductions that preserve approximation; these are reductions from a problem Π to another problem Π′ such that if there is an approximation algorithm with performance guarantee α for problem Π′, then there is an approximation algorithm with performance guarantee f(α) for problem Π, where f is some function. These results yield hardness theorems via the contrapositive: if there is no f(α)-approximation algorithm for problem Π unless P = NP, then there is no α-approximation algorithm for problem Π′ unless P = NP. Third, we will turn to a definition of NP in terms of probabilistically checkable proofs, or PCPs. These PCPs allow us to prove hardness of approximation results for a number of particular constraint satisfaction problems.

In this chapter, we return to the technique of applying dynamic programming via rounding data. We look at two, more technically difficult, applications of this technique to find polynomial-time approximation schemes for two different problems.

First, we consider the traveling salesman problem, introduced in Section 2.4, for instances in which the cities are points in the Euclidean plane and the cost of traveling between two cities is the Euclidean distance between the corresponding points. In this case the dynamic program works by recursively dividing the plane into squares. Starting with the smallest squares, we compute the least-cost set of paths for visiting all the cities in the squares, then use these to compute solutions for larger squares. We can show that the optimal tour can be modified at low cost such that it doesn't enter and exit any square too many times; this “rounding” of the optimal tour makes it possible to solve the dynamic program in polynomial time. This technique turns out to be widely applicable to problems in the Euclidean plane, including the Steiner tree problem and the k-median problem for Euclidean instances.

Second, we consider the maximum independent set problem in planar graphs. We show that the maximum independent set problem is easy to solve on trees, and can be solved in graphs that are “treelike.” We can measure how close a graph is to being a tree via a parameter called its treewidth, and we give an algorithm to solve the maximum independent set problem in time that is polynomial in the number of vertices and exponential in the treewidth of the input graph.

We have now concluded our initial introduction to the various techniques for designing approximation algorithms. In this second part of the book, we revisit each of these techniques and give additional applications of them. In some cases, these applications are recent or more advanced, but in others they are just a bit more technically involved, or are in some other way “nonintroductory.” Hence, this second part covers “further uses” of each technique, rather than “advanced uses” or “recent uses.”

In this chapter, we look again at greedy and local search algorithms. We revisit the problem of minimizing the maximum degree of a spanning tree, and show that a variant of the local search algorithm described in Section 2.6 in which the local moves are carefully ordered results in a spanning tree whose maximum degree is within 1 of the optimum. When we revisit the technique of deterministic rounding in Chapter 11, we will show a similar result for a version of the problem in which there are costs on the edges.

The bulk of this chapter is spent on greedy and local search algorithms for the uncapacitated facility location problem and the k-median problem. Simple local search algorithms for these problems have been known since the early 1960s. It is only relatively recently, however, that it has been shown that these algorithms produce provably near-optimal solutions. In Section 9.1, we show that a local search algorithm for the uncapacitated facility location problem gives a (3 + ε)-approximation algorithm for that problem.

We have described an iterative method that is versatile in its applicability to showing several results in exact optimization and approximation algorithms. The key step in applying this method uses the elementary Rank Lemma to show sparsity of the support of extreme point solutions for a wide variety of problems. The method follows a natural sequence of formulating a tractable LP relaxation of the problem, examining the structure of tight constraints to demonstrate an upper bound on the rank of the tight subsystem defining the extreme point, using the Rank Lemma to imply an upper bound on the support, and finally using this sparsity of the support to find an element in the support with high (possible fractional) value.

The two key steps in the method are upper bounding the rank of the tight constraints and using the sparsity of the support to imply a high-valued element. Various uncrossing techniques in combinatorial optimization are very useful in the first step. However, new ideas are typically required in the second step, which can be usually carried out with a “token charging” argument to prove by contradiction the presence of a high-value element in the support: Assign a set number, k, of tokens to each support variable (now assumed to be low valued for contradiction) and redistribute these tokens so as to collect k tokens per tight constraints and show some leftover tokens for the contradiction.

In previous chapters, we have used the iterative relaxation method to obtain approximation algorithms for degree bounded network design problems. In this chapter, we illustrate that similar techniques can be applied to other constrained optimization problems. In the first part, we study the partial vertex cover problem and show an iterative 2-approximation algorithm for the problem. In the second part, we study the multicriteria spanning tree problem and present a polynomial time approximation scheme for the problem.

Vertex cover

We first give a simple iterative 2-approximation algorithm for the vertex cover problem, and then show that it can be extended to the partial vertex cover problem.

Given a graph G = (V, E) and a nonnegative cost function c on vertices, the goal in the vertex cover problem is to find a set of vertices with minimum cost that covers every edge (i.e., for every edge at least one endpoint is in the vertex cover). In Chapter 3, we showed that the vertex cover problem in bipartite graphs is polynomial time solvable and gave an iterative algorithm for finding the minimum cost vertex cover. In general graphs, the vertex cover problem is NP-hard. Nemhauser and Trotter [105] gave a 2-approximation for the problem. Indeed, they prove a stronger property of half-integrality of the natural linear programming relaxation. We prove this result and its extensions to the partial vertex cover problem in the next section.

In this chapter, we study the survivable network design problem. Given an undirected graph G = (V, E) and a connectivity requirement ruv for each pair of vertices u, v, a Steiner network is a subgraph of G in which there are at least ruv edge-disjoint paths between u and v for every pair of vertices u, v. The survivable network design problem is to find a Steiner network with minimum total cost. In the first part of this chapter, we will present the 2-approximation algorithm given by Jain [75] for this problem. We will present his original proof, which introduced the iterative rounding method to the design of approximation algorithms.

Interestingly, we will see a close connection of the survivable network design problem to the traveling salesman problem. Indeed the linear program, the characterization results, and presence of edges with large fractional value are identical for both problems. In the (symmetric) TSP, we are given an undirected graph G = (V, E) and cost function c: E → ℝ+, and the task is to find a minimum-cost Hamiltonian cycle. In the second part of this chapter, we will present an alternate proof of Jain's result, which also proves a structural result about extreme point solutions to the traveling salesman problem.

In the final part of this chapter, we consider the minimum bounded degree Steiner network problem, where we are also given a degree upper bound Bv for each vertex v ∈ V, and the task is to find a minimum-cost Steiner network satisfying all the degree bounds.

In this chapter, we present 2-approximation algorithms for three “cut” problems: the triangle cover problem, the feedback vertex set problem on bipartite tournaments, and the node multiway cut problem. All the algorithms are based on iterative rounding but require an additional step: As usual the algorithms will pick variables with large fractional values and compute a new optimal fractional solution iteratively, but unlike previous problems we do not show that an optimal extreme point solution must have a variable with large fractional value. Instead, when every variable in an optimal fractional solution has a small fractional value, we will use the complementary slackness conditions to show that there are some special structures that can be exploited to finish rounding the fractional solution. These algorithms do not use the properties of extreme point solutions, but we will need the complementary slackness conditions stated in Section 2.1.4. The results in this chapter illustrate an interesting variant and the flexibility of the iterative rounding method.

Triangle cover

Given an undirected graph with weights on the edges, the triangle cover problem is to find a subset of edges F with minimum total weight that intersects all the triangles (3-cycles) of the graph (i.e., G – F is triangle-free).

Linear programming relaxation

The following is a simple linear programming formulation for the triangle cover problem, denoted by LPtri(G), in which xe is a variable for edge e and we is the weight of edge e.

In this chapter we consider two very closely related problems, maximum weighted matching and minimum cost vertex cover in bipartite graphs. Linear programming duality plays a crucial role in understanding the relationship between these problems. We will show that the natural linear programming relaxations for both the matching problem and the vertex cover problem are integral, and then use duality to obtain a min–max relation between them. Nevertheless, our proofs of integrality use the iterative method by arguing the existence of 1-elements in an extreme point solution.

In the first section, we show the integrality of the more standard maximization version of the matching problem. In the following sections, we show two applications of the proof technique for integrality to derive approximation results for NP-hard problems. We first present a new proof of an approximation result for the generalized assignment problem and then present an approximation result for the budgeted allocation problem. The proofs of both of these results develop on the integrality result for the bipartite matching problem and introduce the iterative relaxation method. Following this, we discuss the integrality of the bipartite vertex cover problem formulation and conclude with a short section on the duality relation between these problems and some historical notes.

Matchings in bipartite graphs

In this section, we show that the matching polytope in bipartite graphs is integral. Given a bipartite graph G = (V1 ∪ V2, E) and a weight function w: E → ℛ, the maximum matching problem is to find a set of vertex-disjoint edges of maximum total weight.

Audience

As teachers and students of combinatorial optimization, we have often looked for material that illustrates the elegance of classical results on matchings, trees, matroids, and flows, but also highlights methods that have continued application. With the advent of approximation algorithms, some techniques from exact optimization such as the primal-dual method have indeed proven their staying power and versatility. In this book, we describe what we believe is a simple and powerful method that is iterative in essence and useful in a variety of settings.

The core of the iterative methods we describe relies on a fundamental result in linear algebra that the row rank and column rank of a real matrix are equal. This seemingly elementary fact allows us via a counting argument to provide an alternate proof of the previously mentioned classical results; the method is constructive and the resulting algorithms are iterative with the correctness proven by induction. Furthermore, these methods generalize to accommodate a variety of additional constraints on these classical problems that render them NP-hard – a careful adaptation of the iterative method leads to very effective approximation algorithms for these cases.

Our goal in this book has been to highlight the commonality and uses of this method and convince the readers of the generality and potential for future applications. We have used an elementary presentation style that should be accessible to anyone with introductory college mathematics exposure in linear algebra and basic graph theory.

In this chapter we study problems in directed graphs and see how the techniques developed in previous chapters generalize to problems on directed graphs. We first consider exact formulations for the arborescence problem and a vertex connectivity problem in directed graphs. For the latter, we demonstrate the iterative method in the more sophisticated uncrossing context which is applied to biset families instead of set families as in previous chapters. We then extend these results to degree bounded variants of the problems and use the iterative method to obtain bicriteria results unlike previous chapters where the algorithm would be optimal on the cost and only violate the degree constraints.

Given a directed graph D = (V, A) and a root vertex r ∈ V, a spanning r-arborescence is a subgraph of D so that there is a directed path from r to every vertex in V – r. The minimum spanning arborescence problem is to find a spanning r-arborescence with minimum total cost. We will show an integral characterization using iterative proofs, and extend this result in two directions. Given a directed graph D and a root vertex r, a rooted k-connected subgraph is a subgraph of D so that there are k internally vertex-disjoint directed paths from r to every vertex in V – r. The minimum rooted k-connected subgraph problem is to find a rooted k-connected subgraph with minimum total cost. We extend the proofs in the minimum arborescence problem to show an integral characterization in this more general setting.