For this penultimate chapter, we turn from techniques for designing good approximation algorithms to techniques for proving that problems are hard to approximate within certain factors. Our coverage of this topic will be relatively brief: indeed, another book could be written on this subject alone.

We will look at several ways in which these results are proven. First, we start with reductions from NP-complete problems. We have already seen a few examples of such reductions in this book; for example, in Theorem 2.4, we showed that there is no α-approximation algorithm for the k-center problem for α < 2 unless P = NP, and in Theorem 2.9 we argued that there is no O(2n)-approximation algorithm for the general case of the traveling salesman problem unless P = NP. Second, we will look at reductions that preserve approximation; these are reductions from a problem Π to another problem Π′ such that if there is an approximation algorithm with performance guarantee α for problem Π′, then there is an approximation algorithm with performance guarantee f(α) for problem Π, where f is some function. These results yield hardness theorems via the contrapositive: if there is no f(α)-approximation algorithm for problem Π unless P = NP, then there is no α-approximation algorithm for problem Π′ unless P = NP. Third, we will turn to a definition of NP in terms of probabilistically checkable proofs, or PCPs. These PCPs allow us to prove hardness of approximation results for a number of particular constraint satisfaction problems.

In this chapter, we return to the technique of applying dynamic programming via rounding data. We look at two, more technically difficult, applications of this technique to find polynomial-time approximation schemes for two different problems.

First, we consider the traveling salesman problem, introduced in Section 2.4, for instances in which the cities are points in the Euclidean plane and the cost of traveling between two cities is the Euclidean distance between the corresponding points. In this case the dynamic program works by recursively dividing the plane into squares. Starting with the smallest squares, we compute the least-cost set of paths for visiting all the cities in the squares, then use these to compute solutions for larger squares. We can show that the optimal tour can be modified at low cost such that it doesn't enter and exit any square too many times; this “rounding” of the optimal tour makes it possible to solve the dynamic program in polynomial time. This technique turns out to be widely applicable to problems in the Euclidean plane, including the Steiner tree problem and the k-median problem for Euclidean instances.

Second, we consider the maximum independent set problem in planar graphs. We show that the maximum independent set problem is easy to solve on trees, and can be solved in graphs that are “treelike.” We can measure how close a graph is to being a tree via a parameter called its treewidth, and we give an algorithm to solve the maximum independent set problem in time that is polynomial in the number of vertices and exponential in the treewidth of the input graph.

We have now concluded our initial introduction to the various techniques for designing approximation algorithms. In this second part of the book, we revisit each of these techniques and give additional applications of them. In some cases, these applications are recent or more advanced, but in others they are just a bit more technically involved, or are in some other way “nonintroductory.” Hence, this second part covers “further uses” of each technique, rather than “advanced uses” or “recent uses.”

In this chapter, we look again at greedy and local search algorithms. We revisit the problem of minimizing the maximum degree of a spanning tree, and show that a variant of the local search algorithm described in Section 2.6 in which the local moves are carefully ordered results in a spanning tree whose maximum degree is within 1 of the optimum. When we revisit the technique of deterministic rounding in Chapter 11, we will show a similar result for a version of the problem in which there are costs on the edges.

The bulk of this chapter is spent on greedy and local search algorithms for the uncapacitated facility location problem and the k-median problem. Simple local search algorithms for these problems have been known since the early 1960s. It is only relatively recently, however, that it has been shown that these algorithms produce provably near-optimal solutions. In Section 9.1, we show that a local search algorithm for the uncapacitated facility location problem gives a (3 + ε)-approximation algorithm for that problem.

We have described an iterative method that is versatile in its applicability to showing several results in exact optimization and approximation algorithms. The key step in applying this method uses the elementary Rank Lemma to show sparsity of the support of extreme point solutions for a wide variety of problems. The method follows a natural sequence of formulating a tractable LP relaxation of the problem, examining the structure of tight constraints to demonstrate an upper bound on the rank of the tight subsystem defining the extreme point, using the Rank Lemma to imply an upper bound on the support, and finally using this sparsity of the support to find an element in the support with high (possible fractional) value.

The two key steps in the method are upper bounding the rank of the tight constraints and using the sparsity of the support to imply a high-valued element. Various uncrossing techniques in combinatorial optimization are very useful in the first step. However, new ideas are typically required in the second step, which can be usually carried out with a “token charging” argument to prove by contradiction the presence of a high-value element in the support: Assign a set number, k, of tokens to each support variable (now assumed to be low valued for contradiction) and redistribute these tokens so as to collect k tokens per tight constraints and show some leftover tokens for the contradiction.

In previous chapters, we have used the iterative relaxation method to obtain approximation algorithms for degree bounded network design problems. In this chapter, we illustrate that similar techniques can be applied to other constrained optimization problems. In the first part, we study the partial vertex cover problem and show an iterative 2-approximation algorithm for the problem. In the second part, we study the multicriteria spanning tree problem and present a polynomial time approximation scheme for the problem.

Vertex cover

We first give a simple iterative 2-approximation algorithm for the vertex cover problem, and then show that it can be extended to the partial vertex cover problem.

Given a graph G = (V, E) and a nonnegative cost function c on vertices, the goal in the vertex cover problem is to find a set of vertices with minimum cost that covers every edge (i.e., for every edge at least one endpoint is in the vertex cover). In Chapter 3, we showed that the vertex cover problem in bipartite graphs is polynomial time solvable and gave an iterative algorithm for finding the minimum cost vertex cover. In general graphs, the vertex cover problem is NP-hard. Nemhauser and Trotter [105] gave a 2-approximation for the problem. Indeed, they prove a stronger property of half-integrality of the natural linear programming relaxation. We prove this result and its extensions to the partial vertex cover problem in the next section.

In this chapter, we study the survivable network design problem. Given an undirected graph G = (V, E) and a connectivity requirement ruv for each pair of vertices u, v, a Steiner network is a subgraph of G in which there are at least ruv edge-disjoint paths between u and v for every pair of vertices u, v. The survivable network design problem is to find a Steiner network with minimum total cost. In the first part of this chapter, we will present the 2-approximation algorithm given by Jain [75] for this problem. We will present his original proof, which introduced the iterative rounding method to the design of approximation algorithms.

Interestingly, we will see a close connection of the survivable network design problem to the traveling salesman problem. Indeed the linear program, the characterization results, and presence of edges with large fractional value are identical for both problems. In the (symmetric) TSP, we are given an undirected graph G = (V, E) and cost function c: E → ℝ+, and the task is to find a minimum-cost Hamiltonian cycle. In the second part of this chapter, we will present an alternate proof of Jain's result, which also proves a structural result about extreme point solutions to the traveling salesman problem.

In the final part of this chapter, we consider the minimum bounded degree Steiner network problem, where we are also given a degree upper bound Bv for each vertex v ∈ V, and the task is to find a minimum-cost Steiner network satisfying all the degree bounds.

In this chapter, we present 2-approximation algorithms for three “cut” problems: the triangle cover problem, the feedback vertex set problem on bipartite tournaments, and the node multiway cut problem. All the algorithms are based on iterative rounding but require an additional step: As usual the algorithms will pick variables with large fractional values and compute a new optimal fractional solution iteratively, but unlike previous problems we do not show that an optimal extreme point solution must have a variable with large fractional value. Instead, when every variable in an optimal fractional solution has a small fractional value, we will use the complementary slackness conditions to show that there are some special structures that can be exploited to finish rounding the fractional solution. These algorithms do not use the properties of extreme point solutions, but we will need the complementary slackness conditions stated in Section 2.1.4. The results in this chapter illustrate an interesting variant and the flexibility of the iterative rounding method.

Triangle cover

Given an undirected graph with weights on the edges, the triangle cover problem is to find a subset of edges F with minimum total weight that intersects all the triangles (3-cycles) of the graph (i.e., G – F is triangle-free).

Linear programming relaxation

The following is a simple linear programming formulation for the triangle cover problem, denoted by LPtri(G), in which xe is a variable for edge e and we is the weight of edge e.

In this chapter we consider two very closely related problems, maximum weighted matching and minimum cost vertex cover in bipartite graphs. Linear programming duality plays a crucial role in understanding the relationship between these problems. We will show that the natural linear programming relaxations for both the matching problem and the vertex cover problem are integral, and then use duality to obtain a min–max relation between them. Nevertheless, our proofs of integrality use the iterative method by arguing the existence of 1-elements in an extreme point solution.

In the first section, we show the integrality of the more standard maximization version of the matching problem. In the following sections, we show two applications of the proof technique for integrality to derive approximation results for NP-hard problems. We first present a new proof of an approximation result for the generalized assignment problem and then present an approximation result for the budgeted allocation problem. The proofs of both of these results develop on the integrality result for the bipartite matching problem and introduce the iterative relaxation method. Following this, we discuss the integrality of the bipartite vertex cover problem formulation and conclude with a short section on the duality relation between these problems and some historical notes.

Matchings in bipartite graphs

In this section, we show that the matching polytope in bipartite graphs is integral. Given a bipartite graph G = (V1 ∪ V2, E) and a weight function w: E → ℛ, the maximum matching problem is to find a set of vertex-disjoint edges of maximum total weight.

Audience

As teachers and students of combinatorial optimization, we have often looked for material that illustrates the elegance of classical results on matchings, trees, matroids, and flows, but also highlights methods that have continued application. With the advent of approximation algorithms, some techniques from exact optimization such as the primal-dual method have indeed proven their staying power and versatility. In this book, we describe what we believe is a simple and powerful method that is iterative in essence and useful in a variety of settings.

The core of the iterative methods we describe relies on a fundamental result in linear algebra that the row rank and column rank of a real matrix are equal. This seemingly elementary fact allows us via a counting argument to provide an alternate proof of the previously mentioned classical results; the method is constructive and the resulting algorithms are iterative with the correctness proven by induction. Furthermore, these methods generalize to accommodate a variety of additional constraints on these classical problems that render them NP-hard – a careful adaptation of the iterative method leads to very effective approximation algorithms for these cases.

Our goal in this book has been to highlight the commonality and uses of this method and convince the readers of the generality and potential for future applications. We have used an elementary presentation style that should be accessible to anyone with introductory college mathematics exposure in linear algebra and basic graph theory.

In this chapter we study problems in directed graphs and see how the techniques developed in previous chapters generalize to problems on directed graphs. We first consider exact formulations for the arborescence problem and a vertex connectivity problem in directed graphs. For the latter, we demonstrate the iterative method in the more sophisticated uncrossing context which is applied to biset families instead of set families as in previous chapters. We then extend these results to degree bounded variants of the problems and use the iterative method to obtain bicriteria results unlike previous chapters where the algorithm would be optimal on the cost and only violate the degree constraints.

Given a directed graph D = (V, A) and a root vertex r ∈ V, a spanning r-arborescence is a subgraph of D so that there is a directed path from r to every vertex in V – r. The minimum spanning arborescence problem is to find a spanning r-arborescence with minimum total cost. We will show an integral characterization using iterative proofs, and extend this result in two directions. Given a directed graph D and a root vertex r, a rooted k-connected subgraph is a subgraph of D so that there are k internally vertex-disjoint directed paths from r to every vertex in V – r. The minimum rooted k-connected subgraph problem is to find a rooted k-connected subgraph with minimum total cost. We extend the proofs in the minimum arborescence problem to show an integral characterization in this more general setting.

Given a weighted undirected graph, the maximum matching problem is to find a matching with maximum total weight. In his seminal paper, Edmonds [35] described an integral polytope for the matching problem, and the famous Blossom Algorithm for solving the problem in polynomial time.

In this chapter, we will show the integrality of the formulation given by Edmonds [35] using the iterative method. The argument will involve applying uncrossing in an involved manner and hence we provide a detailed proof. Then, using the local ratio method, we will show how to extend the iterative method to obtain approximation algorithms for the hypergraph matching problem, a generalization of the matching problem to hypergraphs.

Graph matching

Matchings in bipartite graphs are considerably simpler than matchings in general graphs; indeed, the linear programming relaxation considered in Chapter 3 for the bipartite matching problem is not integral when applied to general graphs. See Figure 9.1 for a simple example.

Linear programming relaxation

Given an undirected graph G = (V, E) with a weight function w: E → ℛ on the edges, the linear programming relaxation for the maximum matching problem due to Edmonds is given by the following LPM(G). Recall that E(S) denotes the set of edges with both endpoints in S ⊆ V and x(F) is a shorthand for ∑e∈Fxe for F ⊆ E.

Although there are exponentially many inequalities in LPM(G), there is an efficient separation oracle for this linear program, obtained by Padberg and Rao using Gomory-Hu trees.

Even though we mentioned the paper by Jain [75] as the first explicit application of the iterative method to approximation algorithms, several earlier results can be reinterpreted in this light, which is what we set out to do in this chapter. We will first present a result by Beck and Fiala [12] on hypergraph discrepancy, whose proof is closest to other proofs in this book. Then we will present a result by Steinitz [127] on rearrangements of sums in a geometric setting, which is the earliest application that we know of. Then we will present an approximation algorithm by Skutella [123] for the single source unsplittable flow problem. Then we present the additive approximation algorithm for the bin packing problem by Karmarkar and Karp [77], which is still one of the most sophisticated uses of the iterative relaxation method. Finally, we sketch a recent application of the iterative method augmented with randomized rounding to the undirected Steiner tree problem [20] following the simplification due to Chakrabarty et al. [24].

A discrepancy theorem

In this section, we present the Beck–Fiala theorem from discrepancy theory using an iterative method. Given a hypergraph G = (V, E), a 2-coloring of the hypergraph is defined as an assignment ψ: V → {-1, +1} on the vertices. The discrepancy of a hyperedge e is defined as discχ(e) = ∑v∈e ψ(v), and the discrepancy of the hypergraph G is defined as discψ(G) = maxe∈E(G) |{discψ(e)}|.

In this first chapter we motivate our method via the assignment problem. Through this problem, we highlight the basic ingredients and ideas of the method. We then give an outline of how a typical chapter in the rest of the book is structured, and how the remaining chapters are organized.

The assignment problem

Consider the classical assignment problem: Given a bipartite graph G = (V1 ∪ V2, E) with |V1| = |V2| and weight function w: E → ℝ+, the objective is to match every vertex in V1 with a distinct vertex in V2 to minimize the total weight (cost) of the matching. This is also called the minimum weight bipartite perfect matching problem in the literature and is a fundamental problem in combinatorial optimization. See Figure 1.1 for an example of a perfect matching in a bipartite graph.

One approach to the assignment problem is to model it as a linear programming problem. A linear program is a mathematical formulation of the problem with a system of linear constraints that can contain both equalities and inequalities, and also a linear objective function that is to be maximized or minimized. In the assignment problem, we associate a variable xuv for every {u, v} ∈ E. Ideally, we would like the variables to take one of two values, zero or one (hence in the ideal case, they are binary variables). When xuv is set to one, we intend the model to signal that this pair is matched; when xuv is set to zero, we intend the model to signal that this pair is not matched.

In this chapter, we will study the spanning tree problem in undirected graphs. First, we will study an exact linear programming formulation and show its integrality using the iterative method. To do this, we will introduce the uncrossing method, which is a very powerful technique in combinatorial optimization. The uncrossing method will play a crucial role in the proof and will occur at numerous places in later chapters. We will show two different iterative algorithms for the spanning tree problem, each using a different choice of 1-elements to pick in the solution. For the second iterative algorithm, we show three different correctness proofs for the existence of a 1-element in an extreme point solution: a global counting argument, a local integral token counting argument and a local fractional token counting argument. These token counting arguments will be used in many proofs in later chapters.

We then address the degree-bounded minimum-cost spanning tree problem. We show how the methods developed for the exact characterization of the spanning tree polyhedron are useful in designing approximation algorithms for this NP-hard problem. We give two additive approximation algorithm: The first follows the first approach for spanning trees and naturally generalizes to give a simple proof of the additive two approximation result of Goemans [59]; the second follows the second approach for spanning trees and uses the local fractional token counting argument to provide a very simple proof of the additive one approximation result of Singh and Lau [125].

In this chapter, we consider a simple model, based on a directed tree representation of the variables and constraints, called network matrices. We show how this model as well as its dual have integral optima when used as constraint matrices with integral right-hand sides. Finally, we show the applications of these models, especially in proving the integrality of the dual of the matroid intersection problem in Chapter 5, as well as the dual of the submodular flow problem in Chapter 7.

While our treatment of network matrices is based on its relations to uncrossed structures and their representations, they play a crucial role in the characterization of totally unimodular matrices, which are all constraint matrices that yield integral polytopes when used as constraint matrices with integral right-hand sides [121]. Note that total unimodularity of network matrices automatically implies integrality of the dual program when the right-hand sides of the dual are integral.

The integrality of the dual of the matroid intersection and submodular flow polyhedra can be alternately derived by showing the Total Dual Integrality of these systems [121]. Although our proof of these facts uses iterative rounding directly on the dual, there is a close connection between these two lines of proof since both use the underlying structure on span of the constraints defining the extreme points of the corresponding linear program.

Quoting Lovász from his paper “Submodular Functions and Convexity” [94]:

The submodular flow model is an excellent example to illustrate this point. In this chapter, we introduce the submodular flow problem as a generalization of the minimum cost circulation problem. We then show the integrality of its LP relaxation and its dual using the iterative method. We then discuss many applications of the main result. We also show an application of the iterative method to an NP-hard degree bounded generalization and show some applications of this result as well.

The crux of the integrality of the submodular flow formulations will be the property that a maximal tight set of constraints form a cross-free family. This representation allows an inductive token counting argument to show a 1-element in an optimal extreme point solution. We will see that this representation is precisely the one we will eventually encounter in Chapter 8 on network matrices.