This chapter investigates a model of computation called the Boolean circuit, which is a generalization of Boolean formulas and a simplified model of the silicon chips used to make modern computers. It is a natural model for nonuniform computation, which crops up often in complexity theory (e.g., see Chapters 19 and 20). In contrast to the standard (or uniform) TM model where the same TM is used on all the infinitely many input sizes, a nonuniform model allows a different algorithm to be used for each input size. Thus Karp and Lipton's quote above refers to the possibility that there could be a small and efficient silicon chip that is tailor-made to solve every 3SAT problem on say, 100,000 variables. The existence of such chips is not ruled out even if P ≠ NP. As the reader might now have guessed, in this chapter we give evidence that such efficient chip solvers for 3SAT are unlikely to exist, at least as the number of variables in the 3CNF formula starts to get large.

So far, we used the Turing machine (as defined in Chapter 1) as our standard model of computation. But there is one aspect of reality this model does not seem to capture: the ability to make random choices during the computation. (Most programming languages provide a built-in random number generator for this.) Scientists and philosophers may still debate if true randomness exists in the world, but it definitely seems that when tossing a coin (or measuring the results of other physical experiments) we get an outcome that is sufficiently random and unpredictable for all practical purposes.

In Chapter 2 we saw that if there is any NP language that cannot be computed by polynomial-sized circuits, then NP ≠ P. Thus proving circuit lower bounds is a potential approach for proving NP ≠ P. Furthermore, there is a reason to hope that this is a viable approach, since the Karp-Lipton Theorem (Theorem 6.19) shows that if the polynomial hierarchy PH does not collapse, then there exists an NP language that does not have polynomial size circuits.

In the 1970s and 1980s, many researchers came to believe that proving circuit lower bounds represented the best route to resolving P versus NP, since circuits seem easier to reason about than Turing machines. The success in this endeavor was mixed.

Progress on general circuits has been almost nonexistent: a lower bound of n is trivial for any function that depends on all its input bits. We are unable to prove even a superlinear circuit lower bound for any NP problem–the best we can do after years of effort is 5n – o(n).

To make life (comparatively) easier, researchers focused on restricted circuit classes, and were successful in proving some good lower bounds. We prove some of the major results of this area, specifically, for bounded depth circuits (Section 14.1), bounded depth circuits with “counting” gates (Section 14.2), and monotone circuits (Section 14.3). In all these results we have a notion of “progress” of the computation. We show that small circuits simply cannot achieve the amount of progress necessary to compute the output from the inputs.

If you have ever attempted a crossword puzzle, you know that it is much harder to solve it from scratch than to verify a solution provided by someone else. Likewise, solving a math homework problem by yourself is usually much harder than reading and understanding a solution provided by your instructor. The usual explanation for this difference of effort is that finding a solution to a crossword puzzle, or a math problem, requires creative effort. Verifying a solution is much easier since somebody else has already done the creative part.

Chapter 0

0.2 Answers are: (a) n (b) n2 (c) 2n (d) log n (e) n (f) n log n (g) nlog 3 (h) n2.

Chapter 1

1.1 Follow the grade-school algorithms.

1.5 Use the proof of Claim 1.6.

1.6 show that the universal TM U obtained by the proof of Theorem 1.9 can be tweaked to be oblivious.

1.12. b. By possibly changing from S to its complement, we may assume that the empty function φ (that is not defined on any input) is in S and that there is some function f that is defined on some input x that is not in S. Use this to show that an algorithm to compute fS can compute the function HALTx, which outputs 1 on input α iff Mα halts on input x. Then reduce computing HALT to computing HALTx thereby deriving Rice's Theorem from Theorem 1.11.

Chapter 2

2.2 CONNECTED and 2COL are shown to be in P in Exercise 1.14 (though 2COL is called BIPARTITE there). 3COL is shown to be NP-complete in Exercise 2.21, and hence it is unlikely that it is in P.

2.3 First show that for every rational matrix A, the determinant of A can always be represented using a number of bits that is polynomial in the representation of A. Then use Cramer's rule for expressing the solution of linear equations in terms of determinants.

In this chapter we study the memory requirements of computational tasks. To do this we define space-bounded computation, which places limits on the number of tape cells a TM can use during its computation. We define both deterministic and nondeterministic versions of such machines and study complexity classes of problems solvable by such machines. In Sections 4.2.1 and 4.3.2, we show some surprising relations between these variants.

As in the case of NP, we define a notion of complete problems for these classes, and identify concrete and interesting problems that are complete for space-bounded classes. It turns out that for polynomial space bounds, the complete problems involve finding winning strategies in two-player games with perfect information such as Chess and Go (see Section 4.2.2). As pointed out in Even and Tarjan's quote at the beginning of the chapter, our current understanding is that computing such strategies is inherently different from (and possibly more difficult than) solving NP problems such as SAT.

We also study computations that run in sublinear space–in other words, the input is much larger than the algorithm's work space.

Currently, resolving many of the basic questions on the power of Turing machines seems out of reach. Thus it makes sense to study simpler, more limited computing devices as a way to get some insight into the elusive notion of efficient computation. Moreover, such limited computational models often arise naturally in a variety of applications, even outside computer science, and hence studying their properties is inherently worthwhile.

Perhaps the simplest such model is that of decision trees. Here the “complexity” measure for a Boolean function f is the number of bits we need to examine in an input x in order to compute f(x). This chapter surveys the basic results and open questions regarding decision trees. Section 12.1 defines decision trees and decision tree complexity. We also define nondeterministic and probabilistic versions of decision trees just as we did for Turing machines; these are described in Sections 12.2 and 12.3, respectively. Section 12.4 contains some techniques for proving lower bounds on decision trees. We also present Yao's Min Max Lemma (see Note 12.8), which is useful for proving lower bounds for randomized decision tree complexity and, more generally, lower bounds for randomized complexity in other computational models.

Randomization is an exciting and powerful paradigm in computer science and, as we saw in Chapter 7, often provides the simplest or most efficient algorithms for many computational problems. In fact, in some areas of computer science, such as distributed algorithms and cryptography, randomization is proven to be necessary to achieve certain tasks or achieve them efficiently. Thus it's natural to conjecture (as many scientists initially did) that at least for some problems, randomization is inherently necessary: One cannot replace the probabilistic algorithm with a deterministic one without a significant loss of efficiency. One concrete version of this conjecture would be that BPP ⊈ P (see Chapter 7 for definition of BPP). Surprisingly, recent research has provided more and more evidence that this is likely to be false. As we will see in this chapter, under very reasonable complexity assumptions, there is in fact a way to derandomize (i.e., transform into a deterministic algorithm) every probabilistic algorithm of the BPP type with only a polynomial loss of efficiency. Thus today most researchers believe that BPP = P. We note that this need not imply that randomness is useless in every setting–we already saw in Chapter 8 its crucial role in the definition of interactive proofs.

Communication complexity concerns the following scenario. There are two players with unlimited computational power, each of whom holds an n bit input, say x and y. Neither knows the other's input, and they wish to collaboratively compute f(x, y) where the function f:{0, 1}n × {0, 1}n → {0, 1} is known to both. Furthermore, they had foreseen this situation (e.g., one of the parties could be a spacecraft and the other could be the base station on earth), so they had already–before they knew their inputs x, y–agreed upon a protocol for communication. The cost of this protocol is the number of bits communicated by the players for the worst-case choice of inputs x, y.

Researchers have studied many modifications of this basic scenario, including randomized protocols, nondeterministic protocols, and average-case protocols. Furthermore, lower bounds on communication complexity have uses in a variety of areas, including lower bounds for parallel and VLSI computation, circuit lower bounds, polyhedral theory, data structure lower bounds, and more. Communication complexity has been one of the most successful models studied in complexity, as it strikes the elusive balance of being simple enough so that we can actually prove strong lower bounds, but general enough so we can obtain important applications of these lower bounds.

Cryptography is much older than computational complexity. Ever since people began to write, they invented method's for “secret writing” that would be difficult to decipher for others. But the numerous methods of encryption or “secret writing” devised over the years all had one common characteristic–sooner or later they were broken. But everything changed in 1970s, when, thanks to the works of several researchers, modern cryptography was born, whereby computational complexity was used to argue about the security of the encryption schemes.

Why have we not been able to prove strong lower bounds for general circuits? Despite the dramatic success in proving lower bounds on restricted circuit classes as described in Chapter 14, we seem utterly at a loss when it comes to showing limitations of general Boolean circuits.

In 1994 Razborov and Rudich [RR94] described what they view as the main technical limitation of current approaches for proving circuit lower bounds. They defined a notion of “natural mathematical proofs” for a circuit lower bound. They pointed out that current lower bound arguments involve such mathematical proofs, and showed that obtaining strong lower bound with such proof techniques would violate a stronger form of the P ≠ NP conjecture–specifically, the conjecture that strong one-way functions exist that cannot be inverted by algorithms running in subexponential time. Since current evidence suggests that such strong one-way functions do exist (factoring integers, discrete log, etc., as described in Chapter 9), we conclude that current techniques are inherently incapable of proving strong lower bounds on general circuits.

The Razborov-Rudich result may be viewed as a modern analog of the 1970s results on the limits of diagonalization (see Chapter 3). What is particularly striking is that computational complexity (namely, the existence of strong one-way functions) is used here to shed light on a metamathematical question about computational complexity: “Why have we been unable to prove P ≠ NP?”

The class NP captures the difficulty of finding certificates. However, in many contexts, one is interested not just in a single certificate but actually in counting the number of certificates. This chapter studies #P (pronounced “sharp p”), a complexity class that captures this notion.

Counting problems arise in diverse fields such as statistical estimation, statistical physics, network design, and economics, often in situations having to do with estimations of probability. Counting problems are also studied in a field of mathematics called enumerative combinatorics, which tries to obtain closed-form mathematical expressions for counting problems. To give an example, in 1847 Kirchoff showed how the resistance of a network can be determined by counting the number of spanning trees in it, for which he gave a formula involving a simple determinant computation. Results in this chapter will show that for many other natural counting problems, such efficiently computable expressions are unlikely to exist.

In Section 17.1 we give an informal introduction to counting problems and how they arise in statistical estimation. We also encounter an interesting phenomenon: A counting problem can be difficult even though the corresponding decision problem is easy.

We saw in Chapter 11 that the PCP Theorem implies that computing approximate solutions to many optimization problems is NP-hard. This chapter gives a complete proof of the PCP Theorem. In Chapter 11 we also mentioned that the PCP Theorem does not suffice for proving several other similar results, for which we need stronger (or simply different) “PCP Theorems.” In this chapter we survey some such results and their proofs. The two main results are Raz's Parallel Repetition Theorem (see Section 22.3) and Håstad's Three-Bit PCP Theorem (Theorem 22.16). Raz's theorem leads to strong hardness results for the 2CSP problem over large alphabets. Håstad's theorem shows that certificates for NP languages can be probabilistically checked by examining only three bits in them.

The standard notion of a mathematical proof is closely related to the certificate definition of NP. To prove that a statement is true one provides a sequence of symbols on a piece of paper, and the verifier checks that they represent a valid proof/certificate. A valid proof/certificate exists only for true statements. However, people often use a more general way to convince one another of the validity of statements: they interact with one another, where the person verifying the proof (called verifier from now on) asks the person providing it (called prover from now on) for a series of explanations before he is convinced.

It seems natural to try to understand the power of such interactive proofs from the complexity-theoretic perspective. For example, can one prove in a succinct way that a given formula is not satisfiable? This problem is coNP-complete, and hence is believed to not have a polynomial-sized proof in the traditional sense.