In this chapter we shall discuss the complexity of Frege systems without any restrictions on the depth. There is some nontrivial information, in particular nontrivial upper bounds, but no nontrivial lower bounds are known at present (only bounds from Lemma 4.4.12).

Counting in Frege systems

Theorems 9.1.5 and 9.1.6 are useful sufficient conditions guaranteeing the existence of the polynomial size EF-proofs and of quasipolynomial size F-proofs, respectively. For example, U11 proves the pigeonhole principle PHP(R) and hence there are quasipolynomial size F-proofs of PHPn. A subtheory of corresponding to the polynomial size F-proofs, based on a version of inductive definitions, was considered by Arai (1991); see Section 9.6. Its axiomatization however, stresses a logical construction, whereas we would like a theory based on a more combinatorial principle.

The most important property of a Frege system relevant for the upper bounds is that it can count. We shall make this precise by showing that F simulates an extension of I△0(α) by counting functions, and that F p-simulates a propositional proof system cutting planes.

Definition 13.1.1.

(a) Let L0 be the language of the second order bounded arithmetic but without the symbol #.

Ten years ago I had the wonderful opportunity to attend a series of lectures given by Jeff Paris in Prague on his and Alec Wilkie's work on bounded arithmetic and its relations to complexity theory. Their work produced fundamental information about the strength and properties of these weak systems, and they developed a variety of basic methods and extracted inspiring problems.

At that time Pavel Pudlak studied sequential theories and proved interesting results about the finitistic consistency statements and interpretability (Pudlak 1985, 1986, 1987). A couple of years later Sam Buss's Ph.D. thesis (Buss 1986) came out with an elegant proof-theoretic characterization of the polynomial time computations. Then I learned about Cook (1975), predating the above developments and containing fundamental ideas about the relation of weak systems of arithmetic, propositional logic, and feasible computations. These ideas were developed already in the late 70s by some of his students but unfortunately remained, to a large extent, unavailable to a general audience. New connections and opportunities opened up with Miki Ajtai's entrance with powerful combinatorics applied earlier in Boolean complexity (Ajtai 1988).

The work of these people attracted other researchers and allowed, quite recently, further fundamental results.

It appears to me that with a growing interest in the field a text surveying some basic knowledge could be helpful. The following is an outline of the book.

Bounded arithmetic was proposed in Parikh (1971), in connection with length-ofproofs questions. He called his system PB, presumably as the alphabetical successor to PA, but we shall stay with the established name I Δ0 (for “induction for Δ0 formulas”). This theory and its extensions by axioms saying that some particular recursive function is total were studied and developed in the fundamental work of J. Paris and A. Wilkie, and their students C. Dimitracopoulos, R. Kaye, and A. Woods.

They studied this theory both from the logical point of view, in connections with models of arithmetic, and in connection with computational complexity theory, mostly reflected by the definability of various complexity classes by subclasses of bounded formulas. They also investigated the relevance of Gödel's theorem to these weak subtheories of PA and closely related interpretability questions.

Further impetus to the development of bounded arithmetic came with Buss (1986), who formulated a bounded arithmetic system S2, a conservative extension of the system I Δ0 + Ω1 investigated earlier by J. Paris and A. Wilkie, and its various subsystems and second order extensions. The particular choice of the language and the definition of suitable subtheories of S2 allowed him to formulate a very precise relation between the quantifier complexity of a bounded formula and the complexity of the relation it defines, measured in terms of the levels of the polynomial time hierachy PH.

This chapter considers various witnessing theorems, which are theorems characterizing functions definable in various systems of arithmetic in terms of their computational complexity. A prototype of such a theorem (and its proof) is the characterization of primitive recursive functions as provably total recursive functions in fragment of PA (cf. Parsons 1970, Takeuti 1975, and Mints 1976).

There are other approaches to proving witnessing theorems, for example, skolemizing the given theory by Skolem functions from a particular class and then applying Herbrand's theorem. Or there are intrigued model-theoretic constructions. I shall mention these methods too, but my opinion is that one really has to know in advance which class of functions one targets before formulating an argument while the methods based on cut-elimination (Section 7.1) and generalizing Theorem 7.2.3 help to discover the right class. This certainly was the case for all witnessing theorems discussed in this chapter.

Cut-elimination for bounded arithmetic

We first extend the sequent predicate calculus by rules allowing the introduction of bounded quantifiers and by the induction rules and then we prove the cutelimination for such a system.

The predicate calculus LK extends the propositional LK from Section 4.3 by four rules for introducing quantifiers to a sequent as in Definition 4.6.2:

From Section 10.4 we know that all theories (R) and (R) are distinct. In this chapter we examine specific, more direct independence proofs for theories (R), (R), and(R), and we strengthen Corollary 10.4.3.

Herbrandization of induction axioms

In this section we shall examine the following idea for independence proofs: Take an induction axiom for a (α)-formula. It has the complexity (α). Introduce a new function symbol to obtain a Herbrand form of the axiom, as at the beginning of Section 7.3. But this time we reduce the axiom to an existential formula. This allows us to use a simpler witnessing theorem (Theorem 7.2.3) than the original form of the axiom would require.

Consider first the simplest case (which will turn out to be the only one for which the idea works). Let α(x, y) be a binary predicate. Then the herbrandization of the induction axiom for the formula A(a) ≔ ∃u ≥ a, α(u, a)

is the formula

Denote this formula JNDH(A(a)).

Theorem 11.1.1. The formula INDH(A(a)) is provable in (α, f) but not in (α, f). Hence (α, f) is not (α, f)-conservative over (α, f).

We shall define in this chapter two translations of bounded arithmetic formulas into propositional formulas and, more importantly, we shall also define translations of proofs in various systems of bounded arithmetic into propositional proofs in particular proof systems.

In the first section we shall consider the case when the language of I△0 is augmented by new predicate or function symbols, and the case of the theories and. In the second section we treat formulas in the language L and the theories, and.

In the third section we study the provability of the reflection principles for propositional proof systems in bounded arithmetic and the relation of these reflection principles to the polynomial simulations. In the fourth section we present some model-theoretic proofs for statements obtained earlier. The final section then suggests another relation of arithmetic proofs to Boolean logic, namely the relation between witnessing arguments and test (decision) trees.

Bounded formulas with a predicate

First we shall treat the theory I△0(R) and then generalize the treatment to the theories and. Instead of I△0(R) we could consider the theory but the presentation for the former is simpler. The language LPA(R) of I△0(R) is the language LPA augmented by a new binary predicate symbol R(x, y).

The central problem of complexity theory is the relation of deterministic and nondeterministic computations: whether P equals NP, and generally whether the polynomial time hierarchy PH collapses. The famous P versus NP problem is often regarded as one of the most important and beautiful open problems in contemporary mathematics, even by nonspecialists (see, for example, Smale [1992]).

The central problem of bounded arithmetic is whether it is a finitely axiomatizable theory. That amounts to deciding whether there is a model of the theory in which the polynomial time hierarchy does not collapse.

The central problem of propositional logic is whether there is a proof system in which every tautology has a proof of size polynomial in the size of the tautology. In this generality the question is equivalent to asking whether the class NP is closed under complementation. Particular cases of the problem, to establish lower bounds for usual calculi, are analogous to constructing models of associated systems of bounded arithmetic in which NP ≠ coNP.

Notions, problems, and results about complexity (of predicates, functions, proofs, …) are deep-rooted in mathematical logic, and (good) theorems about them are among the most profound results in the field. Bounded arithmetic and propositional logic are closely interrelated and have several explicit and implicit connections to the computational complexity theory around the P versus NP problem.

Knowledge and belief play an important role in everyday life. In fact, most of what we do has to do with the things we know or believe. Likewise, it is not so strange that when we have to specify the behaviour of artificial agents in order to program or implement them in some particular way, it is thought to be important to be interested in the ‘knowledge’ and ‘belief’ of such an agent. In many areas of computer science and artificial intelligence one is concerned with the description or representation of knowledge of users or even the systems themselves. For example, in database theory one tries to model knowledge about parts of reality in certain formal ways to render it implementable and accessible to users. In AI one tries to design knowledge-based decision-support systems that are intended to assist professional users in some specialists field when making decisions by providing pieces of knowledge and preferably some deductions from the input data by means of some inference mechanism. The representation and manipulation of knowledge of some sort is ubiquitous in the information sciences.

This book is not about knowledge representation in general, but rather concentrates on the logic of knowledge and belief. What (logical) properties do knowledge and belief have? What is the difference between knowledge and belief? We do not intend to answer these questions in a deep philosophical discussion of these notions.