This chapter will present basic propositional calculus. By that I mean properties of propositional calculus established by direct combinatorial arguments as distinguished from high level arguments involving concepts (or motivations) from other parts of logic (bounded arithmetic) and complexity theory.

Examples of the former are various simulation results or the lower bound for resolution from Haken (1985). Examples of the latter are the simulation of the Frege system with substitution by the extended Frege system (Lemma 4.5.5 and Corollary 9.3.19), or the construction of the provably hardest tautologies from the fmitistic consistency statements (Section 14.2).

We shall define basic propositional proof systems: resolution R, extended resolution ER, Frege system F, extended Frege system EF, Frege system with the substitution rule SF, quantified propositional calculus G, and Gentzen's sequent calculus LK. We begin with the general concept of a propositional proof system.

Propositional proof systems

A property of the usual textbook calculus is that it can be checked in deterministic polynomial time whether a string of symbols is a proof in the system or not. This is generalized into the following basic definition of Cook and Reckhow (1979).

Definition 4.1.1. Let TA UT be the set of propositional tautologies in the language with propositional connectives: constants 0 (FALSE) and 1 (TRUE), ¬ (negation), ∨ (disjunction), and & (conjunction), and atoms p1, p2,…

A propositional proof system is a polynomial time function P whose range is the set TAUT.

This chapter is devoted primarily to proving several definability and witnessing theorems for the second order system and analogous to those in Chapters 6 and 7.

Our tool is the RSUV isomorphism (Theorem 5.5.13), or rather the definition of (Definition 5.5.3), together with the model-theoretic construction of Lemma 5.5.4.

The first section discusses and defines the second order computations. In the second section are proved some definability and witnessing theorems for the second order systems and further conservation results for first order theories (Corollaries 8.2.5-8.2.7). The proofs are sketched and the details of the RSUV isomorphism arguments are left to the reader.

Second order computations

Let A (a, βt(b)) be a second order bounded formula and (K, X) a model of. By Definition 5.5.3 we may think of K as of K = Log(M) for some M ⊨, with X being the subsets of K coded in M. Pick some a, b ∈ K of length n and some βt(b). Then

if and only if (see Theorem 5.5.13 for the notation)

where u codes βt(b) The length of u is thus.

In this chapter we briefly review the basic notions and facts from logic and complexity theory whose knowledge is assumed throughout the book. We shall always sketch important arguments, both from logic and from complexity theory, and so a determined reader can start with only a rough familiarity with the notions surveyed in the next two sections and pick the necessary material along the way.

For those readers who prefer to consult relevant textbooks we recommend the following books: The best introduction to logic are parts of Shoenfield (1967); for elements of structural complexity theory I recommend Balcalzár, Diáz, and Gabbarró (1988, 1990); for NP-completeness Garey and Johnson (1979); and for a Boolean complexity theory survey of lower bounds Boppana and Sipser (1990) or the comprehensive monograph Wegener (1987). A more advanced (but selfcontained) text on logic of first order arithmetic theories is Hájek and Pudlák (1993).

Logic

We shall deal with first order and second order theories of arithmetic. The second order theories are, in fact, just two-sorted first order theories: One sort are numbers; the other are finite sets. This phrase means that the underlying logic is always the first order predicate calculus; in particular, no set-theoretic assumptions are a part of the underlying logic.

From basic theorems we shall use Gödel completeness and incompleteness theorems, Tarski's undefinability of truth, and, in arithmetic, constructions of partial truth definitions.

In §2.5 we saw why it is useful to transform arbitrary binary sequences into sequences that obey certain constraints. In particular, we used Modified Frequency Modulation to code binary sequences into sequences from the (1,3) run-length limited shift. This gave a more efficient way to store binary data so that it is not subject to clock drift or intersymbol interference.

Different situations in data storage and transmission require different sets of constraints. Thus our general problem is to find ways to transform or encode sequences from the full n-shift into sequences from a preassigned sofic shift X. In this chapter we will describe one encoding method called a finite-state code. The main result, the Finite-State Coding Theorem of §5.2, says that we can solve our coding problem with a finite-state code precisely when h(X) ≥ log n. Roughly speaking, this condition simply requires that X should have enough “information capacity” to encode the full n-shift.

We will begin by introducing in §5.1 two special kinds of labelings needed for finite-state codes. Next, §5.2 is devoted to the statement and consequences of the Finite-State Coding Theorem. Crucial to the proof is the notion of an approximate eigenvector, which we discuss in §5.3. The proof itself occupies §5.4, where an approximate eigenvector is used to guide a sequence of state splittings that converts a presentation of X into one with out-degree at least n at every state.

Entropy measures the complexity of mappings. For shifts, it also measures their “information capacity,” or ability to transmit messages. The entropy of a shift is an important number, for it is invariant under conjugacy, can be computed for a wide class of shifts, and behaves well under standard operations like factor codes and products. In this chapter we first introduce entropy and develop its basic properties. In order to compute entropy for irreducible shifts of finite type and sofic shifts in §4.3, we describe the Perron–Frobenius theory of nonnegative matrices in §4.2. In §4.4 we show how general shifts of finite type can be decomposed into irreducible pieces and compute entropy for general shifts of finite type and sofic shifts. In §4.5 we describe the structure of the irreducible pieces in terms of cyclically moving states.

Definition and Basic Properties

Before we get under way, we review some terminology and notation from linear algebra.

Recall that the characteristic polynomial of a matrix A is defined to be χA(t) = det(tId – A), where Id is the identity matrix. The eigenvalues of A are the roots of χA(t). An eigenvector of A corresponding to eigenvalue λ is a vector v, not identically 0, such that Av = λv.

We say that a (possibly rectangular) matrix A is (strictly) positive if each of its entries is positive. In this case we write A > 0.

In this chapter we shall discuss the complexity of Frege systems without any restrictions on the depth. There is some nontrivial information, in particular nontrivial upper bounds, but no nontrivial lower bounds are known at present (only bounds from Lemma 4.4.12).

Counting in Frege systems

Theorems 9.1.5 and 9.1.6 are useful sufficient conditions guaranteeing the existence of the polynomial size EF-proofs and of quasipolynomial size F-proofs, respectively. For example, U11 proves the pigeonhole principle PHP(R) and hence there are quasipolynomial size F-proofs of PHPn. A subtheory of corresponding to the polynomial size F-proofs, based on a version of inductive definitions, was considered by Arai (1991); see Section 9.6. Its axiomatization however, stresses a logical construction, whereas we would like a theory based on a more combinatorial principle.

The most important property of a Frege system relevant for the upper bounds is that it can count. We shall make this precise by showing that F simulates an extension of I△0(α) by counting functions, and that F p-simulates a propositional proof system cutting planes.

Definition 13.1.1.

(a) Let L0 be the language of the second order bounded arithmetic but without the symbol #.

Although we have seen many aspects of symbolic dynamics, there are still many more that we have not mentioned. This final chapter serves as a guide to the reader for some of the more advanced topics. Our treatment of each topic only sketches some of its most important features, and we have not included some important topics. For each topic we have tried to give sufficient references to research papers so that the reader may learn more. In many places we refer to papers for precise proofs and sometimes even for precise definitions. The survey paper of Boyle [Boy5] contains descriptions of some additional topics.

More on Shifts of Finite Type and Sofic Shifts

THE CORE MATRIX

Any shift of finite type X can be recoded to an edge shift XG, and we can associate the matrix AG to X. This matrix is not unique, but any two such matrices are shift equivalent, and in particular they must have the same Jordan form away from zero. This gives us a way of associating to X a particular Jordan form, or, equivalently, a particular similarity class of matrices. By Theorem 7.4.6, this similarity class is an invariant of conjugacy, and, by Proposition 12.2.3, it gives a constraint on finite-to-one factors between irreducible shifts of finite type.

Ten years ago I had the wonderful opportunity to attend a series of lectures given by Jeff Paris in Prague on his and Alec Wilkie's work on bounded arithmetic and its relations to complexity theory. Their work produced fundamental information about the strength and properties of these weak systems, and they developed a variety of basic methods and extracted inspiring problems.

At that time Pavel Pudlak studied sequential theories and proved interesting results about the finitistic consistency statements and interpretability (Pudlak 1985, 1986, 1987). A couple of years later Sam Buss's Ph.D. thesis (Buss 1986) came out with an elegant proof-theoretic characterization of the polynomial time computations. Then I learned about Cook (1975), predating the above developments and containing fundamental ideas about the relation of weak systems of arithmetic, propositional logic, and feasible computations. These ideas were developed already in the late 70s by some of his students but unfortunately remained, to a large extent, unavailable to a general audience. New connections and opportunities opened up with Miki Ajtai's entrance with powerful combinatorics applied earlier in Boolean complexity (Ajtai 1988).

The work of these people attracted other researchers and allowed, quite recently, further fundamental results.

It appears to me that with a growing interest in the field a text surveying some basic knowledge could be helpful. The following is an outline of the book.