Recall from Chapter 6 that we regard two dynamical systems as being “the same” if they are conjugate and otherwise “different.” In Chapter 7 we concentrated on conjugacy for edge shifts, and found that two edge shifts are conjugate if and only if the adjacency matrices of their defining graphs are strong shift equivalent.

We also saw that it can be extremely difficult to decide whether two given matrices are strong shift equivalent. Thus it makes sense to ask if there are ways in which two shifts of finite type can be considered “nearly the same,” and which can be more easily decided. This chapter investigates one way called finite equivalence, for which entropy is a complete invariant. Another, stronger way, called almost conjugacy, is treated in the next chapter, where we show that entropy together with period form a complete set of invariants.

In §8.1 we introduce finite-to-one codes, which are codes used to describe “nearly the same.” Right-resolving codes are basic examples of finite-to-one codes, and in §8.2 we describe a matrix formulation for a 1-block code from one edge shift to another to be finite-to-one. In §8.3 we introduce the notion of finite equivalence between sofic shifts, and prove that entropy is a complete invariant. A stronger version of finite equivalence is discussed and characterized in §8.4.

Finite-to-One Codes

We begin this chapter by introducing finite-to-one sliding block codes.

Bounded arithmetic was proposed in Parikh (1971), in connection with length-ofproofs questions. He called his system PB, presumably as the alphabetical successor to PA, but we shall stay with the established name I Δ0 (for “induction for Δ0 formulas”). This theory and its extensions by axioms saying that some particular recursive function is total were studied and developed in the fundamental work of J. Paris and A. Wilkie, and their students C. Dimitracopoulos, R. Kaye, and A. Woods.

They studied this theory both from the logical point of view, in connections with models of arithmetic, and in connection with computational complexity theory, mostly reflected by the definability of various complexity classes by subclasses of bounded formulas. They also investigated the relevance of Gödel's theorem to these weak subtheories of PA and closely related interpretability questions.

Further impetus to the development of bounded arithmetic came with Buss (1986), who formulated a bounded arithmetic system S2, a conservative extension of the system I Δ0 + Ω1 investigated earlier by J. Paris and A. Wilkie, and its various subsystems and second order extensions. The particular choice of the language and the definition of suitable subtheories of S2 allowed him to formulate a very precise relation between the quantifier complexity of a bounded formula and the complexity of the relation it defines, measured in terms of the levels of the polynomial time hierachy PH.

This chapter considers various witnessing theorems, which are theorems characterizing functions definable in various systems of arithmetic in terms of their computational complexity. A prototype of such a theorem (and its proof) is the characterization of primitive recursive functions as provably total recursive functions in fragment of PA (cf. Parsons 1970, Takeuti 1975, and Mints 1976).

There are other approaches to proving witnessing theorems, for example, skolemizing the given theory by Skolem functions from a particular class and then applying Herbrand's theorem. Or there are intrigued model-theoretic constructions. I shall mention these methods too, but my opinion is that one really has to know in advance which class of functions one targets before formulating an argument while the methods based on cut-elimination (Section 7.1) and generalizing Theorem 7.2.3 help to discover the right class. This certainly was the case for all witnessing theorems discussed in this chapter.

Cut-elimination for bounded arithmetic

We first extend the sequent predicate calculus by rules allowing the introduction of bounded quantifiers and by the induction rules and then we prove the cutelimination for such a system.

The predicate calculus LK extends the propositional LK from Section 4.3 by four rules for introducing quantifiers to a sequent as in Definition 4.6.2:

From Section 10.4 we know that all theories (R) and (R) are distinct. In this chapter we examine specific, more direct independence proofs for theories (R), (R), and(R), and we strengthen Corollary 10.4.3.

Herbrandization of induction axioms

In this section we shall examine the following idea for independence proofs: Take an induction axiom for a (α)-formula. It has the complexity (α). Introduce a new function symbol to obtain a Herbrand form of the axiom, as at the beginning of Section 7.3. But this time we reduce the axiom to an existential formula. This allows us to use a simpler witnessing theorem (Theorem 7.2.3) than the original form of the axiom would require.

Consider first the simplest case (which will turn out to be the only one for which the idea works). Let α(x, y) be a binary predicate. Then the herbrandization of the induction axiom for the formula A(a) ≔ ∃u ≥ a, α(u, a)

is the formula

Denote this formula JNDH(A(a)).

Theorem 11.1.1. The formula INDH(A(a)) is provable in (α, f) but not in (α, f). Hence (α, f) is not (α, f)-conservative over (α, f).

Symbolic dynamics is part of a larger theory of dynamical systems. This chapter gives a brief introduction to this theory and shows where symbolic dynamics fits in. Certain basic dynamical notions, such as continuity, compactness, and topological conjugacy, have already appeared in a more concrete form. So our development thus far serves to motivate more abstract dynamical ideas. On the other hand, applying basic concepts such as continuity to the symbolic context shows why the objects we have been studying, such as sliding block codes, are natural and inevitable.

We begin by introducing in §6.1 a class of spaces for which there is a way to measure the distance between any pair of points. These are called metric spaces. Many spaces that you are familiar with, such as subsets of 3-dimensional space, are metric spaces, using the usual Euclidean formula to measure distance. But our main focus will be on shift spaces as metric spaces. We then discuss concepts from analysis such as convergence, continuity, and compactness in the setting of metric spaces. In §6.2 we define a dynamical system to be a compact metric space together with a continuous map from the space to itself. The shift map on a shift space is the main example for us. When confronted with two dynamical systems, it is natural to wonder whether they are “the same,” i.e., different views of the same underlying process.

Shift spaces are to symbolic dynamics what shapes like polygons and curves are to geometry. We begin by introducing these spaces, and describing a variety of examples to guide the reader's intuition. Later chapters will concentrate on special classes of shift spaces, much as geometry concentrates on triangles and circles. As the name might suggest, on each shift space there is a shift map from the space to itself. Together these form a “shift dynamical system.” Our main focus will be on such dynamical systems, their interactions, and their applications.

In addition to discussing shift spaces, this chapter also connects them with formal languages, gives several methods to construct new shift spaces from old, and introduces a type of mapping from one shift space to another called a sliding block code. In the last section, we introduce a special class of shift spaces and sliding block codes which are of interest in coding theory.

Full Shifts

Information is often represented as a sequence of discrete symbols drawn from a fixed finite set. This book, for example, is really a very long sequence of letters, punctuation, and other symbols from the typographer's usual stock. A real number is described by the infinite sequence of symbols in its decimal expansion. Computers store data as sequences of 0's and 1's. Compact audio disks use blocks of 0's and 1's, representing signal samples, to digitally record Beethoven symphonies.

We shall define in this chapter two translations of bounded arithmetic formulas into propositional formulas and, more importantly, we shall also define translations of proofs in various systems of bounded arithmetic into propositional proofs in particular proof systems.

In the first section we shall consider the case when the language of I△0 is augmented by new predicate or function symbols, and the case of the theories and. In the second section we treat formulas in the language L and the theories, and.

In the third section we study the provability of the reflection principles for propositional proof systems in bounded arithmetic and the relation of these reflection principles to the polynomial simulations. In the fourth section we present some model-theoretic proofs for statements obtained earlier. The final section then suggests another relation of arithmetic proofs to Boolean logic, namely the relation between witnessing arguments and test (decision) trees.

Bounded formulas with a predicate

First we shall treat the theory I△0(R) and then generalize the treatment to the theories and. Instead of I△0(R) we could consider the theory but the presentation for the former is simpler. The language LPA(R) of I△0(R) is the language LPA augmented by a new binary predicate symbol R(x, y).

Symbolic dynamics is a rapidly growing part of dynamical systems. Although it originated as a method to study general dynamical systems, the techniques and ideas have found significant applications in data storage and transmission as well as linear algebra. This is the first general textbook on symbolic dynamics and its applications to coding, and we hope that it will stimulate both engineers and mathematicians to learn and appreciate the subject.

Dynamical systems originally arose in the study of systems of differential equations used to model physical phenomena. The motions of the planets, or of mechanical systems, or of molecules in a gas can be modeled by such systems. One simplification in this study is to discretize time, so that the state of the system is observed only at discrete ticks of a clock, like a motion picture. This leads to the study of the iterates of a single transformation. One is interested in both quantitative behavior, such as the average time spent in a certain region, and also qualitative behavior, such as whether a state eventually becomes periodic or tends to infinity. Symbolic dynamics arose as an attempt to study such systems by means of discretizing space as well as time. The basic idea is to divide up the set of possible states into a finite number of pieces, and keep track of which piece the state of the system lies in at every tick of the clock.

In Chapters 7 and 8 we studied two notions of equivalence for shifts of finite type and sofic shifts: conjugacy and finite equivalence. Conjugacy is the stronger and more fundamental notion, while finite equivalence is more decidable. In this chapter, we introduce an intermediate concept of equivalence called almost conjugacy, which was motivated by constructions of codes in ergodic theory.

For a finite-to-one factor code on an irreducible shift of finite type or sofic shift, there is, by definition, a uniform upper bound on the number of pre-images of points in the image. Thus, the number of pre-images can vary through a finite set of positive integers. In §9.1 we show that certain points which are “representative” of the range shift all have the same number of pre-images. This number is called the degree of the code. In §9.2 we focus on codes of degree one, called almost invertible codes, and show how under certain circumstances finite-to-one codes can be replaced by almost invertible codes without changing the domain and range. In §9.3 we introduce the notion of almost conjugacy, which by definition is a finite equivalence in which both legs are almost invertible. We show that entropy and period form a complete set of almost conjugacy invariants for irreducible shifts of finite type and for irreducible sofic shifts. Finally, in §9.4 we conclude with results which assert that the “representative” points are really “typical” points in a probabilistic sense.

The central problem of complexity theory is the relation of deterministic and nondeterministic computations: whether P equals NP, and generally whether the polynomial time hierarchy PH collapses. The famous P versus NP problem is often regarded as one of the most important and beautiful open problems in contemporary mathematics, even by nonspecialists (see, for example, Smale [1992]).

The central problem of bounded arithmetic is whether it is a finitely axiomatizable theory. That amounts to deciding whether there is a model of the theory in which the polynomial time hierarchy does not collapse.

The central problem of propositional logic is whether there is a proof system in which every tautology has a proof of size polynomial in the size of the tautology. In this generality the question is equivalent to asking whether the class NP is closed under complementation. Particular cases of the problem, to establish lower bounds for usual calculi, are analogous to constructing models of associated systems of bounded arithmetic in which NP ≠ coNP.

Notions, problems, and results about complexity (of predicates, functions, proofs, …) are deep-rooted in mathematical logic, and (good) theorems about them are among the most profound results in the field. Bounded arithmetic and propositional logic are closely interrelated and have several explicit and implicit connections to the computational complexity theory around the P versus NP problem.

The Lower Entropy Factor Theorem of §10.3 completely solves the problem of when one irreducible shift of finite type factors onto another of strictly lower entropy. In contrast, the corresponding problem for equal entropy does not yet have a satisfactory solution. This chapter contains some necessary conditions for an equal-entropy factor code, and also some sufficient conditions. We will also completely characterize when one shift of finite type “eventually” factors onto another of the same entropy.

In §12.1 we state a necessary and sufficient condition for one irreducible shift of finite type to be a right-closing factor of another. While the proof of this is too complicated for inclusion here, we do prove an “eventual” version: for irreducible shifts of finite type X and Y with equal entropy we determine when Ym is a right-closing factor of Xm for all sufficiently large m. In §12.2 we extend this to determine when Xm factors onto Ym for all sufficiently large m (where the factor code need not be right-closing). This is analogous to Theorem 7.5.15, which showed that two irreducible edge shifts are eventually conjugate if and only if their associated matrices are shift equivalent.

At the end of §10.3 we proved a generalization of the Finite-State Coding Theorem from Chapter 5, where the sofic shifts had different entropies. This amounted to the construction of right-closing finite equivalences. In §12.3 we show that given two irreducible edge shifts with equal entropy log λ, the existence of a right-closing finite equivalence forces an arithmetic condition on the entries of the corresponding Perron eigenvectors.

Knowledge and belief play an important role in everyday life. In fact, most of what we do has to do with the things we know or believe. Likewise, it is not so strange that when we have to specify the behaviour of artificial agents in order to program or implement them in some particular way, it is thought to be important to be interested in the ‘knowledge’ and ‘belief’ of such an agent. In many areas of computer science and artificial intelligence one is concerned with the description or representation of knowledge of users or even the systems themselves. For example, in database theory one tries to model knowledge about parts of reality in certain formal ways to render it implementable and accessible to users. In AI one tries to design knowledge-based decision-support systems that are intended to assist professional users in some specialists field when making decisions by providing pieces of knowledge and preferably some deductions from the input data by means of some inference mechanism. The representation and manipulation of knowledge of some sort is ubiquitous in the information sciences.

This book is not about knowledge representation in general, but rather concentrates on the logic of knowledge and belief. What (logical) properties do knowledge and belief have? What is the difference between knowledge and belief? We do not intend to answer these questions in a deep philosophical discussion of these notions.