In Lecture 7 we discussed the relationship between classifications and the Boolean operations. In this lecture, we study the corresponding relationship for theories. In particular, we discuss Boolean operations that take theories to theories, as well as what it would mean for operations to be Boolean operations in the context of a particular theory. In this way, we begin to see how the traditional rules of inference emerge from an informational perspective. The topic is a natural one but it is not central to the main development so this lecture could be skipped.

Boolean Operations on Theories

Given a regular theory T = (Σ, ⊢), one may define a consequence relation on the set pow(Σ) of subsets of Σ in one of two natural ways, depending on whether one thinks of the sets of types disjunctively or conjunctively. This produces two new theories, ∨T and ∧T, respectively.

These operations should fit with the corresponding power operations ∨A and ∧A on classifications A; we want vTh(A) to be the same as the theory Th(∨A), for example. Thus, to motivate our definitions, we begin by investigating the relationship of the theory Th(A) of a classification to the theories Th(∨A) and Th(∧A) of its two power classifications.

Definition 11.1. Given a set Γ of subsets of Σ, a set Y is a choice set on Γ if X ∩ Y ≠ Ø for each X ∈ Γ.

Information and talk of information is everywhere nowadays. Computers are thought of as information technology. Each living thing has a structure determined by information encoded in its DNA. Governments and companies spend vast fortunes to acquire information. People go to prison for the illicit use of information. In spite of all this, there is no accepted science of information. What is information? How is it possible for one thing to carry information about another? This book proposes answers to these questions.

But why does information matter, why is it so important? An obvious answer motivates the direction our theory takes. Living creatures rely on the regularity of their environment for almost everything they do. Successful perception, locomotion, reasoning, and planning all depend on the existence of a stable relationship between the agents and the world around them, near and far. The importance of regularity underlies the view of agents as information processors. The ability to gather information about parts of the world, often remote in time and space, and to use that information to plan and act successfully, depends on the existence of regularities. If the world were a completely chaotic, unpredictable affair, there would be no information to process.

Still, the place of information in the natural world of biological and physical systems is far from clear. A major problem is the lack of a general theory of regularity.

The view of information put forward here associates information flow with distributed systems. Such a system A, we recall, consists of an indexed family cla(A) = {Ai}i∈I of classifications together with a set inf (A) of isomorphisms, all of which have both a domain and a codomain in cla(A). With any such a system we want to associate a systemwide logic Log(A) on the sum ∑i∈I,Ai of the classifications in the system. The constraints of Log(A) should use the lawlike regularities represented by the system as a whole. The normal tokens of Log(A) model those indexed families of tokens to which the constraints are guaranteed to apply, by virtue of the structure of the system.

If we consider a given component classification Ai of A, there are at least two sensible logics on Ai that we might want to incorporate into Log(A), the a priori logic AP(Ai) and the natural logic Log(Ai). The former assumes we are given no information about the constraints of Ai except for the trivial constraints. The latter assumes perfect information about the constraints of Ai. There is typically quite a difference. But really these are just two extremes in our ordering of sound local logics on Ai. After all, in dealing with a distributed system, we may have not just the component classifications and their informorphisms, but also local logics on the component classifications. We want the systemwide logic to incorporate these local logics.

State-space models are one of the most prevalent tools in science and applied mathematics. In this lecture, we show how state spaces are related to classifications and how systems of state spaces are related to information channels. As a result, we will discover that state spaces provide a rich source of information channels. In later lectures, we will exploit the relationship between state spaces and classifications in our study of local logics.

State Spaces and Projections

Definition 8.1. A state space is a classification S for which each token is of exactly one type. The types of a state space are called states, and we say that a is in state σ if a ⊨s σ. The state space S is complete if every state is the state of some token.

Example 8.2. In Example 4.5 we pointed out that for any function f : A → B, there is a classification whose types are elements of B and whose tokens are elements of A and such that a ⊨b if and only if b = f(a). This classification is a state space and every state space arises in this way, so another way to put the definition is to say that a state space is a classification S in which the classification relation ⊨s is a total function. For this reason, we write states(a) for the state σ of a in S.

To understand the account presented here, it is useful to distinguish two questions about information flow in a given system. What information flows through the system? Why does it flow? This book characterizes the first question in terms of a “local logic” and answers the second with the related notion of an “information channel.” Within the resulting framework one can understand the basic structure of information flow. The local logic of a system is a model of the regularities that support information flow within the system, as well as the exceptions to these regularities. The information channel is a model of the system of connections within the system that underwrite this information flow.

The model of information flow developed here draws on ideas from the approaches to information discussed in Lecture 1 and, in the end, can be seen as a theory that unifies these various apparently competing theories. The model also draws on ideas from classical logic and from recent work in computer science. The present lecture gives an informal overview of this framework.

Classifications and Infomorphisms

Fundamental to the notions of information channel and local logic are the notions of “classification” and “infomorphisms.” These terms may be unfamiliar, but the notions have been around in the literature for a long time.

Paying Attention to Particulars

We begin by introducing one of the distinctive features of the present approach, namely its “two-tier” nature, paying attention to both types and particulars.

The concepts of information and representation are, of course, closely related. Indeed, Jerry Fodor feels that they are so closely related as to justify the slogan “No information without representation.” Though we do no go that far, we do think of the two as intimately connected, as should be clear from our account. In this lecture, we sketch the beginnings of a theory of representation within the framework presented in Part II. We have three motives for doing so. One is to suggest what we think such a theory might look like. The second is to explain some interesting recent work on inference by Shimojima. The third is to show that Shimojima's work has a natural setting in the theory presented here.

Modeling Representational Systems

When we think of information flow involving humans, some sort of representational system is typically, if not always, involved: Spoken or written language, pictures, maps, diagrams, and the like are all examples of representations. So representations should fit into our general picture of information flow.

A theory of representation must be compatible with the fact that representation is not always veridical. People often misrepresent things, inadvertently or otherwise. For this reason, we model representation systems as certain special kinds of information systems where unsound logics can appear. We begin with our model of a representational system.

A standard objection to classical logic has been its failure to come to grips with vague predicates and their associated problems and paradoxes. An analysis of the vague predicates “low,” “medium,” and “high” (as applied to brightness of light bulbs) was implicit in Lecture 3. In this lecture we want to make the idea behind this treatment more explicit, thereby suggesting an information-theoretic line of research into vagueness. At best, this line of development would allow the information-flow perspective to contribute to the study of vagueness. At the very least, it should show that vagueness is not an insurmountable problem to the perspective offered in this book.

In this lecture we explore a different family of related vague predicates, “short,” “medium,” “tall,” “taller,” and “same height as.” This family is simple enough to treat in some detail but complicated enough to exhibit three problems that are typical of vague predicates.

Information Flow Between Perspectives

The first problem is that different people, with differing circumstances, often have different standards in regard to what counts as being short or tall. In spite of the lack of any absolute standard, though, information flow is possible between people using these predicates. If Jane informs me that Mary is of medium height while she, Jane, is short, and if I consider Jane to be tall, then I know that I would consider Mary as tall as well. How is such reliable information flow possible between people with quite different standards of what counts as being tall?

The notion of a classification does not build in any assumptions about closure under the usual Boolean operations. It is natural to ask What role do the usual Boolean connectives play in information and its flow? This lecture takes an initial step toward answering this question. We will return to this question in later chapters as we develop more tools. It is not a central topic of the book, but it is one that needs to be addressed in a book devoted to the logic of information flow.

Actually, there are two ways of understanding Boolean operations on classifications. There are Boolean operations mapping classifications to classifications, and there are Boolean operations internal to (many) classifications. Because there is a way to explain the latter in terms of the former, we first discuss the Boolean operations on classifications.

Boolean Operations on Classifications

Given a set Φ of types in a classification, it is often useful to group together the class of tokens that are of every type in Φ. In general, there is no type in the classification with this extension. As a remedy, we can always construct a classification in which such a type exists. Likewise, we can construct a classification in which there is a type whose extension consists of all those tokens that are of at least one of the types in Φ.

In this chapter we present some problems for which their NC approximations are obtained using other techniques like “step by step” parallelization of their sequential approximation algorithms. This statement does not implythat the PRAM implementation is trivial. In some cases, several tricks must be used to get it. A difference from previous chapters is the fact that we shall also consider “heuristic” algorithms. In the first section we present two positive parallel approximation results for the Minimum Metric Traveling Salesperson problem. We defer the non-parallel approximability results on the Minimum Metric Traveling Salesperson problem until the next chapter. The following section deals with an important problem we already mentioned at the end of Section 4.1; the Bin Packing problem. We present a parallelization to the asymptotic approximation scheme. We finish by giving a state of the art about parallel approximation algorithms for some other problems, where the techniques used do not fit into any of the previous paradigms of parallel approximation, and which present some interesting open problems.

The Minimum Metric Traveling Salesperson Problem

Let us start by considering an important problem, the Minimum Metric Traveling Salesperson problem (MTSP). It is well known the problem is in APX. There are several heuristics to do the job. The most famous of them is the constant approximation due to Christofides [Chr76]. Moreover the problem is known to be APX-complete [PY93]. Due to the importance of the problem, quite a few heuristics have been developed for the MTSP problem. For instance, the nearest neighbor heuristics, starting at a given vertex, among all the vertices not yet visited, choose as the next vertex the one that is closest to the current vertex.

The linear programming primal-dual method has been extensively used to obtain sequential exact algorithms (see for example [Chv79], [PS82]). This algorithm keeps both a primal solution and a dual solution. When the solutions together satisfy the complementary slackness conditions, then they are mutually optimal. Otherwise either the primal solution is augmented or the dual solution is improved. The primal-dual method was originally due to Dantzing, Ford and Fulkerson [DFF56]. Unless P=NP, the primaldual method cannot be used to solve exactly in polynomial time NP-hard problems.

The primal-dual framework has been particularly useful to obtain polynomial time approximation algorithms for some NP-hard combinatorial optimization problems (see for example the forthcoming survey by Goemans and Williamson [GW96]). For those problems that can be formulated as integer programming problems, the approach works with the linear programming relaxation and its dual, and seeks for an integral extension of the linear programming solution. Furthermore the use of the combinatorial structure of each problem determines how to design the improvement steps\ and how to carry on the proof of approximation guarantee. There is no general primal-dual approximate technique; however, some approaches can be seen as producing both primal and dual feasible solutions, until some conditions are met. Those last conditions insure that the values of the primal and dual solutions are within 1 + ε of each other. As the optima of the primal and dual problems are the same, the primal and dual feasible solutions produced by the algorithm have a value within 1 + ε of optimal value.

An interesting question in graph theory is to find whether a given graph contains a vertex induced subgraph satisfying a certain property. We shall consider the special case where the property depends only on the value of a certain parameter that can take (positive) integer values. Such properties are described as weighted properties. For a given graph G = (V,E), G′ ⊆ G means that G′ is a vertex induced subgraph of G. The generic Induced Subgraph of High Weight problem (ISHW) consists in, given a graph G = (V, E), a weighted property W on the set of graphs, and an integer κ, to decide if G contains a vertex induced subgraph H such that W(H) ≥ κ. A first concrete example of the Induced Subgraph of High Weight problem is the case when W is the minimum degree of a graph. This instance of the problem is known as the High Degree Subgraph problem (HDS). Historically, this is the first problem shown to be P-complete and approximated in parallel ([AM86] and [HS87]). Another instance of the Induced Subgraph of High Weight is the case when W is the vertex (or edge) connectivity of G. These problems are known as the High Vertex Connected Subgraph problem (HVCS) (respectively the High Edge Connected Subgraph problem (HECS)). As we shall show in Chapter 8, these problems are also P-complete for any κ ≥ 3.

Anderson and Mayr studied the High Degree Subgraph problem and found that the approximability of the problem exhibits a threshold type behavior. This behavior implies that below a certain value of the absolute performance ratio, it remains P-complete, even for fixed value κ.

The approximation techniques that we consider in this chapter apply to problems that can be formulated as profit/cost problems, defined as follows: given an n × n positive matrix C and two positive vectors p and b, find an n-bit vector ƒ such that its cost C . ƒ is bounded by b and its profit p . ƒ is maximized. The value of ƒ is computed incrementally. Starting from the 0 vector, the algorithm analyzes all possible extensions incrementing one bit at a time, until it covers the n bits. Notice that the new set can have twice the size of the previous one thus leading to an exponential size set in the last step. However, we can discard some of the new assignments according to some criteria to get polynomial size sets.

Consider the interval determined by the minimum current profit and the maximum one. In the interval partitioning technique this interval is divided into small subintervals. The interval partition also gives a partition of the current set of assignment. From each class the assignment that dominates all others is selected, and the remainder discarded.

In the separation technique, the criterion used to discard possible completions insures that there are no duplicated profit values, and the profits of no three partial solutions are within a factor of each other.

The rounding/scaling technique is used to deal with problems that are hard due to the presence of large weights in the problem instance. The technique modifies the problem instance in order to produce a second instance that has no large weights, and thus can be solved efficiently.

This monograph surveys the recent developments in the field of parallel approximability to combinatorial optimization problems. The book is designed not specifically as a textbook, but as a sedimentation of a great deal of research done on this specific topic. Accordingly, we do not include a collection of exercises, but throughout the text we comment about different open research problems in the field. The monograph can easily be used as a support for advanced courses in either parallel or sequential algorithms, or for a graduate course on the topic.

In Chapter 1, we motivate the topic of parallel approximability, using an example. In the same chapter we keep the reasoning at a very intuitive level, and we survey some of the techniques and concepts that will be assumed throughout the book, like randomization and derandomization. In Chapter 2 we give a formal presentation of algorithms for the Parallel Random Access Machines model. We survey some of the known results on sequential approximability, and introduce the basic definitions and classes used in the theory of Parallel Approximability. The core of the book, Chapters 3 to 7, forms a collection of paradigms to produce parallel approximations. The word “paradigm” has a different semantic from the one used by Kuhn, in the sense that we use paradigm as a particular property or technique that can be used to give approximations in parallel for a whole class of problems. Chapter 3 presents the use of extremal graph theory, to obtain approximability to some graph problems. Chapter 4 presents parallel approximability results for profit/cost problems using the rounding, interval partitioning and separation techniques.