Introduction

There has recently been much interest in “artificial neural networks,” machines (or models of computation) based loosely on the ways in which the brain is believed to work. Neurobiologists are interested in using these machines as a means of modeling biological brains, but much of the impetus comes from their applications. For example, engineers wish to create machines that can perform “cognitive” tasks, such as speech recognition, and economists are interested in financial time series prediction using such machines.

In this chapter we focus on individual “artificial neurons” and feed-forward artificial neural networks. We are particularly interested in cases where the neurons are linear threshold neurons, sigmoid neurons, polynomial threshold neurons, and spiking neurons. We investigate the relationships between types of artificial neural network and classes of Boolean function. In particular, we ask questions about the type of Boolean functions a given type of network can compute, and about how extensive or expressive the set of functions so computable is.

Artificial Neural Networks

Introduction

It appears that one reason why the human brain is so powerful is the sheer complexity of connections between neurons. In computer science parlance, the brain exhibits huge parallelism, with each neuron connected to many other neurons. This has been reflected in the design of artificial neural networks. One advantage of such parallelism is that the resulting network is robust: in a serial computer, a single fault can make computation impossible, whereas in a system with a high degree of parallelism and many computation paths, a small number of faults may be tolerated with little or no upset to the computation.

Introduction

A fundamental objective of cryptography is to enable two persons to communicate over an insecure channel (a public channel such as the internet) in such a way that any other person is unable to recover their message (called the plaintext) from what is sent in its place over the channel (the ciphertext). The transformation of the plaintext into the ciphertext is called encryption, or enciphering. Encryption-decryption is the most ancient cryptographic activity (ciphers already existed four centuries b.c.), but its nature has deeply changed with the invention of computers, because the cryptanalysis (the activity of the third person, the eavesdropper, who aims at recovering the message) can use their power.

The encryption algorithm takes as input the plaintext and an encryption key KE, and it outputs the ciphertext. If the encryption key is secret, then we speak of conventional cryptography, of private key cryptography, or of symmetric cryptography. In practice, the principle of conventional cryptography relies on the sharing of a private key between the sender of a message (often called Alice in cryptography) and its receiver (often called Bob). If, on the contrary, the encryption key is public, then we speak of public key cryptography. Public key cryptography appeared in the literature in the late 1970s.

The first part of the book, “Algebraic Structures,” deals with compositions and decompositions of Boolean functions.

A set F of Boolean functions is called complete if every Boolean function is a composition of functions from F; it is a clone if it is composition-closed and contains all projections. In 1921, E. L. Post found a completeness criterion, that is, a necessary and sufficient condition for a set F of Boolean functions to be complete. Twenty years later, he gave a full description of the lattice of Boolean clones. Chapter 1, by Reinhard Pöschel and Ivo Rosenberg, provides an accessible and self-contained discussion of “Compositions and Clones of Boolean Functions” and of the classical results of Post.

Functional decomposition of Boolean functions was introduced in switching theory in the late 1950s. In Chapter 2, “Decomposition of Boolean Functions,” Jan C. Bioch proposes a unified treatment of this topic. The chapter contains both a presentation of the main structural properties of modular decompositions and a discussion of the algorithmic aspects of decomposition.

Part II of the collection covers topics in logic, where Boolean models find their historical roots.

In Chapter 3, “Proof Theory,” Alasdair Urquhart briefly describes the more important proof systems for propositional logic, including a discussion of equational calculus, of axiomatic proof systems, and of sequent calculus and resolution proofs. The author compares the relative computational efficiency of these different systems and concludes with a presentation of Haken's classical result that resolution proofs have exponential length for certain families of formulas.

Introduction

This chapter explores the learnability of Boolean functions. Broadly speaking, the problem of interest is how to infer information about an unknown Boolean function given only information about its values on some points, together with the information that it belongs to a particular class of Boolean functions. This broad description can encompass many more precise formulations, but here we focus on probabilistic models of learning, in which the information about the function value on points is provided through its values on some randomly drawn sample, and in which the criteria for successful “learning” are defined using probability theory. Other approaches, such as “exact query learning” (see [1, 18, 20] and Chapter 7 in this volume, for instance) and “specification,” “testing,” or “learning with a helpful teacher” (see [12, 4, 16, 21, 26]) are possible, and particularly interesting in the context of Boolean functions. Here, however, we focus on probabilistic models and aim to give a fairly thorough account of what can be said in two such models.

In the probabilistic models discussed, there are two separate, but linked, issues of concern. First, there is the question of how much information is needed about the values of a function on points before a good approximation to the function can be found. Second, there is the question of how, algorithmically, we might find a good approximation to the function. These two issues are usually termed the sample complexity and computational complexity of learning.

Introduction

Let f: Dn → R be a finite function: that is, D and R are finite sets. Such a function can be represented by the table of all (a, f (a)), a Є Dn, which always has an exponential size of ∣D∣n. Therefore, we are interested in representations that for many important functions are much more compact. The best-known representations are circuits and decision diagrams. Circuits are a hardware model reflecting the sequential and parallel time to compute f (a) froma (see Chapter 11). Decision diagrams (DDs), also called branching programs (BPs), are nonuniform programs for computing f (a) from a based on only two types of instructions represented by nodes in a graph (see also Figure 10.1):

• Decision nodes: depending on the value of some input variable xi the next node is chosen.

• Output nodes (also called sinks): a value from R is presented as output.

A decision diagram is a directed acyclic graph consisting of decision nodes and output nodes. Each node v represents a function fv defined in the following way. Let a = (a1, …, an) Є Dn. At decision nodes, choose the next node as described before. The value of fv(a) is defined as the value of the output node that is finally reached when starting at v. Hence, for each node each input a Є Dn activates a unique computation path that we follow during the computation of fv(a). An edge e = (v,w) of the diagram is called activated by a if the computation path starting at v runs via e.