Introduction

Peer-to-peer (P2P) network systems use an application-level organization of the network overlay for flexibly sharing resources (e.g., files and multimedia documents) stored across network-wide computers. In contrast to the client–server model, any node in a P2P network can act as a server to others and, at the same time, act as a client. Communication and exchange of information is performed directly between the participating peers and the relationships between the nodes in the network are equal. Thus, P2P networks differ from other Internet applications in that they tend to share data from a large number of end users rather than from the more central machines and Web servers. Several well known P2P networks that allow P2P file-sharing include Napster, Gnutella, Freenet, Pastry, Chord, and CAN.

Traditional distributed systems used DNS (domain name service) to provide a lookup from host names (logical names) to IP addresses. Special DNS servers are required, and manual configuration of the routing information is necessary to allow requesting client nodes to navigate the DNS hierarchy. Further, DNS is confined to locating hosts or services (not data objects that have to be a priori associated with specific computers), and host names need to be structured as per administrative boundary regulations. P2P networks overcome these drawbacks, and, more importantly, allow the location of arbitrary data objects.

In this chapter, we first study a methodical framework in which distributed algorithms can be classified and analyzed. We then consider some basic distributed graph algorithms. We then study synchronizers, which provide the abstraction of a synchronous system over an asynchronous system. Finally, we look at some practical graph problems, to appreciate the necessity of designing efficient distributed algorithms.

Topology abstraction and overlays

The topology of a distributed system can be typically viewed as an undirected graph in which the nodes represent the processors and the edges represent the links connecting the processors. Weights on the edges can represent some cost function we need to model in the application. There are usually three (not necessarily distinct) levels of topology abstraction that are useful in analyzing the distributed system or a distributed application. These are now described using Figure 5.1. To keep the figure simple, only the relevant end hosts participating in the application are shown. The WANs are indicated by ovals drawn using dashed lines. The switching elements inside the WANs, and other end hosts that are not participating in the application, are not shown even though they belong to the physical topological view. Similarly, all the edges connecting all end hosts and all edges connecting to all the switching elements inside the WANs also belong to the physical topology view even though only some edges are shown.

Background

The field of distributed computing covers all aspects of computing and information access across multiple processing elements connected by any form of communication network, whether local or wide-area in the coverage. Since the advent of the Internet in the 1970s, there has been a steady growth of new applications requiring distributed processing. This has been enabled by advances in networking and hardware technology, the falling cost of hardware, and greater end-user awareness. These factors have contributed to making distributed computing a cost-effective, high-performance, and fault-tolerant reality. Around the turn of the millenium, there was an explosive growth in the expansion and efficiency of the Internet, which was matched by increased access to networked resources through the World Wide Web, all across the world. Coupled with an equally dramatic growth in the wireless and mobile networking areas, and the plummeting prices of bandwidth and storage devices, we are witnessing a rapid spurt in distributed applications and an accompanying interest in the field of distributed computing in universities, governments organizations, and private institutions.

Advances in hardware technology have suddenly made sensor networking a reality, and embedded and sensor networks are rapidly becoming an integral part of everyone's life – from the home network with the interconnected gadgets to the automobile communicating by GPS (global positioning system), to the fully networked office with RFID monitoring. In the emerging global village, distributed computing will be the centerpiece of all computing and information access sub-disciplines within computer science.

Problem definition

Agreement among the processes in a distributed system is a fundamental requirement for a wide range of applications. Many forms of coordination require the processes to exchange information to negotiate with one another and eventually reach a common understanding or agreement, before taking application-specific actions. A classical example is that of the commit decision in database systems, wherein the processes collectively decide whether to commit or abort a transaction that they participate in. In this chapter, we study the feasibility of designing algorithms to reach agreement under various system models and failure models, and, where possible, examine some representative algorithms to reach agreement.

We first state some assumptions underlying our study of agreement algorithms:

• Failure models Among the n processes in the system, at most f processes can be faulty. A faulty process can behave in any manner allowed by the failure model assumed. The various failure models – fail-stop, send omission and receive omission, and Byzantine failures – were discussed in Chapter 5. Recall that in the fail-stop model, a process may crash in the middle of a step, which could be the execution of a local operation or processing of a message for a send or receive event. In particular, it may send a message to only a subset of the destination set before crashing. In the Byzantine failure model, a process may behave arbitrarily.

• […]

Introduction

Distributed systems today are ubiquitous and enable many applications, including client–server systems, transaction processing, the World Wide Web, and scientific computing, among many others. Distributed systems are not fault-tolerant and the vast computing potential of these systems is often hampered by their susceptibility to failures. Many techniques have been developed to add reliability and high availability to distributed systems. These techniques include transactions, group communication, and rollback recovery. These techniques have different tradeoffs and focus. This chapter covers the rollback recovery protocols, which restore the system back to a consistent state after a failure.

Rollback recovery treats a distributed system application as a collection of processes that communicate over a network. It achieves fault tolerance by periodically saving the state of a process during the failure-free execution, enabling it to restart from a saved state upon a failure to reduce the amount of lost work. The saved state is called a checkpoint, and the procedure of restarting from a previously checkpointed state is called rollback recovery. A checkpoint can be saved on either the stable storage or the volatile storage depending on the failure scenarios to be tolerated.

In distributed systems, rollback recovery is complicated because messages induce inter-process dependencies during failure-free operation. Upon a failure of one or more processes in a system, these dependencies may force some of the processes that did not fail to roll back, creating what is commonly called a rollback propagation.

In a distributed system, processes make local decisions based on their limited view of the system state. A process learns of new facts when it receives messages from other processes, and can reason only with the additional knowledge available to it. This chapter provides a formal framework in which it is easier to understand the role of knowledge in the system, and how processes can reason with such knowledge. The first three sections are based on the book by Fagin et al. The logic of knowledge, classically termed as epistemic logic, is the formal logical analysis of reasoning about knowledge. Epistemic knowledge first received much attention from philosophers in the mid-twentieth century.

The muddy children puzzle

Consider the classical “muddy children” puzzle of Halpern and Moses and Halpern and Fagin. Imagine there are n children who return from playing outdoors, and k, k ≥ 1, of the n children have mud on their foreheads. Let Ψ denote the fact “at least one child has a muddy forehead.” Assume that each child can see all other children and their foreheads, but not their own forehead. We also assume that the children are intelligent and truthful, and answer any question asked of them, simultaneously. We now consider two scenarios.

Inter-process communication via message-passing is at the core of any distributed system. In this chapter, we will study non-FIFO, FIFO, causal order, and synchronous order communication paradigms for ordering messages. We will then examine protocols that provide these message orders. We will also examine several semantics for group communication with multicast – in particular, causal ordering and total ordering. We will then look at how exact semantics can be specified for the expected behavior in the face of processor or link failures. Multicasts are required at the application layer when superimposed topologies or overlays are used, as well as at the lower layers of the protocol stack. We will examine some popular multicast algorithms at the network layer. An example of such an algorithm is the Steiner tree algorithm, which is useful for setting up multi-party teleconferencing and videoconferencing multicast sessions.

Notation

As before, we model the distributed system as a graph (N, L). The following notation is used to refer to messages and events:

• When referring to a message without regard for the identity of the sender and receiver processes, we use mi. For message mi, its send and receive events are denoted as si and ri, respectively.

• More generally, send and receive events are denoted simply as s and r. When the relationship between the message and its send and receive events is to be stressed, we also use M, send(M), and receive(M), respectively.

An alphabet is a set of symbols. Some alphabets are infinite, such as the set of real numbers or the set of complex numbers. Usually, we will be interested in finite alphabets. A sequence is a string of symbols from a given alphabet. A sequence may be of infinite length. An infinite sequence may be periodic or aperiodic; infinite aperiodic sequences may become periodic after some initial segment. Any infinite sequence that we will consider has a fixed beginning, but is unending. It is possible, however, that an infinite sequence has neither a beginning nor an end.

A finite sequence is a string of symbols of finite length from the given alphabet. The blocklength of the sequence, denoted n, is the number of symbols in the sequence. Sometimes the blocklength is not explicitly specified, but is known implicitly only by counting the number of symbols in the sequence after that specific sequence is given. In other situations, the blocklength n is explicitly specified, and only sequences of blocklength n are under consideration.

There are a great many aspects to the study of sequences. One may study the structure and repetition of various subpatterns within a given sequence of symbols. Such studies do not need to presuppose any algebraic or arithmetic structure on the alphabet of the sequence.

Decoding large linear codes, in general, is a formidable task. For this reason, the existence of a practical decoding algorithm for a code can be a significant factor in selecting a code. Reed–Solomon codes – and other cyclic codes – have a distance structure that is closely related to the properties of the Fourier transform. Accordingly, many good decoding algorithms for Reed–Solomon codes are based on the Fourier transform.

The algorithms described in this chapter form the class of decoding algorithms known as “locator decoding algorithms”. This is the richest, the most interesting, and the most important class of algebraic decoding algorithms. The algorithms for locator decoding are quite sophisticated and mathematically interesting. The appeal of locator decoding is that a certain seemingly formidable nonlinear problem is decomposed into a linear problem and a well structured and straightforward nonlinear problem. Within the general class of locator decoding algorithms, there are many options, and a variety of algorithms exist.

Locator decoding can be used whenever the defining set of a cyclic code is a set of consecutive zeros. It uses this set of consecutive zeros to decode, and so the behavior of locator decoding is closely related to the BCH bound rather than to the actual minimum distance. Locator decoding, by itself, reaches the BCH radius, which is the largest integer smaller than half of the BCH bound, but reaches the packing radius of the code only if the packing radius is equal to the BCH radius.

Codes based on the two-dimensional Fourier transform can be decoded by methods analogous to those methods discussed in Chapter 3 for decoding codes that are based on the one-dimensional Fourier transform, such as the Reed–Solomon codes and other BCH codes. Just as for decoding one-dimensional cyclic codes, the task of finding the errors may be divided into two subtasks: finding the locations of the errors, then finding the magnitudes of the errors. In particular, the family of locator decoding algorithms introduces the notion of a bivariate error-locator polynomial, Λ(x, y), into one step of the decoding. However, we no longer have the neat equality that we had in one dimension between the degree of the locator polynomial Λ(x) and the number of its zeros. It now takes several polynomials to specify a finite number of bivariate zeros, and so, in two dimensions, we use the locator ideal instead of a locator polynomial as was used in one dimension. Now we have a neat equality between the number of zeros of the locator ideal {Λℓ(x, y) | ℓ = 1, …, L} and the area of the locator footprint.

The methods for decoding two-dimensional bicyclic codes can also be applied to the decoding of codes on curves. However, restricting a code to a curve in general increases the minimum distance. This means that the decoding algorithm must then be strengthened to reach the minimum distance of the code.

The geometric structure of a code over a finite field consists of a finite set of points in a finite vector space with the separation between any two points described by the Hamming distance between them. A linear code has the important property that every codeword sees the same pattern of other codewords surrounding it. A tabulation of all codeword weights provides a great deal of information about the geometry of a linear code. The number of codewords of weight ω in a linear code is equal to the number of codewords at distance ω from an arbitrary codeword.

Given any element, which we regard geometrically as a point, of the vector space, not necessarily a codeword, the task of decoding is to find the codeword that is closest to the given point. The (componentwise) difference between the given point and the closest codeword is the (presumed) error pattern. A bounded-distance decoder corrects all error patterns of weight not larger than some fixed integer τ, called the decoding radius. A bounded-distance decoder usually uses a decoding radius equal to the packing radius t, though this need not always be true. In this chapter, we shall study both the case in which τ is smaller than t and the case in which τ is larger than t, though the latter case has some ambiguity. In many applications, a bounded-distance decoder is preferred to a complete decoder.

This book began as notes for a collection of lectures given as a graduate course in the summer semester (April to July) of 1993 at the Swiss Federal Institute of Technology (ETH), Zurich, building on a talk that I gave in Brazil in 1992. Subsequently, in the fall of 1995 and again in the spring of 1998, the course notes were extensively revised and expanded for an advanced topics course in the Department of Electrical and Computer Engineering at the University of Illinois, from which course has evolved the final form of the book that appears here. These lectures were also given in various forms at Eindhoven University, Michigan Technological University, Binghamton University, Washington University, and the Technical University of Vienna. The candid reactions of some who attended these lectures helped me greatly in developing the unique (perhaps idiosyncratic) point of view that has evolved, a view that insists on integrating recent developments in the subject of algebraic codes on curves into the classical engineering framework and terminology of the subject of error-control codes. Many classes of error-control codes and their decoding algorithms can be described in the language of the Fourier transform. This approach merges much of the theory of error-control codes with the subject of signal processing, and makes the central ideas more readily accessible to the engineer.

An ideal in the ring F[x, y] is defined as any set of bivariate polynomials that satisfies a certain pair of closure conditions. Examples of ideals can arise in several ways. The most direct way to specify concretely an ideal in the ring F[x, y] is by giving a set of generator polynomials. The ideal is then the set of all polynomial combinations of the generator polynomials. These generator polynomials need not necessarily form a minimal basis. We may wish to compute a minimal basis for an ideal by starting with a given set of generator polynomials. We shall describe an algorithm, known as the Buchberger algorithm, for this computation. Thus, given a set of generator polynomials for an ideal, the Buchberger algorithm computes another set of generator polynomials for that ideal that is a minimal basis.

A different way of specifying an ideal in the ring F[x, y] is as a locator ideal for the nonzeros of a given bivariate polynomial. We then may wish to express this ideal in terms of a set of generator polynomials for it, preferably a set of minimal polynomials. Again, we need a way to compute a minimal basis, but starting now from a different specification of the ideal. We shall describe an algorithm, known as the Sakata algorithm, that performs this computation.

Now that we have studied the ring of bivariate polynomials and its ideals in some detail, we are nearly ready to resume our study of codes. In Chapter 10, we shall construct linear codes as vector spaces on plane curves. This means that the components of the vector space are indexed by the points of the curve. Over a finite field, a curve can have only a finite number of points, so a vector space on a curve in a finite field always has a finite dimension.

Before we can study codes on curves, however, we must study the curves themselves. In this chapter, we shall study curves over a finite field, specifically curves lying in a plane. Such curves, called planar curves or plane curves, are defined by the zeros of a bivariate polynomial. We shall also study vectors defined on curves – that is, vectors whose components are indexed by the points of the curve – and the weights of such vectors. Bounds on the weight of a vector on a curve will be given in terms of the pattern of zeros of its two-dimensional Fourier transform. These bounds are companions to the bounds on the weight of a vector on a line, which were given in Chapter 1, and bounds on the weight of an array on a plane, which were given in Chapter 4.

Given the field F, the vector space Fn exists for every positive integer n, and a linear code of blocklength n is defined as any vector subspace of Fn. Subspaces of dimension k exist in Fn for every integer k ≤ n. In fact, very many subspaces of dimension k exist. Each subspace has a minimum Hamming weight, defined as the smallest Hamming weight of any nonzero vector in that subspace. We are interested in those subspaces of dimension k over GF(q) for which the minimum Hamming weight is large.

In the study of Fn and its subspaces, there is no essential restriction on n. This remark is true in the finite field GF(q) just as in any other field. However, in the finite field, it is often useful to index components of the vector space GF(q)n by the elements of the field GF(q), when n = q, or by the nonzero elements of the field GF(q), when n = q − 1. The technique of using the elements of GF(q) to index the components of the vector over GF(q) is closely related both to the notion of a cyclic code and to polynomial evaluation. The essential idea of using nonzero field elements as indices can be extended to blocklength n = (q−1)2 by indexing the components of the vector υ by pairs of nonzero elements of GF(q). Then the vector υ is displayed more naturally as a two-dimensional array.

Codes on curves, along with their decoding algorithms, have been developed in recent years by using rather advanced topics of mathematics from the subject of algebraic geometry, which is a difficult and specialized branch of mathematics. The applications discussed in this book may be one of the few times that the somewhat inaccessible topics of algebraic geometry, such as the Riemann–Roch theorem, have entered the engineering literature. With the benefit of hindsight, we shall describe the codes in a more elementary way, without much algebraic geometry, emphasizing connections with bicyclic codes and the two-dimensional Fourier transform.

We shall discuss the hermitian codes as our primary example and the Klein codes as our secondary example. The class of hermitian codes, in its fullest form, is probably large enough to satisfy whatever needs may arise in communication systems of the near future. Moreover, this class of codes can be used to illustrate general methods that apply to other classes of codes. The Klein codes comprise a small class of codes over GF (8) with a rather rich and interesting structure, though probably not of practical interest.

An hermitian code is usually defined on a projective plane curve or on an affine plane curve. These choices for the definition are most analogous to the definitions of a doubly extended or singly extended Reed–Solomon code.