In all of the preceding chapters, we have examined the identification problem for block-oriented systems of various forms, that are characterized by a one-dimensional input process. In numerous applications, we confront the problem of identifying a system that has multiple inputs and multiple interconnecting signals. The theory and practical algorithms for identification of multivariate linear systems have been thoroughly examined in the literature [332]. On the other hand, the theory of identification of multivariate nonlinear systems has been far less explored. This is mainly due to the mathematical and computational difficulties appearing in multivariate problems. In this chapter, we examine some selected multivariate nonlinear models that are natural generalizations of the previously introduced block-oriented connections. An apparent curse of dimensionality that takes place in high-dimensional estimation problems forces us to focus on low-dimensional counterparts of the classical block-oriented structures. In particular, we examine a class of additive models, which provides a parsimonious representation for multivariate systems. Indeed, we show that the additive systems provide simple and interpretable structures, which also give a reasonable trade-off between the systematic modeling error and the estimation error of an identification algorithm. The theory of finding an optimal additive model is examined.

Multivariate nonparametric regression

As in all of the previous chapters, we will make use of the notion of a regression function.

The aim of this book is to show that the nonparametric regression can be applied successfully to nonlinear system identification. It gathers what has been done in the area so far and presents main ideas, results, and some new recent developments.

The study of nonparametric regression estimation began with works published by Cencov, Watson, and Nadaraya in the 1960s. The history of nonparametric regression in system identification began about ten years later. Such methods have been applied to the identification of composite systems consisting of nonlinear memoryless systems and linear dynamic ones. Therefore, the approach is strictly connected with so-called block-oriented methods developed since Narendra and Gallman's work published in 1966. Hammerstein and Wiener structures are most popular and have received the greatest attention in numerous applications. Fundamental for nonparametric methods is the observation that the unknown characteristic of the nonlinear subsystem or its inverse can be represented as regression functions.

In terms of the a priori information, standard identification methods and algorithms work when it is parametric, that is, when our knowledge about the system is rather large; for example, when we know that the nonlinear subsystem has a polynomial characteristic. In this book, the information is much smaller, nonparametric. The mentioned characteristic can be, for example, any integrable or bounded or, even, any Borel function.

Introduction

Mutual exclusion is a fundamental problem in distributed computing systems. Mutual exclusion ensures that concurrent access of processes to a shared resource or data is serialized, that is, executed in a mutually exclusive manner. Mutual exclusion in a distributed system states that only one process is allowed to execute the critical section (CS) at any given time. In a distributed system, shared variables (semaphores) or a local kernel cannot be used to implement mutual exclusion. Message passing is the sole means for implementing distributed mutual exclusion. The decision as to which process is allowed access to the CS next is arrived at by message passing, in which each process learns about the state of all other processes in some consistent way. The design of distributed mutual exclusion algorithms is complex because these algorithms have to deal with unpredictable message delays and incomplete knowledge of the system state. There are three basic approaches for implementing distributed mutual exclusion:

1. Token-based approach.

2. Non-token-based approach.

3. Quorum-based approach.

In the token-based approach, a unique token (also known as the PRIVILEGE message) is shared among the sites. A site is allowed to enter its CS if it possesses the token and it continues to hold the token until the execution of the CS is over. Mutual exclusion is ensured because the token is unique.

Abstraction and advantages

Distributed shared memory (DSM) is an abstraction provided to the programmer of a distributed system. It gives the impression of a single monolithic memory, as in traditional von Neumann architecture. Programmers access the data across the network using only read and write primitives, as they would in a uniprocessor system. Programmers do not have to deal with send and receive communication primitives and the ensuing complexity of dealing explicitly with synchronization and consistency in the messagepassing model. The DSM abstraction is illustrated in Figure 12.1. A part of each computer's memory is earmarked for shared space, and the remainder is private memory. To provide programmers with the illusion of a single shared address space, a memory mapping management layer is required to manage the shared virtual memory space.

DSM has the following advantages:

1. Communication across the network is achieved by the read/write abstraction that simplifies the task of programmers.

2. A single address space is provided, thereby providing the possibility of avoiding data movement across multiple address spaces, and simplifying passing-by-reference and passing complex data structures containing pointers.

3. If a block of data needs to be moved, the system can exploit locality of reference to reduce the communication overhead.

4. DSM is often cheaper than using dedicated multiprocessor systems, because it uses simpler software interfaces and off-the-shelf hardware.

5. There is no bottleneck presented by a single memory access bus.

6. […]

Introduction

The idea of self-stabilization in distributed computing was first proposed by Dijkstra in 1974. The concept of self-stabilization is that, regardless of its initial state, the system is guaranteed to converge to a legitimate state in a bounded amount of time by itself without any outside intervention. A non-self-stabilizing system may never reach a legitimate state or it may reach a legitimate state only temporarily. The main complication in designing a self-stabilizing distributed system is that nodes do not have a global memory that they can access instantaneoulsy. Each node must make decisions based on the local knowledge available to it and actions of all nodes must achieve a global ojective.

The definition of legitimate and illegitimate states depends on the particular application. Generally, all illegitimate states are defined to be those states which are not legitimate states. Dijkstra also gave an example of the concept of self-stabilization using a self-stabilizing token ring system. For any given token ring when there are multiple tokens or there is no token, then such global states are known as illegitimate states. When we consider a distributed system where a large number of systems are widely distributed and communicate with each other using message passing or shared memory approach, there is a possibility for these systems to go into an illegitimate state, for example, if a message is lost. The concept of self-stabilization can help us recover from such situations in distributed system.

Introduction

In distributed processing systems, a problem is typically solved in a distributed manner with the cooperation of a number of processes. In such an environment, inferring if a distributed computation has ended is essential so that the results produced by the computation can be used. Also, in some applications, the problem to be solved is divided into many subproblems, and the execution of a subproblem cannot begin until the execution of the previous subproblem is complete. Hence, it is necessary to determine when the execution of a particular subproblem has ended so that the execution of the next subproblem may begin. Therefore, a fundamental problem in distributed systems is to determine if a distributed computation has terminated.

The detection of the termination of a distributed computation is non-trivial since no process has complete knowledge of the global state, and global time does not exist. A distributed computation is considered to be globally terminated if every process is locally terminated and there is no message in transit between any processes. A “locally terminated” state is a state in which a process has finished its computation and will not restart any action unless it receives a message. In the termination detection problem, a particular process (or all of the processes) must infer when the underlying computation has terminated.

When we are interested in inferring when the underlying computation has ended, a termination detection algorithm is used for this purpose.

Introduction

A fundamental concern in building a secure distributed system is the authentication of local and remote entities in the system. In a distributed system, the hosts communicate by sending and receiving messages over the network. Various resources (such as files and printers) distributed among the hosts are shared across the network in the form of network services provided by servers. The entities in a distributed system, such as users, clients, servers, and processes, are collectively referred to as principals. A distributed system is susceptible to a variety of threats mounted by intruders as well as legitimate users of the system.

In an environment where a principal can impersonate another principal, principals must adopt a mutually suspicious attitude toward one another and authentication becomes an important requirement. Authentication is a process by which one principal verifies the identity of another principal. For example, in a client–server system, the server may need to authenticate the client. Likewise, the client may want to authenticate the server so that it is assured that it is talking to the right entity. Authentication is needed for both authorization and accounting functions. In one-way authentication, only one principal verifies the identity of the other principal, while in mutual authentication both communicating principals verify each other's identity. A user gains access to a distributed system by logging on to a host in the system.