Introduction

A neural network is a network of subunits, called “formal neurons,” processing input signals to output signals, which are coupled through “synapses.” The synapses are the nodes of this particular kind of network, the “strength” of which, called the synaptic weight, codes the “knowledge” of the network and controls the processing of the signals.

Let us be clear at the outset that the resemblance of a formal neuron to an animal-brain neuron is not well established, but that is not essential at this stage of abstraction. However, this terminology can be justified to some extent, and it is by now widely accepted, as discussed later. Chapter 8 develops this issue.

Also, there is always a combination of two basic motivations for dealing with neural networks - one attempting to model actual biological nervous systems, the other being content with implementation of neural-like systems on computers. Every model lies between these two requirements – the first constraining the modeling, the second allowing more freedom in the choice of a particular representation.

There are so many different versions of neural networks that it is difficult to find a common framework to unify all of them at a rather concrete level. But one can regard neural networks as dynamical systems (discrete or continuous), the states of which are the signals, and the controls of which are the synaptic weights, which regulate the flux of transmitters from one neuron to another.

This book is devoted to some mathematical methods that arise in two domains of artificial intelligence: neural networks and qualitative physics (which here we shall call “qualitative analysis”). These two topics are treated independently. Rapid advances in these two areas have left unanswered many mathematical questions that should motivate and challenge a wide range of mathematicians. The mathematical techniques that I choose to present in this book are as follows:

1. control and viability theory in neural networks and cognitive systems, regarded as dynamical systems controlled by synaptic matrices.

2. set-valued analysis, which plays a natural and crucial role in qualitative analysis and simulation by emphasizing properties common to a class of problems, data, and solutions. Set-valued analysis also underlies mathematical morphology, which provides useful techniques for image recognition.

This allows us to present in a unified way many examples of neural networks and to use several results on the control of linear and nonlinear systems to obtain a learning algorithm of pattern-classification problems (including time series in forecasting), such as the back-propagation formula, in addition to learning algorithms concerning feedback-regulation laws for solutions to control systems subject to state constraints (inverse dynamics).

Introduction

We investigate in this chapter the case of linear neural networks, named associative memories by T. Kohonen (Figure 3.1). We begin by specializing the heavy algorithm we have studied in the general case of adaptive systems to the case of neural networks, where controls are matrices. It shows how to modify the last synaptic matrix that has learned a set of patterns for learning a new pattern without forgetting the previous patterns.

Because right-inverses of tensor products are tensor products of right-inverses, we observe that the heavy algorithm has a Hebbian character: The heavy algorithm states that the correction of a synaptic matrix during learning is the product of activities in both presynaptic and postsynaptic neurons. This added feature that plain vectors do not enjoy justifies the specifics of systems controlled by matrices instead of vectors.

We then proceed with associative memories with postprocessing, with multilayer and continuous-layer associative memories. We conclude this chapter with associative memories with gates, where the synaptic matrices link conjuncts (i.e., subsets) of presynaptic neurons with each postsynaptic neuron. They allow computation of any Boolean function. They require a short presentation of fuzzy sets.

Introduction

We present in this appendix the tests of the external and internal algorithms conducted by Nicolas Seube at Thomson-SINTRA to control the tracking of an exosystem by an autonomous underwater vehicle (AUV). This system has three degrees of freedom (planar motion), six state variables (positions, heading, and their derivatives), and three controls (thruster forces). The dynamics of an AUV are highly nonlinear, coupled, and sometimes fully interacting, thus making it difficult to control by the usual methods. Moreover, the dynamics are poorly known, because only approximate hydrodynamic models are available for realworld vehicles. Finally, we need to involve the marine currents that can significantly perturb the dynamics of the AUV.

In addition, the problem of controlling an AUV cannot be linearized about a single velocity axis because all vehicle velocities usually have the same range; conventional linear control techniques clearly are unable to provide adequate performance by the control systems.

We shall present three different learning rules that address the problems of uniform minimization and adaptive learning by a set-valued feedback control map. The three classes of algorithms presented here have been tested in the case of the Japanese Dolphin AUV.

In particular, it is shown that the gradient step size is critical for the external rule, but is not critical for the uniform external algorithm. The latter could also be applied to pattern-classification problems, and may provide a plausible alternative method to stochastic gradient algorithms.

Introduction

We propose in this chapter a speculative dynamical description of an abstract cognitive system that goes beyond neural networks to attempt to take into account some features of nervous systems and, in particular, adaptations to environmental constraints. This personal viewpoint of the author is but one of the several attempts to model cognitive processes mathematically. It is presented primarily for the purpose of stirring up reaction and prompting further research involving other techniques and other approaches to this wide field.

Before we look at the evolution of nervous systems for useful suggestions regarding the means they have used to master more and more complex cognitive faculties, we shall start from the fact that an organism must adapt to environmental constraints by perceiving them and recognizing them through “metaphors” with what we shall call “conceptual controls.” This problem of adaptation is not dealt with explicitly in most studies of neural networks. This chapter is devoted to highlighting the roles of cognitive systems in this process.

The variables of the cognitive system are described by its state and a regulatory control (conceptual control). The state of the system (henceforth called the sensorimotor state) is described by

• the state and the variations of the environment on which the cognitive system acts,

• the state of cerebral motor activity of the cognitive system, which guides an individual's action on the environment.