Basic concepts

Approximate string matching, also called “string matching allowing errors,” is the problem of finding a pattern p in a text T when a limited number k of differences is permitted between the pattern and its occurrences in the text.

From the many existing models defining a “difference,” we focus on the most popular one, called Levenshtein distance or edit distance [Lev65]. Other more complex models exist, especially in computational biology, but the edit distance model has received the most attention and the most effective algorithms have been developed for it. Some of these algorithms can be extended to more complex models.

Under edit distance, one difference equals one edit operation: a character insertion, deletion, or substitution. That is, the edit distance between two strings x and y, ed(x, y), is the minimum number of edit operations required to convert x into y, or vice versa. For example, ed(annual, annealing) = 4. The approximate string matching problem becomes that of finding all occurrences in T of every p′ that satisfies ed(p,p′) ≤ k. To ensure a linear size output it is customary to report only the starting or ending positions of the occurrences.

Note that the problem only makes sense for 0 < k < m, because otherwise every text substring of length m can be converted into p by substituting the m characters. The case k = 0 corresponds to exact string matching. We call α = k/m the “error level.” It gives a measure of the “fraction” of the pattern that can be altered.

We concentrate on algorithms that are the fastest in the cases that are likely to be of use in some foreseeable application, particularly text retrieval and computational biology. In particular, α < 1/2 in most cases of interest.

We present four approaches. The first approach, which is also the oldest and most flexible, adapts a dynamic programming algorithm that computes edit distance.

We have seen in Chapter 6 that not all tasks can be implemented by a single CNN template; the XOR function is a typical example.

There are many tasks which are solved by applying several templates, or by applying one template several times. If we consider a template as an instruction with well-defined input and output, we can define a CNN subroutine or function (as in C-like languages) when applying several templates. We can build up processes and complete programs from functions and other instructions.

We define a subroutine by specifying the following items:

• the input/output parameters,

• the global task,

• the informal description of the algorithm,

• the CNN implementation.

In this chapter the CNN implementation is given by three equivalent ways:

the hardware schematics, supposing each CNN template (placed in the CNN Software Library) is implemented by a separate device containing discrete hard-wired cells and additional local (cell by cell) and global devices.

a flow diagram of the CNN algorithm, and

a list of consecutive instructions, henceforth called a program, written in a simple vocabulary involving the CNN analog and logic operations, henceforth called an analogic CNN language, or simply “α” language.

An Alpha Compiler is supposed to exist to translate the code into executable programs on CNN chips. We will describe this process later in Chapter 9.

Indeed, we follow the theory and practice of digital computers.

Scenario

Recent history of the electronic and computer industry can be viewed as three waves of revolutionary processes. The first revolution, making cheap computing power available via microprocessors in the 1970s, led to the PC industry of the 1980s. The cheap laser and fiber optics, which resulted in cheap bandwidth at the end of the 1980s, led to the Internet industry of the 1990s. The third wave, the sensor revolution at the end of the 1990s, will also provide for a new industry. Sensor revolution means that cheap sensor and MEMS (micro-electro-mechanical system) arrays are proliferating in almost all the conceivable forms. Artificial eyes, nose, ears, taste, and somatosensory devices as well as sensing all physical, chemical, and biological parameters, together with microactuators, etc. are becoming commodities. Thousands and millions of generically analog signals are produced waiting for processing. A new computing paradigm is needed. The cited technology assessment reads:

For processing analog array signals, the revolutionary Analogic Cellular Computer paradigm is a major candidate. The core of this computer is a Cellular Nonlinear/neural network (CNN), an array of analog dynamic processors or cells. The computer architecture is the CNN Universal Machine, with its various physical implementations. At the same time, Analogic CNN computers mimic the anatomy and physiology of many sensory and processing organs with an additional capability of stored programmability.

Appendix A: TEMLIB, a CNN Template Library

Under the name TEMLIB, within the Software Library for analogic cellular (CNN) computers, a set of fairly standard types of CNN template data are contained. The template names in TEMLIB can be used in the template and algorithm simulators defined in Appendix B.

Appendix B: TEMPO, template optimization tools

Under the name TEMMASTER, a student version of a program for template optimization and design is available. It is used mainly for Boolean CNN and for robust template design.

Appendix C: CANDY, a simulator for CNN templates and analogic CNN algorithms

Under the name CANDY (CNN Analogic Dynamics), a student version of a software simulator system is available. Multi-layer CNN templates as well as analogic CNN algorithms (defined on the CNN Universal Machine having a one layer, first-order dynamics CNN core) can be simulated. An easy to use Template Runner program as well as a high-level language compiler (Alpha) help the user to analyze complex spatial-temporal dynamics easily and with expressive visualization tools.

During the first years after the introduction of the CNN paradigm, many templates were designed by cut-and-try techniques, playing with a few nonzero template elements, and using a simulator to calculate the CNN dynamics. After a while, some systematic design methodologies emerged. Today several methods are available for generating CNN templates or algorithms, even for complex tasks.

Various design techniques

The main classes of design techniques are as follows:

• systematic methods for binary I/O function via Boolean description and decomposition techniques using uncoupled CNN (see Chapters 5, 6, 7)

• systematic methods for binary I/O function using coupled CNN (see also Chapter 12)

• global optimization techniques as parameter optimization

• genetic algorithms for designing the template elements/synaptic weights

• matching with the spatially discrete representations of partial differential equations (PDEs)

• matching with some neuromorphic models of a living organism, typically the nervous system, in particular the visual pathway of vertebrates (see Chapter 16)

• fuzzy design techniques

• neural network techniques

• matching with existing 2D or 3D algorithms, including techniques in signal processing, telecommunications, adaptive control, nonlinear spatio-temporal dynamical systems, etc.

We have to emphasize, however, that, in spite of the many design techniques, new methods are emerging day by day based on the intuition and skill of the designers. A good example for this is a recent method using active waves applied for a while and combining/colliding with other waves, as well as a method in which a wave metric is used for complex pattern recognition tasks.

The complete stability phenomenon

Our main objective in this section is to derive and prove a general theorem which unifies all of the CNN templates presented in the preceding chapter, and numerous others, under one umbrella. In particular, the mathematical analyses presented for all templates in Chapter 5 could be greatly simplified. We did not present this fundamental theorem earlier for pedagogical reasons: it is essential for the uninitiated students of CNN to understand and appreciate the fundamental notion and power of the shifted DP plots and their dynamic routes.

We have been thoroughly exposed to these rather simple concepts and have learned to exploit the dynamic routes not only for the analysis of the nonlinear dynamics (i.e., transient and asymptotic behaviors), but also as a highly intuitive and potent design tool for deriving optimum and robust CNN templates.

The alert students who have gone over the previous chapter would have recognized that, except for a degenerate case, no matter what the initial conditions are, the solution always converges to a globally asymptotically stable and hence unique equilibrium point (monostable case), or to one of two locally stable equilibrium points (bistable case). Although this latter “convergence” property is rather unusual for nonlinear dynamic circuits having multiple equilibria, it is a gift that nature (i.e., the physical laws) has endowed upon an important class of CNNs on which all current nonlinear information processing applications are based.

Chapter 2

Exercise 2.1 (Simple morph)

Given: two gray-scale images: P1 and P2

Input: U(t) = P1

Initial state: X(0) = P2

Boundary conditions: white frame

Output: Y(t) = a transition from P2 to P1.

Task

Design a single template, which implements this transition.

Example

Exercise 2.2 (Hexagonal neighborhood)

The standard CNN definition specifies that the cells form a rectangular grid. Anther feasible form could be a hexagonal grid.

Task

Give a formula for the side length and the area of a hexagon (measured in cells) in the case of a hexagonal cell grid, when the sphere of influence equals r.

Exercise 2.3 (Triangular neighborhood)

The standard CNN definition specifies that the cells form a rectangular grid. There are only three possibilities to cover the plane. These are rectangular, hexagonal, and triangular.

Task

Give a formula for the area of a triangle in the case of a triangular cell grid, when the sphere of influence equals r.

Chapter 3

Exercise 3.1 (Separate connected objects)

The problem to be solved is to separate connected objects. The example shows a test image where objects are all similar in size. All objects should be separated but their sizes must be preserved.

Digital technology has the key advantage that if a few building blocks are implemented then any complex system can be built from these by

• wiring and

• programming.

Moreover, most of the digital building blocks are placed in a regular arrangement: a simple block is repeated many times in a matrix arrangement (e.g. memories, PLAs, etc.).

The CNN core and the CNN Universal Machine architecture, containing also analog building blocks, possess the very same properties. Due to their special nature, however, they have orders of magnitude advantages in speed, power, and area (SPA) in some standard physical implementations. In many applications, like image flow computing, this advantage might be mission critical.

As a revolutionary feature, stored programmability can be introduced in the analog domain as well. This makes it possible to fabricate visual microprocessors.

In what follows, first, we show the building blocks and their simple CMOS implementation examples, without going into the details of their design issues. The emulated digital implementation will be only briefly reviewed. As to this and the optical implementation, we refer to the literature.

As a summary: using only six simple circuit building blocks, namely:

• resistor,

• capacitor,

• switch,

• VCCS (Voltage Controlled Current Source),

• logic register, and

• logic gate,

the most complex CNN array computer chip can be built in a VLSI friendly, regular structure.

Next, the visual microprocessor and its computational infrastructure is described.