When the input to a quantizer is a sampled time series represented by x1, x2, x3, …, the quantization noise is a time series represented by ν1, ν2, ν3, … Suppose that the input time series is stationary and that its statistics satisfy the conditions for multivariable QT II (it would be sufficient that two–variable QT II conditions were satisfied for x1 and x2, x1 and x3, x1 and x4, and so forth, because of stationarity). As such, the quantization noise will be uncorrelated with the quantizer input, and the quantization noise will be white, i.e. uncorrelated over time. The PQN model applies. The autocorrelation function of the quantizer output will be equal to the autocorrelation function of the input plus the autocorrelation function of the quantization noise.

Fig. 20.1(a) is a sketch of an autocorrelation function of a quantizer input signal. Fig. 20.1(b) shows the autocorrelation function of the quantization noise when the PQN model applies. Fig. 20.1(c) shows the corresponding autocorrelation function of the quantizer output.

Corresponding to the autocorrelation functions of Fig. 20.1, the power spectrum of the quantizer output is equal to the power spectrum of the input plus the power spectrum of the quantization noise. This spectrum is flat, with a total power of q2/12.

For many years, rumors have been circulating in the realm of digital signal processing about quantization noise:

1. (a) the noise is additive and white and uncorrelated with the signal being quantized, and

2. (b) the noise is uniformly distributed between plus and minus half a quanta, giving it zero mean and a mean square of one–twelfth the square of a quanta.

Many successful systems incorporating uniform quantization have been built and placed into service worldwide whose designs are based on these rumors, thereby reinforcing their veracity. Yet simple reasoning leads one to conclude that:

1. (a) quantization noise is deterministically related to the signal being quantized and is certainly not independent of it,

2. (b) the probability density of the noise certainly depends on the probability density of the signal being quantized, and

3. (c) if the signal being quantized is correlated over time, the noise will certainly have some correlation over time.

In spite of the “simple reasoning,” the rumors are true under most circumstances, or at least true to a very good approximation. When the rumors are true, wonderful things happen:

1. (a) digital signal processing systems are easy to design, and

2. (b) systems with quantization that are truly nonlinear behave like linear systems.

In order for the rumors to be true, it is necessary that the signal being quantized obeys a quantizing condition. There actually are several quantizing conditions, all pertaining to the probability density function (PDF) and the characteristic function (CF) of the signal being quantized.

We have seen when discussing the sampling theorem in Section 2.2 that the conditions of the theorem are exactly met only if the signal being sampled is perfectly bandlimited. This is rarely the case, since perfect bandlimitedness implies that the signal cannot be time–limited. Such a signal can be easily defined mathematically, but measured signals are always time–limited, so the condition of the sampling theorem can be met only approximately. While the sinc function wave is theoretically bandlimited, its time truncated versions are not, so the sampling theorem can be applied only approximately. However, sampling theory proved to be very powerful despite its approximate applicability.

The situation is similar with the quantizing theorems. Bandlimitedness of the CF would imply that the PDF is not amplitude–limited. Since measured signals are always amplitude–limited, the quantization theorems can be applied only approximately. Similarly to the sampling theorem, this does not prevent the quantizing theorems from being very powerful in many applications.

Most distributions that are applied in practice, like the Gaussian, exponential or chi–squared are not bandlimited. This fact does not prevent the application of the quantizing theorems if the quantum step size is significantly smaller than the standard deviation. Nevertheless, it is of interest to investigate the question whether there are distributions whose characteristic functions are perfectly bandlimited, similarly to the sinc function. In the following paragraphs we will discuss some examples of distributions whose CFs are perfectly bandlimited.

In the previous chapter, certain relations between the quantization noise and the quantizer input and output were established. These relations are in the form of moments, particularly crosscorrelations and covariances. These moments are extremely useful for the analysis of quantization and for the analysis of control and signal processing systems containing quantizers. The similarities between quantization and the addition of independent uniformly distributed noise when certain quantizing theorems are satisfied are very useful for analysis of quantization.

The properties of these moments are not totally descriptive of the behavior of the quantizer however, because the quantization noise is deterministically related to the quantizer input, and the moments do not show this. To gain a complete and deeper understanding, we dedicate this chapter to the derivation of joint PDFs and CFs of quantization noise and the quantizer input and output.

JOINT PDF AND CF OF THE QUANTIZER INPUT AND OUTPUT

In this section, we will derive the joint PDF and CF of x and x', i.e. fx,x' (x, x'), and Φx,x' (ux, ux'). The statistical relationship between x and x' is like a statistical version of an input–output “transfer function” for the quantizer.

The derivation will proceed in the following manner. Refer to Fig. 7.1. This figure compares quantization (the addition of actual quantization noise ν to x) with the PQN model (the addition of independent noise n to x).

Scientific computation is done these days with floating–point arithmetic. Generally double precision is used. Of concern are quantization effects in the solutions of equations caused by limited precision in the representation of dependent variables (equation solutions) and limited precision in the representation of coefficients (equation parameters).

The full scope of the issues raised here is so great that it would be impossible to encompass it all in a single chapter. There are too many different kinds of equations to be dealt with. Some are linear, some nonlinear. Some have a unique solution, some have solutions of many values such as sampled time functions. Some have driving functions, some are homogeneous with given initial conditions. Some have feedback from the output to decrease the error, some are open–loop. Some have a single input and a single output, some have many inputs and many outputs, etc.

In this brief chapter, we will employ what we have learned about floating–point quantization to analyze roundoff errors in a nonlinear system. The ideas will be developed by studying a few simple cases. It is hoped that the reader will be able to use these ideas in solving new and unusual problems.

DEFINITION OF THE QUANTIZER

Quantization or roundoff occurs whenever physical quantities are represented numerically. The time displayed by a digital watch, the temperature indicated by a digital thermometer, the distances given on a map etc. are all examples of analog values represented by discrete numbers.

The values of measurements may be designated by integers corresponding to their nearest numbers of units. Roundoff errors have values between plus and minus one half unit, and can be made small by choice of the basic unit. It is apparent, however, that the smaller the size of the unit, the larger will be the numbers required to represent the same physical quantities and the greater will be the difficulty and expense in storing and processing these numbers. Often, a balance has to be struck between accuracy and economy. In order to establish such a balance, it is necessary to have a means of evaluating quantitatively the distortion resulting from rough quantization. The analytical difficulty arises from the inherent nonlinearities of the quantization process.

For purposes of analysis, it has been found convenient to define the quantizer as a nonlinear operator having the input–output staircase relation shown in Fig. 1.1(a). The quantizer output x' is a single–valued function of the input x, and the quantizer has an “average gain” of unity. The basic unit of quantization is designated by q.

System identification, as a particular process of statistical inference, exploits two types of information. The first is experiment; the other, called a priori, is known before making any measurements. In a wide sense, the a priori information concerns the system itself and signals entering the system. Elements of the information are, for example:

• the nature of the signals, which may be random or nonrandom, white or correlated, stationary or not, their distributions can be known in full or partially (up to some parameters) or completely unknown,

• general information about the system, which can be, for example, continuous or discrete in the time domain, stationary or not,

• the structure of the system, which can be of the Hammerstein or Wiener type, or other,

• the knowledge about subsystems, that is, about nonlinear characteristics and linear dynamics.

In other words, the a priori information is related to the theory of the phenomena taking place in the system (a real physical process) or can be interpreted as a hypothesis (if so, results of the identification should be necessarily validated) or can be abstract in nature.

This book deals with systems consisting of nonlinear memoryless and linear dynamic subsystems, for example, Hammerstein and Wiener systems and other related structures.

In this chapter, we discuss the problem of identification of a class of semiparametric block-oriented systems. This class of block-oriented systems can be restricted to a parameterization that includes a finite-dimensional parameter and nonlinear characteristics that run through a nonparametric class of mostly univariate functions. The parametric part of a semiparametric model defines characteristics of linear dynamical subsystems and low-dimensional projections of multivariate nonlinearities. The nonparametric part of the model comprises all static nonlinearities defined by functions of a single variable. A general methodology for identifying semiparametric block-oriented systems is developed. This includes a semiparametric version of least squares and a direct method using the concept of the average derivative of a regression function. These general approaches are applied in cases of semiparametric versions of Wiener, Hammerstein, and parallel systems. Section 14.2 gives examples of semiparametric block-oriented systems. This includes the multivariate version of Hammerstein and Wiener systems. In Section 14.3, we give a general approach to semiparametric inference. Section 14.4 is devoted to an important case study concerning the semiparametric Wiener system. Sections 14.5 and 14.6 provide similar considerations for semiparametric Hammerstein and parallel systems. In Section 14.7, we derive direct estimation methods for semiparametric nonlinear systems.

Introduction

In all of the preceding chapters, we have examined various fully nonparametric block-oriented systems.