Fourier series

Introduction

Now that you know how to find Fourier transforms of functions on ℝ, you can quickly learn to find Fourier transforms of functions on Tp, i.e., to construct the Fourier series

when f is given. In principle, you can always obtain F by evaluating the integrals from the analysis equation

but this is often quite tedious. We will present several other methods for finding these coefficients. You can then select the procedure that requires the least amount of work!

You will recall from your study of Chapter 1 that the synthesis equation (1) for f on Tp can be written as the analysis equation

for F on ℤ. In view of this duality, every Fourier series (1) simultaneously tells us that

Direct integration

You can evaluate the integrals (2) with the techniques from elementary calculus when the function f is a linear combination of segments of

You will use the integration by parts formula

for such calculations. Here q(-1), q(-2), … are successive antiderivatives of

q(x) = e2πkx/p or cos(2πkx/p) or sin(2πkx/p), k=±1, ±2, ….

When f is a polynomial, the integrated term will eventually disappear from the right side of (4), and the resulting identity,

is known as Kronecker's rule. The k = 0 integral is usually done separately.

Probability density functions on ℝ

Introduction

A liter bottle filled with air (at room temperature and pressure) holds approximately 2.7 · 1022 molecules of N2, O2, CO2, … that fly about at high-speed colliding with one another and with the walls of the bottle. We cannot hope to keep track of every particle in such a large ensemble, but in the mid-19th century, Maxwell showed that we can deduce many statistical properties about such a system. For example, he found the probability density

for the speed V of a given gas molecule, see Fig. 12.1. (The parameter

depends on the mass m of the molecule, the absolute temperature T and Boltzmann's constant k = 1.3806 … · 10-23 joule/K.) We use the integral

to find the probability that a molecule will have a speed in the interval v1 < V < v2.

Example Use Maxwell's density to find the fraction of N2 molecules (with m = 4.65 · 10-26 kg) in a warm room (with T = 300 K) that have speeds less than v0 = 1000 km/hr = 278 m/sec.

Solution We use (1) with the parameter

and the Maclaurin series for the exponential to compute

Approximately one-sixth of the N2 molecules have speeds less than 1000 km/hr! ▪

We can use the density function (1) to compute the average (or expected) value of certain functions of the molecular speed V.

Example Use Maxwell's density to find the average speed and the average kinetic energy for an N2 molecule when T = 300 K.

Pre-FFT computation of the DFT

Introduction

In this chapter we will study the problem of computing the components

of the discrete Fourier transform of given complex numbers f[0], f[1], …, f[N-1]. We write these relations in the compact form

F = Ff,

where

are complex N-component column vectors and where the N × N DFT matrix

is expressed in terms of powers of

ω ≔ e-2π/N = cos(2π/N) - i sin(2π/N).

We will use indices 0, 1, …, N - 1 (rather than 1, 2, …, N) for the rows of vectors and for the rows and columns of matrices. When it is necessary, we will use a subscript to specify the size of a matrix, e.g., I8, F16 will denote the 8 × 8 identity matrix and the 16 × 16 DFT matrix, respectively.

Given an N × N matrix

and an N-vector

we can evaluate the components of

by using the algorithm

The cost of this computation is approximately N2 operations when we define an operation to be the work we do as we execute the statement

S ≔ S + akn · bn

from the inner loop. [More specifically, we fetch akn, bn, and the “old” value of S from storage; we form the product akn · bn and the sum S + (akn · bn); and we store this result as the “new” value of S.] Of course, complex arithmetic requires more effort than real arithmetic, and by using the real-imaginary decomposition

we verify that

1 complex operation = 4 real operations.

The Haar wavelets

Introduction

We use dilates of the complex exponential wave

w(x) ≔ e2πix, -∈ < x < ∈

when we write the familiar Fourier synthesis equation

for a suitably regular function f on ℝ. (The function F is the Fourier transform of f.) For example, the identity

shows how to synthesize a normal density by combining waves of constant amplitude,

which stretch from x = -∞ to x = +∞. When 0 < σ ≪ 1, this density is an approximate delta with 99.7% of its integral in the tiny interval -3σ ≤ x ≤ 3σ. Almost perfect destructive interference must occur at every point |x| > 3σ. Such a synthesis has mathematical validity (a proof is given in Section 1.5), but it is physically unrealistic. In practice, we cannot produce the audio signal for a 1-ms “click” by having tubas, trombones, …, piccolos play sinusoidal tones of constant amplitude for all eternity!

In this chapter you will learn to synthesize a function f with localized, oscillatory basis functions called wavelets. We use the prototype Haar wavelet

to introduce the fundamental concepts from an exciting new branch of analysis created by mathematicians, electrical engineers, physicists, … during the last two decades of the 20th century. More sophisticated wavelets for audio signal processing, image compression, etc. are described in the following sections.

You will observe that the wavelet (1) oscillates. The positive and negative parts of ψ have the same size in the sense that

It is easy to produce a function y on R that interpolates f at the sample points, i.e., …

Sampling and interpolation

Introduction

As I speak the word Fourier, a microphone converts the pressure wave from my voice into an electrical voltage f(t), 0 ≤ t ≤.5 sec. The sound card in my computer discretizes this signal by producing the samples

Figure 8.1 shows the (overlapping) line segments joining

You can identify the hissy initial consonant “f”, the three long vowels “o”, “e”, “a” (as in boat, beet, bait), and the semivowel “r” (as in burr).

An expanded 80-sample segment of the 10-msec interval .33 sec t 34 sec (a part of the “e” sound) is shown in Fig. 8.2. It would appear that we have enough sample points to construct a good approximation to the original audio signal.

In practice, we work with a quantized approximation of the sample f(nT). For example, the Fourier recording uses 1 byte ≔ 8 bits per sample, with one bit specifying the sign of f(nT) and with seven bits specifying the modulus |f(nT)| to within 1 part in 127 (of some fixed maximum modulus). It takes 4000 bytes of storage for the Fourier recording, more than we need for a page of this text with

A digitized sound file that uses 8000 8-bit samples/sec is of telephone quality. For high fidelity we must increase the sampling rate and reduce the quantization error. A compact disk recording uses 44100 16-bit samples/sec or

Digitized sound files are very big!

Shannon's hypothesis

Let f be a function on ℝ, let T > 0, and let

It is easy to produce a function y on R that interpolates f at the sample points, i.e., …

The concept of an operator identity

Introduction

In the preceding two chapters we developed a calculus for finding Fourier transforms of functions on ℝ, Tp, ℤ, and ℙN, and the corresponding rules are succinctly stated in Appendix 3. Each of these rules involves a pair of function-to-function mappings, i.e., a pair of operators. In this chapter we will study these operators and the elementary relations that link them to one another. The change in emphasis will enable us to characterize the symmetry properties associated with Fourier analysis, to deepen and unify our understanding of the transformation rules that we use so often in practice, and to facilitate our study of the related sine, cosine, Hartley, and Hilbert transforms. Later on, we will use operators to describe fast algorithms for computing the DFT, to describe fast algorithms for computing with wavelets, to analyze thin lens systems in optics, etc.

Operators applied to functions on ℙN

It is easy to illustrate these ideas when we work with functions defined on ℙN, i.e., with functions that can be identified with complex N-vectors. From linear algebra we know that any linear mappingA : ℂN → ℂN can be represented by an N × N matrix of complex coefficients. In particular, the discrete Fourier transform operator F defined by the analysis equation is a linear operator that is represented by the N × N complex matrix

The reflection operator R, defined by writing Rf[n] ≔ f[-n], n=0,±1,±2, …

The concept of a generalized function

Introduction

Let y(t) be the displacement at time t of a mass m that is attached to a spring having the force constant k as shown in Fig. 7.1. We assume that the mass is at rest in its equilibrium position [i.e., y(t) = 0] for all t ≤ 0. At time t = 0 we begin to subject the mass to an impulsive driving force

When the duration ∈ < 0 is “small,” this force simulates the tap of a hammer that transfers the momentum

to the mass and “rapidly” changes its velocity from y′(0) = 0 to y′(∈) ≈ p/m.

You should have no trouble verifying that

is a twice continuously differentiable function that satisfies the forced differential equation

my″(t) + ky(t) = f∈(t)

for the motion (except at the points t = 0, t = ∈ where y″∈ is not defined), see Ex. 7.1. Here

so that sin(ωt), cos(ωt) are solutions of the unforced differential equation

my″(t) + ky(t) = 0.

Now as ∈ → 0+, the response function (2) has the pointwise limit

and it is natural to think of y0 as the response of the system to an impulse

of strength

that acts only at time t = 0 as illustrated in Fig. 7.2. The physical intuition is certainly valid, and such arguments have been used by physicists and engineers (e.g., Euler, Fourier, Maxwell, Heaviside, Dirac) for more than two centuries. And for most of …

Synthesis and analysis equations

Introduction

In mathematics we often try to synthesize a rather arbitrary function f using a suitable linear combination of certain elementary basis functions. For example, the power functions 1, x, x2, … serve as such basis functions when we synthesize f using the power series representation

f(x) = a0 + a1x + a2x2 + …. (1)

The coefficient ak that specifies the amount of the basis function xk needed in the recipe (1) for constructing f is given by the well-known Maclaurin formula

from elementary calculus. Since the equations for a0, a1, a2,… can be used only in cases where f, f′, f″, … are defined at x = 0, we see that not all functions can be synthesized in this way. The class of analytic functions that do have such power series representations is a large and important one, however, and like Newton [who with justifiable pride referred to the representation (1) as “my method”], you have undoubtedly made use of such power series to evaluate functions, to construct antiderivatives, to compute definite integrals, to solve differential equations, to justify discretization procedures of numerical analysis, etc.

Fourier's representation (developed a century and a half after Newton's) uses as basis functions the complex exponentials

e2πisx≔cos(2πsx) + i·sin(2πsx), (2)

where s is a real frequency parameter that serves to specify the rate of oscillation, and i2 = -1. When we graph this complex exponential, i.e., when we graph …

Using the definition to find Fourier transforms

Introduction

If you want to use Fourier analysis, you must develop basic skills for finding Fourier transforms of functions on ℝ. In principle, you can always use the defining integral from the analysis equation to obtain F when f is given. In practice, you will quickly discover that it is not so easy to find a transform such as

by using the techniques of elementary integral calculus, see Ex. 1.1.

In this chapter, we will present a calculus (i.e., a computational process) for finding Fourier transforms of commonly used functions on ℝ. You will memorize a few Fourier transform pairs f, F and learn certain rules for modifying or combining known pairs to obtain new ones. It is analogous to memorizing that (xn)′ = nxn-1, (sin x)′ = cos x, (ex)′ = ex, … and then using the addition rule, product rule, quotient rule, chain rule, … to find derivatives. You will need to spend a bit of time mastering the details, so do not despair when you see the multiplicity of drill exercises!

Once you learn to find Fourier transforms, you can immediately use Fourier's analysis and synthesis equations, Parseval's identity, and the Poisson relations to evaluate integrals and sums that cannot be found by more elementary methods. You will also need these skills when you study various applications of Fourier analysis in the second part of the course.

The box function

The box function

and the cardinal sine or sinc function

are two of the most commonly used functions in Fourier analysis. We often simplify the definition of such functions …

To the Student

This book is about one big idea: You can synthesize a variety of complicated functions from pure sinusoids in much the same way that you produce a major chord by striking nearby C, E, G keys on a piano. A geometric version of this idea forms the basis for the ancient Hipparchus-Ptolemy model of planetary motion (Almagest, 2nd century see Fig. 1.2). It was Joseph Fourier (Analytical Theory of Heat, 1815), however, who developed modern methods for using trigonometric series and integrals as he studied the flow of heat in solids. Today, Fourier analysis is a highly evolved branch of mathematics with an incomparable range of applications and with an impact that is second to none (see Appendix 1). If you are a student in one of the mathematical, physical, or engineering sciences, you will almost certainly find it necessary to learn the elements of this subject. My goal in writing this book is to help you acquire a working knowledge of Fourier analysis early in your career.

If you have mastered the usual core courses in calculus and linear algebra, you have the maturity to follow the presentation without undue difficulty. A few of the proofs and more theoretical exercises require concepts (uniform continuity, uniform convergence, …) from an analysis or advanced calculus course. You may choose to skip over the difficult steps in such arguments and simply accept the stated results. The text has been designed so that you can do this without severely impacting your ability to learn the important ideas in the subsequent chapters.