The scope of this chapter is the presentation of inpainting methods which use fourth-order (and higher!) partial differential equations (PDEs) to fill in missing image contents in gaps in the image domain. In the following section, we first motivate the use of higher-order flows for image inpainting.

Second- Versus Higher-Order Approaches

In this section we want to emphasise the difference between second-order diffusions as discussed in Chapter 4 and higher-order – in particular, fourth-order – diffusions in inpainting. As we have seen already, second-order inpainting methods (in which the order of the method is determined by the derivatives of highest order in the PDE), such as total variation (TV) inpainting, have drawbacks when it comes to the connection of edges over large distances and the smooth propagation of level lines into the damaged domain – qualities that we agreed an image interpolator which follows the good continuation principle from Chapter 3 should have. The disability, in general, of second-order methods to connect structures across the inpainting domain was demonstrated for harmonic inpainting in Figure 4.1 and for TV inpainting in Figure 4.8. An example of the lack of smoothness of interpolated level lines is given in Figure 4.7 for TV inpainting. In the case of TV inpainting, this behaviour of the interpolator is explained using the co-area formula, Theorem 4.3.6. To remind ourselves, TV inpainting seeks an interpolator whose level lines have minimal length, thus connecting level lines from the boundary of the inpainting domain via the shortest distance (linear interpolation). In [MM98, Mas98 and Mas02], Masnou and Morel propose an extension of the length penalisation in TV inpainting by an additional curvature term that should be small for interpolating level lines.

Digital inpainting methods are being designed with the desire for an automated and visually convincing interpolation of images. In this chapter we give an overview of approaches and trends in digital image inpainting and provide a preview of our discussion in Chapters 4 through 7. Before we start with this, let us raise our consciousness about the challenges and hurdles we might face in the design of inpainting problems.

The first immediate issue of image inpainting is, of course, that we do not know the truth but can only guess. We can make an educated guess, but still it will never be more than a guess. This is so because once something is lost, it is lost, and without additional knowledge (based on the context, e.g., historical facts), the problem of recovering this loss is an ambiguous one. Just look at Figure 2.1, and I ask you: is it a black stripe behind a grey stripe or a grey stripe behind a black stripe? Thus, the challenge of image inpainting is that the answer to the problem might not be unique. We will discuss this and strategies to make ‘good’ guesses based on the way our perception works in Chapter 3.

When inspecting different inpainting methods in the course of this book, you should be aware of the fact that mathematical inpainting methods are designed for inpainting the image completely automatically, that is, without intervention (supervision) by the user. Hence, the art of designing efficient and qualitatively high inpainting methods is really the skill of modelling the mechanisms that influence what the human brain can usually do in an instant. At present, we are still far away from a fair competition with the human brain. Digital inpainting methods are currently not (will never be?) as smart as our brain. In particular, no all-round inpainting model exists that can solve a variety of inpainting problems with sufficient quality. One of the main shortcomings of inpainting methods is their inability to realistically reconstruct both structure and texture simultaneously (see Section 2.2).

In this chapter we consider integral representations of functions using kernels that are ridge functions. The most commonly used integral representation with a ridge function kernel is that given by the Fourier transform. The function ψw := eix·w is, for each fixed w, a ridge function. Under suitable assumptions on an f defined on Rn, we have

For example, the above holds if f ∊ L1(Rn) n C(n) and,. While we have drifted into the complex plane, we can easily rewrite the above using only real-valued functions.

Another integral representation with a ridge function kernel may be found in John [1955], p. 11. We present it here without proof. Assume, i.e., f and its first partial derivatives are continuous functions, and f has compact support. Let Δx denote the Laplacian with respect to the variables x1, …, xn, i.e.,

Let Sn−1 denote the unit sphere in Rn, and let da be uniform measure on Sn−1 of total measure equal to the surface area of the unit sphere Sn−1, i.e., of total measure

For n and k odd positive integers, we have

For n an even positive integer and k any even non-negative integer, we have

The relationship between these integral formulæ and the inverse Radon transform can be found in John [1955], together with other similar integral representations.

In this chapter we review two additional integral representations. The first, using an orthogonal decomposition in terms of Gegenbauer polynomials is from Petrushev [1998]. The second is based upon ridgelets and was presented by Candès [1998] in his doctoral thesis, see also Candès [1999].

This monograph is an attempt to examine and study ridge functions as entities in and of themselves. As such we present what we consider to be various central properties of ridge functions. However, no encyclopedic claims are being made, and the topics chosen are those that we alone considered appropriate. In addition, most chapters contain, either explicitly or implicitly, unresolved questions. It is our hope that this monograph will prove useful and interesting to both researchers and the more casual reader. And, of course, all errors, omissions and other transgressions are totally our responsibility.

No monograph is written in a vacuum, and I would like to especially thank Carl de Boor, Vugar Ismailov and an anonymous referee for various comments and suggestions. Thanks also to Heinz Bauschke, Vitaly Maiorov, Simon Reich and Yuan Xu for their patience, and help with my various inquiries.

This monograph is about Ridge Functions. A ridge function is any multivariate real-valued function

F : Rn → R

of the form

F(x1, …, xn) = f(a1x1 + … + anxn) = f(a · x),

where x = (x1, …, xn) ∊ Rn are the variables, f is a univariate real-valued function, i.e., f : R → R, and a = (a1, …, an) ∊ Rn\{0} is a fixed vector. This vector a ∊ Rn\{0} is generally called the direction. In other words, a ridge function is a multivariate function constant on the parallel hyperplanes a · x = c, c ∊ R. It is one of the simpler multivariate functions. Namely, it is a superposition of a univariate function with one of the simplest multivariate functions, the inner product.

More generally, we can and will consider, for given d, 1 ≤ d ≤ n−1, functions F of the form

F(x) = f(Ax),

where A is a fixed d × n real matrix, and f : Rd → R. We call such functions Generalized Ridge Functions. For d = 1, this reduces to a ridge function.

Motivation

We see specific ridge functions in numerous multivariate settings without considering them of interest in and of themselves. We find them, for example, as kernels in integral formulæ. They appear in the Fourier transform

and its inverse. We see them in the n-dimensional Radon transform

and its inverse. Here the integral is taken with respect to the natural hypersurface measure dσ. It is possible to generalize the Radon transform still further by integrating over (n − d)-dimensional affine subspaces of Rn.

In this chapter we study one of the basic properties of ridge function decomposition, namely smoothness. In the first section we ask the following question. If

is smooth, does this imply that each of the fi is also smooth? In the second section we ask this same question with regard to generalized ridge functions, i.e., linear combinations of functions of the form f(Ax), where the A are d × n real matrices, and f : Rd → R.

Ridge Function Smoothness

Let Ck(Rn), k ∊ Z+, denote the usual class of real-valued functions with all derivatives of order up to and including k being continuous. Assume F ∊ Ck(Rn) is of the form

where r is finite, i.e., F ∊ M(a1, …, ar), and the ai are given pairwise linearly independent vectors in Rn. What can we say about the smoothness of the fi? Do the fi necessarily inherit all the smoothness properties of the F?

When r = 1 the answer is yes, and there is essentially nothing to prove. That is, if

F(x) = f1(a1 · x)

is in Ck(Rn) for some a1 ≠ 0, then for c ∊ Rn satisfying a1 · c = 1 and all t ∊ R we have that F(tc) = f1(t) is in Ck(R). This same result holds when r = 2. As the a1 and a2 are linearly independent, there exists a vector c ∊ Rn satisfying a1 · c = 0 and a2 · c = 1. Thus

F(tc) = f1(a1 · tc) + f2(a2 · tc) = f1(0) + f2(t).

Since F(tc) is in Ck(R), as a function of t, so is f2. The same result holds for f1.