The scope of this chapter is the presentation of inpainting methods which use fourth-order (and higher!) partial differential equations (PDEs) to fill in missing image contents in gaps in the image domain. In the following section, we first motivate the use of higher-order flows for image inpainting.

Second- Versus Higher-Order Approaches

In this section we want to emphasise the difference between second-order diffusions as discussed in Chapter 4 and higher-order – in particular, fourth-order – diffusions in inpainting. As we have seen already, second-order inpainting methods (in which the order of the method is determined by the derivatives of highest order in the PDE), such as total variation (TV) inpainting, have drawbacks when it comes to the connection of edges over large distances and the smooth propagation of level lines into the damaged domain – qualities that we agreed an image interpolator which follows the good continuation principle from Chapter 3 should have. The disability, in general, of second-order methods to connect structures across the inpainting domain was demonstrated for harmonic inpainting in Figure 4.1 and for TV inpainting in Figure 4.8. An example of the lack of smoothness of interpolated level lines is given in Figure 4.7 for TV inpainting. In the case of TV inpainting, this behaviour of the interpolator is explained using the co-area formula, Theorem 4.3.6. To remind ourselves, TV inpainting seeks an interpolator whose level lines have minimal length, thus connecting level lines from the boundary of the inpainting domain via the shortest distance (linear interpolation). In [MM98, Mas98 and Mas02], Masnou and Morel propose an extension of the length penalisation in TV inpainting by an additional curvature term that should be small for interpolating level lines.

Digital inpainting methods are being designed with the desire for an automated and visually convincing interpolation of images. In this chapter we give an overview of approaches and trends in digital image inpainting and provide a preview of our discussion in Chapters 4 through 7. Before we start with this, let us raise our consciousness about the challenges and hurdles we might face in the design of inpainting problems.

The first immediate issue of image inpainting is, of course, that we do not know the truth but can only guess. We can make an educated guess, but still it will never be more than a guess. This is so because once something is lost, it is lost, and without additional knowledge (based on the context, e.g., historical facts), the problem of recovering this loss is an ambiguous one. Just look at Figure 2.1, and I ask you: is it a black stripe behind a grey stripe or a grey stripe behind a black stripe? Thus, the challenge of image inpainting is that the answer to the problem might not be unique. We will discuss this and strategies to make ‘good’ guesses based on the way our perception works in Chapter 3.

When inspecting different inpainting methods in the course of this book, you should be aware of the fact that mathematical inpainting methods are designed for inpainting the image completely automatically, that is, without intervention (supervision) by the user. Hence, the art of designing efficient and qualitatively high inpainting methods is really the skill of modelling the mechanisms that influence what the human brain can usually do in an instant. At present, we are still far away from a fair competition with the human brain. Digital inpainting methods are currently not (will never be?) as smart as our brain. In particular, no all-round inpainting model exists that can solve a variety of inpainting problems with sufficient quality. One of the main shortcomings of inpainting methods is their inability to realistically reconstruct both structure and texture simultaneously (see Section 2.2).

The study of solutions to systems of semi-linear parabolic partial differential equations has attracted considerable attention over the past fifty years.In the case when the nonlinearity satisfies a local Lipschitz condition, the fundamental theory is well developed (see, for example, the texts of Friedman [21], Fife [20], Rothe [65], Smoller [70], Samarskii et al. [67], Volpert et al. [72], Leach and Needham [36], and references therein). The situation when the nonlinearity does not necessarily satisfy a local Lipschitz condition is less well studied, but contributions have been made in the case of specific non-Lipschitz nonlinearities which have aided in particular applications (see, for example, Aguirre and Escobedo [5]; Needham et al. [36], [54], [29], [33], [40], [41], [42], [43] and references therein), and for the corresponding steady state elliptic problems (see, for example, Stakgold [71], Bandle et al [9], [10], [11], [12], [13], Abdullaev [2], [3], [4], [1] and references therein). The aim of this monograph is to exhibit general results concerning semi-linear parabolic partial differential equations that do not necessarily satisfy a local Lipschitz condition. The approach is classical, in the sense that the results relate entirely to the well-posedness criteria for classical solutions, in the sense of Hadamard [39], and the main results are principally established within the framework of real analysis. The approach used to develop the existence theory in this monograph has similarities with the method of successive approximations for systems of first order ordinary differential equations, as detailed in [17] and [16]. Alternative approaches may be possible through the concepts of weak solutions and the framework of semigroup theory. These alternative approaches are amenable, and very effective, in the case of Lipschitz continuous nonlinearities, as exemplified in the monographs by Henry [26] and Pazy [62]. However, the extensions to non-Lipschitz nonlinearities have not been developed and our approach provides an effective development of the classical theory for Lipschitz continuous nonlinearities.