Digital inpainting methods are being designed with the desire for an automated and visually convincing interpolation of images. In this chapter we give an overview of approaches and trends in digital image inpainting and provide a preview of our discussion in Chapters 4 through 7. Before we start with this, let us raise our consciousness about the challenges and hurdles we might face in the design of inpainting problems.

The first immediate issue of image inpainting is, of course, that we do not know the truth but can only guess. We can make an educated guess, but still it will never be more than a guess. This is so because once something is lost, it is lost, and without additional knowledge (based on the context, e.g., historical facts), the problem of recovering this loss is an ambiguous one. Just look at Figure 2.1, and I ask you: is it a black stripe behind a grey stripe or a grey stripe behind a black stripe? Thus, the challenge of image inpainting is that the answer to the problem might not be unique. We will discuss this and strategies to make ‘good’ guesses based on the way our perception works in Chapter 3.

When inspecting different inpainting methods in the course of this book, you should be aware of the fact that mathematical inpainting methods are designed for inpainting the image completely automatically, that is, without intervention (supervision) by the user. Hence, the art of designing efficient and qualitatively high inpainting methods is really the skill of modelling the mechanisms that influence what the human brain can usually do in an instant. At present, we are still far away from a fair competition with the human brain. Digital inpainting methods are currently not (will never be?) as smart as our brain. In particular, no all-round inpainting model exists that can solve a variety of inpainting problems with sufficient quality. One of the main shortcomings of inpainting methods is their inability to realistically reconstruct both structure and texture simultaneously (see Section 2.2).