We provide a short, self-contained introduction to deep neural networks that is aimed at mathematically inclined readers. We promote the use of a vect--matrix formalism that is well suited to the compositional structure of these networks and that facilitates the derivation/description of the backpropagation algorithm. We present a detailed analysis of supervised learning for the two most common scenarios, (i) multivariate regression and (ii) classification, which rely on the minimization of least squares and cross-entropy criteria, respectively.

Since the groundbreaking performance improvement by AlexNet at the ImageNet challenge, deep learning has provided significant gains over classical approaches in various fields of data science including imaging reconstruction. The availability of large-scale training datasets and advances in neural network research have resulted in the unprecedented success of deep learning in various applications. Nonetheless, the success of deep learning appears very mysterious. The basic building blocks of deep neural networks are convolution, pooling, and nonlinearity, which are primitive tools of mathematics. Interestingly, the cascaded connection of these primitive tools results in superior performance over traditional approaches. To understand this mystery, one can go back to the basic ideas of the classical approaches to understand the similarities and differences from modern deep-neural-network methods. In this chapter, we explain the limitations of the classical machine learning approaches, and provide a review of mathematical foundations to understand why deep neural networks have successfully overcome their limitations.

Inspired by the success of deep learning in computer vision tasks, deep learning approaches for various MRI problems have been extensively studied in recent years. Early deep learning studies for MRI reconstruction and enhancement were mostly based on image-domain learning. However, because the MR signal is acquired in the k-space domain, researchers have demonstrated that deep neural networks can be directly designed in k-space to utilize the physics of MR acquisition. In this chapter, the recent trend of k-space deep learning for MRI reconstruction and artifact removal are reviewed. First, scan-specific k-space learning, which is inspired by parallel MRI, is covered. Then we provide an overview of data-driven k-space learning. Subsequently, unsupervised learning for MRI reconstruction and motion artifact removal are discussed.

Ultrasound imaging (US) is susceptible to several types of artifacts. Most artifacts appear because of transducer limitations and simplified assumptions on the wave propagation. The artifacts are sometimes used as a component that contains tissue information; however, they often lead to a misinterpretation in the clinical diagnosis. Therefore, to improve the clinical utility of ultrasound in difficult-to-image patients and settings, a number of artifact removal methods have been proposed that aim at boosting image quality. Classical optimization-based methods have severe limitations due to their limited performance and high computation requirements. Furthermore, it is difficult to obtain parameters for producing high-quality output. A quick remedy for the aforementioned issues is the deep learning approach, which offers high performance compared with the traditional methods despite the significantly reduced runtime complexity. Another big advantage is that the same parameters as those learned during the training phase can be used to process different input images. This has motivated the scientific community to design deep-neural-network-based approaches for US artifact removal tasks.

In this chapter, we provide an overview of a recent image-reconstruction method that uses a deep generative algorithm for dynamic magnetic resonance-imaging (dMRI). We begin by briefly introducing the imaging modality of dMRI, the associated image-reconstruction problem, and existing reconstruction approaches. Next, we introduce the time-dependent deep image prior (TD-DIP), which exploits the structure of convolutional neural networks (CNNs) as a regularizing prior. We show some representative results and discuss the pros and cons of this regularizing paradigm. Finally, we discuss a few potential remaining limitations.

CryoGAN uses ideas from deep generative adversarial learning to perform image reconstruction in single-particle cryo-electron microscopy (cryo-EM). In this chapter, we begin by introducing single-particle cryo-EM. We then formulate the associated image-reconstruction problem and discuss the main solutions found in the literature. Next, we describe the CryoGAN algorithm and show some representative results. Finally, we discuss what our experiences with Cryo-GAN suggest about the advantages and disadvantages of such deep generative adversarial methods in single-particle cryo-EM and beyond.

Quantitative phase imaging (QPI) refers to label-free techniques that produce images containing morphological information. In this chapter, we focus on 2D phase imaging with a holographic setup. In such a setting, the complex-valued measurements contain both intensity and phase information. The phase is related to the distribution of the refractive index of the underlying specimen. In practice, the collected phase happens to be wrapped (i.e., modulo 2π of the original phase) and one gains quantitative information on the sample only once the measurements are unwrapped. The process of phase unwrapping relies on the solution of an inverse problem, for which numerous methods exist. However, it is challenging to unwrap the phases of particularly complex or thick specimens such as organoids. Under such extreme conditions, classical methods often exhibit unwrapping errors. In this chapter we first formulate the problem of phase unwrapping and review the existing methods to solve it. Then, we present an application of a regularizing neural network to phase unwrapping, which allows us to outline the advantages of a training-free approach, i.e., a deep image prior, over classical methods or supervised learning.