Machine Learning for Engineers

This chapter focuses on three key problems that underlie the formulation of many machine learning methods for inference and learning, namely variational inference (VI), amortized VI, and variational expectation maximization (VEM). We have already encountered these problems in simplified forms in previous chapters, and they will be essential in developing the more advanced techniques to be covered in the rest of the book. Notably, VI and amortized VI underpin optimal Bayesian inference, which was used, e.g., in Chapter 6 to design optimal predictors for generative models; and VEM generalizes the EM algorithm that was introduced in Chapter 7 for training directed generative latent-variable models.