This chapter delves into the theory and application of reversible Markov Chain Monte Carlo (MCMC) algorithms, focusing on their role in Bayesian inference. It begins with the Metropolis–Hastings algorithm and explores variations such as component-wise updates, and the Metropolis-Adjusted Langevin Algorithm (MALA). The chapter also discusses Hamiltonian Monte Carlo (HMC) and the importance of scaling MCMC methods for high-dimensional models or large datasets. Key challenges in applying reversible MCMC to large-scale problems are addressed, with a focus on computational efficiency and algorithmic adjustments to improve scalability.

This chapter provides a comprehensive overview of the foundational concepts essential for scalable Bayesian learning and Monte Carlo methods. It introduces Monte Carlo integration and its relevance to Bayesian statistics, focusing on techniques such as importance sampling and control variates. The chapter outlines key applications, including logistic regression, Bayesian matrix factorization, and Bayesian neural networks, which serve as illustrative examples throughout the book. It also offers a primer on Markov chains and stochastic differential equations, which are critical for understanding the advanced methods discussed in later chapters. Additionally, the chapter introduces kernel methods in preparation for their application in scalable Markov Chain Monte Carlo (MCMC) diagnostics.

This chapter focuses on continuous-time MCMC algorithms, particularly those based on piecewise deterministic Markov processes (PDMPs). It introduces PDMPs as a scalable alternative to traditional MCMC, with a detailed explanation of their simulation, invariant distribution, and limiting processes. Various continuous-time samplers, including the bouncy particle sampler and zig-zag process, are compared in terms of efficiency and performance. The chapter also addresses practical aspects of simulating PDMPs, including techniques for exploiting model sparsity and data subsampling. Extensions to these methods, such as handling discontinuous target distributions or distributions defined on spaces of different dimensions, are discussed.

The development of more sophisticated and, especially, approximate sampling algorithms aimed at improving scalability in one or more of the senses already discussed in this book raises important considerations about how a suitable algorithm should be selected for a given task, how its tuning parameters should be determined, and how its convergence should be as- sessed. This chapter presents recent solutions to the above problems, whose starting point is to derive explicit upper bounds on an appropriate distance between the posterior and the approximation produced by MCMC. Further, we explain how these same tools can be adapted to provide powerful post-processing methods that can be used retrospectively to improve approximations produced using scalable MCMC.

This chapter explores the benefits of non-reversible MCMC algorithms in improving sampling efficiency. Revisiting Hamiltonian Monte Carlo (HMC), the chapter discusses the advantages of breaking detailed balance and introduces lifting schemes as a tool to enhance exploration of the parameter space. It reviews non-reversible HMC and alternative algorithms like Gustafson’s method. The chapter also covers techniques like delayed rejection and the discrete bouncy particle sampler, offering a comparison between reversible and non-reversible methods. Theoretical insights and practical implementations are provided to highlight the efficiency gains from non-reversibility.

This chapter introduces stochastic gradient MCMC (SG-MCMC) algorithms, designed to scale Bayesian inference to large datasets. Beginning with the unadjusted Langevin algorithm (ULA), it extends to more sophisticated methods such as stochastic gradient Langevin dynamics (SGLD). The chapter emphasises controlling the stochasticity in gradient estimators and explores the role of control variates in reducing variance. Convergence properties of SG-MCMC methods are analysed, with experiments demonstrating their performance in logistic regression and Bayesian neural networks. It concludes by outlining a general framework for SG-MCMC and offering practical guidance for efficient, scalable Bayesian learning.

In this chapter we present two spatial dependent models: one based on defining a latent variable for each area, and the other by defining one latent variable for each pair of latent areas. We call the latter the latent edges model. We compare both models with a real data set. Extensions to spatio-temporal constructions are also considered.

In this chapter we define what a conjugate family is in a Bayesian analysis context and develop detailed examples of some cases; in particular, we review the beta and binomial case, the Pareto and inverse Pareto case, the gamma and gamma case and the gamma and Poisson case. We conclude by providing a list of conjugate models.