In this chapter, we introduce the field of reinforcement learning and some of its most prominent applications in quantum physics and computing. First, we provide an intuitive description of the main concepts, which we then formalize mathematically. We introduce some of the most widely used reinforcement learning algorithms. Starting with temporal-difference algorithms and Q-learning, followed by policy gradient methods and REINFORCE, and the interplay of both approaches in actor-critic algorithms. Furthermore, we introduce the projective simulation algorithm, which deviates from the aforementioned prototypical approaches and has multiple applications in the field of physics. Then, we showcase some prominent reinforcement learning applications, featuring some examples in games; quantum feedback control; quantum computing, error correction and information; and the design of quantum experiments. Finally, we discuss some potential applications and limitations of reinforcement learning in the field of quantum physics.

This chapter discusses more specialized examples on how machine learning can be used to solve problems in quantum sciences. We start by explaining the concept of differentiable programming and its use cases in quantum sciences. Next, we describe deep generative models, which have proven to be an extremely appealing tool for sampling from unknown target distributions in domains ranging from high-energy physics to quantum chemistry. Finally, we describe selected machine learning applications for experimental setups such as ultracold systems or quantum dots. In particular, we show how machine learning can help in tedious and repetitive experimental tasks in quantum devices or in validating quantum simulators with Hamiltonian learning.

In this chapter, we describe basic machine learning concepts connected to optimization and generalization. Moreover, we present a probabilistic view on machine learning that enables us to deal with uncertainty in the predictions we make. Finally, we discuss various basic machine learning models such as support vector machines, neural networks, autoencoders, and autoregressive neural networks. Together, these topics form the machine learning preliminaries needed for understanding the contents of the rest of the book.

In this chapter, we review the growing field of research aiming to represent quantum states with machine learning models, known as neural quantum states. We introduce the key ideas and methods and review results about the capacity of such representations. We discuss in details many applications of neural quantum states, including but not limited to finding the ground state of a quantum system, solving its time evolution equation, quantum tomography, open quantum system dynamics and steady-state solution, and quantum chemistry. Finally, we discuss the challenges to be solved to fully unleash the potential of neural quantum states.

In this chapter, we introduce the reader to basic concepts in machine learning. We start by defining the artificial intelligence, machine learning, and deep learning. We give a historical viewpoint on the field, also from the perspective of statistical physics. Then, we give a very basic introduction to different tasks that are amenable for machine learning such as regression or classification and explain various types of learning. We end the chapter by explaining how to read the book and how chapters depend on each other.

This chapter gives an overview of the different topics covered in the book. As an outlook, we also share our view on potential exciting directions.

Distinguishing between different phases of matter and detecting phase transitions are some of the most central tasks in many-body physics. Traditionally, these tasks are accomplished by searching for a small set of low-dimensional quantities capturing the macroscopic properties of each phase of the system, so-called order parameters. Because of the large state space underlying many-body systems, success generally requires a great deal of human intuition and understanding. In particular, it can be challenging to define an appropriate order parameter if the symmetry breaking pattern is unknown or the phase is of topological nature and thus exhibits nonlocal order. In this chapter, we explore the use of machine learning to automate the task of classifying phases of matter and detecting phase transitions. We discuss the application of various machine learning techniques, ranging from clustering to supervised learning and anomaly detection, to different physical systems, including the prototypical Ising model that features a symmetry-breaking phase transition and the Ising gauge theory which hosts a topological phase of matter.

This chapter delves into the theory and application of reversible Markov Chain Monte Carlo (MCMC) algorithms, focusing on their role in Bayesian inference. It begins with the Metropolis–Hastings algorithm and explores variations such as component-wise updates, and the Metropolis-Adjusted Langevin Algorithm (MALA). The chapter also discusses Hamiltonian Monte Carlo (HMC) and the importance of scaling MCMC methods for high-dimensional models or large datasets. Key challenges in applying reversible MCMC to large-scale problems are addressed, with a focus on computational efficiency and algorithmic adjustments to improve scalability.