This chapter introduces techniques for bounding random processes. We develop Gaussian interpolation to derive powerful comparison inequalities for Gaussian processes, including the Slepian, Sudakov–Fernique, and Gordon inequalities. We use this to get sharp bounds on the operator norm of Gaussian random matrices. We also prove the Sudakov lower bound using covering numbers. We introduce the concept of Gaussian width, which connects probabilistic and geometric perspectives, and apply it to analyze the size of random projections of high-dimensional sets. Exercises cover symmetrization and contraction inequalities for random processes, the Gordon min–max inequality, sharp bounds for Gaussian matrices, the nuclear norm, effective dimension, random projections, and matrix sketching.

This chapter introduces sub-Gaussian and sub-exponential distributions and develops basic concentration inequalities. We prove the Hoeffding, Chernoff, Bernstein, and Khintchine inequalities. Applications include robust mean estimation and analyzing degrees in random graphs. The exercises explore Mills ratio, small ball probabilities, Le Cam’s two-point method, the expander mixing lemma for random graphs, stochastic dominance, Orlicz norms, and the Bennett inequality.

Most of the material in this chapter is from basic analysis and probability courses. Key concepts and results are recalled here, including convexity, norms and inner products, random variables and random vectors, union bound, conditioning, basic inequalities (Jensen, Minkowski, Cauchy–Schwarz, Hölder, Markov, and Chebyshev), the integrated tail formula, the law of large numbers, the central limit theorem, normal and Poisson distributions, and handy bounds on the factorial.

This chapter presents some foundational methods for bounding random processes. We begin with the chaining technique and prove the Dudley inequality, which bounds a random process using covering numbers. Applications include Monte Carlo integration and uniform bounds for empirical processes. We then develop VC (Vapnik– Chervonenkis) theory, offering combinatorial insights into random processes and applying it to statistical learning. Building on chaining, we introduce generic chaining to obtain optimal two-sided bounds using Talagrand’s g2 functional. A key consequence is the Talagrand comparison inequality, a generalization of the Sudakov–Fernique inequality for sub-Gaussian processes. This is used to derive the Chevet inequality, a powerful tool for analyzing random bilinear forms over general sets. Exercises explore the Lipschitz law of large numbers in higher dimensions, one-bit quantization, and the small ball method for heavy-tailed random matrices.

This chapter begins with Maurey’s empirical method – a probabilistic approach to constructing economical convex combinations. We apply it to bound covering numbers and the volumes of polytopes, revealing their counterintuitive behavior in high dimensions. The exercises refine these bounds and culminate in the Carl–Pajor theorem on the volume of polytopes.

This chapter introduces several basic tools in high-dimensional probability: decoupling, concentration for quadratic forms (the Hanson–Wright inequality), symmetrization, and contraction. These techniques are illustrated through estimates of the operator norm of a random matrix. This is applied to matrix completion, where the goal is to recover a low-rank matrix from a random subset of its entries. Exercises explore variants of the Hanson–Wright inequality, mean estimation, concentration of the norm for anisotropic random vectors, distances to subspaces, graph cutting, the concept of type in normed spaces, non-Euclidean versions of the approximate Caratheodory theorem, and covariance estimation.

This chapter begins the study of random vectors in high dimensions, starting by showing their norm concentrates. We give a probabilistic proof of the Grothendieck inequality and apply it to semidefinite optimization. We explore a semidefinite relaxation for the maximum cut, presenting the Goemans–Williamson randomized approximation algorithm. We also give an alternative proof of the Grothendieck inequality with nearly the best known constant using the kernel trick, a method widely used in machine learning. The exercises explore invariant ensembles of random matrix theory, various versions of the Grothendieck inequality, semidefinite relaxations, and the notion of entropy.

Chapter 2 focuses on background information that is essential to understanding SEMs. This includes providing the general structural equation model that appears throughout the book along with definitions of the notation and the assumptions of the model. The chapter introduces path diagram symbols and their relation to the equation form of the model. It also describes differences between endogenous and exogenous variables and observed and latent variables for both continuous and categorical variables. In addition, the chapter introduces the problems of missing data, outliers and influential cases, and multiple significance testing, issues that are common in all types of models. Finally, basic rules of expected values, variances, and covariances are part of the chapter.

This chapter develops a non-asymptotic theory of random matrices. It starts with a quick refresher on linear algebra, including the perturbation theory for matrices and featuring a short proof of the Davis–Kahan inequality. Three key concepts are introduced – nets, covering numbers, and packing numbers – and linked to volume and error-correcting codes. Bounds on the operator norm and singular values of random matrices are established. Three applications are given: community detection in networks, covariance estimation, and spectral clustering. Exercises explore the power method to compute the top singular value, the Schur bound on the operator norm, Hermitian dilation,Walsh matrices, the Wedin theorem on matrix perturbations, a semidefinite relaxation of the cut norm, the volume of high-dimensional balls, and Gaussian mixture models.

Models with multiple equations rather than a single equation are the subject of Chapter 4. It covers model specification, implied moments, model identification, model estimation, and model interpretation, fit, and diagnostics in the context of such models. The consequences of measurement error and the treatment of mediation effects are part of the chapter. Finally, the chapter compares simultaneous equation models and Directed Acyclic Graphs (DAGs).