Thia chapter considers methods for both regression and classification based on Gaussian process, a stochastic process with Gaussian distribution, of which the mean vector and covariance matrix can be obtained based on the labeled samples in the training set. The resulting Gaussian process serves as a nonlinear regression function that fits the given dataset. This function can be treated as the probability for data samples' the class identity and used for classificationas as shown before. This Gaussian process approach also has some two advantages: first, the certainty (or confidence) of the regression or classification result can be quantitatively measured; second proper tradeoff between overfitting and underfitting can be made by adjusting a parameter for the covariance of the Gaussian process model.

This chapter is solely dedicated to reinforcement learning (RL), one of the three main learning paradigms covered in the book (together with regression and classification). The goal of RL is for an agent to learn from and respond to its environment modeled as a Markov decision process (MDP), by following a set of policies to take the best action at each state of the MDP, in order to receive the maximum total accumulated reward. The utmost goal is to come up with the optimal policy in terms of the best action to take at each state. Different from all optimization problems previously considered for maximizing (or minimizing) certain objective functions, RL achieves its goal by the general method of dynamic programming (while linear and quadratic programmings are for constrained optimization), which solves a complex problem by breaking it up and solving a set of subproblems recursively. Specifically, the main method for RL is the Q-learning algorithm which finds the optimal policy that takes the best action selected based on the expected values of the total reward at all states and all actions at each state. Toward to end of the chapter, various more advanced versions of RL are briefly discussed based on some previously learned methods such as neural networks and deep learning.

The goal of this chapter is to prepare for the future discussion of various artificial neural network (ANN) learning algorithms by introducing some basic concepts in neural networks and some biologically inspired examples the Habbian and Hopfield networks to illustrate how an ANN based on some simple learning rule can achieve meaningful results, although they are not actually widely used in machine learning practice. Specifically, the behavior of the Hebbian learning network mimics the associative nature of brain, as a simple model of associative memory, and the Hopfield network further shows how a pattern can be stored and then recalled based on a noisy and imcomplete copy of itself, a function that is commenly demonstratedof the brain.

The sixth chapter provides a deeper exploration of probabilistic models, building upon concepts encountered earlier in the text. The chapter illustrates how to construct diverse models, particularly by employing the notion of conditional independence. It also outlines standard methods for estimating parameters and hidden states, as well as techniques for sampling. The chapter concludes by discussing and implementing applications such as Kalman filtering and Gibbs sampling. The chapter covers a range of topics, including parametric families of probability distributions, maximum likelihood estimation, modeling complex dependencies using conditional independence and marginalization, and applications such as linear-Gaussian models and Kalman filtering.

This chapter introduces the mathematics of data through the example of clustering, a fundamental technique in data analysis and machine learning. The chapter begins with a review of essential mathematical concepts, including matrix and vector algebra, differential calculus, optimization, and elementary probability, with practical Python examples. The chapter then delves into the k-means clustering algorithm, presenting it as an optimization problem and deriving Lloyd's algorithm for its solution. A rigorous analysis of the algorithm's convergence properties is provided, along with a matrix formulation of the k-means objective. The chapter concludes with an exploration of high-dimensional data, demonstrating through simulations and theoretical arguments how the "curse of dimensionality" can affect clustering outcomes.

This chapter explores the behavior of random walks on graphs, framed within the broader context of Markov chains. It introduces finite-state Markov chains, explaining key concepts such as transition matrices, the Markov property, and the computation of stationary distributions. The chapter then discusses the long-term behavior of Markov chains, including the convergence to equilibrium under conditions of irreducibility and aperiodicity. The chapter delves into the application of random walks on graphs, particularly in the context of PageRank, a method for identifying central nodes in a network. It also discusses Markov chain Monte Carlo (MCMC) methods, specifically the Metropolis–Hastings algorithm and Gibbs sampling, which are used to generate samples from complex probability distributions. The chapter concludes by illustrating the application of Gibbs sampling to generate images of handwritten digits using a restricted Boltzmann machine.

Chapter 2 explores the fundamental concept of least squares, covering its geometric, algebraic, and numerical aspects. The chapter begins with a review of vector spaces and matrix inverses, then introduces the geometry of least squares through orthogonal projections. It presents the QR decomposition and Householder transformations as efficient methods for solving least-squares problems. The chapter concludes with an application to regression analysis, demonstrating how to fit linear and polynomial models to data. Key topics include the normal equations, orthogonal decomposition, and the Gram–Schmidt algorithm. The chapter also addresses the issue of overfitting in polynomial regression, highlighting the importance of model selection in data analysis. The chapter includes practical Python implementations and numerical examples to reinforce the theoretical concepts.

This chapter introduces the foundational mathematical concepts behind neural networks, backpropagation, and stochastic gradient descent (SGD). It begins by generalizing the Chain Rule and providing a brief overview of automatic differentiation, which is essential for efficiently computing derivatives in machine learning models. The chapter then explains backpropagation within the context of multilayer neural networks, specifically focusing on multilayer perceptrons (MLPs). It covers the implementation of SGD, highlighting its advantages in optimizing large datasets. Practical examples using the PyTorch library are provided, including the classification of images from the Fashion-MNIST dataset. The chapter provides a solid foundation in the mathematical tools and techniques that underpin modern AI.

This chapter focuses on the core concepts of optimization theory and its application in data science and AI. It begins with a review of differentiable functions of several variables, including the gradient and Hessian matrices, and key results like the Chain Rule and the Mean Value Theorem. The chapter then introduces optimality conditions for unconstrained optimization, explaining first-order and second-order conditions, and the role of convexity in ensuring global optimality. A detailed discussion of the gradient descent algorithm is provided, including its convergence analysis under different assumptions. The chapter concludes with an application to logistic regression, demonstrating how gradient descent is used to optimize the cross-entropy loss function in a supervised learning context. Practical Python examples are integrated throughout to illustrate the theoretical concepts.

The fifth chapter explores the application of spectral graph theory to network data analysis. The chapter begins with an introduction to fundamental graph theory concepts, including undirected and directed graphs, graph connectivity, and matrix representations such as the adjacency and Laplacian matrices. It then discusses the variational characterization of eigenvalues and their significance in understanding the structure of graphs. The chapter highlights the spectral properties of the Laplacian matrix, particularly its role in graph connectivity and partitioning. Key applications, such as spectral clustering for community detection and the analysis of random graph models like Erdős–Rényi random graphs and stochastic blockmodels, are presented. The chapter concludes with a detailed exploration of graph partitioning algorithms and their practical implementations using Python.

The fourth chapter introduces the singular value decomposition (SVD), a fundamental matrix factorization with broad applications in data science. The chapter begins by reviewing key linear algebra concepts, including matrix rank and the spectral theorem. It then explores the problem of finding the best low-dimensional approximating subspace to a set of data points, leading to the formal definition of the SVD. The power iteration method is presented as an efficient way to compute the top singular vectors and values. The chapter then demonstrates the application of SVD to principal components analysis (PCA), a dimensionality reduction technique that identifies the directions of maximum variance in data. Further applications of the SVD are discussed, including low-rank matrix approximations and ridge regression, a regularization technique for handling multicollinearity in linear systems.