We illustrated in Example 63.2 one limitation of linear separation surfaces by considering the XOR mapping (63.11). The example showed that certain feature spaces are not linearly separable and cannot be resolved by the perceptron algorithm. The result in the example was used to motivate one powerful approach to nonlinear separation surfaces by means of kernel methods.

In the immediate past chapters we developed several techniques for the design of linear classifiers, such as logistic regression, perceptron, and support vector machines (SVM). These algorithms are suitable for data that are linearly separable; otherwise, their performance degrades significantly. In this chapter we explain how the methods can be adjusted to determine nonlinear separation surfaces.

In most multistage decision problems, we are interested in determining the optimal strategy, ${\pi }^{\star }\left(a\right|s)$ (i.e., the optimal actions to follow in the state–action space). Most of the algorithms described in the previous chapters focused on evaluating the state and state–action value functions, ${\upsilon }^{\pi }\left(s\right)$ and $q^{\pi }(s,a)$, for a given policy $\pi \left(a\right|s)$. More is needed to learn the optimal policy.

We derived in the previous two chapters procedures for assessing the performance of strategies used by agents interacting with a Markov decision process (MDP), including obtaining optimal policies. Among other methods, we discussed the policy evaluation algorithm (44.116) and the value and policy iterations (45.23) and (45.43), respectively.

In this chapter, we describe a tensor network (TN) based common language established between machine learning and many-body physics, which allows for bidirectional contributions. By showing that many-body wave functions are structurally equivalent to mappings of convolutional and recurrent networks, we bring forth quantum entanglement measures as natural quantifiers of dependencies modeled by such networks. Accordingly, we propose a novel entanglement-based deep learning design scheme that sheds light on the success of popular architectural choices made by deep learning practitioners and suggests new practical prescriptions. In the other direction, we construct TNs corresponding to deep recurrent and convolutional networks. This allows us to theoretically demonstrate that these architectures are powerful enough to represent highly entangled quantum systems polynomially more efficiently than previously employed architectures. We thus provide theoretical motivation to shift neural-network-based wave function representations closer to state-of-the-art deep learning architectures.

Principal component analysis (PCA) is a formidable tool for dimensionality reduction. Given feature vectors $\left\{h_n\right\}$ in $M$‐dimensional space, PCA replaces them by lower‐dimensional vectors $\left\{h_n^{\prime }\right\}$ of size $M^{\prime }\ll M$ each.

Markov decision processes (MDPs) are at the core of reinforcement learning theory. Similar to Markov chains, MDPs involve an underlying Markovian process that evolves from one state to another, with the probability of visiting a new state being dependent on the most recent state. Different from Markov chains, MDPs involve both agents and actions taken by these agents. As a result, the next state is dependent on which action was chosen at the state preceding it. MDPs therefore provide a powerful framework to explore state spaces and to learn from actions and rewards.

In the feedforward networks and convolutional neural networks (CNNs) studied in the previous chapters, the training data was assumed to be static, with no sequential relation among the samples. Using the data, we were able to train the networks to perform reliable classification tasks. There are many applications, however, where the input data will be sequential in nature, with one sample following another in some ordered manner, as happens with words in a sentence.

The material in the last three chapters focused on the use of neural network structures for the solution of inference (regression and classification) problems. In this chapter, we use the same networks to develop two generative methods whose purpose is to generate samples from the same underlying distribution as the training data.

We studied in Chapters 29 and 30 the mean‐square error (MSE) criterion in some detail, and applied it to the problem of inferring an unknown (or hidden) variable $x$ from the observation of another variable $x$ when $\left\{x,y\right\}$ are related by means of a linear regression model or a state‐space model.

The mean-square-error (MSE) criterion (27.17) is one notable example of the Bayesian approach to statistical inference. In the Bayesian approach, both the unknown quantity, $\mathit{x}$, and the observation, $\mathit{y}$, are treated as random variables and an estimator $\widehat{\mathit{x}}$ for $\mathit{x}$ is sought by minimizing the expected value of some loss function denoted by $Q\left(\mathit{x},\widehat{\mathit{x}}\right)$. In the previous chapter, we focused exclusively on the quadratic loss $Q\left(\mathit{x},\widehat{\mathit{x}}\right)={(\mathit{x}-\widehat{\mathit{x}})}^2$ for scalar $\mathit{x}$. In this chapter, we consider more general loss functions, which will lead to other types of inference solutions such as the mean-absolute error (MAE) and the maximum a-posteriori (MAP) estimators. We will also derive the famed Bayes classifier as a special case when the realizations for $\mathit{x}$ are limited to the discrete values $\mathit{x}\in \left\{\pm 1\right\}$.