So far we have explored classifiers with decision boundaries that are linear, or, in the case of the multiclass logistic regression, a combination of linear segments. In this chapter, we will expand what we have learned so far to classifiers that are capable of learning nonlinear decision boundaries. The classifiers that we will discuss here are called feed-forward neural networks, and are a generalization of both logistic regression and the perceptron. Despite the more complicated structures presented, we show that the key building blocks remain the same: the network is trained by minimizing a cost function. This minimization is implemented with backpropagation, which adapts the gradient descent algorithm introduced in the previous chapter to multilayer neural networks.