Bayesian Reasoning and Machine Learning

Machine learning is the body of research related to automated large-scale data analysis. Historically, the field was centred around biologically inspired models and the long-term goals of much of the community are oriented to producing models and algorithms that can process information as well as biological systems. The field also encompasses many of the traditional areas of statistics with, however, a strong focus on mathematical models and also prediction. Machine learning is now central to many areas of interest in computer science and related large-scale information processing domains.

Styles of learning

Broadly speaking the main two subfields of machine learning are supervised learning and unsupervised learning. In supervised learning the focus is on accurate prediction, whereas in unsupervised learning the aim is to find compact descriptions of the data. In both cases, one is interested in methods that generalise well to previously unseen data. In this sense, one distinguishes between data that is used to train a model and data that is used to test the performance of the trained model, see Fig. 13.1. We discuss first the basic characteristics of some learning frameworks before discussing supervised learning in more detail.

Supervised learning

Consider a database of face images, each represented by a vector x. Along with each image x is an output class y ∈ {male, female} that states if the image is of a male or female.