9.1 Introduction to Classification

We rely on machine learning (ML) to make critical decisions or predictions in the modern world. It is very important to understand how computers by using ML make these predictions. Usually, the predictions made by ML models are classified into two types, i.e., classification and regression. The ML models use various techniques to predict the outcome of an event by analyzing already available data. As machines learn from data, the type of training or input data plays a crucial role in deciding the machine's capability to make accurate decisions and predictions. Usually, this data is available in two forms, i.e., labeled and unlabeled. In label data, we know the value of the output attribute for the sample input attributes, while in unlabeled data, we do not have the output attribute value.

For analyzing labeled data, supervised learning is used. Classification and regression are the two types of supervised learning techniques used to predict the outcome of an unknown instance by analyzing the available labeled input instances. Classification is applied when the outcome is finite or discrete, while the regression model is applied when the outcome is infinite or continuous. For example, a classification model is used to predict whether a customer will buy a product or not. Here the outcome is finite, i.e., buying the product or not buying. In this case, the regression model predicts the number of products that the customer may buy. Here the outcome is infinite, i.e., all possible numbers, since the term quantity refers to a set of continuous numbers.

We can now present a simple method for obtaining O(d1.5 n) regret for losses in Fb with the limitation that the analysis only works in the stochastic setting where ft = f for all rounds.

The tool-chest for convex bandits and zeroth-order optimisation has been steadily growing in recent decades. Nevertheless, there are many intriguing open questions, both theoretical and practical. The purpose of this short chapter is to highlight some of the most important (in the author’s view, of course) open problems.

We start with a simple but instructive algorithm for the one-dimensional stochastic setting. The next assumption is considered global throughout the chapter:

Because the optimistic Gaussian surrogate is only well-behaved on a shrinking ellipsoidal focus region, algorithms that use it are most naturally analysed in the stochastic setting, where it is already a challenge to prove that the optimal action remains in the focus region. In the adversarial setting there is limited hope to ensure the optimal action in hindsight remains in the focus region. The plan is to use a mechanism that detects when the minimiser leaves the focus region and restarts the algorithm. This is combined with an argument that the regret is negative whenever a restart occurs. The formal setting studied in this chapter is characterised by the following assumption:

The purpose of this chapter is to introduce and analyse the surrogate loss functions used in Chapters 10 and 11. The results are stated in as much generality as possible to facilitate their use in future applications. In case you want a quick summary of the results, read this introductory section for the basic definitions and then head directly to Section 12.9.

In this chapterwe introduce an idea that is ubiquitous in zeroth-order optimisation, which is to use a gradient-based algorithm but replace the true gradients with estimated gradients of a smoothed loss. Except for Section 5.4, we assume throughout this chapter that the constraint set contains a euclidean ball of unit radius, the losses are bounded, Lipschitz and there is no noise:

This chapter briefly outlines the key algorithmic ideas and history of bandit convex optimisation. There follow in Section 2.4 and Section 2.5 summary tables of known lower and upper bounds for the various settings studied in this book.

Submodular functions are sometimes viewed as a combinatorial analogue of convexity via a gadget called the Lovászwi extension that we explain in a moment.