When the actions of one player affect the environment of another player, we say that there are externalities. In economics, two types of externalities are often discussed: consumption and production externalities. The former externality describes a situation where a player’s utility function is directly affected by another player’s actions (playing loud music). The latter describes a situation where the production function of a player is directly affected by another player’s actions (polluting). Externalities may arise in economies through public goods, common-pool resources, technological spillover, strategic interactions, and so on.

We can now present a simple method for obtaining O(d1.5 n) regret for losses in Fb with the limitation that the analysis only works in the stochastic setting where ft = f for all rounds.

The tool-chest for convex bandits and zeroth-order optimisation has been steadily growing in recent decades. Nevertheless, there are many intriguing open questions, both theoretical and practical. The purpose of this short chapter is to highlight some of the most important (in the author’s view, of course) open problems.

We start with a simple but instructive algorithm for the one-dimensional stochastic setting. The next assumption is considered global throughout the chapter:

Because the optimistic Gaussian surrogate is only well-behaved on a shrinking ellipsoidal focus region, algorithms that use it are most naturally analysed in the stochastic setting, where it is already a challenge to prove that the optimal action remains in the focus region. In the adversarial setting there is limited hope to ensure the optimal action in hindsight remains in the focus region. The plan is to use a mechanism that detects when the minimiser leaves the focus region and restarts the algorithm. This is combined with an argument that the regret is negative whenever a restart occurs. The formal setting studied in this chapter is characterised by the following assumption:

The purpose of this chapter is to introduce and analyse the surrogate loss functions used in Chapters 10 and 11. The results are stated in as much generality as possible to facilitate their use in future applications. In case you want a quick summary of the results, read this introductory section for the basic definitions and then head directly to Section 12.9.

In this chapterwe introduce an idea that is ubiquitous in zeroth-order optimisation, which is to use a gradient-based algorithm but replace the true gradients with estimated gradients of a smoothed loss. Except for Section 5.4, we assume throughout this chapter that the constraint set contains a euclidean ball of unit radius, the losses are bounded, Lipschitz and there is no noise:

This chapter briefly outlines the key algorithmic ideas and history of bandit convex optimisation. There follow in Section 2.4 and Section 2.5 summary tables of known lower and upper bounds for the various settings studied in this book.

Submodular functions are sometimes viewed as a combinatorial analogue of convexity via a gadget called the Lovászwi extension that we explain in a moment.

This book is about approximately solving problems of the form.