To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
The algorithm based on gradient descent in the previous chapter is simple and computationally efficient, at least provided the projection can be computed. There are two limitations, however.
The purpose of this chapter is to introduce the necessary tools from optimisation, convex geometry and convex analysis. You can safely skip this chapter, referring back as needed. The main concepts introduced are as follows:
We already saw an application of exponential weights to linear and quadratic bandits in Chapter 7. The same abstract algorithm can also be used for convex bandits but the situation is more complicated. Throughout this chapter we assume the losses are bounded and there is no noise:
Function classes like Fb are non-parametric. In this chapter we shift gears by studying two important parametric classes: Fb,lin and Fb,quad. The main purpose of this chapter is to use the machinery designed for linear bandits to prove an upper bound on the minimax regret for quadratic bandits. On the positive side the approach is both elementary and instructive. More negatively, the resulting algorithm is not computationally efficient. Before the algorithms and regret analysis we need three tools: covering numbers, optimal experimental design and the exponential weights algorithm.
Like the bisection method (Chapter 4), cutting plane methods are most naturally suited to the stochastic setting. For the remainder of the chapter we assume the setting is stochastic and the loss function is bounded:
This comprehensive reference brings readers to the frontier of research on bandit convex optimization or zeroth-order convex optimization. The focus is on theoretical aspects, with short, self-contained chapters covering all the necessary tools from convex optimization and online learning, including gradient-based algorithms, interior point methods, cutting plane methods and information-theoretic machinery. The book features a large number of exercises, open problems and pointers to future research directions, making it ideal for students as well as researchers.
This tutorial guide introduces online nonstochastic control, an emerging paradigm in control of dynamical systems and differentiable reinforcement learning that applies techniques from online convex optimization and convex relaxations to obtain new methods with provable guarantees for classical settings in optimal and robust control. In optimal control, robust control, and other control methodologies that assume stochastic noise, the goal is to perform comparably to an offline optimal strategy. In online control, both cost functions and perturbations from the assumed dynamical model are chosen by an adversary. Thus, the optimal policy is not defined a priori and the goal is to attain low regret against the best policy in hindsight from a benchmark class of policies. The resulting methods are based on iterative mathematical optimization algorithms and are accompanied by finite-time regret and computational complexity guarantees. This book is ideal for graduate students and researchers interested in bridging classical control theory and modern machine learning.