Impact Statement

We confront the challenge of designing efficient and robust algorithms for grid operation with the paradigm of learning-augmented algorithms, which leverage the excellent performance of machine learning while provably guaranteeing worst-case performance bounds and providing significant computational speedups over standard algorithms. Beyond the setting of generation dispatch, we propose that learning-augmented algorithms hold promise for safely utilizing the power of machine learning in a broader array of energy and sustainability-related control and optimization tasks.

2. Model and Preliminaries

We consider the problem of operating a combined cycle cogeneration power plant to meet both electricity and steam demand in the presence of exogenous variable renewable generation; see Figure 1 for a schematic. Specifically, we consider a plant with three gas turbines and a single steam turbine; we index the gas and steam turbines as $ \left\{\mathrm{1,2,3}\right\} $ and $ \left\{4\right\} $ , respectively. At each time $ t\in \left\{1,\dots, T\right\} $ , the plant operator observes an electricity demand (net of renewables) $ {d}_t\in {\mathrm{\mathbb{R}}}_{+} $ and a steam demand $ {q}_t\in {\mathrm{\mathbb{R}}}_{+} $ . In response, they choose energy dispatch setpoints $ {p}_t^{(i)}\in \left[{\underline{p}}_i,{\overline{p}}_i\right] $ and steam dispatch setpoints $ {s}_t^{(i)}\in \left[{\underline{s}}_i,{\overline{s}}_i\right] $ for all the turbines $ i=1,\dots, 4 $ , where $ {\underline{p}}_i,{\overline{p}}_i $ are the lower and upper bounds, respectively, for the energy dispatch, and $ {\underline{s}}_i,{\overline{s}}_i $ are the lower and upper bounds, respectively, for the steam dispatch. Note that while the gas turbines $ i=1,2,3 $ produce steam, and thus have positive steam dispatches, the steam turbine $ i=4 $ consumes steam, so $ {s}_t^{(4)}<0 $ . The plant operator’s goal is to minimize the total cost of its dispatch decisions and its cost for ramping electricity generation while producing sufficient electricity and steam to meet demand. Formally, the plant operator faces the following constrained minimization problem:

(2.1) $$ \underset{\begin{array}{c}{\mathbf{p}}_1,\dots, {\mathbf{p}}_T\in [\underline {\mathbf{p}},\overline{\mathbf{p}}]\\ {}{\mathbf{s}}_1,\dots, {\mathbf{s}}_T\in \left[\underline {\mathbf{s}},\overline{\mathbf{s}}\right]\end{array}}{\min}\hskip1em \sum \limits_{t=1}^T{f}_t\left({\mathbf{p}}_t,{\mathbf{s}}_t\right)+\alpha \parallel {\mathbf{p}}_t-{\mathbf{p}}_{t-1}{\parallel}_1 $$

(2.2) $$ \mathrm{s}.\mathrm{t}.\hskip1em \sum \limits_{i=1}^4{p}_t^{(i)}={d}_t\hskip2em \mathrm{for}\hskip0.42em \mathrm{all}\hskip0.42em t=1,\dots, T $$

(2.3) $$ \sum \limits_{i=1}^4{s}_t^{(i)}={q}_t\hskip2em \mathrm{for}\hskip0.42em \mathrm{all}\hskip0.42em t=1,\dots, T, $$

where $ {\mathbf{p}}_t={\left({p}_t^{(i)}\right)}_{i=1}^4 $ , $ {\mathbf{s}}_t={\left({s}_t^{(i)}\right)}_{i=1}^4 $ , $ {f}_t $ is a per-round fuel cost function that may depend on ambient conditions such as temperature, pressure, and humidity, and $ \alpha $ is a parameter determining the extent to which ramping energy generation is penalized. Importantly, the demands $ {d}_t,{q}_t $ and the cost functions $ {f}_t $ are not all known in advance, so the plant operator cannot solve problem (2.1) all at once. Instead, the plant operator only knows the current timestep’s problem parameters $ {f}_t,{d}_t $ , and $ {q}_t $ exactly, though they may have (possibly noisy) access to predictions of these parameters in a short lookahead window.

Figure 1. Schematic of the cogeneration power plant model. The plant operator chooses how much electricity (yellow arrow) and steam (blue arrow) the three gas turbines (left cooling tower) produce, as well as how much steam is directed to the steam turbine (right cooling tower) to produce additional electricity. At each time $ t $ , ambient conditions (e.g., temperature) together with electricity and steam demand are represented by the vector $ {\boldsymbol{\theta}}_t $ , and electricity and steam dispatch decisions are represented by the vector $ {\mathbf{x}}_t $ .

The cogeneration dispatch problem we consider can be formulated as a specific instance of the more general problem of online optimization with switching costs (Chen et al., Reference Chen, Comden, Liu, Gandhi and Wierman2016). Abstractly, online optimization with switching costs can be considered as a game in which at each time $ t\in \left\{1,\dots, T\right\} $ , a decision-maker receives a vector $ {\boldsymbol{\theta}}_t\in {\mathrm{\mathbb{R}}}^n $ parametrizing a cost function $ f\left(\cdot; {\boldsymbol{\theta}}_t\right):{\mathrm{\mathbb{R}}}^d\to {\mathrm{\mathbb{R}}}_{+} $ . In response, the decision-maker must choose a decision $ {\mathbf{x}}_t\in {\mathrm{\mathbb{R}}}^d $ and incurs the hitting cost $ f\left({\mathbf{x}}_t;{\boldsymbol{\theta}}_t\right) $ as well as the switching cost $ \parallel {\mathbf{x}}_t-{\mathbf{x}}_{t-1}\parallel $ resulting from that decision, where $ \parallel \cdot \parallel $ is some norm. We assume that the decision $ {\mathbf{x}}_t $ does not impact future parameters $ {\boldsymbol{\theta}}_{\tau } $ for $ \tau >t $ . In the context of the cogeneration dispatch application, $ {\boldsymbol{\theta}}_t $ is a vector containing all ambient conditions such as temperature, pressure, humidity, and the electricity and steam demands $ {d}_t,{q}_t $ at time $ t $ , $ {\mathbf{x}}_t $ is a vector including the plant operator’s energy and steam dispatch decisions at time $ t $ , and the function $ f $ maps ambient conditions and generator dispatches to a fuel cost while penalizing violation of constraints including capacity limits and supply–demand balance. The switching term $ \parallel {\mathbf{x}}_t-{\mathbf{x}}_{t-1}\parallel $ represents the ramping cost.Footnote ² The problem is online, meaning that the decision-maker only has access to the parameters $ {\boldsymbol{\theta}}_1,\dots, {\boldsymbol{\theta}}_t $ that have been revealed by time $ t $ when making the decision $ {\mathbf{x}}_t $ . However, the decision-maker may have access to (possibly inaccurate) forecasts $ {\hat{\boldsymbol{\theta}}}_{t+1\mid t},\dots, {\hat{\boldsymbol{\theta}}}_{t+w\mid t} $ of parameters within a lookahead window of length $ w\in \mathrm{\mathbb{N}} $ , which can help them anticipate and reduce future switching costs. Such forecasts could be obtained using standard ML methods for predicting near-term weather or energy demand.

We consider two standard algorithms for the problem of online optimization with switching costs. The first, Greedy, is a myopic algorithm that simply chooses the decision $ {\mathbf{x}}_t $ that minimizes $ f\left(\cdot; {\boldsymbol{\theta}}_t\right) $ at each time $ t $ . This algorithm has worst-case cost guarantees under mild assumptions on the structure of the cost function $ f $ (Zhang et al., Reference Zhang, Jiang, Lu and Yang2021), and it resembles the single-stage dispatch algorithm widely used by power system operators. Its behavior is characterized formally as follows:

$$ \mathrm{G}\mathrm{REEDY}:{\boldsymbol{\theta}}_t\hskip0.5em \mapsto \hskip0.5em \underset{\mathbf{x}\in {\mathrm{\mathbb{R}}}^d}{\arg \hskip0.1em \min }\ f\left(\mathbf{x};{\boldsymbol{\theta}}_t\right)=:{\mathbf{x}}_t. $$

That is, Greedy can be viewed as a function that, when provided with parameter vector $ {\boldsymbol{\theta}}_t\in {\mathrm{\mathbb{R}}}^n $ , returns the minimizer of $ f\left(\cdot; {\boldsymbol{\theta}}_t\right) $ as a dispatch decision. Note that, since we assume that $ f\left(\cdot; {\boldsymbol{\theta}}_t\right) $ penalizes all constraints from the original cogeneration problem formulation, Greedy will necessarily satisfy these constraints.

The second algorithm we consider is model predictive control (MPC). MPC solves a lookahead optimization problem using near-term predictions of parameters to choose a decision. Formally, at time $ t $ , given the perfectly known current parameter vector $ {\boldsymbol{\theta}}_t $ and forecasts $ {\hat{\boldsymbol{\theta}}}_{t+1\mid t},\dots, {\hat{\boldsymbol{\theta}}}_{t+w\mid t} $ of parameters over the next $ w $ timesteps, MPC chooses its decision as the optimal solution of the following optimization problem:

(2.4) $$ \mathrm{MPC}:\Theta \hskip0.5em \mapsto \underset{\begin{array}{c}\mathbf{x}\in {\mathrm{\mathbb{R}}}^d\\ {}{\mathbf{y}}_1,\dots, {\mathbf{y}}_w\in {\mathrm{\mathbb{R}}}^d\end{array}}{\arg \hskip0.1em \min}\hskip0.24em f\left(\mathbf{x};{\boldsymbol{\theta}}_t\right)+\sum \limits_{\tau =1}^wf\left({\mathbf{y}}_{\tau };{\hat{\boldsymbol{\theta}}}_{t+\tau \mid t}\right)+\parallel {\mathbf{y}}_{\tau }-{\mathbf{y}}_{\tau -1}\parallel =:{\mathbf{x}}_t $$

where $ {\mathbf{y}}_0:=\mathbf{x} $ and we have concatenated the parameter vectors $ \left({\boldsymbol{\theta}}_t,{\hat{\boldsymbol{\theta}}}_{t+1\mid t},\dots, {\hat{\boldsymbol{\theta}}}_{t+w\mid t}\right) $ into a single entity $ \Theta \in {\left({\mathrm{\mathbb{R}}}^n\right)}^{w+1} $ for brevity. Note that only the minimizer $ \mathbf{x} $ corresponding to the decision made for time $ t $ is binding, i.e., the chosen decision $ {\mathbf{x}}_t $ is simply the optimal $ \mathbf{x} $ in the above optimization; all of the other variables $ {\mathbf{y}}_1,\dots, {\mathbf{y}}_w $ are ignored after the solution is obtained. Note that, similar to Greedy, MPC produces decisions that satisfy the constraints of the original cogeneration problem.

As previously discussed, MPC can be computationally prohibitive if $ f\left(\cdot; \boldsymbol{\theta} \right) $ is nonconvex. However, there has been a great deal of recent work on learning proxies for power systems-related constrained optimization problems that operate more quickly than traditional optimization solvers (Donti et al., Reference Donti, Rolnick and Kolter2020; Park & Van Hentenryck, Reference Park and Van Hentenryck2023; Zhang et al., Reference Zhang, Tabas and Zhang2023). Following this approach, in our work, we train an ML model to approximate the input–output behavior of MPC in an unsupervised fashion. That is, given a dataset $ \mathcal{D}={\left\{{\Theta}_i\right\}}_{i=1}^N $ of cost function parameters, we train a neural network $ \mathrm{ML}:{\left({\mathrm{\mathbb{R}}}^n\right)}^{w+1}\to {\left({\mathrm{\mathbb{R}}}^d\right)}^{w+1} $ to minimize MPC’s lookahead objective (2.4) on average over this dataset. We employ the method of Zhang et al. (Reference Zhang, Tabas and Zhang2023) and include a gauge map in the output layer of the network to ensure that the network always produces decisions that satisfy the capacity limits and supply–demand balance constraints for both energy (2.2) and steam (2.3).

While this unsupervised training approach may yield low empirical error on the training set and the gauge map ensures constraint satisfaction, this does not guarantee that the ML algorithm will perform well on out-of-sample instances or under distribution shift. In the next section, we introduce an algorithm to address this challenge and provide provable cost guarantees for the learned proxy ML.

We conclude this section with some notation. For an algorithm Alg producing decisions $ {\mathbf{x}}_1,\dots, {\mathbf{x}}_T $ , define $ {\mathrm{ALG}}_t:={\mathbf{x}}_t $ as Alg’s decision at time $ t $ , and define $ {\mathrm{C}}_{\mathrm{ALG}}\left(s,t\right):={\sum}_{\tau =s}^tf\left({\mathbf{x}}_{\tau };{\boldsymbol{\theta}}_{\tau}\right)+\parallel {\mathbf{x}}_{\tau }-{\mathbf{x}}_{\tau -1}\parallel $ as Alg’s cost from time $ s $ through $ t $ . For brevity, we write the total cost as $ {\mathrm{C}}_{\mathrm{ALG}}:={\mathrm{C}}_{\mathrm{ALG}}\left(1,T\right) $ .

3. Algorithm

We propose in Algorithm 1 a novel algorithm, RobustML, that “robustifies” the algorithm ML, giving a sequence of dispatch decisions with cost that can be bounded in terms of the costs incurred by both ML and Greedy. RobustML behaves as follows: it begins by following ML’s decisions, but if ML surpasses a cost threshold and Greedy is performing relatively well, RobustML will switch to following Greedy’s decisions (line 8). However, if Greedy begins to perform worse relative to ML, then RobustML will switch back to following ML (line 15). The specific thresholds for switching are determined by the parameters $ \unicode{x025B}, \delta >0 $ , which reflect the algorithm’s confidence in ML. That is, when $ \unicode{x025B} $ and $ \delta $ are small (close to $ 0 $ ), RobustML will spend more time following the decisions of ML; on the other hand, when $ \unicode{x025B} $ and $ \delta $ are large, RobustML will spend more of its time following the decisions of Greedy. Tuning of $ \unicode{x025B} $ and $ \delta $ thus enables the plant operator to choose the particular tradeoff they wish to achieve between exploiting the (typically excellent) performance of ML and obtaining worst-case performance guarantees comparable to that of Greedy, even in cases of distribution shift.

Our main analytic result on the performance of RobustML is the following bound on its cost. Note that we assume that the decision space has diameter $ D $ , so $ \parallel {\mathrm{ML}}_t-{\mathrm{GREEDY}}_t\parallel \le D $ for all $ t\in \left\{1,\dots, T\right\} $ .

$$ {\mathrm{C}}_{ROBUSTML}\le \min \left\{\left(1+\epsilon +\delta \right){\mathrm{C}}_{ML},\left(1+\frac{1+\epsilon }{\delta}\right){\mathrm{C}}_{GREEDY}+\left(1+\frac{2}{\varepsilon}\right)D\right\}. $$

We present a proof of Theorem 3.1 in Section 6. In particular, the theorem tells us that by selecting $ \epsilon, \delta $ arbitrarily small, RobustML can achieve performance arbitrarily close to that achieved by ML, at the cost of possibly worse performance relative to Greedy. However, by selecting moderate $ \epsilon, \delta $ , it is possible to trade off the exploitation of ML with robustness with respect to Greedy’s performance. Moreover, rather than selecting fixed $ \epsilon, \delta $ a priori to achieve a particular, fixed tradeoff, it is possible to apply techniques from online learning to optimally tune these parameters in an adaptive fashion (see, e.g., Khodak et al., Reference Khodak, Balcan, Talwalkar and Vassilvitskii2022; Li et al., Reference Li, Yang, Qu, Shi, Yu, Wierman and Low2022; Zeynali et al., Reference Zeynali, Sun, Hajiesmaili and Wierman2021).

It is also important to note that the above result does not depend in any crucial way on the choice of the algorithm Greedy. That is, the plant operator can choose any strategy as their baseline “robust” algorithm, including heuristic methods that are currently employed in practice, and the performance bound given by Theorem 3.1 on Algorithm 1 will hold, with $ {\mathrm{C}}_{\mathrm{GREEDY}} $ replaced by the cost of the operator’s chosen strategy. Thus, Algorithm 1 gives a method for power plant operators to take advantage of machine learning’s good performance while maintaining worst-case guarantees relative to their existing strategies, whatever they may be.

4. Experimental Results and Discussion

We investigate the performance of our algorithm for cogeneration dispatch on a modified form of the CogenEnv environment from SustainGym, a reinforcement learning benchmarks suite (Yeh et al., Reference Yeh, Li, Datta, Arroyo, Christianson, Zhang, Chen, Hosseini, Golmohammadi, Shi, Yue and Wierman2023); all of the code, models, and data employed in conducting experiments can be found at https://doi.org/10.5281/zenodo.13623809. This cogeneration plant has behavior modeled in a black-box fashion via neural networks, and thus the fuel cost is differentiable as a function of the dispatch decisions. We train an ML model to replicate the behavior of MPC with lookahead $ w=6 $ in an unsupervised fashion as described in the previous section. The model is trained on a dataset of 100 days of ambient conditions (temperature, pressure, humidity) and municipal demands for energy and steam on a 15 minute basis. To solve for the Greedy and MPC decisions, we use sequential least-squares programming in SciPy (Virtanen et al., Reference Virtanen, Gommers, Oliphant, Haberland, Reddy, Cournapeau, Burovski, Peterson, Weckesser, Bright, van der Walt, Brett, Wilson, Millman, Mayorov, Nelson, Jones, Kern and Larson2020).

We begin by comparing the runtimes of Greedy, MPC, and ML to produce a day’s worth of dispatch decisions on an AWS EC2 instance with 4 virtual CPU cores and an NVIDIA T4 GPU (Figure 2). We find that while Greedy can produce a day of dispatches in about 30 seconds, MPC with lookahead $ w=6 $ takes nearly 100 times as long ( $ \sim $ 45 minutes). On the other hand, ML is three orders of magnitude faster than MPC, returning a day’s worth of decisions in under two seconds on average. This is unsurprising, since while Greedy and MPC require solving optimization problems to produce dispatch decisions, ML simply requires a neural network’s forward pass.

Figure 2. Number of seconds (mean and standard deviation) for each algorithm to produce a day’s worth of dispatch decisions.

We next examine the performance of ML, Greedy, and RobustML on the baseline system (no renewables) when the lookahead predictions are unreliable, as in the case when there is distribution shift. That is, we compare the setting of perfect predictions (i.e., $ {\hat{\boldsymbol{\theta}}}_{t+1\mid t}={\boldsymbol{\theta}}_{t+1},\dots, {\hat{\boldsymbol{\theta}}}_{t+w\mid t}={\boldsymbol{\theta}}_{t+w} $ ) to settings with i.i.d. Gaussian noise and increasing standard deviation $ \sigma $ on the predictions (i.e., $ {\hat{\boldsymbol{\theta}}}_{t+1\mid t}={\boldsymbol{\theta}}_{t+1}+{\mathbf{z}}_{t+1\mid t},\dots, {\hat{\boldsymbol{\theta}}}_{t+w\mid t}={\boldsymbol{\theta}}_{t+w}+{\mathbf{z}}_{t+w\mid t} $ , with $ {\mathbf{z}}_{\tau \mid t}\overset{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim}\mathcal{N}\left(\mathbf{0},\sigma \mathbf{I}\right) $ ). We show the results in Figure 3 for several choices of the RobustML parameters $ \unicode{x025B}, \delta $ . In particular, we observe that while ML performs better than Greedy when predictions are good ( $ \sigma \le 30 $ ), its performance degrades significantly as the noise grows ( $ \sigma \to 100 $ ). Nonetheless, for the cases of larger $ \unicode{x025B} $ and $ \delta $ – specifically, $ \unicode{x025B} =\delta =0.7 $ and $ \unicode{x025B} =\delta =1.0 $ – RobustML gracefully transitions between the good performance of ML for small $ \sigma $ to nearly matching or beating the performance of Greedy in the large $ \sigma $ regime. Thus, even though the quality of predictions is unknown a priori, RobustML exploits the good performance of ML when predictions are good while preserving robustness in the worst case. Notably, however, this is not the case when $ \unicode{x025B} =\delta $ is small, as can be seen in the top row of Figure 3; in this case, RobustML stays too close to the ML algorithm, and its performance suffers as a result.

Figure 3. Cost of each algorithm under increasing noise $ \sigma $ on the lookahead predictions, normalized by Greedy’s cost. Curves indicate mean cost and shaded regions cover $ \pm $ one standard deviation. Several choices of parameters are shown for RobustML: $ \unicode{x025B} =\delta =0.1 $ (top left), $ \unicode{x025B} =\delta =0.4 $ (top right), $ \unicode{x025B} =\delta =0.7 $ (bottom left), $ \unicode{x025B} =\delta =1.0 $ (bottom right).

We further examine the performance of the algorithms under increasing penetration of wind energy, displaying the results in Figure 4. We find that the efficiency improvement of ML over Greedy widens for moderate wind penetration (up to 200 MW), highlighting the value of using lookahead alongside ML to increase efficiency when renewable generation leads to more frequent ramp events. Notably, however, ML’s performance gets closer to that of Greedy when wind penetration nears 400 MW, possibly reflecting the challenge of adequately anticipating future ramp needs in a high-renewables environment even when equipped with moderate lookahead. Nonetheless, we find that when $ \unicode{x025B} =\delta =1.0 $ , RobustML yields uniformly improved performance over both ML and Greedy and thus enables better efficiency even in challenging high-renewables scenarios. While surprising, this improvement highlights that adaptive switching between the ML and Greedy algorithms enables RobustML to exploit each algorithm when it performs well, leading to better average performance than either algorithm.

Figure 4. Cost of each algorithm under increasing wind penetration, normalized by Greedy’s cost. Curves indicate mean cost and shaded regions cover $ \pm $ one standard deviation. Several choices of parameters are shown for RobustML: $ \unicode{x025B} =\delta =0.1 $ (top left), $ \unicode{x025B} =\delta =0.4 $ (top right), $ \unicode{x025B} =\delta =0.7 $ (bottom left), $ \unicode{x025B} =\delta =1.0 $ (bottom right).

6. Proof of Theorem 3.1

We begin by showing $ {\mathrm{C}}_{\mathrm{ROBUSTML}}\le \left(1+\unicode{x025B} +\delta \right){\mathrm{C}}_{\mathrm{ML}} $ . Note that the algorithm consists of phases in which RobustML first coincides with ML, and then switches to following Greedy, before switching back to ML, and so on. We will assume that RobustML ends the instance coinciding with ML, so $ {\mathbf{x}}_T={\mathbf{a}}_T $ ; the case in which RobustML ends at $ {\mathbf{r}}_T $ is similar. Let $ {t}_i $ denote the timestep in which RobustML switches from Greedy back to ML for the $ i $ th time, with $ {t}_0:=1 $ since RobustML always begins by following ML. Similarly, let $ {m}_i $ denote the timestep in which RobustML switches from ML to Greedy for the $ i $ th time. Clearly, we have $ 1={t}_0<{m}_1<{t}_1<\cdots <{m}_k<{t}_k\le T $ , for some $ k\in \mathrm{\mathbb{N}} $ . Even though RobustML ends at ML, define $ {m}_{k+1}:=T+1 $ for notational simplicity. Then the cost of RobustML may be written as

(6.1) $$ {\displaystyle \begin{array}{l}{\mathrm{C}}_{\mathrm{ROBUSTML}}=\sum \limits_{\tau =1}^{m_1-1}{f}_{\tau}\left({\mathbf{a}}_{\tau}\right)+\parallel {\mathbf{a}}_{\tau }-{\mathbf{a}}_{\tau -1}\parallel \\ {}\hskip8.0em +\sum \limits_{i=1}^k\left({f}_{m_i}\left({\mathbf{r}}_{m_i}\right)+\parallel {\mathbf{r}}_{m_i}-{\mathbf{a}}_{m_i-1}\parallel +\sum \limits_{\tau ={m}_i+1}^{t_i-1}{f}_{\tau}\left({\mathbf{r}}_{\tau}\right)+\parallel {\mathbf{r}}_{\tau }-{\mathbf{r}}_{\tau -1}\parallel \right.\\ {}\hskip13.8em \left.+{f}_{t_i}\left({\mathbf{a}}_{t_i}\right)+\parallel {\mathbf{a}}_{t_i}-{\mathbf{r}}_{t_i-1}\parallel +\sum \limits_{\tau ={t}_i+1}^{m_{i+1}-1}{f}_{\tau}\left({\mathbf{a}}_{\tau}\right)+\parallel {\mathbf{a}}_{\tau }-{\mathbf{a}}_{\tau -1}\parallel \right)\\ {}\hskip6.4em \le {\mathrm{C}}_{\mathrm{ML}}\left(1,{m}_1-1\right)+\sum \limits_{i=1}^k\left({\mathrm{C}}_{\mathrm{GREEDY}}\left({m}_i,{t}_i-1\right)+\parallel {\mathbf{r}}_{m_i-1}-{\mathbf{a}}_{m_i-1}\parallel \right.\\ {}\hskip22.5em \left.+{\mathrm{C}}_{\mathrm{ML}}\left({t}_i,{m}_{i+1}-1\right)+\parallel {\mathbf{a}}_{t_i-1}-{\mathbf{r}}_{t_i-1}\parallel \right)\end{array}} $$

(6.2) $$ \hskip8.5em \le {\mathrm{C}}_{\mathrm{ML}}\left(1,{m}_1-1\right)+2 kD+\sum \limits_{i=1}^k{\mathrm{C}}_{\mathrm{GREEDY}}\left({m}_i,{t}_i-1\right)+{\mathrm{C}}_{\mathrm{ML}}\left({t}_i,{m}_{i+1}-1\right) $$

(6.3) $$ \hskip-11.4em \le (1+\epsilon ){\mathrm{C}}_{\mathrm{ML}}+\sum \limits_{i=1}^k{\mathrm{C}}_{\mathrm{GREEDY}}({m}_i,{t}_i-1) $$

(6.4) $$ \hskip-25em \le (1+\epsilon +\delta ){\mathrm{C}}_{\mathrm{ML}} $$

where (6.1) follows from the triangle equality on $ \parallel {\mathbf{r}}_{m_i}-{\mathbf{a}}_{m_i-1}\parallel $ and $ \parallel {\mathbf{a}}_{t_i}-{\mathbf{r}}_{t_i-1}\parallel $ , and (6.2) follows by the diameter bound. The inequality (6.3) follows by line 6 of the algorithm, which states that the algorithm will switch from following ML to following Greedy at time $ t $ only if $ {\mathrm{C}}_{\mathrm{ML}}\left(s,t\right)\ge \frac{2D}{\unicode{x025B}} $ . Noting that at the start of any timestep $ t $ , $ s $ is exactly

$$ s=\underset{i:{m}_i+1\le t}{\max }{m}_i+1 $$

(with $ {m}_0:=0 $ for notational convenience), it follows that for each $ i\in \left[k\right] $ , $ {\mathrm{C}}_{\mathrm{ML}}\left({m}_{i-1}+1,{m}_i\right)\ge \frac{2D}{\unicode{x025B}} $ . Thus

$$ 2 kD\le \unicode{x025B} \sum \limits_{i=1}^k{\mathrm{C}}_{\mathrm{ML}}\left({m}_{i-1}+1,{m}_i\right)=\unicode{x025B} \cdot {\mathrm{C}}_{\mathrm{ML}}\left(1,{m}_k\right)\le \unicode{x025B} \cdot {\mathrm{C}}_{\mathrm{ML}}. $$

Finally, (6.4) follows from

$$ \sum \limits_{i=1}^k{\mathrm{C}}_{\mathrm{GREEDY}}\left({m}_i,{t}_i-1\right)\le {\mathrm{C}}_{\mathrm{GREEDY}}\left(1,{t}_k-1\right)<\delta \cdot {\mathrm{C}}_{\mathrm{ML}}\left(1,{t}_k-1\right)\le \delta \cdot {\mathrm{C}}_{\mathrm{ML}}, $$

since by definition, $ {\mathbf{x}}_{t_k-1}={\mathbf{r}}_{t_k-1} $ , which by line 12 of the algorithm means that $ {\mathrm{C}}_{\mathrm{GREEDY}}\left(1,{t}_k-1\right)<\delta \cdot {\mathrm{C}}_{\mathrm{ML}}\left(1,{t}_k-1\right) $ . Thus, we have proved the desired bound $ {\mathrm{C}}_{\mathrm{ROBUSTML}}\le \left(1+\unicode{x025B} +\delta \right){\mathrm{C}}_{\mathrm{ML}} $ .

We now turn to showing $ {\mathrm{C}}_{\mathrm{ROBUSTML}}\le \left(1+\frac{1+\unicode{x025B}}{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}+\left(1+\frac{2}{\unicode{x025B}}\right)D $ . First suppose RobustML finishes the instance coinciding with ML, so $ {\mathbf{x}}_T={\mathbf{a}}_T $ . Let $ \tau \in \left\{0,\dots, T-1\right\} $ denote the last time at which RobustML coincided with Greedy, or that $ {\mathbf{x}}_{\tau }={\mathbf{r}}_{\tau } $ . Thus the cost can be bounded as

(6.5) $$ {\displaystyle \begin{array}{cc}{\mathrm{C}}_{\mathrm{ROBUSTML}}& ={\mathrm{C}}_{\mathrm{ROBUSTML}}(1,\tau +1)+{\mathrm{C}}_{\mathrm{ROBUSTML}}(\tau +2,T)\\ {}& \le (1+\epsilon +\delta ){\mathrm{C}}_{\mathrm{ML}}(1,\tau +1)+{\mathrm{C}}_{\mathrm{ML}}(\tau +2,T)\end{array}} $$

(6.6) $$ \le \max \left\{\left(1+\frac{1+\epsilon }{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}(1,\tau +1)+\frac{2D}{\epsilon },\left(1+\frac{1+\epsilon }{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}\right\} $$

(6.7) $$ \le \left(1+\frac{1+\epsilon }{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}+\frac{2D}{\epsilon } $$

where (6.5) follows via the previously proved inequality $ {\mathrm{C}}_{\mathrm{ROBUSTML}}\le \left(1+\unicode{x025B} +\delta \right){\mathrm{C}}_{\mathrm{ML}} $ , and (6.6) follows by the fact (according to line 14 of the algorithm) that RobustML switching from Greedy to ML at time $ \tau +1 $ means that $ {\mathrm{C}}_{\mathrm{GREEDY}}\ge \delta \cdot {\mathrm{C}}_{\mathrm{ML}}\left(1,\tau +1\right) $ , as well as from the following observation: since RobustML coincides with ML between times $ \tau +1 $ and $ T $ , line 6 of the algorithm tells us that either $ {\mathrm{C}}_{\mathrm{ML}}\left(\tau +2,T\right)<\frac{2D}{\unicode{x025B}} $ or $ {\mathrm{C}}_{\mathrm{GREEDY}}\ge \delta \cdot {\mathrm{C}}_{\mathrm{ML}} $ _.

Finally, suppose RobustML finishes the instance coinciding with Greedy, so $ {\mathbf{x}}_T={\mathbf{r}}_T $ . Let $ \sigma \in \left\{0,\dots, T-1\right\} $ denote the last time at which RobustML coincided with ML, or that $ {\mathbf{x}}_{\sigma }={\mathbf{a}}_{\sigma } $ . By the previous case’s inequality (6.7), we have

$$ {\displaystyle \begin{array}{c}{\mathrm{C}}_{\mathrm{ROBUSTML}}={\mathrm{C}}_{\mathrm{ROBUSTML}}\left(1,\sigma \right)+{\mathrm{C}}_{\mathrm{ROBUSTML}}\left(\sigma +1,T\right)\\ {}\le \left(1+\frac{1+\unicode{x025B}}{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}\left(1,\sigma \right)+\frac{2D}{\unicode{x025B}}+{f}_{\sigma +1}\left({\mathbf{r}}_{\sigma +1}\right)+\parallel {\mathbf{r}}_{\sigma +1}-{\mathbf{a}}_{\sigma}\parallel +{\mathrm{C}}_{\mathrm{GREEDY}}\left(\sigma +2,T\right)\\ {}\le \left(1+\frac{1+\unicode{x025B}}{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}\left(1,\sigma \right)+\frac{2D}{\unicode{x025B}}+D+{\mathrm{C}}_{\mathrm{GREEDY}}\left(\sigma +1,T\right)\\ {}\le \left(1+\frac{1+\unicode{x025B}}{\delta}\right){\mathrm{C}}_{\mathrm{GREEDY}}+\left(1+\frac{2}{\unicode{x025B}}\right)D.\end{array}} $$