2. Problem

Let $X=\{X_t \colon t \geq 0 \}$ be a (one-dimensional) Lévy process defined on a probability space $(\Omega , \mathcal{F}, \mathbb{P})$ . For $x\in\mathbb{R}$ , we let $\mathbb{P}_x$ denote the law of X when its initial value is x, and write $\mathbb{P} = \mathbb{P}_0$ for the case $x = 0$ . Let $\Psi$ be the characteristic exponent of X, i.e. ${\mathrm{e}}^{-t\Psi(\lambda)}=\mathbb{E} [{\mathrm{e}}^{{\mathrm{i}}\lambda X_t}]$ , $\lambda \in \mathbb{R}$ and $t\geq 0$ . It is known to admit the form

\begin{align*}\Psi (\lambda) \,:\!=\, -{\mathrm{i}}\gamma\lambda +\dfrac{1}{2}\sigma^2 \lambda^2+\int_{\mathbb{R} \backslash \{0\} } (1-{\mathrm{e}}^{{\mathrm{i}}\lambda z}+{\mathrm{i}}\lambda z1_{\{|z| \lt 1\}}) \Pi(\mathrm{d} z) , \quad \lambda\in\mathbb{R}, \end{align*}

for some $\gamma\in\mathbb{R}$ , $\sigma\geq 0$ , and a Lévy measure $\Pi $ on $\mathbb{R} \backslash \{0\}$ satisfying $\int_{\mathbb{R}\backslash \{ 0\}}(1\land z^2) \Pi (\mathrm{d} z) \lt \infty$ .

We consider a version of the stochastic control problem defined as follows. The set of control opportunities

\[\mathcal{T}_{\eta}\,:\!=\, \{T(k)\,\colon k\in\mathbb{N} \} \]

are given by the arrival times of a Poisson process $N^{\eta}=\{N^{\eta}_t \,\colon t\geq 0\}$ with intensity $\eta \gt 0$ which is independent of X. In other words, the interarrival times $\{T(k)-T(k-1)\colon k \in\mathbb{N} \}$ are an i.i.d. sequence of exponential random variables with intensity $\eta$ , where we let $T(0)=0$ for notational convenience. Let $\mathbb{F} \,:\!=\, \{ {\mathcal{F}_t}\colon t \geq 0 \}$ be the natural filtration generated by $(X, N^{\eta})$ . A strategy, representing the cumulative amount of controlling, $\pi=\{ R^\pi_t\colon t\geq 0\}$ , is a process of the form

(2.1) \begin{align} R^\pi_t = \int_{[0,t]} \nu^\pi_s \,\mathrm{d} N^{\eta}_s = \sum_{0 \leq s \leq t\colon N^\eta_s \neq N^\eta_{s-} } \nu_s^\pi, \quad t\geq 0, \end{align}

for some càglàd (left-continuous with right limits) and non-negative $\mathbb{F}$ -adapted process $\nu^\pi=\{\nu^\pi_t \colon t\geq 0\}$ , where it is understood that $N^\eta_{0-}=0$ . The corresponding controlled process becomes

\begin{align*}U^\pi_t=X_t +R^\pi_t,\quad t\geq 0.\end{align*}

We focus on the case where we can control the state process in one direction, and hence $\nu_s^\pi \geq 0$ a.s. for all $\pi \in \mathcal{A}$ , which is standard as in [Reference Benkherouf and Bensoussan5–Reference Bensoussan, Liu and Sethi7] and [Reference Hernández-Hernández, Pérez and Yamazaki17]. Such an assumption is applicable in many inventory models where only replenishment is allowed, as well as in dam management scenarios where the water level can only be decreased by the decision maker.

For a given discount factor $q \gt 0$ and initial value $x \in \mathbb{R}$ , the objective is to minimize

\begin{align*}v_\pi (x)&\,:\!=\, \mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-qt}f(U^\pi_t)\,\mathrm{d} t + C\int_{0}^\infty {\mathrm{e}}^{-qt} \,\mathrm{d} R^\pi_t \biggr] \\&= \mathbb{E}_x \Biggl[ \int_0^\infty {\mathrm{e}}^{-qt}f(U^\pi_t)\,\mathrm{d} t + C\sum_{0 \leq t \lt \infty\colon N^\eta_t \neq N^\eta_{t-} } {\mathrm{e}}^{-qt } \nu_t^\pi \Biggr],\end{align*}

which is the sum of running costs for a given measurable function $f \colon \mathbb{R} \to\mathbb{R}$ and a controlling cost/reward for a unit cost/reward $C \in \mathbb{R}$ (cost if it is positive and reward if negative). Let $\mathcal{A}$ be the set of all admissible strategies satisfying the constraints described above as well as the integrability condition:

\begin{align*}&\mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-qt}{|f(U^\pi_t)|}\,\mathrm{d} t + \int_0^\infty {\mathrm{e}}^{-qt} \,\mathrm{d} R^\pi_t \biggr] \\&\quad =\mathbb{E}_x \Biggl[ \int_0^\infty {\mathrm{e}}^{-qt}{|f(U^\pi_t)|}\,\mathrm{d} t + \sum_{0 \leq t \lt \infty\colon N^\eta_t \neq N^\eta_{t-} } {\mathrm{e}}^{-qt } \nu_t^\pi \Biggr]\notag \\&\quad \lt \infty. \end{align*}

The aim of the problem is to obtain the (optimal) value function

\begin{align*}v(x)\,:\!=\, \inf_{\pi\in\mathcal{A}}v_\pi(x),\quad x\in\mathbb{R},\end{align*}

and an optimal strategy $\pi^\ast$ such that $v_{\pi^*}(x) = v(x)$ (if such a strategy exists).

For the running cost function f, the unit cost/reward C and the Lévy process X, we impose the same conditions as those assumed in [Reference Noba and Yamazaki25]; similar conditions are commonly assumed in the literature (see [Reference Benkherouf and Bensoussan5], [Reference Bensoussan, Liu and Sethi7], [Reference Hernández-Hernández, Pérez and Yamazaki17], and [Reference Yamazaki37]).

Note that the right- and left-hand derivatives $f^\prime_+ (x)$ and $f^\prime_- (x)$ , respectively, for all $x \in \mathbb{R}$ as well as their limits are well-defined by Assumption 2.1(1). Assumption 2.1(3) is necessary to avoid the optimality of a trivial strategy and the case optimal strategy does not exist; see [Reference Noba and Yamazaki25, Remark 1]. More precisely, when this assumption is violated, the optimal strategy is never to modify the process, or to move the process to an arbitrarily large value.

Remark 2.2. By Assumption 2.2(1) and [Reference Bertoin8, Proposition I.15], the potential measure of X has no atoms. This also shows that

\[ \mathbb{P}_x (X_{T(1)}= b) ={\eta}\mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-{\eta}t} 1_{\{X_t =b\}} \,\mathrm{d} t\biggr]=0 \quad\text{for all $x,b \in \mathbb{R}$.} \]

Assumption 2.2(2) together with [Reference Kyprianou19, Theorem 3.6] guarantees the finiteness of $\mathbb{E} [ \exp (\bar{\theta} {|X_1|})]$ and also that of $\mathbb{E} [|X_1|]$ (since $\exp(x) \geq x$ for $x\geq0$ ).

Remark 2.3. From Assumptions 2.1(2) and 2.2(2) and by the proof of [Reference Yamazaki37, Lemma 11], the expectation

\[\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} |f(X_t)|\,\mathrm{d} t \biggr]\]

is finite and it is at most of polynomial growth as $x\uparrow \infty$ and $x\downarrow-\infty$ .

3. Periodic barrier strategies

Our objective is to show the optimality of a periodic barrier strategy $\pi^b$ for a suitable selection of the barrier $b \in \mathbb{R}$ in the considered stochastic control problem.

Fix $b \in \mathbb{R}$ . A periodic barrier strategy $\pi^b$ pushes the process upward to b whenever it is observed to be below b (see Figure 1). The epochs of controlling $\bigl\{T_b^{(n)}\colon n\in\mathbb{N} \bigr\} \subset{\mathcal{T}_{\eta}}$ are given by a sequence of $\mathbb{F}$ -stopping times, recursively defined as follows: with $T_b^{(0)} \,:\!=\, 0$ ,

\begin{align*}\begin{aligned}T_b^{(n)} &= \inf\bigl\{t \in \mathcal{T}_{\eta}\colon t \gt T_b^{(n-1)},\tilde{X}_t^{(n-1),b} \lt b\bigr\},\quad n\geq 1,\end{aligned}\end{align*}

where

\[\tilde{X}_t^{(m),b} \,:\!=\,\begin{cases}X_{t}, & m = 0,\\b+\bigl(X_t-X_{T_b^{(m)}}\bigr), & m \geq 1,\end{cases}\]

is a parallel shift of X so that it starts from b at $T_b^{(m)}$ when $m \geq 1$ . The strategy $\pi^b$ modifies X by adding at $T_b^{(n)}$ the shortage $b - \tilde{X}_{T_b^{(n)}}^{(n-1),b}$ so that the path of the controlled process is the concatenation of $(\tilde{X}^{(m),b})_{m \geq 1}$ . The corresponding control and controlled processes, respectively, can be written as

\begin{align*}R_t^b &\,:\!=\, R_t^{\pi^b} = \sum_{n = 1}^\infty \Bigl(b - \tilde{X}_{T_b^{(n)}}^{(n-1),b}\Bigr) 1_{\{ T_b^{(n)} \leq t \}}, \\U^b_t &\,:\!=\, U^{\pi^b}_t = X_t + R^b_t= \sum_{n=1}^\infty \tilde{X}_t^{(n-1),b} 1_{\{ t\in [T^{(n-1)}_b, T^{(n)}_b )\}}.\end{align*}

Note that X and $N^\eta$ do not jump at the same time.

Figure 1.

Sample path of $U^b$ . Control opportunities $\mathcal{T}_{\eta}$ are shown by dotted vertical lines. The control times $\{T_b^{(n)}\colon n\in\mathbb{N} \}$ and control sizes $\Delta R^b$ are indicated by the vertical red lines.

Alternatively, in terms of the càglàd $\mathbb{F}$ -adapted process $\nu^b=\{\nu^b_t \colon t\geq 0\}$ with

\begin{align*}\nu^b_t =\begin{cases}(b - X_{t-})^+, & t \in \bigl[ 0 , T^{(1)}_b \bigr],\\\Bigl( X_{T_b^{(n-1)}}- X_{t-}\Bigr)^+, & t\in\bigl(T^{(n-1)}_b, T^{(n)}_b \bigr]\text{ with } n\geq 2 ,\end{cases}\end{align*}

where $x^+ \,:\!=\, x \vee 0$ and it is understood that $X_{0-}=X_0$ , it can be also written as

(3.1) \begin{align}R^b_t = \int_{[0,t]} \nu^b_s \,\mathrm{d} N^{\eta}_s, \quad t\geq 0. \end{align}

Remark 3.1. In terms of the minimum of X observed until time t, we can also write

\[R_t^b = \max_{1 \leq k \leq N_t^\eta} (b-X_{T(k)})^+, \quad t \geq 0,\]

and $T^{(n)}_b$ as the nth jump time of $R^b$ . This expression will be used to show the convergence to the classical case in Section 5.

For the rest of the paper, we denote the expected total cost under the periodic barrier strategy $\pi^b$ by

\begin{align*}v_b (x)\,:\!=\, v_{\pi^b} (x), \quad x \in \mathbb{R}.\end{align*}

We now show the admissibility of periodic barrier strategies along with related results. The proof of the following lemma is deferred to Appendix A.1.

As a corollary of the above, we also have the following. Thanks to Assumption 2.1(1), this can be shown exactly in the same way as the proof of [Reference Noba and Yamazaki25, Lemma 4] and thus we omit the proof.

Corollary 3.1. For $x,b \in \mathbb{R}$ , we have

\[\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} |f^\prime_+ (U^b_t)|\,\mathrm{d} t \biggr] \lt \infty.\]

Lemma 3.1 together with (3.1) shows the following.

Let $T_b \,:\!=\, T_b^{(1)}$ be the first control time under the policy $\pi^b$ . We conclude this section with the expression of the slope of $v_b$ written in terms of $T_b$ and the uncontrolled Lévy process X.

Proposition 3.2. For $b \in \mathbb{R}$ , the function $v_b$ is continuously differentiable with its derivative

(3.2) \begin{align}v_b^\prime (x)= \mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(X_t) \,\mathrm{d} t \biggr]-C\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_b}\bigr], \quad x \in \mathbb{R}. \end{align}

The proof of Proposition 3.2 requires the following continuity result of $T_b$ ; its proof is deferred to Appendix A.2.

Proof of Proposition 3.2. By Lemma 3.1, we can decompose the expected costs as follows:

\begin{align*}v_b (x) = v_b^{(1)}(x) + C v_b^{(2)}(x), \quad x\in\mathbb{R},\end{align*}

where we write

\begin{align*}v_b^{(1)}(x) \,:\!=\, \mathbb{E}_x\biggl[ \int_0^\infty {\mathrm{e}}^{-qt} f(U^b_t)\,\mathrm{d} t \biggr], \quad v_b^{(2)}(x) \,:\!=\, \mathbb{E}_x\biggl[ \int_0^\infty {\mathrm{e}}^{-qt} \,\mathrm{d} R^b_t \biggr].\end{align*}

For $y\in\mathbb{R}$ , we write $X^{[y]}_t\,:\!=\, X_t +y$ , $t\geq 0$ , and let $U^{[y],b}$ , $R^{[y],b}_t$ , $T^{[y]}_b$ be those corresponding to this shifted process.

(i) Fix $b \in \mathbb{R}$ and $\varepsilon \gt 0$ . We show that $t \mapsto U^{[\varepsilon], b}_t-U^b_t= \varepsilon+ R^{[\varepsilon], b}_t-R^b_t$ is non-increasing and always lies on $[0, \varepsilon]$ . Because this difference is a step function in t with jump times contained in the set $\mathcal{T}_\eta$ , it suffices to show that

\[\zeta(k) \,:\!=\, U^{[\varepsilon],b}_{T(k)}-U^b_{T(k)}=\varepsilon + R^{[\varepsilon],b}_{T(k)}-R^b_{T(k)}, \quad k \geq 0,\]

is non-increasing in k and takes values only on $[0,\varepsilon]$ . We show this claim by induction.

First it holds trivially when $k = 0$ with $\zeta(0) = {\varepsilon} \in [0,\varepsilon]$ .

Now, suppose it holds that $\zeta(k) \in [0,\varepsilon]$ for some $k \geq 0$ . With the set of indices of controlling,

\[A^{[\delta]} \,:\!=\, \bigl\{ k \geq 1\colon \Delta R_{T(k)}^{[\delta], b} \gt 0\bigr\} = \bigl\{ k \geq 1\colon U^{[\delta],b}_{T(k-1)}+(X_{T(k)} - X_{T(k-1)}) \lt b \bigr\}, \quad \delta = 0, \varepsilon,\]

we have

(3.3) \begin{align}\{ k+1 \in A^{[\varepsilon]} \}&= \{ U^{b}_{T(k)} + \zeta(k) +(X_{T(k+1)} - X_{T(k)}) \lt b \} \nonumber \\ &\subset \{ U^{b}_{T(k)}+(X_{T(k+1)} - X_{T(k)}) \lt b \} \nonumber \\ &= \{ k+1 \in A^{[0]} \}. \end{align}

(A) Suppose $k+1 \in A^{[\varepsilon]}$ so that $U^{[\varepsilon],b}_{T(k+1)} = b$ . By (3.3), this also implies $k+1 \in A^{[0]}$ or equivalently $U^{b}_{T(k+1)} = b$ . Hence, $\zeta(k+1) = 0$ .

(B) Suppose $k+1 \notin A^{[\varepsilon]}$ so that

(3.4) \begin{align}U^{[\varepsilon],b}_{T(k)}+(X_{T(k+1)} - X_{T(k)}) \geq b. \end{align}

1. (a) Suppose $k+1 \notin A^{[0]}$ , then because $\Delta R^{[\varepsilon],b}_{T(k+1)} = \Delta R^{b}_{T(k+1)} = 0$ , we have $\zeta(k+1) = \zeta(k) \in [0,\varepsilon]$ .

2. (b) Suppose $k+1 \in A^{[0]}$ . Then, clearly $\zeta(k+1) = \zeta(k) - \Delta R^{b}_{T(k+1)} \lt \zeta(k)$ . In addition, by (3.4),

   \[ \zeta(k+1) = \bigl(U^{[\varepsilon],b}_{T(k)}+(X_{T(k+1)} - X_{T(k)})\bigr) - b \geq 0. \]

In sum, in all cases we have $\zeta(k+1) \leq \zeta(k)$ , and in addition, $\zeta(k+1) \in [0, \varepsilon]$ . By mathematical induction we have that $\zeta$ is non-increasing and always lies in $[0, \varepsilon]$ . In view of (3.3), this also shows $A^{[\varepsilon]} \subset A^{[0]}$ .

At the moment $T^{[\varepsilon]}_b = \inf_{k \in A^{[\varepsilon]}} T(k)$ with $\inf \varnothing = \infty$ , the difference between $U^{[\varepsilon], b}$ and $U^{b}$ becomes 0 and must stay at 0 afterwards. On the other hand, before $T_{b}$ there is no control for both and the difference is $\varepsilon$ . In sum,

(3.5) \begin{align}U^{[\varepsilon], b}_t-U^b_t=\begin{cases}\varepsilon, &t\in[0, T_b), \\0, &t\in\bigl[{T^{[\varepsilon]}_b}, \infty\bigr),\end{cases}\quad R^{[\varepsilon], b}_t-R^b_t=\begin{cases}0, &t\in[0, T_b), \\-\varepsilon, &t\in\bigl[{T^{[\varepsilon]}_b}, \infty\bigr).\end{cases}\end{align}

(ii) By (3.5), we have

\begin{align*}\dfrac{v^{(1)}_b(x+\varepsilon)-v^{(1)}_b(x)}{\varepsilon} & = \mathbb{E}_x \biggl[ \int_0^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t + \varepsilon)-f(U^{b}_t ) }{\varepsilon}\,\mathrm{d} t \biggr] \\&\quad + \mathbb{E}_x \biggl[ \int_{T_b}^{T_b^{[\varepsilon]}} {\mathrm{e}}^{-qt} \dfrac{f\bigl(U^{[\varepsilon],b}_t \bigr)-f(U^{b}_t ) }{\varepsilon}\,\mathrm{d} t \biggr].\end{align*}

By (i) (in particular that the process $\{U^{[\varepsilon],b}_t-U^b_t \colon t\geq 0\}$ is non-increasing), mean value theorem, and the convexity of f, for all $0 \lt \varepsilon \lt \bar{\varepsilon}$ ,

\begin{align*}\biggl| \mathbb{E}_x \biggl[ \int_{T_b}^{T_b^{[\varepsilon]}} {\mathrm{e}}^{-qt} \dfrac{f\bigl(U^{[\varepsilon],b}_t \bigr)-f(U^{b}_t ) }{\varepsilon}\,\mathrm{d} t \biggr] \biggr| & \leq \mathbb{E}_x \biggl[ \int_{T_b}^{T_b^{[\varepsilon]}} {\mathrm{e}}^{-qt} \dfrac{|f\bigl(U^{[\varepsilon],b}_t \bigr)-f(U^{b}_t )| }{\varepsilon}\,\mathrm{d} t \biggr] \\& \leq \mathbb{E}_x \biggl[ \int_{T_b}^{T_b^{[\varepsilon]}} {\mathrm{e}}^{-qt} \sup_{y \in [U_t^b, U_t^b+\bar{\varepsilon}]} |f^{\prime}_+(y)| \,\mathrm{d} t\biggr] \\& \leq \mathbb{E}_x \biggl[\int_{T_b}^{T_{b-\varepsilon}} {\mathrm{e}}^{-qt} (|f^{\prime}_+(U_t^b)| + |f^{\prime}_+(U_t^b+\bar{\varepsilon})|) \,\mathrm{d} t\biggr] \\& \xrightarrow{\varepsilon \downarrow 0} 0,\end{align*}

where $T_b^{[\varepsilon]} = T_{b-\varepsilon}$ holds because

\[\bigl\{t \lt T_b^{[\varepsilon]} \bigr\} = \Bigl\{ \min_{1 \leq k \leq N_t^\eta} X_{T(k)}^{[\varepsilon]} \geq b\Bigr\} = \Bigl\{ \min_{1 \leq k \leq N_t^\eta} X_{T(k)} \geq b-\varepsilon\Bigr\} = \{t \lt {T_{b-\varepsilon}}\}\]

(see Remark 3.1) and the last limit holds by monotone convergence and Lemma 3.2. Note that the finiteness of the expectations above hold by Corollary 3.1. Now, by the convexity of f, monotone convergence gives

\begin{align*}\lim_{\varepsilon\downarrow 0}\dfrac{v^{(1)}_b(x+\varepsilon)-v^{(1)}_b(x)}{\varepsilon}&= \lim_{\varepsilon\downarrow 0}\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t+\varepsilon)-f(U^b_t)}{\varepsilon}\,\mathrm{d} t\biggr]\\&= \mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(U^b_t) \,\mathrm{d} t \biggr].\end{align*}

In the same way, we compute the left derivative. By (i), with x changed to $x-\varepsilon$ , we have

\begin{align*}\dfrac{v^{(1)}_b(x)-v^{(1)}_b(x-\varepsilon)}{\varepsilon} = \mathbb{E}_x \biggl[ \int_0^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t )-f(U^{b}_t -\varepsilon) }{\varepsilon}\,\mathrm{d} t \biggr] + {h(\varepsilon)},\end{align*}

where

\[h(\varepsilon) \,:\!=\, \mathbb{E}_x \biggl[ \int_{T_b^{[-\varepsilon]}}^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t )-f\bigl(U^{[-\varepsilon],b}_t \bigr) }{\varepsilon}\,\mathrm{d} t \biggr]-\mathbb{E}_x \biggl[ \int_{T^{[-\varepsilon]}_b}^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t )-f(U^{b}_t -\varepsilon) }{\varepsilon}\,\mathrm{d} t \biggr].\]

For all $0 \lt \varepsilon \lt \bar{\varepsilon}$ , the mean value theorem and the convexity of f give

\begin{align*}\biggl| \dfrac{f(U^{b}_t )-f(U^{b}_t -\varepsilon) }{\varepsilon} \biggr| \lor\biggl| \dfrac{f(U^{b}_t )-f\bigl(U^{[-\varepsilon],b}_t \bigr) }{\varepsilon}\biggr| \leq|f^{\prime}_+(U_t^b-\bar{\varepsilon})| + |f^{\prime}_+(U_t^b)|, \quad t \geq 0,\end{align*}

and thus

\[ {|h(\varepsilon)|}\leq 2\mathbb{E}_x \biggl[\int_{ {T_{b+\varepsilon}}}^{T_b} {\mathrm{e}}^{-qt} (|f^{\prime}_+(U_t^b-\bar{\varepsilon})| + |f^{\prime}_+(U_t^b)|) \,\mathrm{d} t\biggr] \xrightarrow{\varepsilon \downarrow 0} 0 , \]

where we used $T^{[-\varepsilon]}_b=T_{b+\varepsilon}$ and Lemma 3.2. Therefore, as in the case of the right derivative, we have by monotone convergence

\begin{align*}\lim_{\varepsilon\downarrow 0}\dfrac{v^{(1)}_b(x)-v^{(1)}_b(x-\varepsilon)}{\varepsilon}&= \lim_{\varepsilon\downarrow 0}\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} \dfrac{f(U^{b}_t)-f(U^b_t-\varepsilon)}{\varepsilon}\,\mathrm{d} t\biggr]\\&= \mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_-(U^b_t) \,\mathrm{d} t \biggr].\end{align*}

Because the right and left derivatives coincide thanks to Remark 2.2 and $U^{b}_t = X_t$ for $t \lt T_b$ ,

\[v_b^{(1) \prime} (x)=\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(U_t^b) \,\mathrm{d} t \biggr]=\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(X_t) \,\mathrm{d} t \biggr].\]

(iii) We now show

(3.6) \begin{align}\lim_{\varepsilon\downarrow0}\dfrac{v^{(2)}_{b}(x+\varepsilon)-v^{(2)}_b(x)}{\varepsilon}=\lim_{\varepsilon\downarrow0}\dfrac{v^{(2)}_{b}(x)-v^{(2)}_b(x-\varepsilon)}{\varepsilon}= -\mathbb{E}_x \bigl[{\mathrm{e}}^{-qT_b}\bigr]. \end{align}

Indeed, since the process $\bigl\{R^{[\varepsilon],b}_t-R^{b}_t \colon t\geq 0\bigr\}$ is non-increasing by (i) and from (3.5), we have

(3.7) \begin{align}\begin{aligned}- \mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_b}\bigr]\leq\dfrac{v^{(2)}_{b}(x+\varepsilon)-v^{(2)}_b(x)}{\varepsilon}\leq -\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_{b+\varepsilon}}\bigr].\end{aligned}\end{align}

By Lemma 3.2 and (3.7), we have that the first term in (3.6) is equal to the third term in (3.6). By changing from x to $x-\varepsilon$ in the above argument, we have the second equality in (3.6).

From (ii) and (iii), we obtain (3.2).

(iv) It remains to show that $x \mapsto v^\prime_b(x)$ is continuous. We have

\begin{align*}&|v_{b}^\prime (x+\varepsilon)-v_{b}^\prime (x)|\\&\quad \leq\biggl|\mathbb{E}_{x+\varepsilon} \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(X_t) \,\mathrm{d} t \biggr]-\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(X_t) \,\mathrm{d} t \biggr]\biggr|\\&\quad\quad+|C|\bigl|\mathbb{E}_{x+\varepsilon} \bigl[ {\mathrm{e}}^{-qT_b}\bigr]-\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_b}\bigr]\bigr|\\& \quad=\biggl|\mathbb{E}_{x} \biggl[\int_0^{T_{b-\varepsilon}} {\mathrm{e}}^{-qt} f^\prime_+(X_t+\varepsilon) \,\mathrm{d} t \biggr]-\mathbb{E}_x \biggl[\int_0^{T_b} {\mathrm{e}}^{-qt} f^\prime_+(X_t) \,\mathrm{d} t \biggr]\biggr|\\&\quad\quad+|C|\biggl|\mathbb{E}_{x} \bigl[ {\mathrm{e}}^{-qT_{b-\varepsilon}}\bigr]-\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_b}\bigr]\biggr|\\&\quad\leq\mathbb{E}_x \biggl[ \int_0^{T_b} {\mathrm{e}}^{-qt} | f^\prime_+(X_t+\varepsilon)- f^\prime_+(X_t) | \,\mathrm{d} t \biggr]+\mathbb{E}_x \biggl[ \int_{T_b}^{T_{b-\varepsilon}} {\mathrm{e}}^{-qt} | f^\prime_+(X_t+\varepsilon) | \,\mathrm{d} t \biggr]\\&\quad \quad +|C|\bigl|\mathbb{E}_{x} \bigl[ {\mathrm{e}}^{-qT_{b-\varepsilon}}\bigr]-\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_b}\bigr]\bigr|.\end{align*}

As $\varepsilon \searrow 0$ , the first expectation converges to zero by monotone convergence, the second expectation converges to zero by monotone convergence and Lemma 3.2. The last expectation converges to zero by Lemma 3.2. By replacing x with $x-\varepsilon$ and again using Remark 2.2, we also have the left continuity.

4. The optimal barrier $ \boldsymbol{b}^\ast$ in the periodic barrier strategies

In this section, we show the optimality of a periodic barrier strategy. Define

(4.1) \begin{align}\rho(b)\,:\!=\, \mathbb{E}_b \biggl[\int_0^\infty {\mathrm{e}}^{-qt} f^\prime_+ (U^b_t) \,\mathrm{d} t \biggr],\quad b\in\mathbb{R}, \end{align}

which takes real values by Corollary 3.1. Our candidate optimal barrier is

(4.2) \begin{align}b^\ast\,:\!=\, \inf\{b\in\mathbb{R}\colon \rho (b) +C\geq 0\},\end{align}

which is well-defined by Lemma 4.1 below; see Appendix A.3 for the proof.

We now state the main result of the paper.

In the remaining part, we show Theorem 4.1. Acting on a measurable function $g\colon \mathbb{R}\to\mathbb{R}$ belonging to $C^1(\mathbb{R})$ (resp. $C^2(\mathbb{R})$ ) when X has bounded (resp. unbounded) variation paths with at most polynomial growth, define the operator

\begin{align*} \mathcal{L} g(x)& \,:\!=\, \gamma g^\prime (x)+\dfrac{1}{2}\sigma^2 g^{\prime \prime}(x) \\&\quad +\int_{\mathbb{R} \backslash \{0\} } (g(x+z)-g(x) -g^\prime (x) z1_{\{|z| \lt 1\}}) \Pi(\mathrm{d} z), \quad x \in \mathbb{R}.\end{align*}

Let $(\mathcal{L} - q) g \,:\!=\, \mathcal{L} g - q g$ . Define also, for any measurable function $g\colon \mathbb{R}\to\mathbb{R}$ ,

\begin{align*}\mathcal{M} g(x)&\,:\!=\, \inf_{l\geq 0}\{Cl +g(x+l)\}, \quad x \in \mathbb{R}.\end{align*}

The following verification lemma gives a sufficient condition for optimality. The proof is the same as that for the spectrally negative case in [Reference Pérez, Yamazaki and Bensoussan30, Lemma 3.1] and hence we omit it.

Lemma 4.2. (Verification lemma.) Let $w\colon \mathbb{R} \to \mathbb{R}$ be of polynomial growth and belong to $C^1(\mathbb{R})$ (resp. $C^2(\mathbb{R})$ ) when X has bounded (resp. unbounded) variation paths. If it satisfies

\begin{align*}(\mathcal{L} -q)w(x)+ {\eta} (\mathcal{M} w(x) -w (x)) + f(x)=0, \quad x\in\mathbb{R}, \end{align*}

then we have $w(x) \leq v(x)$ for $x\in\mathbb{R}$ .

Before confirming these sufficient conditions for $w = v_{b^*}$ , we explicitly compute $\mathcal{M} v_{b^*}$ . To this end, we show the following, which is a direct consequence of Proposition 3.2.

Lemma 4.3. For $x\in\mathbb{R}$ , we have

\[v_{b^\ast}^\prime (x) = \mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} f^\prime_+ (U^{b^\ast}_t) \,\mathrm{d} t\biggr].\]

Proof. By Lemma 4.1 and the definition of $b^\ast$ , we have $\rho (b^\ast)+C=0$ . This together with the strong Markov property gives

\[-C\mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_{b^\ast}}\bigr] = \mathbb{E}_x \bigl[ {\mathrm{e}}^{-qT_{b^\ast}}\bigr]\rho (b^\ast) =\mathbb{E}_x \biggl[\int_{T_{b^\ast}}^\infty {\mathrm{e}}^{-qt} f^\prime_+(U^{b^\ast}_t) \,\mathrm{d} t \biggr],\]

where we recall that $T_{b^\ast} \,:\!=\, T_{b^\ast}^{(1)}$ is the first control time under the policy $\pi^{b^\ast}$ . Substituting this in (3.2) gives the result.

From part (i) of the proof of Proposition 3.2, for each $t \geq 0$ , $U_t^{b^*}$ is monotonically increasing in the start value $X_0 = x$ . By this and Lemma 4.3, the derivative $v_{b^\ast}^{\prime}$ is non-decreasing. In addition, by the definition of $b^\ast$ and the continuity of $\rho$ as in Lemma 4.1, we have $v_{b^\ast}^\prime(b^\ast)=-C$ . Thus we have

\begin{align*}v_{b^\ast}^\prime (x)\begin{cases}\leq -C, &x\lt b^\ast, \\\geq -C, &x\geq b^\ast.\end{cases}\end{align*}

Since the derivative of the function $l \mapsto C l + v_{b^\ast} (x+l)$ is equal to $l\mapsto C+v_{b^\ast}^\prime (x+l)$ , it is minimized when $l = (b^* - x)^+$ , showing the following.

Proposition 4.1. We have

\[\mathcal{M} v_{b^\ast}(x)= \begin{cases}v_{b^*}(x), & x \geq b^*, \\ C (b^\ast - x ) + v_{b^*}(b^*), & x \lt b^*.\end{cases}\]

Regarding the smoothness of $v_{b^*}$ , it belongs to $C^1(\mathbb{R})$ by Proposition 3.2. This is sufficient for the case of bounded variation, but care is needed for the unbounded variation case. We temporarily assume the following, to first consider the case when the $C^2$ property of $v_{b^*}$ is guaranteed.

The proof of the following lemma is deferred to Appendix A.4.

We assume Condition 4.1 temporarily for Lemma 4.5. However, Condition 4.1 can be completely relaxed by following the arguments in Section 4.2 of [Reference Noba and Yamazaki25]. We later provide a brief remark on how Condition 4.1 can be removed in the proof of Theorem 4.1.

By Proposition 3.2 and Lemma 4.4, the function ${x \mapsto v_{b^\ast}(x)}$ is sufficiently smooth to apply $\mathcal{L}$ (under Condition 4.1).

Proof. It suffices to show $\mathcal{L} v_{b^\ast}(x) - (q+\eta)v_{b^\ast}(x) + h(x)=0$ with

\begin{align*}h(x) \,:\!=\, f(x) + \eta\mathcal{M} v_{b^\ast}(x) = f(x)+ {\eta} C (b^\ast -x)^+ + {\eta} v_{b^\ast} (x\lor b^\ast),\end{align*}

where the second equality holds by Proposition 4.1. Because $v_{b^*}$ is smooth enough to apply Ito’s formula (see Proposition 3.2 and Lemma 4.4), it is enough to show that the process $\{M_t\colon t\geq 0 \}$ , where

\begin{align*}M_t \,:\!=\, {\mathrm{e}}^{-(q+\eta)t}v_{b^\ast}(X_t)+\int_0^t {\mathrm{e}}^{-(q+\eta)s} h(X_s) \,\mathrm{d} s,\end{align*}

is a local martingale with respect to the natural filtration $\{\mathcal{F}^X_t \colon t\geq 0\}$ generated by X. See the proof of [Reference Biffis and Kyprianou9, (12)].

By the strong Markov property and because $U^{b^*}_t = X_t$ for $t \lt T(1)$ , we have, for $x \in\mathbb{R}$ ,

\begin{align*}v_{b^\ast}(x) &= \mathbb{E}_x\biggl[\int_0^{T(1)} {\mathrm{e}}^{-qt} f({X_t}) \,\mathrm{d} t \biggr]+C\mathbb{E}_x \bigl[ {\mathrm{e}}^{-q T(1)} ({b^\ast- X_{T(1)}} )^+ \bigr]\\&\quad +\mathbb{E}_x \bigl[{\mathrm{e}}^{-qT(1)} v_{b^\ast}({ X_{T(1)}\vee b^*} ) \bigr]\\&=\mathbb{E}_x\biggl[\int_0^{\infty} {\mathrm{e}}^{-(q+\eta)t} f(X_t) \,\mathrm{d} t \biggr]+ \eta C\mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-(q+\eta) t} (b^\ast- X_{t} )^+ \,\mathrm{d} t \biggr] \\&\quad+ \eta \mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-(q+\eta)t} v_{b^\ast}(X_t \lor b^\ast ) \,\mathrm{d} t \biggr]\\&=\mathbb{E}_x\biggl[\int_0^{\infty} {\mathrm{e}}^{-(q+\eta)t} h(X_t) \,\mathrm{d} t \biggr].\end{align*}

This together with the strong Markov property gives, for $t\geq 0$ and $\tau_{[n]}\,:\!=\, \inf\{t \gt 0\colon |X_t| \gt n\}$ with $n\in\mathbb{N}$ ,

\begin{align*}&\mathbb{E}_x\biggl[\int_0^{\infty} {\mathrm{e}}^{-(q+\eta) s} h(X_s) \,\mathrm{d} s \mid \mathcal{F}^X_{t\land \tau_{[n]}} \biggr]\\&\quad=\int_0^{t\land \tau_{[n]}} {\mathrm{e}}^{-(q+\eta)s} h(X_s) \,\mathrm{d} s +\mathbb{E}_x\biggl[\int_{t\land \tau_{[n]}}^{\infty} {\mathrm{e}}^{-(q+\eta)s} h(X_s) \,\mathrm{d} s \mid \mathcal{F}^X_{t\land \tau_{[n]}} \biggr]\\&\quad =\int_0^{t\land \tau_{[n]}} {\mathrm{e}}^{-(q+\eta)s} h(X_s) \,\mathrm{d} s +{\mathrm{e}}^{-(q+\eta) (t\land \tau_{[n]})}v_{b^\ast} (X_{t\land \tau_{[n]}}) = M_{t\land \tau_{[n]}}.\end{align*}

By the tower property of conditional expectations, M is a local martingale.

5. Convergence as $\eta \to \infty$

In this section we verify the convergence of the optimal solutions to the classical case [Reference Noba and Yamazaki25] as the rate of observation $\eta \to \infty$ .

Recall that in the classical case strategy $R^\pi$ is any adapted (with respect to the natural filtration of X) and non-decreasing process, which does not have to be of the form (2.1). The classical barrier strategy with barrier $b \in \mathbb{R}$ is given by $R_t^{b,\infty} = (b-\underline{X}_t)^+$ , where $\underline{X}$ is the running infimum process of X and the corresponding controlled process is $U_t^{b,\infty} = X_t + R_t^{b,\infty}$ , $t \geq 0$ . As obtained in [Reference Noba and Yamazaki25], the barrier strategy $\bigl\{R_t^{b^*_\infty,\infty}\colon t \geq 0\bigr\}$ with barrier

\begin{align*}b^\ast_\infty \,:\!=\, \inf\{b\in\mathbb{R}\colon \rho_\infty (b) +C \geq 0\} \quad \text{for}\ \rho_\infty(b) \,:\!=\, \mathbb{E}_b \biggl[\int_0^\infty {\mathrm{e}}^{-qt} f^\prime_+ (U^{b,\infty}_t) \,\mathrm{d} t \biggr], \quad b \in \mathbb{R},\end{align*}

is optimal; the value function becomes

\[v_\infty^* (x)\,:\!=\, \mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-qt}f\bigl(U_t^{b^\ast_\infty,\infty}\bigr)\,\mathrm{d} t + C\int_{0}^\infty {\mathrm{e}}^{-qt} \,\mathrm{d} R_t^{b^\ast_\infty,\infty}\biggr]\quad\text{for $x \in \mathbb{R}$.}\]

Solely in this section, to spell out the dependence on the rate $\eta$ , we add super/subscript $\eta$ in an obvious way and add $\infty$ for the classical case.

The objective is to show the convergence $b^\ast_{\eta} \to b^*_\infty$ and $v^{*}_{\eta} \to v^{*}_\infty$ , where $b^\ast_{\eta}$ is as defined in (4.2). The results hold except for a very particular case when X is the negative of a subordinator (where the reflected process becomes a constant).

Proof. Let $\{\eta_n\colon {n \in \{0\} \cup \mathbb{N}} \}$ be a strictly increasing (deterministic) sequence such that $\eta_0 = 0$ and $\eta_n \xrightarrow{n \uparrow \infty} \infty$ . Consider, for each n, a Poisson process $M^{n}$ with rate $\lambda_n \,:\!=\, \eta_n - \eta_{n-1} \gt 0$ independent of X, and let

\begin{align*}N^{\eta_n}_t = \sum_{k=1}^n M^{k}_t,\quad t\geq 0.\end{align*}

We assume $\{M^n\colon n \geq 1\}$ are mutually independent and also independent of X. Hence, their superposition $N^{\eta_n}$ becomes a Poisson process with rate $\eta_n$ independent of X. We consider the problems driven by these processes (defined on the same probability space) to show the convergence.

(i) Fix $u \geq 0$ . Let $\bar{\sigma}_n(u) \,:\!=\, \inf \{ s \gt u\colon \Delta N^{\eta_n}_s \neq 0 \}$ and $\underline{\sigma}_n(u) \,:\!=\, \sup \{ s \lt u\colon \Delta N^{\eta_n}_s \neq 0 \}$ , respectively, be the first arrival time after u and the last arrival time before u of $N^{\eta_n}$ (with the understanding $\sup \varnothing = 0$ ). Then

\[\mathbb{P} (\bar{\sigma}_n(u) - u \gt \varepsilon) = \mathbb{P} (N^{\eta_n}_{u + \varepsilon} - N^{\eta_n}_u = 0) = {\mathrm{e}}^{-\varepsilon \eta_n} \xrightarrow{n \uparrow \infty} 0, \quad \varepsilon \gt 0.\]

In other words $\bar{\sigma}_n(u) \xrightarrow{n \uparrow \infty} u$ in probability. Because it is decreasing, the convergence also holds in the a.s.-sense. Similarly, we also have $\underline{\sigma}_n(u) \nearrow u$ a.s. as $n \to \infty$ .

Fix $t \gt 0$ and $G(t)\,:\!=\, \sup\{s\in[0, t]\colon X_{s-}\wedge X_s= {\underline{X}_t} \}$ (with $X_{0-}=X_0$ ). Suppose $G(t) {\in (0, t)}$ . If $X_{G(t)-} \geq X_{G(t)}$ (i.e. X is continuous or jumps downward at G(t)), because X is right-continuous a.s., then

\[X_{\bar{\sigma}^n (G(t))} \xrightarrow{n \uparrow \infty} X_{G(t)} = X_{G(t)} \wedge X_{G(t)-} = \underline{X}_t.\]

If $X_{G(t)} \gt X_{G(t)-}$ (i.e. X jumps upward at G(t)), then

\[X_{\underline{\sigma}^n (G(t))} \xrightarrow{n \uparrow \infty} X_{G(t)-} = X_{G(t)} \wedge X_{G(t)-} = \underline{X}_t.\]

These together with Remark 3.1 give, for any $b \in \mathbb{R}$ ,

\begin{align*}R^{b, \eta_n}_t { = \max_{1 \leq k \leq N_t^{\eta_n}} (b-X_{T(k)})^+}\geq (b-X_{\underline{\sigma}^n (G(t))})^+ \vee (b-X_{\bar{\sigma}^n (G(t))})^+ \xrightarrow{n \uparrow \infty} (b - \underline{X}_t)^+ {= R_t^{b,\infty}}.\end{align*}

For the case $G(t) = 0$ (i.e. $\underline{X}_t = X_0$ ), by slightly modifying the arguments,

\[R^{b, \eta_n}_t\geq (b-X_{\bar{\sigma}^n (0)})^+ \xrightarrow{n \uparrow \infty} (b - X_0)^+ {= R_t^{b,\infty}}.\]

If $G(t) = t$ , we have $X_{\underline{\sigma}^n (t)} \xrightarrow{n \uparrow \infty} X_{t-}$ and hence $R^{b, \eta_n}_t \xrightarrow{n \uparrow \infty} (b - \underline{X}_{t-})^+$ , which differs from $R_t^{b,\infty}$ only when X jumps downward at t.

By these and because $n \mapsto R^{b,\eta_n}_t$ is increasing, we have $R^{b,\eta_n}_t \nearrow R^{b,\infty}_t$ and consequently $U^{b,\eta_n}_t \nearrow U^{b,\infty}_t$ as $n \to \infty$ for a.e. $t \gt 0$ (more specifically all $t \gt 0$ except t at which X jumps downward) for all $ b \in \mathbb{R}$ .

By this, together with the fact that $f_+^{\prime}$ is non-decreasing, we have

\[f^{\prime}_- (U_t^{b, \infty}) \leq \lim_{n \to \infty}f^{\prime}_+ (U_t^{b, \eta_n}) \leq f^{\prime}_+ (U_t^{b, \infty})\quad\text{for a.e.\ $t \gt 0$,}\]

and hence, by the monotone convergence theorem, $n \mapsto \rho_{\eta_n}(b)$ is non-decreasing and $\rho_{\infty}(b-) \leq \lim_{n \to \infty} \rho_{\eta_n}(b) \leq \rho_{\infty}(b)$ for all $b \in \mathbb{R}$ .

This shows that $b^\dagger \,:\!=\, \lim_{n \to \infty} b^*_{\eta_n}$ exists and $b^\dagger \geq b^*_\infty$ . The monotonicity also suggests that $\rho_{\eta_n} (b^\dagger) \leq {-C}$ uniformly in n and hence $\rho_{\infty} (b^\dagger{-}) \leq {-C}$ . If $b^\dagger \gt b^*_\infty$ , then we must have $\rho_{\infty} ((b^*_\infty + b^\dagger)/2) \leq {-C}$ . However, as shown in [Reference Noba and Yamazaki25, Lemma 5], $\rho_\infty (b) \gt -C$ for $b \gt b^\ast_\infty$ for the case when X is not the negative of a subordinator. For the case when it is the negative of a subordinator, we have $\rho_\infty(b) = f^{\prime}_+(b)/q$ , which is strictly increasing at $b = b^*_\infty$ by assumption and hence the contradiction can be derived similarly. Hence we must have $b^\dagger = b^*_\infty$ , as desired.

(ii) Fix $N \in \mathbb{N}$ and $x\in\mathbb{N}$ . Because, for $n \geq N$ , $b^*_\infty \leq b^*_{\eta_n} \leq b^*_{\eta_N}$ and hence $U_t^{b^*_\infty,\eta_N} \leq U_t^{b^*_{\eta_n},\eta_n} \leq U_t^{b^*_{\eta_N}, \infty}$ and by the convexity of f,

\begin{align*}&\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} \sup_{n \geq N}\bigl|f \bigl(U_t^{b^*_{\eta_n},\eta_n}\bigr)\bigr| \,\mathrm{d} t\biggr] \\ &\quad \leq \mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} \bigl(\bigl| f \bigl(U_t^{b^*_\infty,\eta_N}\bigr) \bigr| + \bigl| f \bigl(U_t^{b^*_{\eta_N}, \infty}\bigr) \bigr|+c\bigr)\,\mathrm{d} t\biggr] \\ &\quad \lt \infty,\end{align*}

where c is a constant value defined in (A.2). On the other hand,

\[\bigl|U^{b^{\ast}_{\eta_n},\eta_n}_t - U^{b^{\ast}_{\infty},\infty}_t \bigr| \leq \bigl|U^{b^{\ast}_{\eta_n},\eta_n}_t - U^{b^{\ast}_{\infty},\eta_n}_t\bigr| + \bigl|U^{b^{\ast}_{\infty},\eta_n}_t - U^{b^{\ast}_{\infty},\infty}_t\bigr| \xrightarrow{n \uparrow \infty} 0\]

by Remark 3.1 and (i) for a.e. $t \gt 0$ . Hence dominated convergence gives the pointwise convergence of

\[\mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-qt}f\bigl(U_t^{b^\ast_{\eta_n},\eta_n}\bigr)\,\mathrm{d} t \biggr]\]

to

\[\mathbb{E}_x \biggl[ \int_0^\infty {\mathrm{e}}^{-qt}f\bigl(U_t^{b^\ast_\infty,\infty}\bigr)\,\mathrm{d} t \biggr]\quad\text{for all $x \in \mathbb{R}$.}\]

On the other hand, by integration by parts,

\begin{align*}&\biggl| \mathbb{E}_x \biggl[{\int_0^\infty}{\mathrm{e}}^{-qt}\,\mathrm{d} R_t^{b^*_{\eta_n},\eta_n} \biggr] - \mathbb{E}_x \biggl[\int_{[0, \infty)} {\mathrm{e}}^{-qt}\,\mathrm{d} R_t^{b^*_{\infty},\infty} \biggr] \biggr| \\&\quad=q\biggl| \mathbb{E}_x \biggl[{\int_0^\infty} {\mathrm{e}}^{-qt} R_t^{b^*_{\eta_n},\eta_n} \,\mathrm{d} t\biggr] - \mathbb{E}_x \biggl[{\int_0^\infty} {\mathrm{e}}^{-qt} R_t^{b^*_{\infty},\infty} \,\mathrm{d} t\biggr] \biggr|\\&\quad\leq q \mathbb{E}_x \biggl[{\int_0^\infty}{\mathrm{e}}^{-qt} \bigl| R_t^{b^*_{\eta_n},\eta_n}- R_t^{b^*_{\infty},\infty}\bigr| \,\mathrm{d} t\biggr].\end{align*}

Here,

\[\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} \sup_{n \geq N}\bigl| R_t^{b^*_{\eta_n},\eta_n}- R_t^{b^*_{\infty},\infty} \bigr| \,\mathrm{d} t\biggr]\leq 2\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt} R_t^{b^*_{\eta_N},\infty} \,\mathrm{d} t \biggr] \lt \infty\]

thanks to

\[R_t^{b^*_{\eta_n},\eta_n}\lor R_t^{b^*_{\infty},\infty} \leq R_t^{b^*_{\eta_N}, \infty}\quad\text{for all $t \gt 0$.}\]

Thus, by the dominated convergence theorem together with

\[\bigl|R^{b^{\ast}_{\eta_n},\eta_n}_t - R^{b^{\ast}_{\infty},\infty}_t \bigr|=\bigl|U^{b^{\ast}_{\eta_n},\eta_n}_t - U^{b^{\ast}_{\infty},\infty}_t \bigr| \xrightarrow{n \uparrow \infty} 0\quad \text{for a.e.\ $t \gt 0$,}\]

we have the pointwise convergence of

\[\mathbb{E}_x \biggl[\int_0^\infty {\mathrm{e}}^{-qt}\mathrm{ d} R_t^{b^*_{\eta_n},\eta_n} \biggr]\quad\text{to}\quad\mathbb{E}_x \biggl[\int_{[0, \infty)} {\mathrm{e}}^{-qt}\,\mathrm{d} R_t^{b^*_{\infty},\infty} \biggr]\quad\text{for all $x \in \mathbb{R}$.} \]

Finally, because $n \mapsto v^{\ast}_{\eta_n}(x)$ is monotone and each value function is continuous in x, $\lim_{n \uparrow \infty}v^{\ast}_{\eta_n}(x) = v^{\ast}_\infty(x)$ holds uniformly in x on any compact set by Dini’s theorem.

6. Numerical results

In this section we confirm the obtained results through numerical experiments via Monte Carlo simulation (classical Euler scheme). In order to confirm that the results hold for a wide class of Lévy processes, we choose a Lévy process X of the form

\begin{equation*} X_t = X_0 -0.1 t+ 0.2 B_t + \sum_{n=1}^{N_t^+} Z_n^{+} - \sum_{n=1}^{N_t^-} Z_n^{-}, \quad 0\le t \lt \infty, \end{equation*}

where $\{B_t\colon t\ge 0\}$ is a standard Brownian motion and $\{N_t^+\colon t\ge 0 \}$ and $\{N_t^-\colon t\ge 0 \}$ are Poisson processes with arrival rates $0.4$ and $0.6$ , respectively. The upward and downward jumps $\{ Z_n^+\colon n \in \mathbb{N} \}$ and $\{ Z_n^-\colon n \in \mathbb{N} \}$ are i.i.d. sequences of (folded) normal random variables with mean zero and variance 1 and Weibull random variables with shape parameter 2 and scale parameter 1, respectively. These processes are assumed to be mutually independent.

For the running cost function f, we consider the following three cases:

(6.1) \begin{equation} \begin{aligned} f_1(x) &\,:\!=\, x^2, \quad f_2(x) \,:\!=\, x^3 1_{\{x \geq 0\}} + x^2 1_{\{x \lt 0\}}, \\\quad f_3(x) &\,:\!=\, \bigl[ x^2 + {\mathrm{e}}^{-(x-1)} \bigr] 1_{\{x \geq 1\}} + \dfrac {x^2+3} 2 1_{\{x \lt 1\}}, \end{aligned} \end{equation}

for $x \in \mathbb{R}$ , which are convex and continuously differentiable on $\mathbb{R}$ . For other parameters, we set $q=0.05$ and $C = 1$ . For each realization, we truncate the time horizon to $T = 100$ and discretize [0,T] using $N = 10\,000$ equally spaced points with distance $\Delta_t \,:\!=\, T/N$ . Unless stated otherwise, we use $\eta = 1$ .

For the approximation of the expectation, we first obtain a set of $M \,:\!=\, 5\,000$ sample paths of X started at zero, say

\[\hat{\mathbf{X}} \,:\!=\, (\hat{X}^{(1)}, \ldots, \hat{X}^{(M)})\quad{with}\quad \hat{X}^{(m)} = \bigl\{\hat{X}^{(m)}_{n \Delta_t}\colon 1 \leq n \leq N \bigr\}\quad \text{for $1 \leq m \leq M$.}\]

Control opportunities

\[\hat{\mathbf{N}}^\eta \,:\!=\, (\hat{N}^{\eta,(1)}, \ldots, \hat{N}^{\eta,(M)})\quad\text{with}\quad \hat{N}^{\eta,(m)} = \bigl\{\hat{N}^{\eta,(m)}_{n \Delta_t }\colon 1 \leq n \leq N \bigr\}\quad \text{for $1 \leq m \leq M$}\]

are sampled by generating

\[\hat{N}^{\eta,(m)}_{(n+1) \Delta_t} - \hat{N}^{\eta,(m)}_{n \Delta_t} = 1_{\{\mathrm{e} \lt \Delta_t \}}\quad \text{with i.i.d.\ $\mathrm{e} \sim \exp(\eta)$}\]

and their corresponding reflected paths (with barrier zero)

\[\hat{U}^{0,(m)} = \bigl\{\hat{U}^{0,(m)}_{n \Delta_t }\colon 1 \leq n \leq N \bigr\}\]

are then computed. These sample paths can be used commonly for the approximation of the expectation in $\rho(b)$ as in (4.1). In other words, we approximate it by

\[\hat{\rho}_M(b) \,:\!=\, M^{-1} \sum_{m=1}^M \Delta_t \sum_{n=0}^{N} {\mathrm{e}}^{-q n \Delta_t } f^{\prime}\bigl(\hat{U}^{0,(m)}_{n \Delta_t }+b\bigr).\]

As shown in Section 3, $\rho(b)$ is monotone and hence $b^*$ can be obtained by classical bisection. While $\hat{\rho}_M(b)$ for each b is an approximated value, because we are using the same sample paths $(\hat{\mathbf{X}}, \hat{\mathbf{N}}^\eta)$ , the monotonicity of $b \to \hat{\rho}_M(b) $ is still preserved, causing no problem in using bisection methods. Figure 2 shows the plots of $\hat{\rho}_M(b)$ for cases i for $i = 1,2,3$ . It can be confirmed that it is indeed monotonically increasing, and the root becomes $b^*$ . Note that for the case $i=1$ , $\rho_M(b)$ becomes a straight line.

Figure 2.

Plot of $\hat{\rho}_M(b)$ for case i under the cost function $f_i$ as in (6.1), for (a) $i = 1$ , (b) $i = 2$ , (c) $i = 3$ . The root (indicated by a star) becomes an approximation of the optimal barrier $b^*$ .

With the approximated optimal barrier $b^*$ , we shall now confirm the optimality by comparing the expected total costs $v_{b^*}$ with $v_b$ under suboptimal choices of b. In order to compute these, we continue using the set of paths $(\hat{\mathbf{X}}, \hat{\mathbf{N}}^\eta)$ . Figure 3 shows the results. It can be confirmed that the selection $b^*$ indeed minimizes the total expected cost for all starting points.

Figure 3.

Plot of the approximated value functions $v_{b^*}$ (solid) along with $v_b$ (dotted) for $b = b^*-1$ , $b^*-0.5$ , $b^*+0.5$ , $b^*+1.0$ , for case i for (a) $i = 1$ , (b) $i = 2$ , (c) $i = 3$ . The points at the barriers are indicated by stars and circles for $b = b^*$ and $b \neq b^*$ , respectively.

Finally, we confirm the convergence as $\eta \to \infty$ . In Figure 4, we plot the value function when $\eta = 2$ , 5, 10, 20, 50, 100, 200, 500, 1000 together with the classical case whose reflected path with lower barrier b under $\mathbb{P}_x$ is approximated by

\[\bigl(\hat{X}_{n\Delta_t }^{(m)} +x \bigr) + \max_{0 \leq l \leq n} \bigl(b- \bigl(\hat{X}_{l \Delta_t }^{(m)}+x\bigr) \bigr)^+.\]

It is observed in all cases that the optimal barrier and the value function converge decreasingly to those of the classical case, confirming Theorem 5.1.

Figure 4.

Plot of the approximated value functions $v_{b^*}$ (dotted) for $\eta = 2$ , 5, 10, 20, 50, 100, 200, 500, 1000 along with that in the classical case (solid) for case $i$ for (a) $i = 1$ , (b) $i = 2$ , (c) $i = 3$ . The points at the barriers are indicated by stars.

7. Concluding remarks

In this paper we have solved the stochastic control problem of minimizing the sum of running and controlling costs under the constraint that control opportunities are restricted to independent Poisson arrival times. For a general Lévy process model, we showed the optimality of a simple barrier strategy, with its barrier analytically provided as a root of the equality (1). Furthermore, we demonstrated that the optimal solutions in the Poissonian setting converge to those in the continuous-observation setting. These results potentially provide a new approach to the classical case, using techniques developed in this paper for Poisson observation models.

One important extension is to consider the case where we can control the process in both directions. This scenario has been studied in the continuous-observation case driven by spectrally negative Lévy processes, as discussed in [Reference Baurdoux and Yamazaki4], where it is shown that it is optimal to reflect the process at both upper and lower barriers.

Another natural extension is to consider the case with a fixed intervention cost. In this case, the optimal strategy is expected to be of the two-barrier type. More specifically, it is expected to be a variant of the (s,S)-policy (see e.g. [Reference Benkherouf and Bensoussan5], [Reference Bensoussan, Liu and Sethi7]), which moves the process to a certain point, say $\bar{b}$ , whenever it is observed to be below a different point, say $\underline{b}$ , at Poisson observation times. This is a reasonable conjecture based on Yamazaki [Reference Yamazaki37], who showed the optimality of such a policy for the continuous-observation case under a spectrally one-sided Lévy model.