Designing efficient and rigorous numerical methods for sequential decision-making under uncertainty is a difficult problem that arises in many applications frameworks. In this paper we focus on the numerical solution of a subclass of impulse control problems for the piecewise deterministic Markov process (PDMP) when the jump times are hidden. We first state the problem as a partially observed Markov decision process (POMDP) on a continuous state space and with controlled transition kernels corresponding to some specific skeleton chains of the PDMP. We then proceed to build a numerically tractable approximation of the POMDP by tailor-made discretizations of the state spaces. The main difficulty in evaluating the discretization error comes from the possible random jumps of the PDMP between consecutive epochs of the POMDP and requires special care. Finally, we discuss the practical construction of discretization grids and illustrate our method on simulations.

A general way to represent stochastic differential equations (SDEs) on smooth manifolds is based on the Schwartz morphism. In this manuscript, we are interested in SDEs on a smooth manifold $M$ that are driven by p-dimensional Wiener process $W_t \in \mathbb{R}^p$ and time $t$. In terms of the Schwartz morphism, such an SDE is represented by a Schwartz morphism that morphs the semimartingale $(t,W_t)\in\mathbb{R}^{p+1}$ into a semimartingale on the manifold $M$. We show that it is possible to construct such Schwartz morphisms using special maps that we call diffusion generators. We show that one of the ways to construct a diffusion generator is by considering the flow of differential equations. One particular case is the construction of diffusion generators using Lagrangian vector fields. Using the diffusion generator approach, we also give the extended Itô formula (also known as generalized Itô formula or Itô–Wentzell formula) for SDEs on manifolds.

We study a continuous-time mutually catalytic branching model on the $\mathbb{Z}^{d}$. The model describes the behavior of two different populations of particles, performing random walk on the lattice in the presence of branching, that is, each particle dies at a certain rate and is replaced by a random number of offspring. The branching rate of a particle in one population is proportional to the number of particles of another population at the same site. We study the long time behavior for this model, in particular, coexistence and noncoexistence of two populations in the long run. Finally, we construct a sequence of renormalized processes and use duality techniques to investigate its limiting behavior.

We study stationary distributions in the context of stochastic reaction networks. In particular, we are interested in complex balanced reaction networks and the reduction of such networks by assuming that a set of species (called non-interacting species) are degraded fast (and therefore essentially absent from the network), implying that some reaction rates are large relative to others. Technically, we assume that these reaction rates are scaled by a common parameter N and let $N\to\infty$. The limiting stationary distribution as $N\to\infty$ is compared with the stationary distribution of the reduced reaction network obtained by elimination of the non-interacting species. In general, the limiting stationary distribution could differ from the stationary distribution of the reduced reaction network. We identify various sufficient conditions under which these two distributions are the same, including when the reaction network is detailed balanced and when the set of non-interacting species consists of intermediate species. In the latter case, the limiting stationary distribution essentially retains the form of the complex balanced distribution. This finding is particularly surprising given that the reduced reaction network could be non-weakly reversible and might exhibit unconventional kinetics.

In this paper, we introduce a new technique to study the distribution in residue classes of sets of integers with digit and sum-of-digits restrictions. From our main theorem, we derive a necessary and sufficient condition for integers with missing digits to be uniformly distributed in arithmetic progressions, extending previous results going back to the work of Erdős, Mauduit and Sárközy. Our approach uses Markov chains and does not rely on Fourier analysis as many results of this nature do. Our results apply more generally to the class of multiplicatively invariant sets of integers. This class, defined by Glasscock, Moreira and Richter using symbolic dynamics, is an integer analogue to fractal sets and includes all missing digits sets. We address uniform distribution in this setting, partially answering an open question posed by the same authors.

We investigate a specific class of irreducible, level-dependent, discrete-time, GI/M/1-type Markov chains. The transition matrices possess a block lower-Hessenberg structure, which shows asymptotic convergence along the rows as the level approaches infinity. Criteria are presented for recurrence, transience, positive recurrence, geometric ergodicity, and geometric transience in terms of elements of the transition matrices. These criteria are established by employing drift functions and matrix-generating functions. Furthermore, we discuss the extension of the main results to the continuous-time case.

We show that the Potts model on a graph can be approximated by a sequence of independent and identically distributed spins in terms of Wasserstein distance at high temperatures. We prove a similar result for the Curie–Weiss–Potts model on the complete graph, conditioned on being close enough to any of its equilibrium macrostates, in the low-temperature regime. Our proof technique is based on Stein’s method for comparing the stationary distributions of two Glauber dynamics with similar updates, one of which is rapid mixing and contracting on a subset of the state space. Along the way, we prove a new upper bound on the mixing time of the Glauber dynamics for the conditional measure of the Curie–Weiss–Potts model near an equilibrium macrostate.

We study sequential optimal stopping with partial reversibility. The optimal stopping problem is subject to implementation delay, which is random and exponentially distributed. Once the stopping decision is made, the decision maker can, by incurring a cost, call the decision off and restart the stopping problem. The optimization criterion is to maximize the expected present value of the total payoff. We characterize the value function in terms of a Bellman principle for a wide class of payoff functions and potentially multidimensional strong Markov dynamics. We also analyse the case of linear diffusion dynamics and characterize the value function and the optimal decision rule for a wide class of payoff functions.

We prove a scaling limit theorem for two-type Galton–Watson branching processes with interaction. The limit theorem gives rise to a class of mixed-state branching processes with interaction used to simulate evolution for cell division affected by parasites. Such processes can also be obtained by the pathwise-unique solution to a stochastic equation system. Moreover, we present sufficient conditions for extinction with probability 1 and the exponential ergodicity in the $L^1$-Wasserstein distance of such processes in some cases.

In this paper, we introduce a unified framework based on the pathwise expansion method to derive explicit recursive formulas for cumulative distribution functions, option prices, and transition densities in multivariate diffusion models. A key innovation of our approach is the introduction of the quasi-Lamperti transform, which normalizes the diffusion matrix at the initial time. This transformation facilitates expansions using uncorrelated Brownian motions, effectively reducing multivariate problems to one-dimensional computations. Consequently, both the analysis and the computation are significantly simplified. We also present two novel applications of the pathwise expansion method. Specifically, we employ the proposed framework to compute the value-at-risk for stock portfolios and to evaluate complex derivatives, such as forward-starting options. Our method has the flexibility to accommodate models with diverse features, including stochastic risk premiums, stochastic volatility, and nonaffine structures. Numerical experiments demonstrate the accuracy and computational efficiency of our approach. In addition, as a theoretical contribution, we establish an equivalence between the pathwise expansion method and the Hermite polynomial-based expansion method in the literature.

We consider a population consisting of two types of individuals, each of which can produce offspring on two different islands (in particular, the islands can be interpreted as active or dormant individuals). We model the evolution of the population of each type using a two-type Feller diffusion with immigration and study the frequency of one type on each island, when the total population size on each island is forced to be constant at a dense set of times. This leads to the solution of a stochastic differential equation, which we call the asymmetric two-island frequency process. We derive properties of this process and obtain a large population limit as the total size of each island tends to infinity. Additionally, we compute the fluctuations of the process around its deterministic limit. We establish conditions under which the asymmetric two-island frequency process has a moment dual. The dual is a continuous-time two-dimensional Markov chain that can be interpreted in terms of mutation, branching, pairwise branching, coalescence, and a novel mixed selection–migration term.

The hard-core model has as its configurations the independent sets of some graph instance $G$. The probability distribution on independent sets is controlled by a ‘fugacity’ $\lambda \gt 0$, with higher $\lambda$ leading to denser configurations. We investigate the mixing time of Glauber (single-site) dynamics for the hard-core model on restricted classes of bounded-degree graphs in which a particular graph $H$ is excluded as an induced subgraph. If $H$ is a subdivided claw then, for all $\lambda$, the mixing time is $O(n\log n)$, where $n$ is the order of $G$. This extends a result of Chen and Gu for claw-free graphs. When $H$ is a path, the set of possible instances is finite. For all other $H$, the mixing time is exponential in $n$ for sufficiently large $\lambda$, depending on $H$ and the maximum degree of $G$.

We study the local limit in distribution of Bienaymé–Galton–Watson trees conditioned on having large sub-populations. Assuming a generic and aperiodic condition on the offspring distribution, we prove the existence of a limit given by a Kesten’s tree associated with a certain critical offspring distribution.

Consider a subcritical branching Markov chain. Let $Z_n$ denote the counting measure of particles of generation n. Under some conditions, we give a probabilistic proof for the existence of the Yaglom limit of $(Z_n)_{n\in\mathbb{N}}$ by the moment method, based on the spinal decomposition and the many-to-few formula. As a result, we give explicit integral representations of all quasi-stationary distributions of $(Z_n)_{n\in\mathbb{N}}$, whose proofs are direct and probabilistic, and do not rely on Martin boundary theory.

We consider the random series–parallel graph introduced by Hambly and Jordan (2004 Adv. Appl. Probab. 36, 824–838), which is a hierarchical graph with a parameter $p\in [0, \, 1]$. The graph is built recursively: at each step, every edge in the graph is either replaced with probability p by a series of two edges, or with probability $1-p$ by two parallel edges, and the replacements are independent of each other and of everything up to then. At the nth step of the recursive procedure, the distance between the extremal points on the graph is denoted by $D_n (p)$. It is known that $D_n(p)$ possesses a phase transition at $p=p_c \;:\!=\;\frac{1}{2}$; more precisely, $\frac{1}{n}\log {{\mathbb{E}}}[D_n(p)] \to \alpha(p)$ when $n \to \infty$, with $\alpha(p) >0$ for $p>p_c$ and $\alpha(p)=0$ for $p\le p_c$. We study the exponent $\alpha(p)$ in the slightly supercritical regime $p=p_c+\varepsilon$. Our main result says that as $\varepsilon\to 0^+$, $\alpha(p_c+\varepsilon)$ behaves like $\sqrt{\zeta(2) \, \varepsilon}$, where $\zeta(2) \;:\!=\; \frac{\pi^2}{6}$.

We consider an optimal stopping problem of a linear diffusion under Poisson constraint where the agent can adjust the arrival rate of new stopping opportunities. We assume that the agent may switch the rate of the Poisson process between two values. Maintaining the lower rate incurs no cost, whereas the higher rate requires effort that is captured by a cost function c. We study a broad class of payoff functions, cost functions and diffusion dynamics, for which we explicitly characterize the solution to the constrained stopping problem. We also characterize the case where switching to the higher rate is always suboptimal. The results are illustrated with two examples.

We prove two-sided bounds on the expected values of several geometric functionals of the convex hull of Brownian motion in $\mathbb {R}^n$ and their inverse processes. This extends some recent results of McRedmond and Xu (2017), Jovalekić (2021), and Cygan, Šebek, and the first author (2023) from the plane to higher dimensions. Our main result shows that the average time required for the convex hull in $\mathbb {R}^n$ to attain unit volume is at most $n\sqrt [n]{n!}$. The proof relies on a novel procedure that embeds an n-simplex of prescribed volume within the convex hull of the Brownian path run up to a certain stopping time. All of our bounds capture the correct order of asymptotic growth or decay in the dimension n.

Let $\pi$ be a probability distribution in $\mathbb{R}^d$ and f a test function, and consider the problem of variance reduction in estimating $\mathbb{E}_\pi(f)$. We first construct a sequence of estimators for $\mathbb{E}_\pi (f)$, say $({1}/{k})\sum_{i=0}^{k-1} g_n(X_i)$, where the $X_i$ are samples from $\pi$ generated by the Metropolized Hamiltonian Monte Carlo algorithm and $g_n$ is the approximate solution of the Poisson equation through the weak approximate scheme recently invented by Mijatović and Vogrinc (2018). Then we prove under some regularity assumptions that the estimation error variance $\sigma_\pi^2(g_n)$ can be as arbitrarily small as the approximation order parameter $n\rightarrow\infty$. To illustrate, we confirm that the assumptions are satisfied by two typical concrete models, a Bayesian linear inverse problem and a two-component mixture of Gaussian distributions.

Recently, there has been much progress in understanding stationary measures for colored (also called multi-species or multi-type) interacting particle systems, motivated by asymptotic phenomena and rich underlying algebraic and combinatorial structures (such as nonsymmetric Macdonald polynomials). In this paper, we present a unified approach to constructing stationary measures for most of the known colored particle systems on the ring and the line, including (1) the Asymmetric Simple Exclusion Process (multi-species ASEP, or mASEP); (2) the $q$-deformed Totally Asymmetric Zero Range Process (TAZRP) also known as the $q$-Boson particle system; (3) the $q$-deformed Pushing Totally Asymmetric Simple Exclusion Process ($q$-PushTASEP). Our method is based on integrable stochastic vertex models and the Yang–Baxter equation. We express the stationary measures as partition functions of new ‘queue vertex models’ on the cylinder. The stationarity property is a direct consequence of the Yang–Baxter equation. For the mASEP on the ring, a particular case of our vertex model is equivalent to the multiline queues of Martin (Stationary distributions of the multi-type ASEP, Electron. J. Probab. 25 (2020), 1–41). For the colored $q$-Boson process and the $q$-PushTASEP on the ring, we recover and generalize known stationary measures constructed using multiline queues or other methods by Ayyer, Mandelshtam and Martin (Modified Macdonald polynomials and the multispecies zero range process: II, Algebr. Comb. 6 (2022), 243–284; Modified Macdonald polynomials and the multispecies zero-range process: I, Algebr. Comb. 6 (2023), 243–284) and Bukh and Cox (Periodic words, common subsequences and frogs, Ann. Appl. Probab. 32 (2022), 1295–1332). Our proofs of stationarity use the Yang–Baxter equation and bypass the Matrix Product Ansatz (used for the mASEP by Prolhac, Evans and Mallick (The matrix product solution of the multispecies partially asymmetric exclusion process, J. Phys. A. 42 (2009), 165004)). On the line and in a quadrant, we use the Yang–Baxter equation to establish a general colored Burke’s theorem, which implies that suitable specializations of our queue vertex models produce stationary measures for particle systems on the line. We also compute the colored particle currents in stationarity.

We show that on every affine Weyl group natural random walks are noise sensitive in total variation.