The two-sided nonlinear boundary crossing probabilities for one-dimensional Brownian motion and related processes have been studied in Fu and Wu (2010) based on the finite Markov chain imbedding technique. It provides an efficient numerical method to computing the boundary crossing probabilities. In this paper we extend the above results for high-dimensional Brownian motion. In particular, we obtain the rate of convergence for high-dimensional boundary crossing probabilities. Numerical results are also provided to illustrate our results.

Inspired by the works of Landriault et al. (2011), (2014), we study the Gerber–Shiu distribution at Parisian ruin with exponential implementation delays for a spectrally negative Lévy insurance risk process. To be more specific, we study the so-called Gerber–Shiu distribution for a ruin model where at each time the surplus process goes negative, an independent exponential clock is started. If the clock rings before the surplus becomes positive again then the insurance company is ruined. Our methodology uses excursion theory for spectrally negative Lévy processes and relies on the theory of so-called scale functions. In particular, we extend the recent results of Landriault et al. (2011), (2014).

We study a classical multitype Galton–Watson process with mutation and selection. The individuals are sequences of fixed length over a finite alphabet. On the sharp peak fitness landscape together with independent mutations per locus, we show that, as the length of the sequences goes to ∞ and the mutation probability goes to 0, the asymptotic relative frequency of the sequences differing on k digits from the master sequence approaches (σe-a - 1)(ak/k!)∑i≥ 1ik/σi, where σ is the selective advantage of the master sequence and a is the product of the length of the chains with the mutation probability. The probability distribution Q(σ, a) on the nonnegative integers given by the above equation is the quasispecies distribution with parameters σ and a.

In this paper we focus on a class of linear Hawkes processes with general immigrants. These are counting processes with shot-noise intensity, including self-excited and externally excited patterns. For such processes, we introduce the concept of the age pyramid which evolves according to immigration and births. The virtue of this approach that combines an intensity process definition and a branching representation is that the population age pyramid keeps track of all past events. This is used to compute new distribution properties for a class of Hawkes processes with general immigrants which generalize the popular exponential fertility function. The pathwise construction of the Hawkes process and its underlying population is also given.

We investigate the dynamics of a susceptible infected recovered (SIR) epidemic model on small networks with different topologies, as a stepping stone to determining how the structure of a contact network impacts the transmission of infection through a population. For an SIR model on a network of $N$ nodes, there are $3^{N}$ configurations that the network can be in. To simplify the analysis, we group the states together based on the number of nodes in each infection state and the symmetries of the network. We derive analytical expressions for the final epidemic size of an SIR model on small networks composed of three or four nodes with different topological structures. Differential equations which describe the transition of the network between states are also derived and solved numerically to confirm our analysis. A stochastic SIR model is numerically simulated on each of the small networks with the same initial conditions and infection parameters to confirm our results independently. We show that the structure of the network, degree of the initial infectious node, number of initial infectious nodes and the transmission rate all significantly impact the final epidemic size of an SIR model on small networks.

A set is shy or Haar null (in the sense of Christensen) if there exists a Borel set and a Borel probability measure μ on C[0, 1] such that and for all f ∈ C[0, 1]. The complement of a shy set is called a prevalent set. We say that a set is Haar ambivalent if it is neither shy nor prevalent.

The main goal of the paper is to answer the following question: what can we say about the topological properties of the level sets of the prevalent/non-shy many f ∈ C[0, 1]?

The classical Bruckner–Garg theorem characterizes the level sets of the generic (in the sense of Baire category) f ∈ C[0, 1] from the topological point of view. We prove that the functions f ∈ C[0, 1] for which the same characterization holds form a Haar ambivalent set.

In an earlier paper, Balka et al. proved that the functions f ∈ C[0, 1] for which positively many level sets with respect to the Lebesgue measure λ are singletons form a non-shy set in C[0, 1]. The above result yields that this set is actually Haar ambivalent. Now we prove that the functions f ∈ C[0, 1] for which positively many level sets with respect to the occupation measure λ ◦ f –1 are not perfect form a Haar ambivalent set in C[0, 1].

We show that for the prevalent f ∈ C[0, 1] for the generic y ∈ f([0, 1]) the level set f –1(y) is perfect. Finally, we answer a question of Darji and White by showing that the set of functions f ∈ C[0, 1] for which there exists a perfect set Pf ⊂ [0, 1] such that fʹ(x) = ∞ for all x ∈ Pf is Haar ambivalent.

We establish a convexity property for the hitting probabilities of discrete random walks in ${\mathbb Z}^d$ (discrete harmonic measures). For d = 2 this implies a recent result on the convexity of the density of certain harmonic measures.

Let N and M be positive integers satisfying 1≤ M≤ N, and let 0< p0 < p1 < 1. Define a process {Xn}n=0∞ on ℤ as follows. At each step, the process jumps either one step to the right or one step to the left, according to the following mechanism. For the first N steps, the process behaves like a random walk that jumps to the right with probability p0 and to the left with probability 1-p0. At subsequent steps the jump mechanism is defined as follows: if at least M out of the N most recent jumps were to the right, then the probability of jumping to the right is p1; however, if fewer than M out of the N most recent jumps were to the right then the probability of jumping to the right is p0. We calculate the speed of the process. Then we let N→ ∞ and M/N→ r∈[0,1], and calculate the limiting speed. More generally, we consider the above questions for a random walk with a finite number l of threshold levels, (Mi,pi) i=1l, above the pre-threshold level p0, as well as for one model with l=N such thresholds.

We consider Markov processes, which describe, e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit steady-state distribution of product form available in the literature, we note that this steady-state distribution does not provide information about the correlation structure in time and space (over nodes). We study this correlation structure via one-step correlations for the queueing-environment process. Although formulas for absolute values of these correlations are complicated, the differences of correlations of related networks are simple and have a nice structure. We therefore compare two networks in a random environment having the same invariant distribution, and focus on the time behaviour of the processes when in such a network the environment changes or the rules for travelling are perturbed. Evaluating the comparison formulas we compare spectral gaps and asymptotic variances of related processes.

In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.

We propose a novel theoretical characterization of the optimal 'Gittins index' policy in multi-armed bandit problems with non-Gaussian, infinitely divisible reward distributions. We first construct a continuous-time, conditional Lévy process which probabilistically interpolates the sequence of discrete-time rewards. When the rewards are Gaussian, this approach enables an easy connection to the convenient time-change properties of a Brownian motion. Although no such device is available in general for the non-Gaussian case, we use optimal stopping theory to characterize the value of the optimal policy as the solution to a free-boundary partial integro-differential equation (PIDE). We provide the free-boundary PIDE in explicit form under the specific settings of exponential and Poisson rewards. We also prove continuity and monotonicity properties of the Gittins index in these two problems, and discuss how the PIDE can be solved numerically to find the optimal index value of a given belief state.

We consider a class of multi-type particle systems whose structure is similar to that of a contact process and show that additivity is equivalent to the existence of a dual process, extending a result of Harris. We prove a necessary and sufficient condition for the model to preserve positive correlations. We then show that complete convergence on Zd holds for a large subclass of models including the two-stage contact process and a household model, and give examples.

Recent inapproximability results of Sly (2010), together with an approximation algorithm presented by Weitz (2006), establish a beautiful picture of the computational complexity of approximating the partition function of the hard-core model. Let λc ( $\mathbb{T}_{\Delta}$ ) denote the critical activity for the hard-model on the infinite Δ-regular tree. Weitz presented an FPTAS for the partition function when λ < λc ( $\mathbb{T}_{\Delta}$ ) for graphs with constant maximum degree Δ. In contrast, Sly showed that for all Δ ⩾ 3, there exists εΔ > 0 such that (unless RP = NP) there is no FPRAS for approximating the partition function on graphs of maximum degree Δ for activities λ satisfying λc ( $\mathbb{T}_{\Delta}$ ) < λ < λc ( $\mathbb{T}_{\Delta}$ ) + εΔ.

We prove that a similar phenomenon holds for the antiferromagnetic Ising model. Sinclair, Srivastava and Thurley (2014) extended Weitz's approach to the antiferromagnetic Ising model, yielding an FPTAS for the partition function for all graphs of constant maximum degree Δ when the parameters of the model lie in the uniqueness region of the infinite Δ-regular tree. We prove the complementary result for the antiferromagnetic Ising model without external field, namely, that unless RP = NP, for all Δ ⩾ 3, there is no FPRAS for approximating the partition function on graphs of maximum degree Δ when the inverse temperature lies in the non-uniqueness region of the infinite tree $\mathbb{T}_{\Delta}$ . Our proof works by relating certain second moment calculations for random Δ-regular bipartite graphs to the tree recursions used to establish the critical points on the infinite tree.

Density dependent Markov population processes in large populations of size N were shown by Kurtz (1970), (1971) to be well approximated over finite time intervals by the solution of the differential equations that describe their average drift, and to exhibit stochastic fluctuations about this deterministic solution on the scale √N that can be approximated by a diffusion process. Here, motivated by an example from evolutionary biology, we are concerned with describing how such a process leaves an absorbing boundary. Initially, one or more of the populations is of size much smaller than N, and the length of time taken until all populations have sizes comparable to N then becomes infinite as N → ∞. Under suitable assumptions, we show that in the early stages of development, up to the time when all populations have sizes at least N 1-α for 1/3 < α < 1, the process can be accurately approximated in total variation by a Markov branching process. Thereafter, it is well approximated by the deterministic solution starting from the original initial point, but with a random time delay. Analogous behaviour is also established for a Markov process approaching an equilibrium on a boundary, where one or more of the populations become extinct.

We give an exponential tail approximation for the extinction time of a subcritical multitype branching process arising from the SIR epidemic model on a random graph with given degrees, where the type corresponds to the vertex degree. As a corollary we obtain a Gumbel limit law for the extinction time, when beginning with a large population. Our contribution is to allow countably many types (this corresponds to unbounded degrees in the random graph epidemic model, as the number of vertices tends to∞). We only require a second moment for the offspring-type distribution featuring in our model.

In this paper we consider an M/M/c queue modified to allow both mass arrivals when the system is empty and the workload to be removed. Properties of queues which terminate when the server becomes idle are firstly developed. Recurrence properties, equilibrium distribution, and equilibrium queue-size structure are studied for the case of resurrection and no mass exodus. All of these results are then generalized to allow for the removal of the entire workload. In particular, we obtain the Laplace transformation of the transition probability for the absorptive M/M/c queue.

In this paper we focus on the finite-horizon optimality for denumerable continuous-time Markov decision processes, in which the transition and reward/cost rates are allowed to be unbounded, and the optimality is over the class of all randomized history-dependent policies. Under mild reasonable conditions, we first establish the existence of a solution to the finite-horizon optimality equation by designing a technique of approximations from the bounded transition rates to unbounded ones. Then we prove the existence of ε (≥ 0)-optimal Markov policies and verify that the value function is the unique solution to the optimality equation by establishing the analog of the Itô-Dynkin formula. Finally, we provide an example in which the transition rates and the value function are all unbounded and, thus, obtain solutions to some of the unsolved problems by Yushkevich (1978).

In this paper we describe a perfect simulation algorithm for the stable M/G/c queue. Sigman (2011) showed how to build a dominated coupling-from-the-past algorithm for perfect simulation of the super-stable M/G/c queue operating under first-come-first-served discipline. Sigman's method used a dominating process provided by the corresponding M/G/1 queue (using Wolff's sample path monotonicity, which applies when service durations are coupled in order of initiation of service). The method exploited the fact that the workload process for the M/G/1 queue remains the same under different queueing disciplines, in particular under the processor sharing discipline, for which a dynamic reversibility property holds. We generalise Sigman's construction to the stable case by comparing the M/G/c queue to a copy run under random assignment. This allows us to produce a naïve perfect simulation algorithm based on running the dominating process back to the time it first empties. We also construct a more efficient algorithm that uses sandwiching by lower and upper processes constructed as coupled M/G/c queues started respectively from the empty state and the state of the M/G/c queue under random assignment. A careful analysis shows that appropriate ordering relationships can still be maintained, so long as service durations continue to be coupled in order of initiation of service. We summarise statistical checks of simulation output, and demonstrate that the mean run-time is finite so long as the second moment of the service duration distribution is finite.

This paper considers Markov decision processes (MDPs) with unbounded rates, as a function of state. We are especially interested in studying structural properties of optimal policies and the value function. A common method to derive such properties is by value iteration applied to the uniformised MDP. However, due to the unboundedness of the rates, uniformisation is not possible, and so value iteration cannot be applied in the way we need. To circumvent this, one can perturb the MDP. Then we need two results for the perturbed sequence of MDPs: 1. there exists a unique solution to the discounted cost optimality equation for each perturbation as well as for the original MDP; 2. if the perturbed sequence of MDPs converges in a suitable manner then the associated optimal policies and the value function should converge as well. We can model both the MDP and perturbed MDPs as a collection of parametrised Markov processes. Then both of the results above are essentially implied by certain continuity properties of the process as a function of the parameter. In this paper we deduce tight verifiable conditions that imply the necessary continuity properties. The most important of these conditions are drift conditions that are strongly related to nonexplosiveness.