In this paper, we consider denumerable-state continuous-time Markov decision processes with (possibly unbounded) transition and reward rates and general action space under the discounted criterion. We provide a set of conditions weaker than those previously known and then prove the existence of optimal stationary policies within the class of all possibly randomized Markov policies. Moreover, the results in this paper are illustrated by considering the birth-and-death processes with controlled immigration in which the conditions in this paper are satisfied, whereas the earlier conditions fail to hold.

The additive-increase multiplicative-decrease (AIMD) schemes designed to control congestion in communication networks are investigated from a probabilistic point of view. Functional limit theorems for a general class of Markov processes that describe these algorithms are obtained. The asymptotic behaviour of the corresponding invariant measures is described in terms of the limiting Markov processes. For some special important cases, including TCP congestion avoidance, an important autoregressive property is proved. As a consequence, the explicit expression of the related invariant probabilities is derived. The transient behaviour of these algorithms is also analysed.

The purpose of this paper is to study ageing properties of first-passage times of increasing Markov chains. We extend the literature to some new ageing classes, such as the IFR(2), NBU(2), DRLLtand NBULt classes. We also give sufficient conditions in the finite case, that are more efficient computationally, just in terms of the transition matrix K, in the discrete case, or the generator matrix Q, in the continuous case. For the uniformizable, continuous-time Markov processes, we derive conditions in terms of the discrete uniformized Markov chain for the NBU(2) and the NBULtclasses. In the last section, a review of the main results in this direction in the literature is given, and we compare some of the conditions stated in this paper with others given in the literature about some other ageing classes. Some examples where these results are applied are given.

This paper presents an algorithmic procedure to calculate the delay distribution of a type k customer in a first-come-first-served (FCFS) discrete-time queueing system with multiple types of customers, where each type has different service requirements (the MMAP[K]/PH[K]/1 queue). First, we develop a procedure, using matrix analytical methods, to handle arrival processes that do not allow batch arrivals to occur. Next, we show that this technique can be generalized to arrival processes that do allow batch arrivals to occur. We end the paper by presenting some numerical examples.

The Laplace transform is obtained for the joint length and height of an M/M/∞ queue excursion above occupancy level C.

Empirical studies of data traffic in high-speed networks suggest that network traffic exhibits self-similarity and long-range dependence. Cumulative network traffic has been modeled using the so-called ON/OFF model. It was shown that cumulative network traffic can be approximated by either fractional Brownian motion or stable Lévy motion, depending on how many sources are active in the model. In this paper we consider exceedances of a high threshold by the sequence of lengths of ON-periods. If the cumulative network traffic converges to stable Lévy motion, the number of exceedances converges to a Poisson limit. The same holds in the fractional Brownian motion case, provided a very high threshold is used. Finally, we show that the number of exceedances obeys the central limit theorem.

Patch clamp recordings from ion channels often show periods of repetitive activity, known as bursts, which are noticeably separated from each other by periods of inactivity. Depending on the type of channel, such recordings may exhibit (conductance) levels between the closed (zero) level and the fully open level. Properties of bursts are less subject to problems that arise from recording than are properties for individual sojourns at different levels, and study of bursting behaviour provides important information about the finer structure of the underlying channel gating process. For a general finite state space continuous-time Markov chain model allowing one or more nonzero conductance levels, the present paper establishes results about the semi-Markov structure of a single burst and of a sequence of bursts, and uses this in a unified approach to properties of both theoretical and empirical bursts. The distribution and moments of particular burst properties, including the total charge transfer, the total sojourn time and the total number of visits to specified conductance levels during a burst, are derived. Various extensions are also described.

In this note, we give new proofs of the closure property of ageing classes NBUC and NBU(2) under convolution to make up the gaps in the proofs of Cao and Wang (1991) and of Li and Kochar (2001).

In this paper, we analyse a model of a regular tree loss network that supports two types of calls: unicast calls that require unit capacity on a single link, and multicast calls that require unit capacity on every link emanating from a node. We study the behaviour of the distribution of calls in the core of a large network that has uniform unicast and multicast arrival rates. At sufficiently high multicast call arrival rates the network exhibits a ‘phase transition’, leading to unfairness due to spatial variation in the multicast blocking probabilities. We study the dependence of the phase transition on unicast arrival rates, the coordination number of the network, and the parity of the capacity of edges in the network. Numerical results suggest that the nature of phase transitions is qualitatively different when there are odd and even capacities on the links. These phenomena are seen to persist even with the introduction of nonuniform arrival rates and multihop multicast calls into the network. Finally, we also show the inadequacy of approximations such as the Erlang fixed-point approximations when multicasting is present.

We study Bernoulli bond percolation on Sierpiński carpet lattices, which is a class of graphs corresponding to generalized Sierpiński carpets. In this paper we give a sufficient condition for the existence of a phase transition on the lattices. The proof is suitable for graphs which have self-similarity. We also discuss the relation between the existence of a phase transition and the isoperimetric dimension.

Let (X t ) and (Y t ) be continuous-time Markov chains with countable state spaces E and F and let K be an arbitrary subset of E x F. We give necessary and sufficient conditions on the transition rates of (X t ) and (Y t ) for the existence of a coupling which stays in K. We also show that when such a coupling exists, it can be chosen to be Markovian and give a way to construct it. In the case E=F and K ⊆ E x E, we see how the problem of construction of the coupling can be simplified. We give some examples of use and application of our results, including a new concept of lumpability in Markov chains.

We consider a finite-capacity queueing system where arriving customers offer rewards which are paid upon acceptance into the system. The gatekeeper, whose objective is to ‘maximize’ rewards, decides if the reward offered is sufficient to accept or reject the arriving customer. Suppose the arrival rates, service rates, and system capacity are changing over time in a known manner. We show that all bias optimal (a refinement of long-run average reward optimal) policies are of threshold form. Furthermore, we give sufficient conditions for the bias optimal policy to be monotonic in time. We show, via a counterexample, that if these conditions are violated, the optimal policy may not be monotonic in time or of threshold form.

We consider the Poisson Boolean model of percolation where the percolating shapes are convex regions. By an enhancement argument we strengthen a result of Jonasson (2000) to show that the critical intensity of percolation in two dimensions is minimized among the class of convex shapes of unit area when the percolating shapes are triangles, and, for any other shape, the critical intensity is strictly larger than this minimum value. We also obtain a partial generalization to higher dimensions. In particular, for three dimensions, the critical intensity of percolation is minimized among the class of regular polytopes of unit volume when the percolating shapes are tetrahedrons. Moreover, for any other regular polytope, the critical intensity is strictly larger than this minimum value.

Suppose that a graph process begins with n isolated vertices, to which edges are added randomly one by one so that the maximum degree of the induced graph is always at most d. In a previous article, the authors showed that as n → ∞, with probability tending to 1, the result of this process is a d-regular graph. This graph is shown to be connected with probability asymptotic to 1.

The asymptotic behaviour of an occupation-time process associated with alternating renewal processes is investigated in the infinite mean cycle case. The limit theorems obtained extend some asymptotic results proved by Dynkin (1955), Lamperti (1958) and Erickson (1970) for the classical spent lifetime process. Some new phenomena are also presented.

In first-passage percolation (FPP) models, the passage time T ℓ from the origin to the point ℓe ℓ satisfies f(ℓ) := ET ℓ = μℓ + o(ℓ ½+ε), where μ ∊ (0,∞) denotes the time constant. Yet, for lattice FPP, it is not known rigorously that f(ℓ) is eventually monotonically increasing. Here, for the Poisson-based Euclidean FPP of Howard and Newman (Prob. Theory Relat. Fields108 (1997), 153–170), we prove an explicit formula for f′(ℓ). In all dimensions, for certain values of the model's only parameter we have f′(ℓ) ≥ C > 0 for large ℓ.

In this paper we study the asymptotic relationship between the loss ratio in a finite buffer system and the overflow probability (the tail of the queue length distribution) in the corresponding infinite buffer system. We model the system by a fluid queue which consists of a server with constant rate c and a fluid input. We provide asymptotic upper and lower bounds on the difference between log P{Q > x} and logP L (x) under different conditions. The conditions for the upper bound are simple and are satisfied by a very large class of input processes. The conditions on the lower bound are more complex but we show that various classes of processes such as Markov modulated and ARMA type Gaussian input processes satisfy them.

We present a new inspection policy useful when testing is needed to detect failures of a single-unit system. It is supposed that tests may fail and give an erroneous result. The inspection policy minimizing cost per unit of time for an infinite time span is also discussed. In addition, we study the behaviour of the optimum policy for some time to failure distributions often assumed in reliability: exponential and Pareto.

In this paper, a single channel FIFO fluid queue with an infinite buffer space and a long-range dependent input is studied. The input traffic is modeled by an average input rate plus a standard fractional Brownian motion as the fluctuation. Lower and upper bounds are derived for the tail distribution of the transient queue length at time T, which result in a logarithmic characterization of the asymptotic behavior of the tail distribution. Furthermore, the exact asymptotic is also obtained. It is observed that the transient queue length under fractional Brownian input does not suffer from the heavy-tail property as does the steady-state queue length. The results are used to compute the equivalent bandwidth requirement for ATM broadband connections with fractional Brownian traffic feed and finite connection holding time.

The contact process is an interacting particle system which models a spatially restricted infection. In the basic contact process the infection can only spread to an uninfected neighbour, but the diffusive contact process allows an infected individual to move to an uninfected site. If the infection rate is too low, the process will die out. If the individual can move (or diffuse), the disease can spread with a lower infection rate. An idea of the relationship between these rates is obtained by obtaining rigorous lower bounds for the critical infection rate for various values of the diffusion rate. In this paper we also improve the lower bound for the critical infection rate for the basic contact process from 1.539 to 1.5517.