Consider a random walk S=(S n : n≥0) that is ‘perturbed’ by a stationary sequence (ξn : n≥0) to produce the process S=(S n +ξn : n≥0). In this paper, we are concerned with developing limit theorems and approximations for the distribution of M n =max{S k +ξk : 0≤k≤n} when the random walk has a drift close to 0. Such maxima are of interest in several modeling contexts, including operations management and insurance risk theory. The associated limits combine features of both conventional diffusion approximations for random walks and extreme-value limit theory.

We consider a Jackson network in which some of the nodes have an infinite supply of work: when all the customers queued at such a node have departed, the node will process a customer from this supply. Such nodes will be processing jobs all the time, so they will be fully utilized and experience a traffic intensity of 1. We calculate flow rates for such networks, obtain conditions for stability, and investigate the stationary distributions. Standard nodes in this network continue to have product-form distributions, while nodes with an infinite supply of work have geometric marginal distributions and Poisson inflows and outflows, but their joint distribution is not of product form.

We consider a two-stage tandem queue with two parallel servers and two queues. We assume that initially all jobs are present and that no further arrivals take place at any time. The two servers are identical and can serve both types of job. The processing times are exponentially distributed. After being served, a job of queue 1 joins queue 2, whereas a job of queue 2 leaves the system. Holding costs per job and per unit time are incurred if there are jobs holding in the system. Our goal is to find the optimal strategy that minimizes the expected total holding costs until the system is cleared. We give a complete solution for the optimal control of all possible parameters (costs and service times), especially for those parameter regions in which the optimal control depends on how many jobs are present in the two queues.

The aim of this paper is to evaluate the performance of the optimal policy (the Gittins index policy) for open tax problems of the type considered by Klimov in the undiscounted limit. In this limit, the state-dependent part of the cost is linear in the state occupation numbers for the multi-armed bandit, but is quadratic for the tax problem. The discussion of the passage to the limit for the tax problem is believed to be largely new; the principal novelty is our evaluation of the matrix of the quadratic form. These results are confirmed by a dynamic programming analysis, which also suggests how the optimal policy should be modified when resources can be freely deployed only within workstations, rather than system-wide.

We consider the lifetimes of systems that can be modeled as particles that move within a bounded region in ℝn . Particles move within the set according to a random walk, and particles that leave the set are lost. We divide the set into equal cells and define the lifetime of the set as the time required for the number of particles in one of the cells to fall below a predetermined threshold. We show that the lifetime of the system, given a sufficiently large number of particles, is Weibull distributed.

We consider a service system (Q S) that operates according to the first-come-first-served (FCFS) discipline, and in which the service rate is an increasing function of the queue length. Customers arrive sequentially at the system, and decide whether or not to join using decision rules based upon the queue length on arrival. Each customer is interested in selecting a rule that meets a certain optimality criterion with regard to their expected sojourn time in the system; as a consequence, the decision rules of other customers must be taken into account. Within a particular class of decision rules for an associated infinite-player game, the structure of the Nash equilibrium routeing policies is characterized. We prove that, within this class, there exist a finite number of Nash equilibria, and that at least one of these is nonrandomized. Finally, with the aid of simulation experiments, we explore the extent to which the Nash equilibria are characteristic of customer joining behaviour under a learning rule based on system-wide data.

We consider a finite dam under the policy, where the input of water is formed by a Wiener process subject to random jumps arriving according to a Poisson process. The long-run average cost per unit time is obtained after assigning costs to the changes of release rate, a reward to each unit of output, and a penalty that is a function of the level of water in the reservoir.

We derive several algorithms for the busy period distribution of the canonical Markovian fluid flow model. One of them is similar to the Latouche-Ramaswami algorithm for quasi-birth-death models and is shown to be quadratically convergent. These algorithms significantly increase the efficiency of the matrix-geometric procedures developed earlier by the authors for the transient and steady-state analyses of fluid flow models.

We consider stationary and ergodic tessellations X = Ξn n≥1 in R d , where X is observed in a bounded and convex sampling window W p ⊂ R d . It is assumed that the cells Ξn of X possess random inner structures, examples of which include point patterns, fibre systems, and tessellations. These inner cell structures are generated, both independently of each other and independently of the tessellation X, by generic stationary random sets that are related to a stationary random vector measure J 0 acting on R d . In particular, we study the asymptotic behaviour of a multivariate random functional, which is determined both by X and by the individual cell structures contained in W p , as W p ↑ R d . It turns out that this functional provides an unbiased estimator for the intensity vector associated with J 0. Furthermore, under natural restrictions, strong laws of large numbers and a multivariate central limit theorem of the normalized functional are proven. Finally, we discuss in detail some numerical examples and applications, for which the inner structures of the cells of X are induced by iterated Poisson-type tessellations.

We consider an M/G/1 queue that is idle at time 0. The number of customers sampled at an independent exponential time is shown to have the same geometric distribution under the preemptive-resume last-in-first-out and the processor-sharing disciplines. Hence, the marginal distribution of the queue length at any time is identical for both disciplines. We then give a detailed analysis of the time until the first departure for any symmetric queueing discipline. We characterize its distribution and show that it is insensitive to the service discipline. Finally, we study the tail behavior of this distribution.

In this paper, we study a monotone process maintenance model for a multistate system with k working states and ℓ failure states. By making different assumptions, we can apply the model to a multistate deteriorating system as well as to a multistate improving system. We show that the monotone process model for a multistate system is equivalent to a geometric process model for a two-state system. Then, for both the deteriorating and the improving system, we analytically determine an optimal replacement policy for minimizing the long-run average cost per unit time.

In this paper, we study the stationary distributions for reflected diffusions with jumps in the positive orthant. Under the assumption that the stationary distribution possesses a density in R + n that satisfies certain finiteness conditions, we characterize the Fokker-Planck equation. We then provide necessary and sufficient conditions for the existence of a product-form distribution for diffusions with oblique boundary reflections and jumps. To do so, we exploit a recent characterization of the boundary properties of such reflected processes. In particular, we show that the conditions generalize those for semimartingale reflecting Brownian motions and reflected Lévy processes. We provide explicit results for some models of interest.

We give a very general reformulation of multi-actor Markov decision processes and show that there is a tendency for the actors to take the same action whenever possible. This considerably reduces the complexity of the problem, either facilitating numerical computation of the optimal policy or providing a basis for a heuristic.

We consider a failure-prone system operating in continuous time. Condition monitoring is conducted at discrete time epochs. The state of the system is assumed to evolve as a continuous-time Markov process with a finite state space. The observation process with continuous-range values is stochastically related to the state process, which, except for the failure state, is unobservable. Combining the failure information and the condition monitoring information, we derive a general recursive filter, and, as special cases, we obtain recursive formulae for the state estimation and other quantities of interest. Updated parameter estimates are obtained using the expectation-maximization (EM) algorithm. Some practical prediction problems are discussed and finally an illustrative example is given using a real dataset.

In this paper we consider a system with two carousels operated by one picker. The items to be picked are randomly located on the carousels and the pick times follow a phase-type distribution. The picker alternates between the two carousels, picking one item at a time. Important performance characteristics are the waiting time of the picker and the throughput of the two carousels. The waiting time of the picker satisfies an equation very similar to Lindley's equation for the waiting time in the PH/U/1 queue. Although the latter equation has no simple solution, we show that the one for the waiting time of the picker can be solved explicitly. Furthermore, it is well known that the mean waiting time in the PH/U/1 queue depends on the complete interarrival time distribution, but numerical results show that, for the carousel system, the mean waiting time and throughput are rather insensitive to the pick-time distribution.

We consider two classes of life distribution, V D and V I , the members of which are defined in terms of the conditional variance σ 2(t) of the remaining lifetime of a system: a life distribution F belongs to V D if is a decreasing function and to V I if is increasing. We study closure properties of these classes under relevant reliability operations such as mixing, convolution and formation of coherent systems. We show, for example, that the class V D is not closed under convolution or mixing, and that the class V I is not closed under formation of coherent systems.

We consider the problem of scheduling the transmissions of multiple data users (flows) sharing the same wireless channel (server). The unique feature of this problem is the fact that the capacity (service rate) of the channel varies randomly with time and asynchronously for different users. We study a scheduling policy called the exponential scheduling rule, which was introduced in an earlier paper. Given a system with N users, and any set of positive numbers {a n }, n = 1, 2,…, N, we show that in a heavy-traffic limit, under a nonrestrictive ‘complete resource pooling’ condition, this algorithm has the property that, for each time t, it (asymptotically) minimizes maxn a n q̃n (t), where q̃n (t) is the queue length of user n in the heavy-traffic regime.

We consider a Brownian motion with time-reversible Markov-modulated speed and two reflecting barriers. A methodology depending on a certain multidimensional martingale together with some linear algebra is applied in order to explicitly compute the stationary distribution of the joint process of the content level and the state of the underlying Markov chain. It is shown that the stationary distribution is such that the two quantities are independent. The long-run average push at the two barriers at each of the states is also computed.

We analyse several aspects of a class of simple counting processes that can emerge in some fields of applications where a change point occurs. In particular, under simple conditions we prove a significant inequality for the stochastic intensity.

Within reliability theory, identifiability problems arise through competing risks. If we have a series system of several components, and if that system is replaced or repaired to as good as new on failure, then the different component failures represent competing risks for the system. It is well known that the underlying component failure distributions cannot be estimated from the observable data (failure time and identity of failed component) without nontestable assumptions such as independence. In practice many systems are not subject to the ‘as good as new’ repair regime. Hence, the objective of this paper is to contrast the identifiability issues arising for different repair regimes. We consider the problem of identifying a model within a given class of probabilistic models for the system. Different models corresponding to different repair strategies are considered: a partial-repair model, where only the failing component is repaired; perfect repair, where all components are as good as new after a failure; and minimal repair, where components are only minimally repaired at failures. We show that on the basis of observing a single socket, the partial-repair model is identifiable, while the perfect- and minimal-repair models are not.