For k-armed Bernoulli bandits with discounting, sharp comparisons are given between average optimal rewards for a gambler and for a ‘perfectly informed' gambler, over natural collections of prior distributions. Some of these comparisons are proved under general discounting, and others under non-increasing discount sequences. Connections are made between these comparisons and the concept of ‘regret' in the minimax approach to bandit processes. Identification of extremal cases in the sharp comparisons is emphasized.

Stopping-allocation problems are concerned with how best to allocate observations among some K competing stochastic populations and when to stop the observation process. The goal of the decision-maker is to choose a stopping–allocation rule to maximize the expected value of a payoff function. First the stopping rule is fixed, and the local and global optimality of the myopic allocation rule are derived under some monotonicity conditions. An application is considered, namely the inspection problem and its use in solving a computer scheduling problem. Next, optimization is done with respect to both the allocation rule and the stopping rule. For any given stopping-allocation rule, it is shown that under some monotonicity conditions, the decision-maker can improve on it by using a ‘partial' myopic allocation rule and a generalized one-stage-look-ahead stopping rule; this result is then extended, under the same conditions and other monotonicity requirements, to derive the joint optimality of the myopic allocation rule and the one-stage-look-ahead stopping rule. Finally this latter result is applied to the inspection problem.

Conditional stochastic ordering is concerned with the stochastic ordering of a pair of probability measures conditional on certain subsets or sub-σ -algebras. Some basic results of conditional stochastic ordering were proved by Whitt. We extend some of Whitt's results and prove a basic relation between stochastic ordering conditional on subsets and stochastic ordering conditional on σ -algebras. In the second part of the paper we consider the ordering of conditional expectations. There are several different formulations of this problem motivated by different types of applications.

This paper extends the asymptotic results for ordinary renewal processes to the superposition of independent renewal processes. In particular, the ordinary renewal functions, renewal equations, and the key renewal theorem are extended to the superposition of independent renewal processes. We fix the number of renewal processes, p, and study the asymptotic behavior of the superposition process when time, t, is large. The key superposition renewal theorem is applied to the study of queueing systems.

A single machine is available to process a collection J of jobs. The machine is free to switch between jobs at any time, but processing must respect a set Γof precedence constraints. Jobs evolve stochastically and earn rewards as they are processed, not otherwise. The theoretical framework of forwards induction/Gittins indexation is used to develop approaches to strategy evaluation for quite general (J,Γ). The performance of both forwards induction strategies and a class of quasi-myopic heuristics is assessed.

In many communication and computer systems, information arrives to a multiplexer, switch or information processor at a rate which fluctuates randomly, often with a high degree of correlation in time. The information is buffered for service (the server typically being a communication channel or processing unit) and the service rate may also vary randomly. Accurate capture of the statistical properties of these fluctuations is facilitated by modeling the arrival and service rates as superpositions of a number of independent finite state reversible Markov processes. We call such models separable Markov-modulated rate processes (MMRP).

In this work a general mathematical model for separable MMRPs is presented, focusing on Markov-modulated continuous flow models. An efficient procedure for analyzing their performance is derived. It is shown that the ‘state explosion' problem typical of systems composed of a large number of subsystems, can be circumvented because of the separability property, which permits a decomposition of the equations for the equilibrium probabilities of these systems. The decomposition technique (generalizing a method proposed by Kosten) leads to a solution of the equilibrium equations expressed as a sum of terms in Kronecker product form. A key consequence of decomposition is that the computational complexity of the problem is vastly reduced for large systems. Examples are presented to illustrate the power of the solution technique.

In this paper, we study a synchronous computer network model which may be described as follows. There are n service stations around a circle and there are k people to be serviced with at most one person per station. Time is slotted and in each time slot b adjacent people with an empty station ahead can move synchronously and simultaneously to the next counterclockwise station with probability pb. We show that the stationary distribution is uniform on all possible states, and using the Descartes' rule of signs we find the optimal value of k (fixed p and n) that yields maximal ‘throughput', i.e., the expected number of people served per time slot in equilibrium. We also briefly study an asynchronous model (introduced by D. Sarkar) where in each time slot a person moves with probability p to the next counterclockwise station if it is empty. We find, among other results, the new stationary distribution (Ramakrishnan et al. (1989) also find independently the stationary distribution and they also find the optimal value of k (fixed p and n) that yields maximal throughput in this model). We give tables comparing the synchronous and asynchronous models. Some applications which motivate this study are briefly presented.

A general framework of continuous-time queueing networks is studied with simultaneous state dependent service completions such as due to concurrent servicing or discrete-time slotting and with state dependent batch routings such as most typically modelling blocking. By using a key notion of group-local-balance, necessary and sufficient conditions are given for the stationary distribution to be of product form. These conditions and a constructive computation of the product form are based upon merely local solutions of the group-local-balance equations which can usually be solved explicitly for concrete networks. Moreover, a decomposition theorem is presented to separate service and routing conditions. General batch service and batch routing examples yielding a product form are hereby concluded. As illustrated by various examples, known results on both discrete- and continuous-time queueing networks are unified and extended.

This paper presents a new approach for obtaining heavy-traffic limits for infinite-server queues and open networks of infinite-server queues. The key observation is that infinite-server queues having deterministic service times can easily be analyzed in terms of the arrival counting process. A variant of the same idea applies when the service times take values in a finite set, so this is the key assumption. In addition to new proofs of established results, the paper contains several new results, including limits for the work-in-system process, limits for steady-state distributions, limits for open networks with general customer routes, and rates of convergence. The relatively tractable Gaussian limits are promising approximations for many-server queues and open networks of such queues, possibly with finite waiting rooms.

In this paper, we compare queueing systems that differ only in their arrival processes, which are special forms of doubly stochastic Poisson (DSP) processes. We define a special form of stochastic dominance for DSP processes which is based on the well-known variability or convex ordering for random variables. For two DSP processes that satisfy our comparability condition in such a way that the first process is more ‘regular' than the second process, we show the following three results: (i) If the two systems are DSP/GI/1 queues, then for all f increasing convex, with V(i), i = 1 and 2, representing the workload (virtual waiting time) in system. (ii) If the two systems are DSP/M(k)/1→ /M(k)/l ∞ ·· ·∞ /M(k)/1 tandem systems, with M(k) representing an exponential service time distribution with a rate that is increasing concave in the number of customers, k, present at the station, then for all f increasing convex, with Q(i), i = 1 and 2, being the total number of customers in the two systems. (iii) If the two systems are DSP/M(k)/1/N systems, with N being the size of the buffer, then where denotes the blocking (loss) probability of the two systems. A model considered before by Ross (1978) satisfies our comparability condition; a conjecture stated by him is shown to be true.