GI/M/1-type Markov chains make up a class of two-dimensional Markov chains. One dimension is usually called the level, and the other is often called the phase. Transitions from states in level k are restricted to states in levels less than or equal to k+1. For given transition probabilities in the interior of the state space, we show that it is always possible to define the boundary transition probabilities in such a way that the level and phase are independent under the stationary distribution. We motivate our analysis by first considering the quasi-birth-and-death process special case in which transitions from any state are restricted to states in the same, or adjacent, levels.

In this paper we study the assignment of packets to M parallel heterogeneous servers with no buffers. The controller does not have knowledge on the state of the servers and bases all decisions on the number of time slots ago that packets have been sent to the different servers. The objective of the controller is to minimize the expected average cost. We consider a general stationary arrival process, with no independence assumptions. We show that the problem can be transformed into a Markov Decision Process (MDP). We further show under quite general conditions on cost functions that only a finite number of states have to be considered in this MDP, which then implies the optimality of periodic policies. For the case of two servers we obtain a more detailed structure of the cost and optimal policies. In particular we show that the average cost function is multimodular, and we obtain expressions for the cost for MMPP and MAP processes. Finally we present an application to optimal robot scheduling for Web search engines.

We study a random record model where the observation Xi has continuous distribution function Fαi (αi > 0) and the number of available observations is random and independent of the observations. We obtain the joint distribution of the record values and inter-record times for our model. We investigate the distribution of the number of records when the number of observations has one of the common distributions and the α's increase geometrically or linearly. A particularly interesting case arises when the observations arrive at time points paced by a Poisson point process. For this model we obtain distributional results for the inter-arrival times of records for a large class of combinations of α structures and intensity functions.

In various stochastic models the random equation of implicit renewal theory appears where the real random variable S and the stochastic process Ψ with index space and state space R are independent. By use of stochastic approximation the distribution function of S is recursively estimated on the basis of independent or ergodic copies of Ψ. Under integrability assumptions almost sure L1-convergence is proved. The choice of gains in the recursion is discussed. Applications are given to insurance mathematics (perpetuities) and queueing theory (stationary waiting and queueing times).

This paper arose from interest in assessing the quality of random number generators. The problem of testing randomness of a string of binary bits produced by such a generator gained importance with the wide use of public key cryptography and the need for secure encryption algorithms. All such algorithms are based on a generator of (pseudo) random numbers; the testing of such generators for randomness became crucial for the communications industry where digital signatures and key management are vital for information processing.

The concept of approximate entropy has been introduced in a series of papers by S. Pincus and co-authors. The corresponding statistic is designed to measure the degree of randomness of observed sequences. It is based on incremental contrasts of empirical entropies based on the frequencies of different patterns in the sequence. Sequences with large approximate entropy must have substantial fluctuation or irregularity. Alternatively, small values of this characteristic imply strong regularity, or lack of randomness, in a sequence. Pincus and Kalman (1997) evaluated approximate entropies for binary and decimal expansions of e, π, √2 and √3 with the surprising conclusion that the expansion of √3 demonstrated much less irregularity than that of π. Tractable small sample distributions are hardly available, and testing randomness is based, as a rule, on fairly long strings. Therefore, to have rigorous statistical tests of randomness based on this approximate entropy statistic, one needs the limiting distribution of this characteristic under the randomness assumption. Until now this distribution remained unknown and was thought to be difficult to obtain. To derive the limiting distribution of approximate entropy we modify its definition. It is shown that the approximate entropy as well as its modified version converges in distribution to a χ2-random variable. The P-values of approximate entropy test statistics for binary expansions of e, π and √3 are plotted. Although some of these values for √3 digits are small, they do not provide enough statistical significance against the randomness hypothesis.

Let I1,I2,…,In be a sequence of independent indicator functions defined on a probability space (Ω, A, P). We say that index k is a success time if Ik = 1. The sequence I1,I2,…,In is observed sequentially. The objective of this article is to predict the lth last success, if any, with maximum probability at the time of its occurrence. We find the optimal rule and discuss briefly an algorithm to compute it in an efficient way. This generalizes the result of Bruss (1998) for l = 1, and is equivalent to the problem of (multiple) stopping with l stops on the last l successes. We then extend the model to a larger class allowing for an unknown number N of indicator functions, and present, in particular, a convenient method for an approximate solution if the success probabilities are small. We also discuss some applications of the results.

This paper studies the connection between the dynamical and equilibrium behaviour of large uncontrolled loss networks. We consider the behaviour of the number of calls of each type in the network, and show that, under the limiting regime of Kelly (1986), all trajectories of the limiting dynamics converge to a single fixed point, which is necessarily that on which the limiting stationary distribution is concentrated. The approach uses Lyapunov techniques and involves the evolution of the transition rates of a stationary Markov process in such a way that it tends to reversibility.

We investigate some preservation properties of two nonparametric classes of survival distributions and their duals, under appropriate reliability operations. The aging properties defining these nonparametric classes are based on comparing the mean life of a new unit to the mean residual life function of the asymptotic remaining survival time of the unit under repeated perfect repairs. They are motivated from a point of view that realistic notions of degradation, applicable to repairable systems, should be based on contrasting some aspect of the remaining life of a repairable unit (under a given repair strategy, such as renewals) to the life of a new unit.

We prove that, under rather general conditions, the law of a continuous Gaussian process represented by a stochastic integral of a deterministic kernel, with respect to a standard Wiener process, can be weakly approximated by the law of some processes constructed from a standard Poisson process. An example of a Gaussian process to which this result applies is the fractional Brownian motion with any Hurst parameter.

We estimate the limiting availability of a system when the operating and repair times form a stationary bivariate sequence. These estimators are shown to be consistent and asymptotically normal under certain conditions. In particular, we estimate the limiting availability for a bivariate exponential autoregressive process.

We consider the uncovered set (i.e. the complement of the union of growing random intervals) in the one-dimensional Johnson-Mehl model. Let S(z,L) be the number of components of this set at time z > 0 which intersect (0, L]. An explicit bound is known for the total variation distance between the distribution of S(z,L) and a Poisson distribution, but due to clumping of the components the bound can be rather large. We here give a bound for the total variation distance between the distribution of S(z,L) and a simple compound Poisson distribution (a Pólya-Aeppli distribution). The bound is derived by interpreting S(z,L) as the number of visits to a ‘rare’ set by a Markov chain, and applying results on compound Poisson approximation for Markov chains by Erhardsson. It is shown that under a mild condition, if z→∞ and L→∞ in a proper fashion, then both the Pólya-Aeppli and the Poisson approximation error bounds converge to 0, but the convergence of the former is much faster.

In this paper, we investigate how fast the stationary distribution π(K) of an embedded Markov chain (time-stationary distribution q(K) of the GI/M/1/K queue converges to the stationary distribution π of the embedded Markov chain (time-stationary distribution q) of the GI/M/1 queue as K tends to infinity. Simonot (1997) proved certain equalities. We obtain sharper results than these by finding limit values limK→∞σ-K||π(K) - π|| and limK→∞σ-K||q(K) - q|| explicitly.

Consider a population of fixed size consisting of N haploid individuals. Assume that this population evolves according to the two-allele neutral Moran model in mathematical genetics. Denote the two alleles by A1 and A2. Allow mutation from one type to another and let 0 < γ < 1 be the sum of mutation probabilities. All the information about the population is recorded by the Markov chain X = (X(t))t≥0 which counts the number of individuals of type A1. In this paper we study the time taken for the population to ‘reach’ stationarity (in the sense of separation and total variation distances) when initially all individuals are of one type. We show that after t∗ = Nγ-1logN + cN the separation distance between the law of X(t∗) and its stationary distribution converges to 1 - exp(-γe-γc) as N → ∞. For the total variation distance an asymptotic upper bound is obtained. The results depend on a particular duality, and couplings, between X and a genealogical process known as the lines of descent process.

In this paper we consider the optimal control of an infinite dam using policies assuming that the input process is a compound Poisson process with a non-negative drift term, and using the total discounted cost and long-run average cost criteria. The results of Lee and Ahn (1998) as well as other well-known results are shown to follow from our results.

This paper studies the dependence structure and bounds of several basic prototypical parallel queueing systems with correlated arrival processes to different queues. The marked feature of our systems is that each queue viewed alone is a standard single-server queuing system extensively studied in the literature, but those queues are statistically dependent due to correlated arrival streams. The major difficulty in analysing those systems is that the presence of correlation makes the explicit computation of a joint performance measure either intractable or computationally intensive. In addition, it is not well understood how and in what sense arrival correlation will improve or deteriorate a system performance measure.

The objective of this paper is to provide a better understanding of the dependence structure of correlated queueing systems and to derive computable bounds for the statistics of a joint performance measure. In this paper, we obtain conditions on arrival processes under which a performance measure in two systems can be compared, in the sense of orthant and supermodular orders, among different queues and over different arrival times. Such strong comparison results enable us to study both spatial dependence (dependence among different queues) and temporal dependence (dependence over different time instances) for a joint performance measure. Further, we derive a variety of upper and lower bounds for the statistics of a stationary joint performance measure. Finally, we apply our results to synchronized queueing systems, using the ideas combined from the theory of orthant and supermodular dependence orders and majorization with respect to weighted trees (Xu and Li (2000)). Our results reveal how a performance measure can be affected, favourably or adversely, by different types of dependencies.

The estimation of critical values is one of the most interesting problems in the study of interacting particle systems. The bounds obtained analytically are not usually very tight and, therefore, computer simulation has been proved to be very useful in the estimation of these values. In this paper we present a new method for the estimation of critical values in any interacting particle system with an absorbing state. The method, based on the asymptotic behaviour of the absorption time of the process, is very easy to implement and provides good estimates. It can also be applied to processes different from particle systems.

We study stochastic dynamic investment games in continuous time between two investors (players) who have available two different, but possibly correlated, investment opportunities. There is a single payoff function which depends on both investors’ wealth processes. One player chooses a dynamic portfolio strategy in order to maximize this expected payoff, while his opponent is simultaneously choosing a dynamic portfolio strategy so as to minimize the same quantity. This leads to a stochastic differential game with controlled drift and variance. For the most part, we consider games with payoffs that depend on the achievement of relative performance goals and/or shortfalls. We provide conditions under which a game with a general payoff function has an achievable value, and give an explicit representation for the value and resulting equilibrium portfolio strategies in that case. It is shown that non-perfect correlation is required to rule out trivial solutions. We then use this general result explicitly to solve a variety of specific games. For example, we solve a probability maximizing game, where each investor is trying to maximize the probability of beating the other's return by a given predetermined percentage. We also consider objectives related to the minimization or maximization of the expected time until one investor's return beats the other investor's return by a given percentage. Our results allow a new interpretation of the market price of risk in a Black-Scholes world. Games with discounting are also discussed, as are games of fixed duration related to utility maximization.

We consider an interacting particle system in which each site of the d-dimensional integer lattice can be in state 0, 1, or 2. Our aim is to model the spread of disease in plant populations, so think of 0 = vacant, 1 = healthy plant, 2 = infected plant. A vacant site becomes occupied by a plant at a rate which increases linearly with the number of plants within range R, up to some saturation level, F1, above which the rate is constant. Similarly, a plant becomes infected at a rate which increases linearly with the number of infected plants within range M, up to some saturation level, F2. An infected plant dies (and the site becomes vacant) at constant rate δ. We discuss coexistence results in one and two dimensions. These results depend on the relative dispersal ranges for plants and disease.

Consider a situation where a known number, n, of objects appear sequentially in a random order. At each stage, the present object is presented to d ≥ 2 different selectors, who must jointly decide whether to select or reject it, irrevocably. Exactly one object must be chosen. The observation at stage j is a d-dimensional vector R(j) = (R1(j),…, Rd(j)), where Ri(j) is the relative rank of the jth object, by the criterion of the ith selector. The decision whether to stop or not at time j is based on the d-dimensional random vectors R(1),…, R(j). The criteria according to which each selector ranks the objects can either be dependent or independent. Although the goal of each selector is to maximize the probability of choosing the best object from his/her point of view, all d selectors must cooperate and chose the same object. The objective studied here is the maximization of the minimum over the d individual probabilities of choosing the best object. We exhibit the structure of the optimal rule. For independent criteria we give a full description of the rule and show that the optimal value tends to d-d/(d-1), as n → ∞. Furthermore, we show that as n → ∞, the liminf of the values under negatively associated criteria is bounded below by d-d/(d-1).

This paper describes the limiting asymptotic behaviour of a long cascade of linear reservoirs fed by stationary inflows into the first reservoir. We show that the storage in the nth reservoir becomes asymptotically deterministic as n → ∞, and establish a central limit theorem for the random fluctuations about the deterministic approximation. In addition, we prove a large deviations theorem that provides precise logarithmic asymptotics for the tail probabilities associated with the storage in the nth reservoir when n is large.