N tasks must be successfully performed for a job to be completed. The tasks may be attempted in any order, where each attempt of task i requires an expected cost ci and is successful with probability pi. Whenever an attempt fails, the job is fed back to the initial stage and the entire sequence starts again. We show that the cost of completing a job is minimized if the tasks are sequenced via increasing values of ci/(l–pi). We further show that the same result holds when the feedback can be either to stage i itself or to the starting task.

The Lagrange-Sylvester interpolation polynomial approach provides a simple, eigenvector-free representation for finite diagonalizable matrices. This paper discusses the Lagrange-Sylvester methodology and applies it to skip free to the right Markov chains. It leads to relatively simple, eigenvalue-based expressions for first passage time distributions and transition probabilities.

When waiting times cannot be observed directly, Little's law can be applied to estimate the average waiting time by the average number in system divided by the average arrival rate, but that simple indirect estimator tends to be biased significantly when the arrival rates are time-varying and the service times are relatively long. Here it is shown that the bias in that indirect estimator can be estimated and reduced by applying the time-varying Little's law (TVLL). If there is appropriate time-varying staffing, then the waiting time distribution may not be time-varying even though the arrival rate is time varying. Given a fixed waiting time distribution with unknown mean, there is a unique mean consistent with the TVLL for each time t. Thus, under that condition, the TVLL provides an estimator for the unknown mean wait, given estimates of the average number in system over a subinterval and the arrival rate function. Useful variants of the TVLL estimator are obtained by fitting a linear or quadratic function to arrival data. When the arrival rate function is approximately linear (quadratic), the mean waiting time satisfies a quadratic (cubic) equation. The new estimator based on the TVLL is a positive real root of that equation. The new methods are shown to be effective in estimating the bias in the indirect estimator and reducing it, using simulations of multi-server queues and data from a call center.

We consider a general discrete-time stochastic recursive model that is influenced by an external Markov chain. Our aim is to investigate the effect that the transition matrix of the external process has on the system states of the model. To answer this question, we use new stochastic ordering concepts. Especially interesting are the results for infinite-stage Markov-modulated models. We illustrate our main results by three applications: an inventory model, a consumption model, and a queueing model for a time division multiplexing system.

In this article we study a firm that is facing demand from two sources: demand for new items and demand to replace failed items under warranty. We model this setting as a multiperiod single-product inventory problem where the demands for new items in different periods are independent and the demands for replacing failed items depend on the number of the items under warranty. We consider backlogging and emergency supply cases and study both discounted-cost and average-cost criteria. We prove the optimality of the w-dependent base stock ordering policy, where the base stock level is a function of w, the number of items currently under warranty. For the special case where the demand for new products is stationary, we prove the optimality of a stationary w-dependent base stock policy for the finite-horizon discounted-cost and the infinite-horizon discounted- and average-cost cases. We compare the integrated inventory policy with the one that neglects demands from items under warranty.

We study a stochastic fluid EOQ-type model operating in a Markovian random environment of alternating good and bad periods determining the demand rate. We deal with the classical problem of “when to place an order” and “how big it should be,” leading to the trade-off between the setup cost and the holding cost. The key functionals are the steady-state mean of the content level, the expected cycle length (which is the time between two large orders), and the expected number of orders in a cycle. These performance measures are derived in closed form by using the level crossing approach in an intricate way. We also present numerical examples and carry out a sensitivity analysis.

A simulation-based algorithm for learning good policies for a discrete-time stochastic control process with unknown transition law is analyzed when the state and action spaces are compact subsets of Euclidean spaces. This extends the Q-learning scheme of discrete state/action problems along the lines of Baker [4]. Almost sure convergence is proved under suitable conditions.

A 2-by-2 buffered switch is the basic element of certain parallel data-processing networks. For a switch fed by two independent Bernoulli input streams, we find the joint distribution of the number of messages waiting in the two buffers at equilibrium, in the form of a bivariate generating function. The derivation uses complex-variable techniques developed by Kingman and by Flatto and McKean for the “shortest queue problem.” A number of asymptotic results are given, the principal one being the variance of the total number of waiting messages in the heavy-traffic limit.

The asymptotic probability of buffer overflow for a queueing system with a Markov fluid input and deterministic service rate is derived by way of large deviation theory. The equations characterizing the deviant behavior are presented and examples are given for which closed-form solutions may be obtained. An independence result extends the analysis to cases where the input is an aggregate of independent Markov fluids.

Limit theorems are established and relatively simple closed-form approximations are developed for the busy-period distribution in single-server queues. For the M/G/l queue, the complementary busy-period c.d.f. is shown to be asymptotically equivalent as t → ∞ to a scaled version of the heavy-traffic limit (obtained as p → 1), where the scaling parameters are based on the asymptotics as t → ∞. We call this the asymptotic normal approximation, because it involves the standard normal c.d.f. and density. The asymptotic normal approximation is asymptotically correct as t → ∞ for each fixed p and as p → 1 for each fixed t and yields remarkably good approximations for times not too small, whereas the direct heavy-traffic (p → 1) and asymptotic (t → ∞) limits do not yield such good approximations. Indeed, even the approximation based on three terms of the standard asymptotic expansion does not perform well unless t is very large. As a basis for generating corresponding approximations for the busy-period distribution in more general models, we also establish a more general heavy-traffic limit theorem.

This paper defines and studies the down entrance state (DES) and the restart entrance state (RES) classes of quasi-skip free (QSF) processes specified in terms of the nonzero structure of the elements of their transition rate matrix Q. A QSF process is a Markov chain with states that can be specified by tuples of the form (m, i), where $m \in {\open Z}$ represents the “current” level of the state and $i \in {\open Z}^{+}$ the current phase of the state, and its transition probability matrix Q does not permit one-step transitions to states that are two or more levels away from the current state in one direction of the level variable m. A QSF process is a DES process if and only if one step “down” transitions from a level m can only reach a single state in level m − 1, for all m. A QSF process is a RES process if and only if one step “up” transitions from a level m can only reach a single set of states in the highest level M2, largest of all m.

We derive explicit solutions and simple truncation bounds for the steady-state probabilities of both DES and RES processes, when in addition Q insures ergodicity. DES and RES processes have applications in many areas of applied probability comprising computer science, queueing theory, inventory theory, reliability, and the theory of branching processes. To motivate their applicability we present explicit solutions for the well-known open problem of the M/Er/n queue with batch arrivals, an inventory model, and a reliability model.

A statistic for testing the null hypothesis that F is the exponential distribution against the alternative of harmonic new better than used in expectation (HNBUE) is proposed. The asymptotic distribution theory for this statistic is derived under the null hypothesis and asymptotic relative efficiency (ARE) with respect to other competing tests for HNBUE is evaluated. This test is applied to the leukemia data described in Bryson and Siddiqui (1969).

Named after the French–Belgian mathematician Eugène Charles Catalan, Catalan's numbers arise in various combinatorial problems [12]. Catalan's triangle, a triangular array of numbers somewhat similar to Pascal's triangle, extends the combinatorial meaning of Catalan's numbers and generalizes them [1,5,11]. A need for a generalization of Catalan's triangle itself arose while conducting a probabilistic analysis of the Asymmetric Simple Inclusion Process (ASIP) — a model for a tandem array of queues with unlimited batch service [7–10]. In this paper, we introduce Catalan's trapezoids, a countable set of trapezoids whose first element is Catalan's triangle. An iterative scheme for the construction of these trapezoids is presented, and a closed-form formula for the calculation of their entries is derived. We further discuss the combinatorial interpretations and applications of Catalan's trapezoids.

In this article, we propose a new continuous-time stochastic inventory model with deterioration and stock-dependent demand items. We then formulate the problem of finding the optimal impulse control schedule that minimizes the total expected return over an infinite horizon, as a quasivariational inequality (QVI) problem. The QVI is shown to lead to an (s, S) policy, where s and S are determined uniquely as a solution of some algebraic equations.

We consider the classical coupon collector's problem in which each new coupon collected is type i with probability pi; ∑i=1npi=1. We derive some formulas concerning N, the number of coupons needed to have a complete set of at least one of each type, that are computationally useful when n is not too large. We also present efficient simulation procedures for determining P(N > k), as well as analytic bounds for this probability.

The distribution of upward first passage times in skip-free Markov chains can be expressed solely in terms of the eigenvalues in the spectral representation, without performing a separate calculation to determine the eigenvectors. We provide insight into this result and skip-free Markov chains more generally by showing that part of the spectral theory developed for birth-and-death processes extends to skip-free chains. We show that the eigenvalues and eigenvectors of skip-free chains can be characterized in terms of recursively defined polynomials. Moreover, the Laplace transform of the upward first passage time from 0 to n is the reciprocal of the nth polynomial. This simple relationship holds because the Laplace transforms of the first passage times satisfy the same recursion as the polynomials except for a normalization.

This paper is divided into two distinct parts: the first considers a loss network, the second a queueing network. In each case, we consider a fully connected network consisting of a large number of links (queues) and operating under a dynamic routing policy known as least busy alternative routing. Using weak convergence results, we can examine the behavior of the networks as the number of links (queues) increases to infinity. We find that, despite the models having similarities and being amenable to the same analytical tools, they exhibit important differences in character.

This paper gives fast and reliable algorithms for the numerical evaluation of the incomplete gamma functions and for auxiliary functions, such as functions related with the gamma function and error function. All these functions are of basic importance in applied probability problems.

Our primary aim in this paper is to study a functional equation that arises in a problem of queueing. Consider a queue with compound Poisson arrivals and general service times with a gating mechanism. The gating mechanism takes in at most m(≤∞) customers at a time for service and serves these customers according to the processor-sharing discipline. In this paper, we examine various performance characteristics for this queue. The characteristics include waiting-time distribution, queue-length distribution, time spent with the server, and batch-size distribution for service.

Items of various types arrive at a bin-packing facility according to random processes and are to be combined with other readily available items of different types and packed into bins using one of a number of possible packings. One might think of a manufacturing context in which randomly arriving subassemblies are to be combined with subassemblies from an existing inventory to assemble a variety of finished products. Packing must be done on-line; that is, as each item arrives, it must be allocated to a bin whose configuration of packing is fixed. Moreover, it is required that the packing be managed in such a way that the readily available items are consumed at predescribed rates, corresponding perhaps to optimal rates for manufacturing these items. At any moment, some number of bins will be partially full. In practice, it is important that the packing be managed so that the expected number of partially full bins remains uniformly bounded in time. We present a necessary and sufficient condition for this goal to be realized and describe an algorithm to achieve it.