In this paper we provide further results on the general d-dimensional correlated random walk. In particular, we prove that the n-step characteristic function of any correlated random walk satisfies a recurrence formula which enables both it and the total characteristic function to be obtained. Some examples are then considered.

In this paper, we discuss a variety of methods for computing the Wiener-Hopf factorization of a finite Markov chain associated to a fluctuating additive functional. The importance of this is that the equilibrium law of a fluid model can be expressed in terms of these Wiener–Hopf factors. The diagonalization methods considered are actually quite efficient, and provide an effective solution to the problem.

In [14] a necessary and sufficient condition was obtained for there to exist uniquely a Q-process with a specified invariant measure, under the assumption that Q is a stable, conservative, single-exit matrix. The purpose of this note is to demonstrate that, for an arbitrary stable and conservative q-matrix, the same condition suffices for the existence of a suitable Q-process, but that this process might not be unique. A range of examples is considered, including pure-birth processes, a birth process with catastrophes, birth-death processes and the Markov branching process with immigration.

We prove that the quasi-invariant measures associated to a Brownian motion with negative drift X form a one-parameter family. The minimal one is a probability measure inducing the transition density of a three-dimensional Bessel process, and it is shown that it is the density of the limit distribution limt→∞Px(X A | τ > t). It is also shown that the minimal quasi-invariant measure of infinite mass induces the density of the limit distribution ) which is the law of a Bessel process with drift.

We consider the two-dimensional process {X(t), V(t)} where {V(t)} is Brownian motion with drift, and {X(t)} is its integral. In this note we derive the joint density function of T and V(T) where T is the time at which the process {X(t)} first returns to its initial value. A series expansion of the marginal density of T is given in the zero-drift case. When V(0) and the drift are both positive there is a positive probability that {Χ (t)} never returns to its initial value. We show how this probability grows for small drift. Finally, using the Kontorovich–Lebedev transform pair we obtain the escape probability explicitly for arbitrary values of the drift parameter.

Krafft and Schaefer [14] considered a two-parameter Ehrenfest urn model and found the n-step transition probabilities using representations by Krawtchouk polynomials. For a special case of the model Palacios [17] calculated some of the expected first-passage times. This note investigates a generalization of the two-parameter Ehrenfest urn model where the transition probabilities pi,i+1 and pi,i+1 are allowed to be quadratic functions of the current state i. The approach used in this paper is based on the integral representations of Karlin and McGregor [9] and can also be used for Markov chains with an infinite state space.

At which (random) sample size will every population element have been drawn at least m times? This special coupon collector's problem is often referred to as the Dixie cup problem. Some asymptotic properties of the Dixie cup problem with unequal sampling probabilities are described.

In this paper we review conditions under which Wald's equation holds, mainly if the expectation of the given stopping time is infinite. As a main result we obtain what is probably the weakest possible version of Wald's equation for the case of independent, identically distributed (i.i.d.) random variables. Moreover, we improve a result of Samuel (1967) concerning the existence of stopping times for which the expectation of the stopped sum of the underlying i.i.d. sequence of random variables does not exist. Finally, we show by counterexamples that it is impossible to generalize a theorem of Kiefer and Wolfowitz (1956) relating the moments of the supremum of a random walk with negative drift to moments of the positive part of X1 to the case where the expectation of X1 is —∞. Here, the Laplace–Stieltjes transform of the supremum of the considered random walk plays an important role.

In this paper, we develop the probabilistic foundations of the dynamic continuous choice problem. The underlying choice set is a compact metric space E such as the unit interval or the unit square. At each time point t, utilities for alternatives are given by a random function . To achieve a model of dynamic continuous choice, the theory of classical vector-valued extremal processes is extended to super-extremal processesY= {Yt, t > 0}. At any t > 0, Yt is a random upper semicontinuous function on a locally compact, separable, metric space E. General path properties of Y are discussed and it is shown that Y is Markov with state-space US(E). For each t > 0, Yt is associated.

For a compact metric E, we consider the argmax process M = {Mt, t > 0}, where . In the dynamic continuous choice application, the argmax process M represents the evolution of the set of random utility maximizing alternatives. M is a closed set-valued random process, and its path properties are investigated.

We present two sufficient conditions for detection of optimal and non-optimal actions in (ergodic) average-cost MDPs. They are easily interpreted and can be implemented as detection tests in both policy iteration and linear programming methods. An efficient implementation of a recent new policy iteration scheme is discussed.

Kalashnikov and Rachev (1986) have proposed a partial ordering of life distributions which is equivalent to an increasing hazard ratio, when the ratio exists. This model can represent the phenomenon of crossing hazards, which has received considerable attention in recent years. In this paper we study this and two other models of relative ageing. Their connections with common partial orderings in the reliability literature are discussed. We examine the closure properties of the three orderings under several operations. Finally, we give reliability and moment bounds for a distribution when it is ordered with respect to a known distribution.

Allocation of a redundant component in a system in order to optimize, in some sense, the lifetime of the system is an important problem in reliability theory, having practical applications. Consider a series system consisting of two components (say C1 and C2), having independent random lifetimes X1 and X2, and suppose a component C having random lifetime X (independent of X1 and X2) is available for active redundancy with one of the components. Let U1 = min(max(X1, X), X2) and U2 = min(X1, max(X2, X)), so that U1 (U2) denote the lifetime of a system obtained by allocating C to C1 (C2). We consider the criterion where C1 is preferred to C2 for redundancy allocation if . Here we investigate the problem of allocating C to C1 or C2, with respect to the above criterion. We also consider the standby redundancy for series and parallel systems with respect to the above criterion. The problem of allocating an active redundant component in order that the resulting system has the smallest failure rate function is also considered and it is observed that unlike stochastic optimization, here the lifetime distribution of the redundant component also plays a role, making the problem of even active redundancy allocation more complex.

This paper studies optimal timing of inspections for units in cold stand-by. The problem is approached in continuous time. The policies developed adapt optimally to information provided by failures of the main unit and by ageing of both the standby and the main unit.

Results include a general form for the optimal policy, consisting of a recursive relation between successive inspection times, and of a renewal-type integral equation to determine the value of the objective function for a given policy. Further results are derived for the case in which the failure densities describing the process are logconcave. Then the stopping rule is easily characterized. It is also shown that the interval between inspections decreases with time, and that the optimal schedule is unique. A binary search algorithm for finding the solution is outlined.

When a mission is assigned, it often is the case that the component used to perform the task is required to work properly during the period of the mission time. In other words, the probability of the event that this component does not fail within the allowable mission time should be as large as possible. This problem is considered for the case when the lifetime of a component has a bathtub-shaped failure rate function, and it is found that burn-in procedure is beneficial. An application of this result to the problem of minimizing the mean number of failures in a given period of mission time is also considered.

In this paper we give a probabilistic interpretation for the function d(f), which has been proposed as a measure of fuzziness of a fuzzy set f. For this we construct random patterns which approximate the fuzzy set f and show that, for large n, the number of possible outcomes of the nth random pattern is approximately 2nd(f). We also give the best possible constant K in a more accurate approximation .

We obtain explicit upper bounds in closed form for the queue length in a slotted time FCFS queue in which the service requirement is a sum of independent Markov processes on the state space {0, 1}, with integral service rate. The bound is of the form [queue length for any where c < 1 and y > 1 are given explicitly in terms of the parameters of the model. The model can be viewed as an approximation for the burst-level component of the queue in an ATM multiplexer. We obtain heavy traffic bounds for the mean queue length and show that for typical parameters this far exceeds the mean queue length for independent arrivals at the same load. We compare our results on the mean queue length with an analytic expression for the case of unit service rate, and compare our results on the full distribution with computer simulations.

Modeling of manufacturing lines and data communications makes frequent use of tandem queues with blocking. Here we present and study such a system with a k-stage blocking scheme in which processing in each station requires attendance of the server of that station together with the servers of the next k – 1 stations. This scheme describes the conventional manufacturing and communications blocking schemes but is also representative of a wider range of applications. Explicit expressions for residence times, throughput, queueing times, waiting times and other measures of performance are obtained for the case of communications type flow and just-in-time input. Use of the results in modeling and analyzing a highway traffic flow situation is presented.

In this paper we study the departure process of M/G/1 queueing models with a single server vacation and multiple server vacations. The arguments employed are direct probability decomposition, renewal theory and the Laplace–Stieltjes transform. We discuss the distribution of the interdeparture time and the expected number of departures occurring in the time interval (0, t] from the beginning of the state i (i = 0, 1, 2, ···), and provide a new method for analysis of the departure process of the single-server queue.

We consider a class of G/G/1 queueing models with independent generalized setup time and exhaustive service. It is shown that a variety of single-server queueing systems with service interruption are special cases of our model. We give a simple computational scheme for the moments of the stationary waiting time and sojourn time. Our numerical investigations indicate that the algorithm is quite accurate and fast in general. For the M/G/1 case, we are able to derive a recursive formula for the moments of the stationary waiting time, which includes the Takács formula as a special case. It immediately results in the stochastic decompòsition property which can be found in the literature.

For a Markov chain in n = 2, drift vectors (conditional expected jumps) on the interior and the boundaries distinguish between recurrence and transience. The result of this paper is that the analogous proposition in the n = 3 case fails.