We consider two models for the control of a satellite–in the first, fuel is expended in a linear fashion to move a satellite following a diffusion–where the aim is to keep the satellite above a critical level for as long as possible (or indeed to reach a higher, ‘safe’ level). Under suitable assumptions for the drift and diffusion coefficients, it is shown that the stochastic maximum of the time to fall below the critical level is attained by a policy which imposes a reflecting boundary at the critical level until the fuel is exhausted and jumps the satellite directly to the safe level if this is ever possible. In the second model, there is a nonlinear response to the expenditure of fuel, and no safe level. It is shown that the optimal policy for maximizing the expected discounted time for the satellite to crash is similar, in that equal packets of fuel are used to jump the satellite upwards each time it reaches the critical level.

We analyse the expected performance of various group testing, or pooling, designs. The context is that of identifying characterized clones in a large collection of clones. Here we choose as performance criterion the expected number of unresolved ‘negative’ clones, and we aim to minimize this quantity. Technically, long inclusion–exclusion summations are encountered which, aside from being computationally demanding, give little inkling of the qualitative effect of parametric control on the pooling strategy. We show that readily-interpreted re-summation can be performed, leading to asymptotic forms and systematic corrections. We apply our results to randomized designs, illustrating how they might be implemented for approximating combinatorial formulae.

We introduce a continuous-time Markov chain model for the evolution of microsatellites, simple sequence repeats in DNA. We prove the existence of a unique stationary distribution for our model, and fit the model to data from approximately 106 base pairs of DNA from fruit flies, mice, and humans. The slippage rates from the best fit for our model are consistent with experimental findings.

Classical results describe the asymptotic behaviour of a Galton–Watson branching process conditioned on non-extinction. We give new proofs of limit theorems in critical and subcritical cases. The proofs are based on the representation of conditioned Galton–Watson generation sizes as a sum of independent increments which is derived from the decomposition of the conditioned Galton–Watson family tree along the line of descent of the left-most particle.

Well-known inequalities for the spectral gap of a discrete-time Markov chain, such as Poincaré's and Cheeger's inequalities, do not perform well if the transition graph of the Markov chain is strongly connected. For example in the case of nearest-neighbour random walk on the n-dimensional cube Poincaré's and Cheeger's inequalities are off by a factor n. Using a coupling technique and a contraction principle lower bounds on the spectral gap can be derived. Finally, we show that using the contraction principle yields a sharp estimate for nearest-neighbour random walk on the n-dimensional cube.

The number Yn of offspring of the most prolific individual in the nth generation of a Bienaymé–Galton–Watson process is studied. The asymptotic behaviour of Yn as n → ∞ may be viewed as an extreme value problem for i.i.d. random variables with random sample size. Limit theorems for both Yn and EYn provided that the offspring mean is finite are obtained using some convergence results for branching processes as well as a transfer limit lemma for maxima. Subcritical, critical and supercritical branching processes are considered separately.

The set of the attainable structures of a continuous time homogeneous Markov system (HMS) with fixed size, is considered as a continuum and the evolution of the HMS in the Euclidean space corresponds to its motion. Taking account of the velocity field of the HMS, a suitable model of continuum–defined by its stress tensor–is proposed in order to explain the motion of the system. The adoption of this model (equivalently of its stress tensor) enables us to establish the concept of the energy of a structure of the HMS.

Given independent random variables X1,…,Xn, with continuous distributions F1,…,Fn, we investigate the order in which these random variables should be arranged so as to minimize the number of upper records. We show that records are stochastically minimized if the sequence F1,…,Fn decreases with respect to a partial order, closely related to the monotone likelihood ratio property. Also, the expected number of records is shown to be minimal when the distributions are comparable in terms of a one-sided hazard rate ordering. Applications to parametric models are considered.

Consider the random walk {Sn} whose summands have the distribution P(X=0) = 1-(2/π), and P(X = ± n) = 2/[π(4n2−1)], for n ≥ 1. This random walk arises when a simple random walk in the integer plane is observed only at those instants at which the two coordinates are equal. We derive the fundamental matrix, or Green function, for the process on the integral [0,N] = {0,1,…,N}, and from this, an explicit formula for the mean time xk for the random walk starting from S0 = k to exit the interval. The explicit formula yields the limiting behavior of xk as N → ∞ with k fixed. For the random walk starting from zero, the probability of exiting the interval on the right is obtained. By letting N → ∞ in the fundamental matrix, the Green function on the interval [0,∞) is found, and a simple and explicit formula for the probability distribution of the point of entry into the interval (−∞,0) for the random walk starting from k = 0 results. The distributions for some related random variables are also discovered.

Applications to stress concentration calculations in discrete lattices are briefly reviewed.

In this paper, we give necessary and sufficient conditions to ensure the validity of confidence intervals, based on the central limit theorem, in simulations of highly reliable Markovian systems. We resort to simulations because of the frequently huge state space in practical systems. So far the literature has focused on the property of bounded relative error. In this paper we focus on ‘bounded normal approximation’ which asserts that the approximation of the normal law, suggested by the central limit theorem, does not deteriorate as the reliability of the system increases. Here we see that the set of systems with bounded normal approximation is (strictly) included in the set of systems with bounded relative error.

An open hierarchical (manpower) system divided into a totally ordered set of k grades is discussed. The transitions occur only from one grade to the next or to an additional (k+1)th grade representing the external environment of the system. The model used to describe the dynamics of the system is a continuous-time homogeneous Markov chain with k+1 states and infinitesimal generator R = (rij) satisfying rij = 0 if i > j or i + 1 < j ≤ k (i, j = 1,…,k+1), the transition matrix P between times 0 and 1 being P = expR. In this paper, two-wave panel data about the hierarchical system are considered and the resulting fact that, in general, the maximum-likelihood estimated transition matrix cannot be written as an exponential of an infinitesimal generator R having the form described above. The purpose of this paper is to investigate when this can be ascribed to the effect of sampling variability.

We present a general recurrence model which provides a conceptual framework for well-known problems such as ascents, peaks, turning points, Bernstein's urn model, the Eggenberger–Pólya urn model and the hypergeometric distribution. Moreover, we show that the Frobenius-Harper technique, based on real roots of a generating function, can be applied to this general recurrence model (under simple conditions), and so a Berry–Esséen bound and local limit theorems can be found. This provides a simple and unified approach to asymptotic theory for diverse problems hitherto treated separately.

In this paper we consider limit theorems for a random walk in a random environment, (Xn). Known results (recurrence-transience criteria, law of large numbers) in the case of independent environments are naturally extended to the case where the environments are only supposed to be stationary and ergodic. Furthermore, if ‘the fluctuations of the random transition probabilities around are small’, we show that there exists an invariant probability measure for ‘the environments seen from the position of (Xn)’. In the case of uniquely ergodic (therefore non-independent) environments, this measure exists as soon as (Xn) is transient so that the ‘slow diffusion phenomenon’ does not appear as it does in the independent case. Thus, under regularity conditions, we prove that, in this case, the random walk satisfies a central limit theorem for any fixed environment.

Repetitive Markov processes form a class of processes where the generator matrix has a particular repeating form. Many queueing models fall in this category such as M/M/1 queues, quasi-birth-and-death processes, and processes with M/G/1 or GI/M/1 generator matrices. In this paper, a new iterative scheme is proposed for computing the stationary probabilities of such processes. An infinite state process is approximated by a finite state process by lumping an infinite number of states into a super-state. What we call the feedback rate, the conditional expected rate of flow from the super-state to the remaining states, given the process is in the super-state, is approximated simultaneously with the steady state probabilities. The method is theoretically developed and numerically tested for quasi-birth-and-death processes. It turns out that the new concept of the feedback rate can be effectively used in computing the stationary probabilities.

Let ξ0,ξ1,ξ2,… be a homogeneous Markov process and let Sn denote the partial sum Sn = θ(ξ1) + … + θ(ξn), where θ(ξ) is a scalar nonlinearity. If N is a stopping time with 𝔼N < ∞ and the Markov process satisfies certain ergodicity properties, we then show that 𝔼SN = [limn→∞𝔼θ(ξn)]𝔼N + 𝔼ω(ξ0) − 𝔼ω(ξN). The function ω(ξ) is a well defined scalar nonlinearity directly related to θ(ξ) through a Poisson integral equation, with the characteristic that ω(ξ) becomes zero in the i.i.d. case. Consequently our result constitutes an extension to Wald's first lemma for the case of Markov processes. We also show that, when 𝔼N → ∞, the correction term is negligible as compared to 𝔼N in the sense that 𝔼ω(ξ0) − 𝔼ω(ξN) = o(𝔼N).

We consider a sequence of observations which is generated by a so-called hidden Markov model. An exponential smoothing procedure applied to such an observation sequence generates an inhomogeneous Markov process as a sequence of smoothed values. If the state sequence of the underlying hidden Markov model is moreover ergodic, then for two classes of smoothing functions the strong ergodicity of the sequence of smoothed values is proved. As a consequence a central limit theorem and a law of large numbers hold true for the smoothed values. The proof uses general results for so-called convergent inhomogeneous Markov processes. The procedure proposed by the author can be applied to some time series discussed in the literature.

Expressions for the multi-dimensional densities of Brownian excursion local time are derived by two different methods: a direct method based on Kac's formula for Brownian functionals and an indirect one based on a limit theorem for Galton–Watson trees.

A large deviation principle (LDP) with an explicit rate function is proved for the estimation of drift parameter of the Ornstein-Uhlenbeck process. We establish an LDP for two estimating functions, one of them being the score function. The first one is derived by applying the Gärtner–Ellis theorem. But this theorem is not suitable for the LDP on the score function and we circumvent this key point by using a parameter-dependent change of measure. We then state large deviation principles for the maximum likelihood estimator and another consistent drift estimator.

We study the records and related variables for sequences with linear trends. We discuss the properties of the asymptotic rate function and relationships between the distribution of the long-term maxima in the sequence and that of a particular observation, including two characterization type results. We also consider certain Markov chains related to the process of records and prove limit theorems for them, including the ergodicity theorem in the regular case (convergence rates are given under additional assumptions), and derive the limiting distributions for the inter-record times and increments of records.

This paper deals with the recurrence of branching random walks on polynomially growing graphs. Amongst other things, we demonstrate the strong recurrence of tree indexed random walks determined by the resistance properties of spherically symmetric graphs. Several branching walk models are considered to show how the branching mechanism influences the recurrence behaviour.