Previous work on the concept of a limiting conditional age distribution of a discrete-state continuous-time Markov process with one absorbing state is generalised. The generalisation allows this process to have a finite number of absorbing states and the associated return process to have an arbitrary initial distribution on the transient states of the absorbing process. If the return process is ρ-recurrent, possesses the strong ratio limit property and satisfies some further requirements then the limiting age distribution exists. The proof of this result requires a new representation of the ρ-invariant measure of the return process.

The following examples are treated, (a) finite state space birth-death processes, (b) Markov branching processes and the linear death process, and (c) the linear birth and death process with killing.

The problem studied is that of controlling a finite Markov chain so as to maximize the long-run expected reward per unit time. The chain's transition probabilities depend upon an unknown parameter taking values in a subset [a, b] of Rn. A control policy is defined as the probability of selecting a control action for each state of the chain. Derived is a Taylor-like expansion formula for the expected reward in terms of policy variations. Based on that result, a recursive stochastic gradient algorithm is presented for the adaptation of the control policy at consecutive times. The gradient depends on the estimated transition parameter which is also recursively updated using the gradient of the likelihood function. Convergence with probability 1 is proved for the control and estimation algorithms.

Two urns initially contain r red balls and n – r black balls respectively. At each time epoch a ball is chosen randomly from each urn and the balls are switched. Effectively the same process arises in many other contexts, notably for a symmetric exclusion process and random walk on the Johnson graph. If Y(·) counts the number of black balls in the first urn then we give a direct asymptotic analysis of its transition probabilities to show that (when run at rate (n – r)/n in continuous time) for as n →∞, where π n denotes the equilibrium distribution of Y(·) and γ α = 1 – α /β (1 – β). Thus for large n the transient probabilities approach their equilibrium values at time log n + log|γ α | (≦log n) in a particularly sharp manner. The same is true of the separation distance between the transient distribution and the equilibrium distribution. This is an explicit analysis of the so-called cut-off phenomenon associated with a wide variety of Markov chains.

We are interested in the rate of convergence of a subordinate Markov process to its invariant measure. Given a subordinator and the corresponding Bernstein function (Laplace exponent), we characterize the convergence rate of the subordinate Markov process; the key ingredients are the rate of convergence of the original process and the (inverse of the) Bernstein function. At a technical level, the crucial point is to bound three types of moment (subexponential, algebraic, and logarithmic) for subordinators as time t tends to ∞. We also discuss some concrete models and we show that subordination can dramatically change the speed of convergence to equilibrium.

Recursive estimation is considered for parameters of certain continuous stochastic models. Several optimality properties are shown to hold for the resulting recursive estimator, where a stochastic approximation viewpoint is taken when deriving statistical properties, like strong consistency and convergence in distribution. Applications are considered throughout, where for example explosion theory for diffusion processes is used as a modeling guide, in a particular application.

Motivated by a problem arising in the mining industry, we estimate the energy ε(η) that is needed to reduce a unit mass to fragments of size at most η in a fragmentation process, when η→0. We assume that the energy used in the instantaneous dislocation of a block of size s into a set of fragments (s1,s2,…) is sβφ(s1/s,s2/s,…), where φ is some cost function and β a positive parameter. Roughly, our main result shows that if α>0 is the Malthusian parameter of an underlying Crump-Mode-Jagers branching process (with α = 1 when the fragmentation is mass-conservative), then there exists a c∈(0,∞) such that ε(η)∼cηβ-α when β<α. We also obtain a limit theorem for the empirical distribution of fragments of size less than η that result from the process. In the discrete setting, the approach relies on results of Nerman for general branching processes; the continuous approach follows by considering discrete skeletons. In the continuous setting, we also provide a direct approach that circumvents restrictions induced by the discretization.

We give three applications of the Pecherskii-Rogozin-Spitzer identity for Lévy processes. First, we find the joint distribution of the supremum and the epoch at which it is ‘attained’ if a Lévy process has phase-type upward jumps. We also find the characteristics of the ladder process. Second, we establish general properties of perturbed risk models, and obtain explicit fluctuation identities in the case that the Lévy process is spectrally positive. Third, we study the tail asymptotics for the supremum of a Lévy process under different assumptions on the tail of the Lévy measure.

We consider a dynamic multilevel population or information system. At each level individuals or information units undergo a Galton–Watson-type branching process in which they can be replicated or removed. In addition, a collection of individuals or information units at a given level constitutes an information unit at the next higher level. Each collection of units also undergoes a Galton–Watson branching process, either dying or replicating. In this paper, we represent this multilevel branching model as a measure-valued stochastic process, study its moment structure, identify the limiting continuous-state approximation and analyse the long-time behavior in both non-critical and critical cases. For example, we obtain an asymptotic expression for the extinction probability for the total population mass process and an analogue of Yaglom's conditioned limit theorem in the critical case.

We aim to link random fields and marked point processes, and, therefore, introduce a new class of stochastic processes which are defined on a random set in . Unlike for random fields, the mark covariance function of a random marked set is in general not positive definite. This implies that in many situations the use of simple geostatistical methods appears to be questionable. Surprisingly, for a special class of processes based on Gaussian random fields, we do have positive definiteness for the corresponding mark covariance function and mark correlation function.

Consider a haploid population of fixed finite size with a finite number of allele types and having Cannings exchangeable genealogy with neutral mutation. The stationary distribution of the Markov chain of allele counts in each generation is an important quantity in population genetics but has no tractable description in general. We provide upper bounds on the distributional distance between the Dirichlet distribution and this finite-population stationary distribution for the Wright–Fisher genealogy with general mutation structure and the Cannings exchangeable genealogy with parent independent mutation structure. In the first case, the bound is small if the population is large and the mutations do not depend too much on parent type; 'too much' is naturally quantified by our bound. In the second case, the bound is small if the population is large and the chance of three-mergers in the Cannings genealogy is small relative to the chance of two-mergers; this is the same condition to ensure convergence of the genealogy to Kingman's coalescent. These results follow from a new development of Stein's method for the Dirichlet distribution based on Barbour's generator approach and a probabilistic description of the semigroup of the Wright–Fisher diffusion due to Griffiths and Li (1983) and Tavaré (1984).

It is known that Bartoszyński’s model for the growth of rabies virus in an infected host is a continuous branching process. We show by explicit construction that any such process is a randomly time-transformed compound Poisson process having a negative linear drift.

This connection is exploited to obtain limit theorems for the population size and for the jump times in the rabies model. Some of these results are obtained in a more general context wherein the compound Poisson process is replaced by a subordinator.

We study Poisson-Voronoi tessellations in three-dimensional hyperbolic spaces, and give explicit expressions for mean surface area, mean perimeter length, and mean number of vertices of their cells. Furthermore we compare these mean characteristics with those for Poisson-Voronoi tessellations in three-dimensional Euclidean spaces. It is shown that, as the absolute value of the curvature of hyperbolic spaces increases from zero to infinity, these mean characteristics increase monotonically from those for the Euclidean case to infinity.

In this paper we study a multi-channel queueing model of type with N waiting places and a non-recurrent input flow dependent on queue length at the time of each arrival. The queue length is treated as a basic process. We first determine explicitly the limit distribution of the embedded Markov chain. Then, by introducing an auxiliary Markov process, we find a simple relationship between the limiting distribution of the Markov chain and the limiting distribution of the original process with continuous time parameter. Here we simultaneously combine two methods: solving the corresponding Kolmogorov system of the differential equations, and using an approach based on the theory of semi-regenerative processes. Among various applications of multi-channel queues with state-dependent input stream, we consider a closed single-server system with reserve replacement and state-dependent service, which turns out to be dual (in a certain sense) in relation to our model; an optimization problem is also solved, and an interpretation by means of tandem systems is discussed.

We address the question of controlling the Brownian path in several dimensions (d≧2) by continually choosing its drift from among vectors of the unit ball in ℝd. The past and present of the path are supposed to be completely observable, while no anticipation of the future is allowed. Imposing a suitable cost on distance from the origin, as well as a cost of effort proportional to the length of the drift vector, ‘reasonable’ procedures turn out to be of the following type: to apply drift of maximal length along the ray towards the origin if the current position is outside a sphere centred at the origin, and to choose zero drift otherwise. It is shown just how to compute the radius of such a sphere in terms of the data of the problem, so that the resulting procedure is optimal.

The kin number problem concerns the relationship between the distribution of the number of offspring of a randomly chosen individual in a population, and that of the number of relatives of various degrees of affinity of a randomly chosen individual (referred to as ‘Ego’). This problem is considered in terms of a population consisting of a large number of simultaneously developing Galton-Watson family trees. By time-reversal starting with the epoch at which Ego is sampled, it is shown that the number of offspring of Ego's parent (Ego plus siblings) and the number of offspring of Ego's grandparent (Ego's parent plus Ego's parent's siblings) have the same distribution, and the probability generating function (p.g.f.) is obtained in terms of the reproduction p.g.f. Further development of the method yields the joint p.g.f. of any number of generations prior, and subsequent, to that of Ego. Means and variances are obtained, and numerical examples are given based on data for the reproduction p.g.f. in two contrasting human populations. Various applications including demography and cancer research are discussed.

Let X be a one-dimensional strong Markov process with continuous sample paths. Using Volterra-Stieltjes integral equation techniques we investigate Hölder continuity and differentiability of first passage time distributions of X with respect to continuous lower and upper moving boundaries. Under mild assumptions on the transition function of X we prove the existence of a continuous first passage time density to one-sided differentiable moving boundaries and derive a new integral equation for this density. We apply our results to Brownian motion and its nonrandom Markovian transforms, in particular to the Ornstein-Uhlenbeck process.

We study the dispersion of a collection of particles carried by an isotropic Brownian flow in Of particular interest are the center of mass and the centered spatial second moments. Their asymptotic behavior depends strongly on the spatial dimension and the largest Lyapunov exponent of the flow. We use estimates for the pair separation process to give a fairly complete picture of this behavior as t → ∞. In particular, for incompressible flows in two dimensions, we show that the variance of the center of mass grows sublinearly, while dispersion relative to the center of mass grows linearly.

Consider N towers each made up of a number of counters. At each step a tower is chosen at random, a counter removed which is then added to another tower also chosen at random. The probability distribution for the time needed to empty one of the towers is obtained in the case N = 3. Arguments are set forward as to why no simple formulae can be expected for N > 3. An asymptotic expression for the mean time before one of the towers becomes empty is derived in the case of four towers when they all initially contain a comparably large number of counters. We then study related problems, in particular the ruin problem for three players. Here we use simple martingale methodology as well as a solution proposed by T. S. Ferguson for a slightly modified problem. Throughout the paper it is our main objective to shed light on the reasons why the case N > 3 is so substantially different from the case N ≤ 3.

In the present study, we establish the existence of site percolation in the Gabriel graph for Poisson and hard-core stationary point processes.

The objective of this study is to examine the asymptotic behavior of Betti numbers of Čech complexes treated as stochastic processes and formed from random points in the d-dimensional Euclidean space ${\mathbb{R}}^d$ . We consider the case where the points of the Čech complex are generated by a Poisson process with intensity nf for a probability density f. We look at the cases where the behavior of the connectivity radius of the Čech complex causes simplices of dimension greater than $k+1$ to vanish in probability, the so-called sparse regime, as well when the connectivity radius is of the order of $n^{-1/d}$ , the critical regime. We establish limit theorems in the aforementioned regimes: central limit theorems for the sparse and critical regimes, and a Poisson limit theorem for the sparse regime. When the connectivity radius of the Čech complex is $o(n^{-1/d})$ , i.e. the sparse regime, we can decompose the limiting processes into a time-changed Brownian motion or a time-changed homogeneous Poisson process respectively. In the critical regime, the limiting process is a centered Gaussian process but has a much more complicated representation, because the Čech complex becomes highly connected with many topological holes of any dimension.