In this paper we study polynomial and geometric (exponential) ergodicity for M/G/1-type Markov chains and Markov processes. First, practical criteria for M/G/1-type Markov chains are obtained by analyzing the generating function of the first return probability to level 0. Then the corresponding criteria for M/G/1-type Markov processes are given, using their h-approximation chains. Our method yields the radius of convergence of the generating function of the first return probability, which is very important in obtaining explicit bounds on geometric (exponential) convergence rates. Our results are illustrated, in the final section, in some examples.

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for continuous-time Markov reward chains with finite state spaces. The results correspond to discrete-time results. In particular, the variance growth rate is shown to be asymptotically linear in time. Expressions are provided to compute this growth rate.

We study exponential families within the class of counting processes and show that a mixed Poisson process belongs to an exponential family if and only if it is either a Poisson process or has a gamma structure distribution. This property can be expressed via exponential martingales.

We consider a buying-selling problem when two stops of a sequence of independent random variables are required. An optimal stopping rule and the value of a game are obtained.

Classical extreme value theory for stationary sequences of random variables can to a large extent be paraphrased as the study of exceedances over a high threshold. A special role within the description of the temporal dependence between such exceedances is played by the extremal index. Parts of this theory can be generalized not only to random variables on an arbitrary state space hitting certain failure sets, but even to a triangular array of rare events on an abstract probability space. In the case of M4 (maxima of multivariate moving maxima) processes, the arguments take a simple and direct form.

We investigate the asymptotic behaviour of homogeneous multidimensional Markov chains whose states have nonnegative integer components. We obtain growth rates for these models in a situation similar to the near-critical case for branching processes, provided that they converge to infinity with positive probability. Finally, the general theoretical results are applied to a class of controlled multitype branching process in which random control is allowed.

Consider N players, respectively owning x1, x2, …, xN monetary units, who play a sequence of games, winning from and losing to each other integer amounts according to fixed rules. The sequence stops as soon as (at least) one player is ruined. We are interested in the ruin process of these N players, i.e. in the probability that a given player is ruined first, and also in the expected ruin time. This problem is called the N-player ruin problem. In this paper, the problem is set up as a multivariate absorbing Markov chain with an absorbing state corresponding to the ruin of each player. This is then discussed in the context of phase-type distributions where each phase is represented by a vector of size N and the distribution has as many absorbing points as there are ruin events. We use this modified phase-type distribution to obtain an explicit solution to the N-player problem. We define a partition of the set of transient states into different levels, and on it give an extension of the folding algorithm (see Ye and Li (1994)). This provides an efficient computational procedure for calculating some of the key measures.

The distribution theory for reward functions on semi-Markov processes has been of interest since the early 1960s. The relevant asymptotic distribution theory has been satisfactorily developed. On the other hand, it has been noticed that it is difficult to find exact distribution results which lead to the effective computation of such distributions. Note that there is no satisfactory exact distribution result for rewards accumulated over deterministic time intervals [0, t], even in the special case of continuous-time Markov chains. The present paper provides neat general results which lead to explicit closed-form expressions for the relevant Laplace transforms of general reward functions on semi-Markov and Markov additive processes.

In this paper we identify a monotonicity in all countable-state-space reversible Markov chains and examine several consequences of this structure. In particular, we show that the return times to every state in a reversible chain have a decreasing hazard rate on the subsequence of even times. This monotonicity is used to develop geometric convergence rate bounds for time-reversible Markov chains. Results relating the radius of convergence of the probability generating function of first return times to the chain's rate of convergence are presented. An effort is made to keep the exposition rudimentary.

In this paper we obtain some ergodic properties and ergodic decompositions of a continuous-time, Borel right Markov process taking values in a locally compact and separable metric space. Initially, we assume that an invariant probability measure (IPM) μ exists for the process and, without making any further assumptions on the transition kernel, obtain some characterization results for the convergence of the expected occupation measure to a limit kernel. Under the same assumption, we present the so-called Yosida decomposition. Next, instead of assuming the existence of an IPM, we assume that the Markov process satisfies a certain condition, named the T'-condition. Under this condition it is shown that the Foster-Lyapunov criterion is necessary and sufficient for the existence of an IPM and that the process admits a Doeblin decomposition. Furthermore, it is shown that in this case the set of ergodic probability measures is countable and that every probability measure for the Markov process is nonsingular with respect to the transition kernel.

We study the genealogy of so-called immortal branching processes, i.e. branching processes where each individual upon death is replaced by at least one new individual, and conclude that their marginal distributions are compound geometric. The result also implies that the limiting distributions of properly scaled supercritical branching processes are compound geometric. We exemplify our results with an expression for the marginal distribution for a class of branching processes that have recently appeared in the theory of coalescent processes and continuous stable random trees. The limiting distribution can be expressed in terms of the Fox H-function, and in special cases by the Meijer G-function.

The problem of computing the moment generating function of the first passage time T to a > 0 or −b < 0 for a one-dimensional Wiener process {X(t), t ≥ 0} is generalized by assuming that the infinitesimal parameters of the process may depend on the sign of X(t). The probability that the process is absorbed at a is also computed explicitly, as is the expected value of T.

We find explicit analytical formulae for the time dependence of the probability of the number of Okazaki fragments produced during the process of DNA replication. This extends a result of Cowan on the asymptotic probability distribution of these fragments.

Local linear approximations have been the main component in the construction of a class of effective numerical integrators and inference methods for diffusion processes. In this note, two local linear approximations of jump diffusion processes are introduced as a generalization of the usual ones. Their rate of uniform strong convergence is also studied.

The northeast model is a spin system on the two-dimensional integer lattice that evolves according to the following rule: whenever a site's southerly and westerly nearest neighbors have spin 1, it may reset its own spin by tossing a p-coin; at all other times, its spin remains frozen. It is proved that the northeast model has a phase transition at pc = 1 - βc, where βc is the critical parameter for oriented percolation. For p < pc, the trivial measure, δ0, that puts mass one on the configuration with all spins set at 0 is the unique ergodic, translation-invariant, stationary measure. For p ≥ pc, the product Bernoulli-p measure on configuration space is the unique nontrivial, ergodic, translation-invariant, stationary measure for the system, and it is mixing. For p > ⅔, it is shown that there is exponential decay of correlations.

We give a criterion for extinction or local extinction of branching symmetric α-stable processes in terms of the principal eigenvalue for time-changed processes of symmetric α-stable processes. Here the branching rate and the branching mechanism are spatially dependent. In particular, the branching rate is allowed to be singular with respect to the Lebesgue measure. We apply this criterion to some branching processes.

We complete a paper written by Edward Pollak in 1974 on a multitype branching process the generating functions of whose birth law are fractional linear functions with the same denominator. The main tool is a parameterization of these functions adapted using the mean matrix M and an element w of the first quadrant. We use this opportunity to give a self-contained presentation of Pollak's theory.

We compare dependence in stochastically monotone Markov processes with partially ordered Polish state spaces using the concordance and supermodular orders. We show necessary and sufficient conditions for the concordance order to hold both in terms of the one-step transition probabilities for discrete-time processes and in terms of the corresponding infinitesimal generators for continuous-time processes. We give examples showing that a stochastic monotonicity assumption is not necessary for such orderings. We indicate relations between dependence orderings and, variously, the asymptotic variance-reduction effect in Monte Carlo Markov chains, Cheeger constants, and positive dependence for Markov processes.

A continuous-state population-size-dependent branching process {Xt} is a modification of the Jiřina process. We prove that such a process arises as the limit of a sequence of suitably scaled population-size-dependent branching processes with discrete states. The extinction problem for the population Xt is discussed, and the limit distribution of Xt / t obtained when Xt tends to infinity.

We study a network of fluid queues in which exogenous arrivals are modulated by a continuous-time Markov chain. Service rates in each queue are proportional to the queue size, and the network is assumed to be irreducible. The queue levels satisfy a linear, vector-valued differential equation. We obtain joint moments of the queue sizes recursively, and deduce the Laplace transform of the queue sizes in the stationary regime.