We consider a problem similar in many respects to a finite horizon Markov decision process, except that the reward to the individual is a strictly concave functional of the distribution of the state of the individual at final time T. Reward structures such as these are of interest to biologists studying the fitness of different strategies in a fluctuating environment. The problem fails to satisfy the usual optimality equation and cannot be solved directly by dynamic programming. We establish equations characterising the optimal final distribution and an optimal policy π*. We show that in general π* will be a Markov randomised policy (or equivalently a mixture of Markov deterministic policies) and we develop an iterative, policy improvement based algorithm which converges to π*. We also consider an infinite population version of the problem, and show that the population cannot do better using a coordinated policy than by each individual independently following the individual optimal policy π*.

For autoregressive time series with positive innovations which either have heavy right or left tails, linear programming parameter estimates of the autoregressive coefficients have good rates of convergence. However, the asymptotic distribution of the estimators depends heavily on the distribution of the process and thus cannot be used for inference. A bootstrap procedure circumvents this difficulty. We verify the validity of the bootstrap and also give some general comments on the bootstrapping of heavy tailed phenomena.

Cross-covariances between the Bernoulli thinned processes of an arbitrary point process are determined. When the point process is renewal it is shown that zero correlation implies independence. An example is given to show that zero covariance between intervals does not imply zero covariance between counts. Mark-dependent thinning of Markov renewal processes is discussed and the results are applied to the overflow queue. Here we give an example of two uncorrelated but dependent renewal processes, neither of which is Poisson, which yield a Poisson process when superposed. Finally, we study Markov-chain thinning of renewal processes.

We investigate the degree profile and total weight in Apollonian networks. We study the distribution of the degrees of vertices as they age in the evolutionary process. Asymptotically, the (suitably-scaled) degree of a node with a fixed label has a Mittag-Leffler-like limit distribution. The degrees of nodes of later ages have different asymptotic distributions, influenced by the time of their appearance. The very late arrivals have a degenerate distribution. The result is obtained via triangular Pólya urns. Also, via the Bagchi–Pal urn, we show that the number of terminal nodes asymptotically follows a Gaussian law. We prove that the total weight of the network asymptotically follows a Gaussian law, obtained via martingale methods. Similar results carry over to the sister structure of the k-trees, with minor modification in the proof methods, done mutatis mutandis.

The explicit form for the characteristic function of a stable distribution on the line is derived analytically by solving the associated functional equation and applying the theory of regular variation, without appeal to the general Lévy‒Khintchine integral representation of infinitely divisible distributions.

Batches of immigrants arrive in a region at event times of a renewal process and individuals grow according to a Bellman-Harris branching process. Tribal emigration allows the possibility that all descendants of a group of immigrants collectively leave the region at some instant.

A number of results are derived giving conditions for the existence of a limiting distribution for the population size. These conditions can be given either in terms of the immigration distribution or in terms of the distribution of emigration times. Some limit theorems are obtained when the latter conditions are not fulfilled.

Intrinsic volumes are key functionals in convex geometry and have also appeared in several stochastic settings. Here we relate them to questions of regularity in Gaussian processes with regard to Itô–Nisio oscillation and metrization of GB/GC indexing sets. Various bounds and estimates are presented, and questions for further investigation are suggested. From alternate points of view, much of the discussion can be interpreted in terms of (i) random sets and (ii) properties of (deterministic) infinite-dimensional convex bodies.

In this paper, we are concerned with preservation properties of first and second order by an operator L representable in terms of a stochastic process Z with non-decreasing right-continuous paths. We introduce the derived operator D of L and the derived process V of Z in order to characterize the preservation of absolute continuity and convexity. To obtain different characterizations of the preservation of convexity, we introduce two kinds of duality, the first referring to the process Z and the second to the derived process V. We illustrate the preceding results by considering some examples of interest both in probability and in approximation theory - namely, mixtures, centred subordinators, Bernstein polynomials and beta operators. In most of them, we find bidensities to describe the duality between the derived processes. A unified approach based on stochastic orders is given.

Let X(t) be a homogeneous and continuous stochastic process with independent increments. The subject of this paper is to characterize the stable process by two identically distributed stochastic integrals formed by means of X(t) (in the sense of convergence in probability). The proof of the main results is based on a modern extension of the Phragmén-Lindelöf theory.

In this paper, we study a maintenance model with general repair and two types of replacement: failure and preventive replacement. When the system fails a decision is made whether to replace or repair it. The repair degree that affects the virtual age of the system is assumed to be a random function of the repair-cost and the virtual age at failure time. The system can be preventively replaced at any time before failure. The objective is to find the repair/replacement policy minimizing the long-run expected average cost per unit time. It is shown that a generalized repair-cost-limit policy is optimal and the preventive replacement time depends on the virtual age of the system and on the length of the operating time since the last repair. Computational procedures for finding the optimal repair-cost limit and the optimal average cost are developed. This model includes many well-known models as special cases and the approach provides a unified treatment of a wide class of maintenance models.

One of two random variables, X and Y, can be selected at each of a possibly infinite number of stages. Depending on the outcome, one's fortune is either increased or decreased by 1. The probability of increase may not be known for either X or Y. The objective is to increase one's fortune to G before it decreases to g, for some integral g and G; either may be infinite.

In the current part of the paper, the distribution of X is unknown and that of Y is known. We characterize the situations in which optimal strategies exist and, for certain kinds of information concerning X and Y, we characterize optimal sequential strategies for choosing to observe X and Y.

In Part II (Berry and Fristedt (1980)), it is known that either X or Y has probability α of increasing the current fortune by 1 and the other has probability β of increasing the fortune by 1, where α and β are known, but which goes with X is not known.

In this paper we introduce multitype branching processes with inhomogeneous Poisson immigration, and consider in detail the critical Markov case when the local intensity r(t) of the Poisson random measure is a regularly varying function. Various multitype limit distributions (conditional and unconditional) are obtained depending on the rate at which r(t) changes with time. The asymptotic behaviour of the first and second moments, and the probability of nonextinction are investigated.

For n independent, identically distributed uniform points in [0, 1]d, d ≥ 2, let Ln be the total distance from the origin to all the minimal points under the coordinatewise partial order (this is also the total length of the rooted edges of a minimal directed spanning tree on the given random points). For d ≥ 3, we establish the asymptotics of the mean and the variance of Ln, and show that Ln satisfies a central limit theorem, unlike in the case d = 2.

A classical result about Markov jump processes states that a certain class of dynamical systems given by ordinary differential equations are obtained as the limit of a sequence of scaled Markov jump processes. This approach fails if the scaling cannot be carried out equally across all entities. In the present paper we present a convergence theorem for such an unequal scaling. In contrast to an equal scaling the limit process is not purely deterministic but still possesses randomness. We show that these processes constitute a rich subclass of piecewise-deterministic processes. Such processes apply in molecular biology where entities often occur in different scales of numbers.

We consider the same circuit switching problem as in Mitra [1]. The calculation of the blocking probabilities is reduced to finding the partition function for a closed exponential pseudo-network with L−1 customers. This pseudo-network differs from that in [1] in one respect only: service rates at nodes 1, 2, …, p depend on the queue length. The asymptotic expansion developed in [1] follows from our exact expression for the partition function.

The gating mechanism of a single ion channel is usually modelled by a continuous-time Markov chain with a finite state space. The state space is partitioned into two classes, termed ‘open’ and ‘closed’, and it is possible to observe only which class the process is in. In many experiments channel openings occur in bursts. This can be modelled by partitioning the closed states further into ‘short-lived’ and ‘long-lived’ closed states, and defining a burst of openings to be a succession of open sojourns separated by closed sojourns that are entirely within the short-lived closed states. There is also evidence that bursts of openings are themselves grouped together into clusters. This clustering of bursts can be described by the ratio of the variance Var (N(t)) to the mean [N(t)] of the number of bursts of openings commencing in (0, t]. In this paper two methods of determining Var (N(t))/[N(t)] and limt→∝ Var (N(t))/[N(t)] are developed, the first via an embedded Markov renewal process and the second via an augmented continuous-time Markov chain. The theory is illustrated by a numerical study of a molecular stochastic model of the nicotinic acetylcholine receptor. Extensions to semi-Markov models of ion channel gating and the incorporation of time interval omission are briefly discussed.

In this paper we study different types of planar random motions (performed with constant velocity) with three directions, defined by the vectors dj = (cos(2πj/3), sin(2πj/3)) for j = 0, 1, 2, changing at Poisson-paced times. We examine the cyclic motion (where the change of direction is deterministic), the completely uniform motion (where at each Poisson event each direction can be taken with probability ) and the symmetrically deviating case (where the particle can choose all directions except that taken before the Poisson event). For each of the above random motions we derive the explicit distribution of the position of the particle, by using an approach based on order statistics. We prove that the densities obtained are solutions of the partial differential equations governing the processes. We are also able to give the explicit distributions on the boundary and, for the case of the symmetrically deviating motion, we can write it as the distribution of a telegraph process. For the symmetrically deviating motion we use a generalization of the Bose-Einstein statistics in order to determine the distribution of the triple (N0, N1, N2) (conditional on N(t) = k, with N0 + N1 + N2 = N(t) + 1, where N(t) is the number of Poisson events in [0, t]), where Nj denotes the number of times the direction dj (j = 0, 1, 2) is taken. Possible extensions to four directions or more are briefly considered.

In this paper, a reflected stochastic differential equation (SDE) with jumps is studied for the case where the constraint acts on the law of the solution rather than on its paths. These reflected SDEs have been approximated by Briand et al. (2016) using a numerical scheme based on particles systems, when no jumps occur. The main contribution of this paper is to prove the existence and the uniqueness of the solutions to this kind of reflected SDE with jumps and to generalize the results obtained by Briand et al. (2016) to this context.

Statistical inference for Markov processes is commonly based on the maximum likelihood method of estimation and the likelihood ratio criterion for testing hypotheses. Construction of estimators and test statistics by these methods require that a model be chosen in the form of a family of transition density functions. In this paper, asymptotic properties of the maximum likelihood estimator and of the likelihood ratio statistic λn are examined when the model chosen for their construction is incorrect—that is, when no density in the model is a density for the transition probability distribution of the Markov process. It is shown that if and λn are constructed from a ‘regular’ incorrect model, then is consistent and asymptotically normally distributed and the asymptotic null distribution of −2 log λn is that of a linear combination of independent chi-squared random variables. These results are applied to propose measures of the performance of the test based on λn when the statistic is constructed from an incorrect model.