The asymptotic behaviour of the cumulative mean of a reward process 𝒵ρ, where the reward function ρ belongs to a rather large class of functions, is obtained. It is proved that E𝒵ρ(t) = C0 + C1t + o(1), t → ∞, where C0 and C1 are fully specified. A section is devoted to the dual process of a semi-Markov process, and a formula is given for the mean of the first passage time from a state i to a state j of the dual process, in terms of the means of passage times of the original process.

We study a not necessarily symmetric random walk with interactions on ℤ, which is an extension of the one-dimensional discrete version of the sausage Wiener path measure. We prove the existence of a repulsion/attraction phase transition for the critical value λc ≡ −μ of the repulsion coefficient λ, where μ is a drift parameter. In the self-repellent case, we determine the escape speed, as a function of λ and μ, and we prove a law of large numbers for the end-point.

A deterministic function of a Markov process is called an aggregated Markov process. We give necessary and sufficient conditions for the equivalence of continuous-time aggregated Markov processes. For both discrete- and continuous-time, we show that any aggregated Markov process which satisfies mild regularity conditions can be directly converted to a canonical representation which is unique for each class of equivalent models, and furthermore, is a minimal parameterization of all that can be identified about the underlying Markov process. Hidden Markov models on finite state spaces may be framed as aggregated Markov processes by expanding the state space and thus also have canonical representations.

This paper introduces a new stochastic process in which the iterates of a dynamical system evolving in discrete time coincide with the events of a Poisson process. The autocovariance function of the stochastic process is studied and a necessary and sufficient condition for it to vanish is deduced. It is shown that the mean function of this process comprises a continuous-time semidynamical system if the underlying dynamical map is linear. The flow of probability density functions generated by the stochastic process is analysed in detail, and the relationship between the flow and the solutions of the linear Boltzmann equation is investigated. It is shown that the flow is a semigroup if and only if the point process defining the stochastic process is Poisson, thereby providing a new characterization of the Poisson process.

Given a sequence of random variables (rewards), the Haviv–Puterman differential equation relates the expected infinite-horizon λ-discounted reward and the expected total reward up to a random time that is determined by an independent negative binomial random variable with parameters 2 and λ. This paper provides an interpretation of this proven, but previously unexplained, result. Furthermore, the interpretation is formalized into a new proof, which then yields new results for the general case where the rewards are accumulated up to a time determined by an independent negative binomial random variable with parameters k and λ.

We consider the problem of predicting integrals of second order processes whose covariances satisfy some Hölder regularity condition of order α > 0. When α is an odd integer, linear estimators based on regular sampling designs were constructed and asymptotic results for the approximation error were derived. We extend this result to any α > 0. When 2K < α ≤ 2K + 2, K a non-negative integer, we use an appropriate predictor based on the Euler-MacLaurin formula of order K with regular sampling designs. We give the corresponding result for the mean square error.

In time series analysis, it is well-known that the differencing operator ∇d may transform a non-stationary series, {Z(t)} say, to a stationary one, {W(t)} = ∇dZ(t)}; and there are many procedures for analysing and modelling {Z(t)} which exploit this transformation. Rather differently, Matheron (1973) introduced a set of measures on Rn that transform an appropriate non-stationary spatial process to stationarity, and Cressie (1988) then suggested that specialized low-order analogues of these measures, called increment-vectors, be used in time series analysis. This paper develops a general theory of increment-vectors which provides a more powerful transformation tool than mere simple differencing. The methodology gives a handle on the second-moment structure and divergence behaviour of homogeneously non-stationary series which leads to many important applications such as determining the correct degree of differencing, forecasting and interpolation.

Let {(Xn,Jn)} be a stationary Markov-modulated random walk on ℝ x E (E is finite), defined by its probability transition matrix measure F = {Fij}, Fij(B) = ℙ[X1 ∈ B, J1 = j | J0 = i], B ∈ B(ℝ), i, j ∈ E. If Fij([x,∞))/(1-H(x)) → Wij ∈ [0,∞), as x → ∞, for some long-tailed distribution function H, then the ascending ladder heights matrix distribution G+(x) (right Wiener-Hopf factor) has long-tailed asymptotics. If 𝔼Xn < 0, at least one Wij > 0, and H(x) is a subexponential distribution function, then the asymptotic behavior of the supremum of this random walk is the same as in the i.i.d. case, and it is given by ℙ[supn≥0Sn > x] → (−𝔼Xn)−1 ∫x∞ ℙ[Xn > u]du as x → ∞, where Sn = ∑1nXk, S0 = 0. Two general queueing applications of this result are given.

First, if the same asymptotic conditions are imposed on a Markov-modulated G/G/1 queue, then the waiting time distribution has the same asymptotics as the waiting time distribution of a GI/GI/1 queue, i.e., it is given by the integrated tail of the service time distribution function divided by the negative drift of the queue increment process. Second, the autocorrelation function of a class of processes constructed by embedding a Markov chain into a subexponential renewal process, has a subexponential tail. When a fluid flow queue is fed by these processes, the queue-length distribution is asymptotically proportional to its autocorrelation function.

Stochastic comparison results for replacement policies are shown in this paper using the formalism of point processes theory. At each failure moment a repair is allowed. It is performed with a random degree of repair including (as special cases) perfect, minimal and imperfect repair models. Results for such repairable systems with schemes of planned replacements are also shown. The results are obtained by coupling methods for point processes.

We consider a class of stationary infinite-order moving average processes with margins in the class of infinitely divisible exponential dispersion models. The processes are constructed by means of the thinning operation of Joe (1996), generalizing the binomial thinning used by McKenzie (1986, 1988) and Al-Osh and Alzaid (1987) for integer-valued time series. As a special case we obtain a class of autoregressive moving average processes that are different from the ARMA models proposed by Joe (1996). The range of possible marginal distributions for the new models is extensive and includes all infinitely divisible distributions with finite moment generating functions, hereunder many known discrete, continuous and mixed distributions.

Explicit formulas are found for the payoff and the optimal stopping strategy of the optimal stopping problem supτE (max0≤t≤τXt − c τ), where X = (Xt)t≥0 is geometric Brownian motion with drift μ and volatility σ > 0, and the supremum is taken over all stopping times for X. The payoff is shown to be finite, if and only if μ < 0. The optimal stopping time is given by τ* = inf {t > 0 | Xt = g* (max0≤t≤sXs)} where s ↦ g*(s) is the maximal solution of the (nonlinear) differential equation under the condition 0 < g(s) < s, where Δ = 1 − 2μ / σ2 and K = Δ σ2 / 2c. The estimate is established g*(s) ∼ ((Δ − 1) / K Δ)1 / Δs1−1/Δ as s → ∞. Applying these results we prove the following maximal inequality: where τ may be any stopping time for X. This extends the well-known identity E (supt>0Xt) = 1 − (σ 2 / 2 μ) and is shown to be sharp. The method of proof relies upon a smooth pasting guess (for the Stephan problem with moving boundary) and the Itô–Tanaka formula (being applied two-dimensionally). The key point and main novelty in our approach is the maximality principle for the moving boundary (the optimal stopping boundary is the maximal solution of the differential equation obtained by a smooth pasting guess). We think that this principle is by itself of theoretical and practical interest.

We present new closed form solutions for partition functions used to normalize the steady-state flow balance equations of certain Markovian type queueing networks. The results focus on single class closed product form networks with state space constraints at the queueing stations. They are achieved by combining the partition function of the open network, having finite local buffers with a delta function in order to fix the number of customers in the system.

We consider the inverse problem of estimating the input random field in a stochastic integral equation relating two random fields. The purpose of this paper is to present an approach to this problem using a Riesz-based or orthonormal-based series expansion of the input random field with uncorrelated random coefficients. We establish conditions under which the input series expansion induces (via the integral equation) a Riesz-based or orthonormal-based series expansion for the output random field. The estimation problem is studied considering two cases, depending on whether data are available from either the output random field alone, or from both the input and output random fields. Finally, we discuss this approach in the case of transmissivity estimation from piezometric head data, which was the original motivation of this work.

We consider the convex ordering for random vectors and some weaker versions of it, like the convex ordering for linear combinations of random variables. First we establish conditions of stochastic equality for random vectors that are ordered by one of the convex orderings. Then we establish necessary and sufficient conditions for the convex ordering to hold in the case of multivariate normal distributions and sufficient conditions for the positive linear convex ordering (without the restriction to multi-normality).

We prove a central limit theorem for conditionally centred random fields, under a moment condition and strict positivity of the empirical variance per observation. We use a random normalization, which fits non-stationary situations. The theorem applies directly to Markov random fields, including the cases of phase transition and lack of stationarity. One consequence is the asymptotic normality of the maximum pseudo-likelihood estimator for Markov fields in complete generality.

The filtering problem concerns the estimation of a stochastic process X from its noisy partial information Y. With the notable exception of the linear-Gaussian situation, general optimal filters have no finitely recursive solution. The aim of this work is the design of a Monte Carlo particle system approach to solve discrete time and nonlinear filtering problems. The main result is a uniform convergence theorem. We introduce a concept of regularity and we give a simple ergodic condition on the signal semigroup for the Monte Carlo particle filter to converge in law and uniformly with respect to time to the optimal filter, yielding what seems to be the first uniform convergence result for a particle approximation of the nonlinear filtering equation.

We study a class of simulated annealing type algorithms for global minimization with general acceptance probabilities. This paper presents simple conditions, easy to verify in practice, which ensure the convergence of the algorithm to the global minimum with probability 1.

Is the Ewens distribution the only one-parameter family of partition structures where the total number of types sampled is a sufficient statistic? In general, the answer is no. It is shown that all counterexamples can be generated via an urn scheme. The urn scheme need only satisfy two general conditions. In fact, the conditions are both necessary and sufficient. However, in particular, for a large class of partition structures that naturally arise in the infinite alleles theory of population genetics, the Ewens distribution is the only one in this class where the total number of types is sufficient for estimating the mutation rate. Finally, asymptotic sufficiency for parametric families of partition structures is discussed.

For truncated birth-and-death processes with two absorbing or two reflecting boundaries, necessary and sufficient conditions on the transition rates are given such that the transition probabilities satisfy a suitable spatial symmetry relation. This allows one to obtain simple expressions for first-passage-time densities and for certain avoiding transition probabilities. An application to an M/M/1 queueing system with two finite sequential queueing rooms of equal sizes is finally provided.

In this paper we study an approximation of system reliability using one-step conditioning. It is shown that, without greatly increasing the computational complexity, the conditional method may be used instead of the usual minimal cut and minimal path bounds to obtain more accurate approximations and bounds. We also study the conditions under which the approximations are bounds on the reliability. Some further extensions are also presented.