In the classical framework, a random walk on a group is a Markov chain with independent and identically distributed increments. In some sense, random walks are time and space homogeneous. This paper is devoted to a class of inhomogeneous random walks on $\mathbb{Z}^d$ termed ‘Markov additive processes’ (also known as Markov random walks, random walks with internal degrees of freedom, or semi-Markov processes). In this model, the increments of the walk are still independent but their distributions are dictated by a Markov chain, termed the internal Markov chain. While this model is largely studied in the literature, most of the results involve internal Markov chains whose operator is quasi-compact. This paper extends two results for more general internal operators: a local limit theorem and a sufficient criterion for their transience. These results are thereafter applied to a new family of models of drifted random walks on the lattice $\mathbb{Z}^d$ .

We suggest four new measures of importance for repairable multistate systems based on the classical Birnbaum measure. Periodic component life cycles and general semi-Markov processes are considered. Similar to the Birnbaum measure, the proposed measures are generic in the sense that they only depend on the probabilistic properties of the components and the system structure. The multistate system model encodes physical properties of the components and the system directly into the structure function. As a result, calculating importance is easy, especially in the asymptotic case. Moreover, the proposed measures are composite measures, combining importance for all component states into a unified quantity. This simplifies ranking of the components with respect to importance. The proposed measures can be characterized with respect to two features: forward-looking versus backward-looking and with respect to how criticality is measured. Forward-looking importance measures focus on the next component states, while backward-looking importance measures focus on the previous component states. Two approaches to measuring criticality are considered: probability of criticality versus expected impact. Examples show that the different importance measures may result in unequal rankings.

The claims reserving problem is currently one of the most debated in actuarial literature. The high level of interest in this topic is due to the fact that Solvency II rules will come into operation in 2014. Indeed, it is expected that quantile computations will be compulsory in the evaluation of company risk and for this reason we think that the construction of the claims reserve random variable distribution assumes a fundamental relevance.

The aim of this paper is to present a method for constructing the claims reserve distribution which can take into account IBNyR (Issued But Not yet Reported) claims in a natural way. The construction of the distribution function for each time of the observed interval is done by means of a Monte Carlo simulation model applied on a backward time semi-Markov process. It should be pointed out that this is the first time that a simulation model based on semi-Markov with backward recurrence time has been presented. The method is totally different from the models given in the current literature.

The most important features given in the paper are:

1) for the first time the Monte Carlo simulation method is applied to a backward semi-Markov environment;

2) the Monte Carlo simulation permits the construction of the random variable of the claims reserve for each year of the studied horizon in a natural way;

3) as already pointed out, the backward process attached to the semi-Markov process permits taking into account the evaluation of the IBNyR claims in a natural way.

In the last part of the paper an applicative example constructed from tables that summarise 4 years of claims from an important Italian insurance company will be given.

We consider the following ordering for stochastic processes as introduced by Irle and Gani (2001). A process (Yt)t is said to be slower in level crossing than a process (Zt)t if it takes (Yt)t stochastically longer than (Zt)t to exceed any given level. In Irle and Gani (2001), this ordering was investigated for Markov chains in discrete time. Here these results are carried over to semi-Markov processes with particular attention to birth-and-death processes and also to Wiener processes.

Daley and Vesilo (1997) introduced long-range count dependence (LRcD) for stationary point processes on the real line as a natural augmentation of the classical long-range dependence of the corresponding interpoint sequence. They studied LRcD for some renewal processes and some output processes of queueing systems, continuing the previous research on such processes of Daley (1968), (1975). Subsequently, Daley (1999) showed that a necessary and sufficient condition for a stationary renewal process to be LRcD is that under its Palm measure the generic lifetime distribution has infinite second moment. We show that point processes dominating, in a sense of stochastic ordering, LRcD point processes are LRcD, and as a corollary we obtain that for arbitrary stationary point processes with finite intensity a sufficient condition for LRcD is that under Palm measure the interpoint distances are positively dependent (associated) with infinite second moment. We give many examples of LRcD point processes, among them exchangeable, cluster, moving average, Wold, semi-Markov processes and some examples of LRcD point processes with finite second Palm moment of interpoint distances. These examples show that, in general, the condition of infiniteness of the second moment is not necessary for LRcD. It is an open question whether the infinite second Palm moment of interpoint distances suffices to make a stationary point process LRcD.

The asymptotic behaviour of the cumulative mean of a reward process 𝒵ρ, where the reward function ρ belongs to a rather large class of functions, is obtained. It is proved that E𝒵ρ(t) = C0 + C1t + o(1), t → ∞, where C0 and C1 are fully specified. A section is devoted to the dual process of a semi-Markov process, and a formula is given for the mean of the first passage time from a state i to a state j of the dual process, in terms of the means of passage times of the original process.

Based on a semi-Markov process J(t), t ≧ 0, a reward process Z(t), t ≧ 0, is introduced where it is assumed that the reward function, p(k, x) is nonlinear; if the reward function is linear, i.e. ρ (k, x) = kx, the reward process Z(t), t ≧ 0, becomes the classical one, which has been considered by many authors. An explicit formula for E(Z(t)) is given in terms of the moments of the sojourn time distribution at t, when the reward function is a polynomial.

The limit behavior of the content of a subcriticai storage model defined on a semi-Markov process is examined. This is achieved by creating a renewal equation using a regeneration point (i0,0) of the process. By showing that the expected return time to (i0, 0) is finite, the conditions needed for the basic renewal theorem are established. The joint asymptotic distribution of the content of the storage at time t and the accumulated amount of the unmet (lost) demands during (0,t) is then established by showing the asymptotic independence of these two.

Let with Z(0) = 0 be a random process under investigation and N be a point process associated with Z. Both Z and N are defined on the same probability space. Let with R0 = 0 denote the consecutive positions of points of N on the half-line . In this paper we present sufficient conditions under which (Z, R) is a new better than used (NBU) process and give several examples of NBU processes satisfying these conditions. In particular we consider the processes in which N is a renewal and a general point process. The NBU property of some semi-Markov processes is also presented.

We consider a risk model in which the claim inter-arrivals and amounts depend on a markovian environment process. Semi-Markov risk models are so introduced in a quite natural way. We derive some quantities of interest for the risk process and obtain a necessary and sufficient condition for the fairness of the risk (positive asymptotic non-ruin probabilities). These probabilities are explicitly calculated in a particular case (two possible states for the environment, exponential claim amounts distributions).

In this part we extend and particularise results developed by the author in Part I (pp. 349–375) for a class of queueing systems which can be formulated as Markov renewal processes. We examine those models where the basic transition consists of only two types: ‘arrivals' and ‘departures'. The ‘arrival lobby' and ‘departure lobby' queue-length processes are shown, using the results of Part I to be Markov renewal. Whereas the initial study focused attention on the behaviour of the embedded discrete-time Markov chains, in this paper we examine, in detail, the embedded continuous-time semi-Markov processes. The limiting distributions of the queue-length processes in both continuous and discrete time are derived and interrelationships between them are examined in the case of continuous-time birth–death queues including the M/M/1/M and M/M/1 variants. Results for discrete-time birth–death queues are also derived.

This paper is concerned with the characterization of the cumulative pensionable service over an individual's working life that is made up of random lengths of service in different employments in a given industry, under partial coverage, transferability, and a uniform vesting rule. This characterization uses some results that are developed in the paper involving a functional and cumulative constrained sojourn times (constrained in the sense that if a sojourn time is less than a given constant it is not counted) in semi-Markov processes.

Nelson [9], [10] has computed the generating function of return probabilities to the initial state for a particular Markov chain on permutations of three objects. The formula obtained is

The present paper studies three distinct Markov chains generalizing the Nelson chain: the so-called three-coloured chain, with some birth-and-death processes on ℤ as a particular case, a chain on a graph close to the graph of the edges of a cube, and the daisy library. Two other themes piece together these chains: the notion of coloured chain and the technique of computation by additive processes.

We show that if the increment distribution of a renewal process has some convolution non-singular with respect to Lebesgue measure, then the skeletons of the forward recurrence time process are φ-irreducible positive recurrent Markov chains. Known convergence properties of such chains give simple proofs of uniform versions of some old and new key renewal theorems; these show in particular that non-singularity assumptions on the increment and initial distributions enable the assumption of direct Riemann integrability to be dropped from the standard key renewal theorem. An application to Markov renewal processes is given.

Let {(Jn, Xn), n ≧ 0} be the standard J–X process of Markov renewal theory. Suppose {Jn, n ≧ 0} is irreducible, aperiodic and positive recurrent. It is shown using the strong mixing condition, that if converges in distribution, where an, bn > 0 (bn → ∞) are real constants, then the limit law F must be stable. Suppose Q(x) = {PijHi(x)} is the semi-Markov matrix of {(JnXn), n ≧ 0}. Then the n-fold convolution, Q∗n(bnx + anbn), converges in distribution to F(x)Π if and only if converges in distribution to F. Π is the matrix of stationary transition probabilities of {Jn, n ≧ 0}. Sufficient conditions on the Hi's are given for the convergence of the sequence of semi-Markov matrices to F(x)Π, where F is stable.

Particles enter a finite-state system and move according to independent sample paths from a semi-Markov process. Strong limit theorems are developed for the ratio of the flow of particles from states i to j and the flow out of When the cumulative arrival of particles into the system up to time t, A (t) ∼ λtα, then a.s. When A (t)∼ λekt, then the flow between states must be normalized by the Laplace–Stieltjes transform of the conditional holding time distribution, in order to make the ratio an unbiased estimator of ρij.

In this paper the limit behaviour of α-recurrent Markov renewal processes and semi-Markov processes is studied by using the recent results on the concept of α-recurrence for Markov renewal processes. Section 1 contains the preliminary results, which are needed later in the paper. In Section 2 we consider the limit behaviour of the transition probabilities Pij(t) of an α-recurrent semi-Markov process. Section 4 deals with quasi-stationarity. Our results extend the results of Cheong (1968), (1970) and of Flaspohler and Holmes (1972) to the case in which the functions to be considered are directly Riemann integrable. We also try to correct the errors we have found in these papers. As a special case from our results we consider continuous-time Markov processes in Sections 3 and 5.

Consider a system that is subject to a sequence of randomly occurring shocks; each shock causes some damage of random magnitude to the system. Any of the shocks might cause the system to fail, and the probability of such a failure is a function of the sum of the magnitudes of damage caused from all previous shocks.

The purpose of this paper is to derive the optimal replacement rule for such a system whose cumulative damage process is a semi-Markov process. This allows for both the time between shocks and the damage due to the next shock to be dependent on the present cumulative damage level.

Only policies within the class of control-limit policies will be considered; namely, policies with which no action is taken if the damage is below a fixed level, and a replacement is made if the damage is above that.

An example will be given illustrating the use of the optimal replacement rule.

A sequence of random variables η0, η1, …, ηn, … is defined by the recurrence formula ηn = max (ηn–1 + ξn, 0) where η0 is a discrete random variable taking on non-negative integers only and ξ1, ξ2, … ξn, … is a semi-Markov sequence of discrete random variables taking on integers only. Define Δ as the smallest n = 1, 2, … for which ηn = 0. The random variable ηn can be interpreted as the content of a dam at time t = n(n = 0, 1, 2, …) and Δ as the time of first emptiness. This paper deals with the determination of the distributions of ηn and Δ by using the method of matrix factorisation.

It is shown that methods used by the author to approximate limit probabilities for Markov processes from their Q-matrices extend to semi-Markov processes. The limit probabilities for semi-Markov processes can be approximated using only truncations of the embedded Markov chain transition matrix and the vector of mean holding times.