Hostname: page-component-cd9895bd7-fscjk Total loading time: 0 Render date: 2024-12-23T01:21:23.959Z Has data issue: false hasContentIssue false

Reward processes for semi-Markov processes: asymptotic behaviour

Published online by Cambridge University Press:  14 July 2016

A. Reza Soltani*
Affiliation:
Shiraz University
K. Khorshidian*
Affiliation:
Shiraz University
*
Postal address: Department of Statistics, Shiraz University, College of Sciences, Shiraz 71454, Iran.
Postal address: Department of Statistics, Shiraz University, College of Sciences, Shiraz 71454, Iran.

Abstract

The asymptotic behaviour of the cumulative mean of a reward process 𝒵ρ, where the reward function ρ belongs to a rather large class of functions, is obtained. It is proved that E𝒵ρ(t) = C0 + C1t + o(1), t → ∞, where C0 and C1 are fully specified. A section is devoted to the dual process of a semi-Markov process, and a formula is given for the mean of the first passage time from a state i to a state j of the dual process, in terms of the means of passage times of the original process.

Type
Research Papers
Copyright
Copyright © Applied Probability Trust 1998 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The research was supported by the Institute for Studies in Theoretical Physics and Mathematics (IPM).

References

Çinlar, E. (1969). Markov renewal theory. Adv. Appl. Prob. 1, 123187.Google Scholar
Çinlar, E. (1975). Markov renewal theory: a survey. Management Sci. 21, 727752.Google Scholar
Çinlar, E. (1975). Introduction to stochastic processes. Prentice-Hall, Englewood Cliffs, NJ.Google Scholar
Keilson, J. (1969). On the matrix renewal function for Markov renewal processes. Ann. Math. Statist. 40, 19011907.Google Scholar
Keilson, J. (1971). A process with chain dependent growth rate. Part II: the ruin and ergodic problems. Adv. Appl. Prob. 3, 315338.Google Scholar
Masuda, Y., and Sumita, U. (1991). A multivariate reward process defined on a semi-Markov process and its first passage time distributions. J. Appl. Prob. 28, 360373.Google Scholar
Mclean, R. A., and Neuts, M. F. (1967). The integral of a step function defined on a semi Markov process. SIAM J. Appl. Math. 15, 726737.Google Scholar
Pyke, R., and Schaufele, R. A. (1964). Limit theorem for Markov renewal processes. Ann. Math. Statist. 35, 17461764.Google Scholar
Soltani, A. R. (1996). Reward processes with nonlinear reward functions. J. Appl. Prob. 33, 10111017.Google Scholar
Sumita, U., and Masuda, Y. (1987). An alternative approach to the analysis of finite semi-Markov and related processes. Commun. Statist.–Stoch. Models 3, 6787.Google Scholar
te Teugels, J. L. (1976). A bibliography on semi-Markov processes. J. Comp. Appl. Math. 2, 125144.Google Scholar