Hostname: page-component-78c5997874-xbtfd Total loading time: 0 Render date: 2024-11-09T20:00:39.717Z Has data issue: false hasContentIssue false

A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices

Published online by Cambridge University Press:  14 July 2016

A. Federgruen
Affiliation:
Mathematisch Centrum, Amsterdam
A. Hordijk
Affiliation:
Rijksuniversiteit Leiden
H. C. Tijms
Affiliation:
Vrije Universiteit, Amsterdam

Abstract

In this paper we consider a set of denumerable stochastic matrices where the parameter set is a compact metric space. We give a number of simultaneous recurrence conditions on the stochastic matrices and establish equivalences between these conditions. The results obtained generalize corresponding results in Markov chain theory to a considerable extent and have applications in stochastic control problems.

Type
Short Communications
Copyright
Copyright © Applied Probability Trust 1978 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Doob, J. L. (1953) Stochastic Processes. Wiley, New York.Google Scholar
[2] Federgruen, A. and Tijms, H. C. (1978) The optimality equation in average cost denumerable state semi-Markov decision problems, recurrency conditions and algorithms. J. Appl. Prob. 15, 356373.CrossRefGoogle Scholar
[3] Federgruen, A., Hordijk, A. and Tijms, H. C. (1978) Recurrence conditions in denumer able state Markov decision processes In Dynamic Programming and its Applications , ed. Puterman, M. L. Academic Press, New York.Google Scholar
[4] Hordijk, A. (1974) Dynamic Programming and Markov Potential Theory. Mathematical Centre Tract No. 51, Mathematisch Centrum, Amsterdam.Google Scholar
[5] Huang, C., Isaacson, D. and Vinograde, B. (1976) The rate of convergence of certain nonhomogeneous Markov chains. Z. Wahrscheinlichkeitsth. 35, 141146.CrossRefGoogle Scholar
[6] Isaacson, D. and Madsen, R. (1974) Positive columns for stochastic matrices. J. Appl. Prob. 11, 829835.CrossRefGoogle Scholar
[7] Neveu, J. (1965) Mathematical Foundations of the Calculus of Probability. Holden-Day, San Francisco.Google Scholar
[8] Rudin, W. (1964) Principles of Mathematical Analysis , 2nd edn. McGraw-Hill, New York.Google Scholar