Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-11-23T03:06:27.710Z Has data issue: false hasContentIssue false
  • English
  • Français

Characterization and sufficient conditions for normed ergodicity of Markov chains

Part of: Operations research and management science Markov processes

Published online by Cambridge University Press:  01 July 2016

A. A. Borovkov  and
A. Hordijk
A. A. Borovkov*
Affiliation:
Sobolev Institute of Mathematics, Novosibirsk
A. Hordijk*
Affiliation:
Leiden University
*
∗ Postal address: Sobolev Institute of Mathematics, Koptjug pr. 4, 630090 Novosibirsk, Russia. Email address: [email protected]
∗∗ Postal address: Mathematical Institute, Leiden University, PO Box 9512, 2300 RA Leiden, The Netherlands. Email address: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Normed ergodicity is a type of strong ergodicity for which convergence of the nth step transition operator to the stationary operator holds in the operator norm. We derive a new characterization of normed ergodicity and we clarify its relation with exponential ergodicity. The existence of a Lyapunov function together with two conditions on the uniform integrability of the increments of the Markov chain is shown to be a sufficient condition for normed ergodicity. Conversely, the sufficient conditions are also almost necessary.

Keywords

Markov chainnormed ergodicityLyapunov function

MSC classification

Primary: 60J05: Discrete-time Markov processes on general state spaces
Secondary: 90B22: Queues and service
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 36 , Issue 1 , March 2004 , pp. 227 - 242
Copyright
Copyright © Applied Probability Trust 2004 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Borovkov, A. A. (1998). Ergodicity and Stability of Stochastic Processes. John Wiley, Chichester.Google Scholar
[2] Borovkov, A. A. and Hordijk, A. (2000). On normed ergodicity of Markov chains. Tech. Rep. MI 2000–40, Leiden University.Google Scholar
[3] Borovkov, A. A. and Hordijk, A. (2004). On linear Lyapunov functions and normed ergodicity for stochastic networks. In preparation.Google Scholar
[4] Dekker, R. and Hordijk, A. (1988). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395–421.Google Scholar
[5] Dekker, R., Hordijk, A. and Spieksma, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539–559.CrossRefGoogle Scholar
[6] Fayolle, G., Malyshev, V. A. and Menshikov, M. V. (1995). Topics in the Constructive Theory of Countable Markov Chains. Cambridge University Press.Google Scholar
[7] Federgruen, A., Hordijk, A. and Tijms, H. C. (1978). Recurrence conditions in denumerable state Markov decision processes. In Dynamic Programming and Its Applications, ed. Puterman, M. L., Academic Press, New York, pp. 3–22.Google Scholar
[8] Federgruen, A., Hordijk, A. and Tijms, H. C. (1978). A note on simultaneous recurrence conditions on a set of denumerable stochastic matrices. J. Appl. Prob. 15, 842–847.Google Scholar
[9] Heidergott, B. and Hordijk, A. (2003). Taylor series expansions for stationary Markov chains. Adv. Appl. Prob. 35, 1046–1070.Google Scholar
[10] Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Measure-valued differentiation for stationary Markov chains. EURANDOM Report 2002–027.Google Scholar
[11] Hordijk, A. (1974). Dynamic Programming and Markov Potential Theory (Math. Centre Tract 51). CWI, Amsterdam.Google Scholar
[12] Hordijk, A. and Dekker, R. (1983). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Report No. 83–36, Institute of Applied Mathematics and Computing Science, Leiden University.Google Scholar
[13] Hordijk, A. and Spieksma, F. M. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Prob. 24, 343–376.CrossRefGoogle Scholar
[14] Hordijk, A. and Yushkevich, A. A. (1999). Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards. Math. Meth. Operat. Res. 50, 421–448.Google Scholar
[15] Hordijk, A., Spieksma, F. M. and Tweedie, R. L. (1995). Uniform stability for general state space Markov decision processes. Tech. Rep., Leiden University.Google Scholar
[16] Kartoschov, N. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with common phase space. Theory Prob. Appl. 30, 247–259.Google Scholar
[17] Meyn, S. P. and Tweedie, R. L. (1993). Markov chains and stochastic stability. Springer, London.Google Scholar
[18] Neveu, J. (1965). Mathematical Foundations of the Calculus of Probability. Holden Day, San Francisco.Google Scholar
[19] Spieksma, F. M. and Tweedie, R. L. (1994). Strenghtening ergodicity to geometric ergodicity for Markov chains. Stoch. Models 10, 45–75.Google Scholar