Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-11T08:01:56.075Z Has data issue: false hasContentIssue false
  • English
  • Français

Taylor series expansions for stationary Markov chains

Part of: Markov processes

Published online by Cambridge University Press:  01 July 2016

Bernd Heidergott  and
Arie Hordijk
Bernd Heidergott*
Affiliation:
Vrije Universiteit Amsterdam and Tinbergen Institute
Arie Hordijk*
Affiliation:
Leiden University
*
∗ Postal address: Vrije Universiteit Amsterdam, Department of Economics, De Boelelaan 1105, 1081 HV Amsterdam, The Netherlands. Email address: [email protected]
∗∗ Postal address: Leiden University, Mathematical Institute, PO Box 9512, 2300 RA Leiden, The Netherlands.
Get access Rights & Permissions [Opens in a new window]

Abstract

We study Taylor series expansions of stationary characteristics of general-state-space Markov chains. The elements of the Taylor series are explicitly calculated and a lower bound for the radius of convergence of the Taylor series is established. The analysis provided in this paper applies to the case where the stationary characteristic is given through an unbounded sample performance function such as the second moment of the stationary waiting time in a queueing system.

Keywords

Markov chainTaylor seriesmeasure-valued differentiationdeviation matrix

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 90C31: Sensitivity, stability, parametric optimization
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 35 , Issue 4 , December 2003 , pp. 1046 - 1070
Copyright
Copyright © Applied Probability Trust 2003 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Ayhan, H. and Baccelli, F. (2001). Expansions for joint Laplace transforms of stationary waiting times in (max,+)-linear systems with Poisson input. Queueing Systems 37, 291–328.Google Scholar
[2] Ayhan, H. and Seo, D. (2001). Laplace transform and moments of waiting times in Poisson driven (max,+)-linear systems. Queueing Systems 37, 405–436.Google Scholar
[3] Ayhan, H. and Seo, D. (2002). Tail probability of transient and stationary waiting times in (max,+)-linear systems. IEEE Trans. Automatic Control 47, 151–157.Google Scholar
[4] Baccelli, F. and Schmidt, V. (1996). Taylor series expansions for Poisson-driven (max,+)-linear systems. Ann. Appl. Prob. 6, 138–185.CrossRefGoogle Scholar
[5] Baccelli, F., Hasenfuss, S. and Schmidt, V. (1997). Transient and stationary waiting times in (max,+)-linear systems with Poisson input. Queueing Systems 26, 301–342.CrossRefGoogle Scholar
[6] Baccelli, F., Hasenfuss, S. and Schmidt, V. (1998). Expansions for steady-state characteristics of (max,+)-linear systems. Commun. Statist. Stoch. Models 14, 1–24.Google Scholar
[7] Borovkov, A. and Hordijk, A. (2004). Characterization and sufficient conditions for normed ergodicity of Markov chains. To appear in Adv. Appl. Prob. 36, No. 1.Google Scholar
[8] Cao, X.-R. (1998). The Maclaurin series for performance functions of Markov chains. Adv. Appl. Prob. 30, 676–692.Google Scholar
[9] Cassandras, C. and Lafortune, S. (1999). Introduction to Discrete Event Systems. Kluwer, Norwell, MA.Google Scholar
[10] Coolen-Schrijner, P. and van Doorn, E. A. (2002). The deviation matrix of a continuous-time Markov chain. Prob. Eng. Inf. Sci. 16, 351–366.Google Scholar
[11] Dekker, R. and Hordijk, A. (1988). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Math. Operat. Res. 13, 395–421.CrossRefGoogle Scholar
[12] Dekker, R., Hordijk, A. and Spieksma, F. M. (1994). On the relation between recurrence and ergodicity properties in denumerable Markov decision chains. Math. Operat. Res. 19, 539–559.Google Scholar
[13] Gong, W.-B. and Hu, J.-Q. (1992). The Maclaurin series of the GI/G/1 queue. J. Appl. Prob. 29, 176–184.Google Scholar
[14] Heidergott, B. and Vázquez-Abad, F. (2000). Measure-valued differentiation for stochastic processes: the finite horizon case. EURANDOM Report 2000-033. Available at http://staff.feweb.vu.nl/bheidergott/.Google Scholar
[15] Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Derivatives of Markov kernels and their Jordan decomposition. EURANDOM Report 2003-001. Available at http://staff.feweb.vu.nl/bheidergott/.Google Scholar
[16] Heidergott, B., Hordijk, A. and Weisshaupt, H. (2002). Measure-valued differentiation for stationary Markov chains. EURANDOM Report 2002-027. Available at http://staff.feweb.vu.nl/bheidergott/.Google Scholar
[17] Ho, Y. and Cao, X. (1991). Perturbation Analysis of Discrete Event Systems. Kluwer, Boston, MA.CrossRefGoogle Scholar
[18] Hordijk, A. and Dekker, R. (1983). Average, sensitive and Blackwell optimal policies in denumerable Markov decision chains with unbounded rewards. Rep. 83–36, Institute of Applied Mathematics and Computing Science, Leiden University.Google Scholar
[19] Hordijk, A. and Puterman, M. L. (1987). On the convergence of policy iteration in finite state undiscounted Markov decision processes: the unichain case. Math. Operat. Res. 12, 163–176.Google Scholar
[20] Hordijk, A. and Spieksma, F. M. (1992). On ergodicity and recurrence properties of a Markov chain with an application to an open Jackson network. Adv. Appl. Prob. 24, 343–376.CrossRefGoogle Scholar
[21] Hu, J.-Q. (1995). Analyticity of single-server queues in light traffic. Queueing Systems 19, 63–80.Google Scholar
[22] Hu, J.-Q. (1996). The departure process of the GI/G/1 queue and its Maclaurin series. Operat. Res. 44, 810–815.CrossRefGoogle Scholar
[23] Kartoschov, N. (1985). Inequalities in theorems of ergodicity and stability for Markov chains with common phase space. Theory Prob. Appl. 30, 247–259.Google Scholar
[24] Koole, G. M. (1998). The deviation matrix of the M/M/1/∞ and M/M/1/N queue, with applications to controlled queueing models. In Proc. 37th IEEE Conf. Decision Control (Tampa, FL), IEEE Press, pp. 56–59.Google Scholar
[25] Koole, G. M. and Spieksma, F. M. (2001). On deviation matrices for birth–death process. Prob. Eng. Inf. Sci. 15, 239–258.Google Scholar
[26] Meyn, S. P. and Tweedie, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.CrossRefGoogle Scholar
[27] Pflug, G. (1996). Optimisation of Stochastic Models. Kluwer, Boston, MA.Google Scholar
[28] Van den Hout, W. (1996). The power-series-algorithm. Doctoral Thesis, Center for Economic Research, Tilburg University.Google Scholar
[29] Zhu, Y. and Li, H. (1993). The Maclaurin expansion for a G/GI/1 queue with Markov-modulated arrivals and services. Queueing Systems 14, 125–134.Google Scholar
[30] Zazanis, M. (1992). Analyticity of Poisson-driven stochastic systems. Adv. Appl. Prob. 24, 532–541.CrossRefGoogle Scholar