Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-28T16:19:31.925Z Has data issue: false hasContentIssue false
  • English
  • Français

A Simple Numerical Approach for Infinite-State Markov Chains

Published online by Cambridge University Press:  27 July 2009

Henk C. Tijms  and
Michel C. T. Van De Coevering
Henk C. Tijms
Affiliation:
Department of Econometrics Vrije University, 1081 HV Amsterdam, The Netherlands
Michel C. T. Van De Coevering
Affiliation:
Department of Econometrics Vrije University, 1081 HV Amsterdam, The Netherlands
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper presents a simple and practical approach to solving the equilibrium equations for a class of Markov chains with an infinite number of states. Markov chains arising in queueing and inventory applications often have the property that the state probabilities exhibit a geometric tail behavior. The basic idea of the approach is to reduce the infinite system of linear equations to a finite system using the geometric tail behavior of the equilibrium probabilities. The reduction typically leads to a remarkably small system of linear equations that can be routinely solved by a Gaussian elimination method. An application is given to the single-server queue with scheduled arrivals.

Type
Articles
Information
Probability in the Engineering and Informational Sciences , Volume 5 , Issue 3 , July 1991 , pp. 285 - 295
Copyright
Copyright © Cambridge University Press 1991

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Barker, G.P. & Plemmons, R.J. (1986). Convergent iterations for computing stationary distributions for Markov chains. SIAM Journal Algebraic Discrete Methods 7: 390–398.CrossRefGoogle Scholar
Chaudry, M.L. & Templeton, J.G.C. (1983). A first course in bulk queues. New York: Wiley.Google Scholar
Everett, J.L. (1954). State probabilities in congestion problems characterized by constant holding times. Operations Research 1: 279–285.Google Scholar
Fredericks, A.A. (1982). A class of approximations for the waiting time distribution in a G1/0/l queueing system. Bell System Technical Journal 61: 295–325.CrossRefGoogle Scholar
Grassmann, W.K., Taksar, M.l., & Heyman, D.P. (1985). Regenerative analysis and steady- state distributions for Markov chains. Operations Research 33: 1107–1116.CrossRefGoogle Scholar
Neuts, M.F. (1984). Matrix-geometric solutions in stochastic models: An algorithmic approach. Baltimore: The Johns Hopkins University Press.Google Scholar
Press, W.H., Flannery, B.P., Teukoisky, S.A., & Vetterling, W.T. (1986), Numerical recipes. Cambridge, England: Cambridge University Press.Google Scholar
Ross, S. (1983). Stochastic processes. New York: Wiley.Google Scholar
Suzuki, T. (1963). Batch-arrival queueing problem. Journal of the Operations Research Society of Japan 5: 137–148.Google Scholar
Takahashi, Y. & Takami, Y. (1976). A numerical method for the steady state probabilities of a G1/G/c queueing system in a general class. Journal of the Operations Research Society of Japan 19: 147–157.CrossRefGoogle Scholar
Turns, H.C. (1986). Stochastic modeling and analysis: A computational approach. New York: Wiley.Google Scholar