Hostname: page-component-cd9895bd7-jkksz Total loading time: 0 Render date: 2024-12-23T16:31:47.191Z Has data issue: false hasContentIssue false
  • English
  • Français

Lumpings of Markov Chains, Entropy Rate Preservation, and Higher-Order Lumpability

Part of: Communication, information Stochastic processes Markov processes Probabilistic methods, simulation and stochastic differential equations

Published online by Cambridge University Press:  30 January 2018

Bernhard C. Geiger  and
Christoph Temmel
Bernhard C. Geiger*
Affiliation:
Graz University of Technology
Christoph Temmel*
Affiliation:
Graz University of Technology and VU University Amsterdam
*
∗ Postal address: Institute for Communications Engineering, Technische Universiät München, Theresienstrasse 90, 80333 Munich. Email address: [email protected]
∗∗ Postal address: Department of Mathematics, VU University Amsterdam, De Boelelaan 1081a, 1081 HV Amsterdam, The Netherlands. Email address: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

A lumping of a Markov chain is a coordinatewise projection of the chain. We characterise the entropy rate preservation of a lumping of an aperiodic and irreducible Markov chain on a finite state space by the random growth rate of the cardinality of the realisable preimage of a finite-length trajectory of the lumped chain and by the information needed to reconstruct original trajectories from their lumped images. Both are purely combinatorial criteria, depending only on the transition graph of the Markov chain and the lumping function. A lumping is strongly k-lumpable, if and only if the lumped process is a kth-order Markov chain for each starting distribution of the original Markov chain. We characterise strong k-lumpability via tightness of stationary entropic bounds. In the sparse setting, we give sufficient conditions on the lumping to both preserve the entropy rate and be strongly k-lumpable.

Keywords

Lumpingentropy rate lossfunctional hidden Markov modelstrong lumpabilityhigher-order Markov chain

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 60G17: Sample path properties 94A17: Measures of information, entropy 60G10: Stationary processes 65C40: Computational Markov chains
Type
Research Article
Information
Journal of Applied Probability , Volume 51 , Issue 4 , December 2014 , pp. 1114 - 1132
Copyright
© Applied Probability Trust 

References

Anderson, B. D. O. (1999). The realization problem for hidden Markov models. Math. Control Signals Systems 12, 80–120.Google Scholar
Brown, P. F. et al. (1992). Class-based n-gram models of natural language. Comput. Linguist. 18, 467–479.Google Scholar
Blackwell, D. (1957). The entropy of functions of finite-state Markov chains. In Trans 1st Prague Conf. Inf. Theory, Statist. Decision Functions, (Liblice, 1956). Publishing House of the Czechoslovak Academy of Sciences, Prague, pp. 13–20.Google Scholar
Blackwell, D. and Koopmans, L. (1957). On the identifiability problem for functions of finite Markov chains. Ann. Math. Statist. 28, 1011–1015.CrossRefGoogle Scholar
Burke, C. J. and Rosenblatt, M. (1958). A Markovian function of a Markov chain. Ann. Math. Statist. 29, 1112–1122.CrossRefGoogle Scholar
Carlyle, J. W. (1967). Identification of state-calculable functions of finite Markov chains. Ann. Math. Statist. 38, 201–205.Google Scholar
Cover, T. M. and Thomas, J. A. (2006). Elements of Information Theory, 2nd edn. John Wiley, Hoboken, NJ.Google Scholar
Ephraim, Y. and Merhav, N. (2002). Hidden Markov processes. IEEE Trans. Inf. Theory 48, 1518–1569.CrossRefGoogle Scholar
Geiger, B. C. and Kubin, G. (2011). Some results on the information loss in dynamical systems. In Proc. IEEE Internat. Symp. Wireless Commun. Systems (ISWSC), IEEE, New York, pp. 794–798, 2011. Extended version available at http://uk.arxiv.org/abs/1106.2404.Google Scholar
Geiger, B. C. and Temmel, C. (2013). Information-preserving Markov aggregation. In Proc. IEEE Information Theory Workshop (ITW), IEEE, New York, pp. 258–262. Extended version available at http://uk.arxiv.org/abs/1304.0920.Google Scholar
Gilbert, E. J. (1959). On the identifiability problem for functions of finite Markov chains. Ann. Math. Statist. 30, 688–697.Google Scholar
Gray, R. M. (1990). Entropy and Information Theory. Springer, New York.Google Scholar
Gurvits, L. and Ledoux, J. (2005). Markov property for a function of a Markov chain: a linear algebra approach. Linear Algebra Appl. 404, 85–117.Google Scholar
Heiner, M., Rohr, C., Schwarick, M. and Streif, S. (2010). A comparative study of stochastic analysis techniques. In Proc. 8th Internat. Conf. Comput. Meth. Systems Biol., ACM, New York, pp. 96–106.Google Scholar
Heller, A. (1965). On stochastic processes derived from Markov chains. Ann. Math. Statist. 36, 1286–1291.Google Scholar
Henzinger, T. A., Mikeev, L., Mateescu, M. and Wolf, V. (2010). Hybrid numerical solution of the chemical master equation. In Proc. 8th Internat. Conf. Comput. Meth. Systems Biol., ACM, New York, pp. 55–65.Google Scholar
Kemeny, J. G. and Snell, J. L. (1976). Finite Markov Chains. Springer, New York.Google Scholar
Kieffer, J. C. and Rahe, M. (1981). Markov channels are asymptotically mean stationary. SIAM J. Math. Anal. 12, 293–305.Google Scholar
Lindqvist, B. (1978). On the loss of information incurred by lumping states of a Markov chain. Scand. J. Statist. 5, 92–98.Google Scholar
Parzen, E. (1999). Stochastic Processes (Classics Appl. Math. 24). Society for Industrial and Applied Mathematics, Philadelphia, PA.Google Scholar
Pinsker, M. S. (1964). Information and Information Stability of Random Variables and Processes. Holden-Day, San Francisco, CA.Google Scholar
Rogers, L. C. G. and Pitman, J. W. (1981). Markov functions. Ann. Prob. 9, 573–582.Google Scholar
Sarukkai, R. R. (2000). Link prediction and path analysis using Markov chains. Comput. Networks 33, 377–386.Google Scholar
Watanabe, S. and Abraham, C. T. (1960). Loss and recovery of information by coarse observation of stochastic chain. Inf. Control 3, 248–278.Google Scholar
Wilkinson, D. J. (2006). Stochastic Modelling for Systems Biology. Chapman & Hall/CRC, Boca Raton, FL.Google Scholar
Woess, W. (2009). Denumerable Markov chains. Generating Functions, Boundary Theory, Random Walks on Trees. European Mathematical Society, Zürich.Google Scholar