Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-16T15:22:08.431Z Has data issue: false hasContentIssue false
  • English
  • Français

Extended Laplace principle for empirical measures of a Markov chain

Part of: Markov processes Limit theorems

Published online by Cambridge University Press:  22 July 2019

Stephan Eckstein
Stephan Eckstein*
Affiliation:
University of Konstanz
*
*Postal address: Department of Mathematics, University of Konstanz, 78464 Konstanz, Germany. Email address: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider discrete-time Markov chains with Polish state space. The large deviations principle for empirical measures of a Markov chain can equivalently be stated in Laplace principle form, which builds on the convex dual pair of relative entropy (or Kullback– Leibler divergence) and cumulant generating functional f ↦ ln ʃ exp (f). Following the approach by Lacker (2016) in the independent and identically distributed case, we generalize the Laplace principle to a greater class of convex dual pairs. We present in depth one application arising from this extension, which includes large deviation results and a weak law of large numbers for certain robust Markov chains—similar to Markov set chains—where we model robustness via the first Wasserstein distance. The setting and proof of the extended Laplace principle are based on the weak convergence approach to large deviations by Dupuis and Ellis (2011).

Keywords

Large deviationsMarkov chainconvex dualitydistributional uncertainty

MSC classification

Primary: 60J05: Discrete-time Markov processes on general state spaces
Secondary: 60F10: Large deviations
Type
Original Article
Information
Advances in Applied Probability , Volume 51 , Issue 1 , March 2019 , pp. 136 - 167
Copyright
© Applied Probability Trust 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The supplementary material for this article can be found at http://doi.org/10.1017/apr.2019.6.

References

Acciaio, B. and Penner, I. (2011). Dynamic risk measures. In Advanced Mathematical Methods for Finance, Springer, Heidelberg, pp. 1–34.Google Scholar
Agueh, M. and Carlier, G. (2011). Barycenters in the Wasserstein space. SIAM J. Math. Anal. 43, 904–924.CrossRefGoogle Scholar
Bartl, D. (2016). Exponential utility maximization under model uncertainty for unbounded endowments. Preprint. Available at https://arxiv.org/abs/1610.00999.Google Scholar
Bartl, D. (2016). Pointwise dual representation of dynamic convex expectations. Preprint. Available at https://arxiv.org/abs/1612.09103v1.Google Scholar
Bartl, D., Drapeau, S. and Tangpi, L. (2017). Computational aspects of robust optimized certainty equivalents. Preprint. Available at https://arxiv.org/abs/1706.10186v1.Google Scholar
Bertsekas, D. P. and Shreve, S. E. (1996). Stochastic Optimal Control: The Discrete-Time Case. Athena Scientific.Google Scholar
Blanchet, A. and Carlier, G. (2016). Optimal transport and Cournot-Nash equilibria. Math. Operat. Res. 41, 125–145.CrossRefGoogle Scholar
Blanchet, J. and Murthy, K. R. A. (2016). Quantifying distributional model risk via optimal transport. Preprint. Available at https://arxiv.org/abs/1604.01446.Google Scholar
Breiman, L. (1992). Probability (Classics Appl. Math. 7). Society for Industrial and Applied Mathematics, Philadelphia, PA.Google Scholar
Cerreia-Vioglio, S., Maccheroni, F. and Marinacci, M. (2016). Ergodic theorems for lower probabilities. Proceedings of the American Mathematical Society 144, 3381–3396.CrossRefGoogle Scholar
Cheridito, P. (2013). Convex analysis. Princeton University lecture notes.Google Scholar
Cheridito, P. and Kupper, M. (2011). Composition of time-consistent dynamic monetary risk measures in discrete time. Internat. J. Theoret. Appl. Finance 14, 137–162.CrossRefGoogle Scholar
De Acosta, A. (1990). Large deviations for empirical measures of Markov chains. J. Theoret. Prob. 3, 395–431.CrossRefGoogle Scholar
De Cooman, G., Hermans, F. and Quaeghebeur, E. (2009). Imprecise Markov chains and their limit behavior. Prob. Eng. Inf. Sci. 23, 597–635.CrossRefGoogle Scholar
Dellacherie, C. and Meyer, P.-A. (1982). Probability and Potential B: Theory of Martingales. Elsevier, Amsterdam.Google Scholar
Dembo, A. and Zeitouni, O. (2010). Large Deviations Techniques and Applications (Stoch. Modelling Appl. Prob. 38). Springer, Berlin.CrossRefGoogle Scholar
Donsker, M. D. and Varadhan, S. R. S. (1975). Asymptotic evaluation of certain Markov process expectations for large time, I. Commun. Pure Appl. Math. 28, 1–47.CrossRefGoogle Scholar
Donsker, M. D. and Varadhan, S. R. S. (1976). Asymptotic evaluation of certain markov process expectations for large time. III. Commun. Pure Appl. Math. 29, 389–461.CrossRefGoogle Scholar
Dudley, R. M. (2002). Real Analysis and Probability (Cambridge Studies in Advanced Mathematics 74). Cambridge University Press.Google Scholar
Dupuis, P. and Ellis, R. S. (2011). A Weak Convergence Approach to the Theory of Large Deviations. John Wiley, Hoboken, NJ.Google Scholar
Eckstein, S. (2019). Extended Laplace principle for empirical measures of a Markov chain. Supplementary material. Available at http://doi.org/10.1017/apr.2019.6.CrossRefGoogle Scholar
Erreygers, A., Rottondi, C., Verticale, G. and De Bock, J. (2018). Imprecise Markov models for scalable and robust performance evaluation of flexi-grid spectrum allocation policies. Preprint. Available at https://arxiv.org/abs/1801.05700.Google Scholar
Esfahani, P. M. and Kuhn, D. (2015). Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Preprint. Available at https://arxiv.org/abs/1505.05116.Google Scholar
Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. John Wiley, New York.CrossRefGoogle Scholar
Gao, R. and Kleywegt, A. J. (2016). Distributionally robust stochastic optimization with Wasserstein distance. Preprint. Available at https://arxiv.org/abs/1604.02199.Google Scholar
Gibbs, A. L. and Su, F. E. (2002). On choosing and bounding probability metrics. Internat. Statist. Rev. 70, 419–435.CrossRefGoogle Scholar
Hanasusanto, G. A., Roitch, V., Kuhn, D. and Wiesemann, W. (2015). A distributionally robust perspective on uncertainty quantification and chance constrained programming. Math. Program. 151, 35–62.CrossRefGoogle Scholar
Hartfiel, D. and Seneta, E. (1994). On the theory of Markov set-chains. Adv. Appl. Prob. 26, 947–964.CrossRefGoogle Scholar
Hartfiel, D. J. (2006). Markov Set-Chains. Springer, Heidelberg.Google Scholar
Jain, N. C. (1990). Large deviation lower bounds for additive functionals of Markov processes. Ann. Prob. 18, 1071–1098.CrossRefGoogle Scholar
Kirkizlar, E., Andradóttir, S. and Ayhan, H. (2010). Robustness of efficient server assignment policies to service time distributions in finite-buffered lines. Naval Res. Logistics 57, 563–582.CrossRefGoogle Scholar
Kurano, M., Song, J., Hosaka, M. and Huang, Y. (1998). Controlled Markov set-chains with discounting. J. Appl. Prob. 35, 293–302.CrossRefGoogle Scholar
Lacker, D. (2015). Law invariant risk measures and information divergences. Preprint. Available at https://arxiv.org/abs/1510.07030.Google Scholar
Lacker, D. (2016). A non-exponential extension of sanov’s theorem via convex duality. Preprint. Available at https://arxiv.org/abs/1609.04744.Google Scholar
Lan, Y. and Zhang, N. (2017). Strong limit theorems for weighted sums of negatively associated random variables in nonlinear probability. Preprint. Available at https://arxiv.org/abs/1706.05788.Google Scholar
Ney, P. and Nummelin, E. (1987). Markov additive processes II. large deviations. Ann. Prob. 15, 593–609.CrossRefGoogle Scholar
Nilim, A. and El Ghaoui, L. (2005). Robust control of Markov decision processes with uncertain transition matrices. Operat. Res. 53, 780–798.CrossRefGoogle Scholar
Peng, S. (2009). Survey on normal distributions, central limit theorem, brownian motion and the related stochastic calculus under sublinear expectations. Sci. China Ser. A 52, 1391–1411.CrossRefGoogle Scholar
Peng, S. (2010). Nonlinear expectations and stochastic calculus under uncertainty. Preprint. Available at https://arxiv.org/abs/1002.4546.Google Scholar
Rottondi, C., Erreygers, A., Verticale, G. and De Bock, J. (2017). Modelling spectrum assignment in a two-service flexi-grid optical link with imprecise continuous-time Markov chains. In 13th Internat. Conf. on Design of Reliable Communication Networks (DRCN 2017), Munich, pp. 1–8.Google Scholar
Škulj, D. (2006). Finite discrete time Markov chains with interval probabilities. In Soft Methods for Integrated Uncertainty Modelling (Adv. Soft Comput. 37), Springer, Heidelberg, pp. 299–306.CrossRefGoogle Scholar
Škulj, D. (2009). Discrete time Markov chains with interval probabilities. Internat. J. Approx. Reason. 50, 1314–1329.CrossRefGoogle Scholar
Škulj, D. (2015). Efficient computation of the bounds of continuous time imprecise Markov chains. Appl. Math. Comput. 250, 165–180.Google Scholar
Varadarajan, V. S. (1958). Weak convergence of measures on separable metric spaces. Sankhyā 19, 15–22.Google Scholar
Villani, C. (2008). Optimal Transport: Old and New, Vol. 338. Springer, Heidelberg.Google Scholar
Wiesemann, W., Kuhn, D. and Rustem, B. (2013). Robust Markov decision processes. Math. Operat. Res. 38, 153–183.CrossRefGoogle Scholar
Yang, I. (2017). A convex optimization approach to distributionally robust Markov decision processes with Wasserstein distance. IEEE Control Systems Lett. 1, 164–169.CrossRefGoogle Scholar
Yu, P. and Xu, H. (2016). Distributionally robust counterpart in Markov decision processes. IEEE Trans. Automatic Control 61, 2538–2543.CrossRefGoogle Scholar
Supplementary material: PDF

Eckstein supplementary material

Supplementary material 1
Download Eckstein supplementary material(PDF)
PDF 221.4 KB