Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-05T15:35:06.257Z Has data issue: false hasContentIssue false
  • English
  • Français

ENTROPY FLOW AND DE BRUIJN'S IDENTITY FOR A CLASS OF STOCHASTIC DIFFERENTIAL EQUATIONS DRIVEN BY FRACTIONAL BROWNIAN MOTION

Published online by Cambridge University Press:  17 December 2019

Michael C.H. Choi Open the ORCID record for Michael C.H. Choi [Opens in a new window] ,
Chihoon Lee  and
Jian Song Open the ORCID record for Jian Song [Opens in a new window]
Michael C.H. Choi
Affiliation:
Institute for Data and Decision Analytics, The Chinese University of Hong Kong, Shenzhen, Guangdong, 518172, P.R. China and Shenzhen Institute of Artificial Intelligence and Robotics for Society E-mail: [email protected]
Chihoon Lee
Affiliation:
School of Business, Stevens Institute of Technology, Hoboken, NJ 07030, USA and Institute for Data and Decision Analytics, The Chinese University of Hong Kong, Shenzhen, Guangdong 518172, P.R. China E-mail: [email protected]
Jian Song
Affiliation:
Research Center for Mathematics and Interdisciplinary Sciences, Shandong University, Qingdao, Shandong, 266237, China and School of Mathematics, Shandong University, Jinan, Shandong, 250100, China E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Motivated by the classical De Bruijn's identity for the additive Gaussian noise channel, in this paper we consider a generalized setting where the channel is modelled via stochastic differential equations driven by fractional Brownian motion with Hurst parameter H ∈ (0, 1). We derive generalized De Bruijn's identity for Shannon entropy and Kullback–Leibler divergence by means of Itô's formula, and present two applications. In the first application we demonstrate its equivalence with Stein's identity for Gaussian distributions, while in the second application, we show that for H ∈ (0, 1/2], the entropy power is concave in time while for H ∈ (1/2, 1) it is convex in time when the initial distribution is Gaussian. Compared with the classical case of H = 1/2, the time parameter plays an interesting and significant role in the analysis of these quantities.

Keywords

de bruijn's identityentropy powerfokker–planck equationfractional brownian motion
Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 35 , Issue 3 , July 2021 , pp. 369 - 380
Copyright
© Cambridge University Press 2019

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Alòs, E., Mazet, O., & Nualart, D. (2001). Stochastic calculus with respect to Gaussian processes. Annals of Probability 29(2): 766–801.CrossRefGoogle Scholar
2.Baudoin, F. & Coutin, L. (2007). Operators associated with a stochastic differential equation driven by fractional Brownian motions. Stochastic Processes and Their Applications 117(5): 550–574.CrossRefGoogle Scholar
3.Beran, J., Sherman, R., Taqqu, M.S., & Willinger, W. (1995). Long-range dependence in variable-bit-rate video traffic. IEEE Transactions on Communications 43(2/3/4): 1566–1579.CrossRefGoogle Scholar
4.Brown, L., DasGupta, A., Haff, L.R., & Strawderman, W.E. (2006). The heat equation and Stein's identity: connections, applications. Journal of Statistical Planning and Inference 136(7): 2254–2278.CrossRefGoogle Scholar
5.Cheridito, P. & Nualart, D. (2005). Stochastic integral of divergence type with respect to fractional Brownian motion with Hurst parameter $H\in (0,\frac {1}{2})$. Annales de l'Institut Henri Poincaré B: Probability and Statistics 41(6): 1049–1081, ISSN 0246–0203.CrossRefGoogle Scholar
6.Costa, M.H.M. (1985). A new entropy power inequality. IEEE Transactions on Information Theory 31(6): 751–760.CrossRefGoogle Scholar
7.Costa, M.H.M. (1985). On the Gaussian interference channel. IEEE Transactions on Information Theory 31(5): 607–615.CrossRefGoogle Scholar
8.Coutin, L. & Qian, Z. (2002). Stochastic analysis, rough path analysis and fractional Brownian motions. Probability Theory and Related Fields 122(1): 108–140.CrossRefGoogle Scholar
9.Cover, T.M. & Thomas, J.A. (2006). Elements of information theory (2nd ed.). Hoboken, NJ: Wiley-Interscience [John Wiley & Sons].Google Scholar
10.Dembo, A. (1989). Simple proof of the concavity of the entropy power with respect to added Gaussian noise. IEEE Transactions on Information Theory 35(4): 887–888.CrossRefGoogle Scholar
11.Gradinaru, M., Nourdin, I., Russo, F., & Vallois, P. (2005). m-order integrals and generalized Itô's formula: the case of a fractional Brownian motion with any Hurst index. Annales de l'Institut Henri Poincaré B: Probability and Statistics 41(4): 781–806, ISSN 0246–0203.CrossRefGoogle Scholar
12.Guo, D., Shamai, S., & Verdú, S. (2005). Mutual information and minimum mean-square error in Gaussian channels. IEEE Transactions on Information Theory 51(4): 1261–1282.CrossRefGoogle Scholar
13.Khoolenjani, N.B. & Alamatsaz, M.H. (2016). A De Bruijn's identity for dependent random variables based on copula theory. Probability in the Engineering and Informational Sciences 30(1): 125–140.CrossRefGoogle Scholar
14.Mandelbrot, B.B. & Van Ness, J.W. (1968). Fractional Brownian motions, fractional noises and applications. SIAM Review 10: 422–437.CrossRefGoogle Scholar
15.Nourdin, I. (2008). A simple theory for the study of SDEs driven by a fractional Brownian motion, in dimension one. In Séminaire de probabilités XLI, vol. 1934, Lecture Notes in Math, Springer, Berlin, pp. 181–197.CrossRefGoogle Scholar
16.Palomar, D.P. & Verdú, S. (2006). Gradient of mutual information in linear vector Gaussian channels. IEEE Transactions on Information Theory 52(1): 141–154.CrossRefGoogle Scholar
17.Park, S., Serpedin, E., & Qaraqe, K. (2012). On the equivalence between Stein and De Bruijn identities. IEEE Transactions on Information Theory 58(12): 7045–7067.CrossRefGoogle Scholar
18.Russo, F. & Vallois, P. (1993). Forward, backward and symmetric stochastic integration. Probability Theory and Related Fields 97(3): 403–421, ISSN 0178–8051.CrossRefGoogle Scholar
19.Stam, A.J. (1959). Some inequalities satisfied by the quantities of information of Fisher and Shannon. Information and Control 2: 101–112.CrossRefGoogle Scholar
20.Sussmann, H.J. (1978). On the gap between deterministic and stochastic ordinary differential equations. Annals of Probability 6(1): 19–41.CrossRefGoogle Scholar
21.Valdivia, A. (2017). Information loss on Gaussian Volterra process. Electronic Communications in Probability 22: 5, Paper No. 60.CrossRefGoogle Scholar
22.Wibisono, A., Jog, V., & Loh, P. (2017). Information and estimation in Fokker-Planck channels. In 2017 IEEE International Symposium on Information Theory, ISIT 2017, IEEE, Aachen, Germany, June 25–30, 2017, pp. 2673–2677.CrossRefGoogle Scholar
23.Willinger, W., Taqqu, M.S., Leland, W.E., & Wilson, D.V. (1995). Self-similarity in high-speed packet traffic: analysis and modeling of Ethernet traffic measurements. Statistical Science 10(1): 67–85.CrossRefGoogle Scholar
24.Young, L.C. (1936). An inequality of the Hölder type, connected with Stieltjes integration. Acta Mathematica 67(1): 251–282, ISSN 0001–5962.CrossRefGoogle Scholar