Hostname: page-component-7479d7b7d-wxhwt Total loading time: 0 Render date: 2024-07-09T18:14:46.678Z Has data issue: false hasContentIssue false
  • English
  • Français

Risk bounds for mixture density estimation

Published online by Cambridge University Press:  15 November 2005

Alexander Rakhlin ,
Dmitry Panchenko  and
Sayan Mukherjee
Alexander Rakhlin
Affiliation:
Center for Biological and Computational Learning, Massachusetts Institute of Technology, Cambridge, MA 02139, USA; [email protected]
Dmitry Panchenko
Affiliation:
Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02143, USA.
Sayan Mukherjee
Affiliation:
Institute of Statistics and Decision Sciences, Institute for Genome Sciences and Policy, Duke University, Durham, NC 27708, USA.
Get access

Abstract

In this paper we focus on the problem of estimating a bounded density using a finite combination of densities from a given class. We consider the Maximum Likelihood Estimator (MLE) and the greedy procedure described by Li and Barron (1999) under the additional assumption of boundedness of densities. We prove an $O(\frac{1}{\sqrt{n}})$ bound on the estimation error which does not depend on the number of densities in the estimated combination. Under the boundedness assumption, this improves the bound of Li and Barron by removing the $\log n$ factor and also generalizes it to the base classes with converging Dudley integral.

Keywords

Mixture density estimationmaximum likelihoodRademacher processes.
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 9 , February 2005 , pp. 220 - 229
Copyright
© EDP Sciences, SMAI, 2005

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Barron, A.R., Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inform. Theory 39 (1993) 930–945. CrossRef
Barron, A.R., Approximation and estimation bounds for artificial neural networks. Machine Learning 14 (1994) 115–133. CrossRef
Birgé, L. and Massart, P., Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 (1993) 113–150. CrossRef
R.M. Dudley, Uniform Central Limit Theorems. Cambridge University Press (1999).
Jones, L.K., A simple lemma on greedy approximation in Hilbert space and convergence rates for Projection Pursuit Regression and neural network training. Ann. Stat. 20 (1992) 608–613. CrossRef
M. Ledoux and M. Talagrand, Probability in Banach Spaces. Springer-Verlag, New York (1991).
J. Li and A. Barron, Mixture density estimation, in Advances in Neural information processings systems 12, S.A. Solla, T.K. Leen and K.-R. Muller Ed. San Mateo, CA. Morgan Kaufmann Publishers (1999).
J. Li, Estimation of Mixture Models. Ph.D. Thesis, The Department of Statistics. Yale University (1999).
C. McDiarmid, On the method of bounded differences. Surveys in Combinatorics (1989) 148–188.
Mendelson, S., On the size of convex hulls of small sets. J. Machine Learning Research 2 (2001) 1–18.
Niyogi, P. and Girosi, F., Generalization bounds for function approximation from scattered noisy data. Adv. Comput. Math. 10 (1999) 51–80. CrossRef
van de Geer, S.A., Rates of convergence for the maximum likelihood estimator in mixture models. Nonparametric Statistics 6 (1996) 293–310. CrossRef
S.A. van de Geer, Empirical Processes in M-Estimation. Cambridge University Press (2000).
A.W. van der Vaart and J.A. Wellner, Weak Convergence and Empirical Processes with Applications to Statistics. Springer-Verlag, New York (1996).
Wong, W.H. and Shen, X., Probability inequalities for likelihood ratios and convergence rates for sieve mles. Ann. Stat. 23 (1995) 339–362. CrossRef