Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-09T00:07:30.606Z Has data issue: false hasContentIssue false
  • English
  • Français

Partition-based conditional density estimation

Published online by Cambridge University Press:  04 November 2013

S. X. Cohen  and
E. Le Pennec
S. X. Cohen
Affiliation:
IPANEMA USR 3461 CNRS/MCC, BP 48 Saint Aubin, F-91192 Gif-sur-Yvette, France
E. Le Pennec
Affiliation:
SELECT/Inria Saclay IdF, Laboratoire de Mathématiques Faculté des Sciences d’Orsay, Université Paris-Sud 11, F-91405 Orsay Cedex, France. [email protected]
Get access

Abstract

We propose a general partition-based strategy to estimate conditional density with candidate densities that are piecewise constant with respect to the covariate. Capitalizing on a general penalized maximum likelihood model selection result, we prove, on two specific examples, that the penalty of each model can be chosen roughly proportional to its dimension. We first study a classical strategy in which the densities are chosen piecewise conditional according to the variable. We then consider Gaussian mixture models with mixing proportion that vary according to the covariate but with common mixture components. This model proves to be interesting for an unsupervised segmentation application that was our original motivation for this work.

Keywords

Conditional density estimationpartitionpenalized likelihoodpiecewise polynomial densityGaussian mixture model
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 17 , 2013 , pp. 672 - 697
Copyright
© EDP Sciences, SMAI, 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akakpo, N., Adaptation to anisotropy and inhomogeneity via dyadic piecewise polynomial selection. Math. Meth. Stat. 21 (2012) 1–28. Google Scholar
Akakpo, N. and Lacour, C., Inhomogeneous and anisotropic conditional density estimation from dependent data. Electon. J. Statist. 5 (2011) 1618–1653. Google Scholar
Antoniadis, A., Bigot, J. and von Sachs, R., A multiscale approach for statistical characterization of functional images. J. Comput. Graph. Stat. 18 (2008) 216–237. Google Scholar
A. Barron, C. Huang, J. Li and X. Luo, MDL Principle, Penalized Likelihood, and Statistical Risk, in Festschrift in Honor of Jorma Rissanen on the Occasion of his 75th Birthday. Tampere University Press (2008).
Bashtannyk, D. and Hyndman, R., Bandwidth selection for kernel conditional density estimation. Comput. Stat. Data Anal. 36 (2001) 279–298. Google Scholar
Bertrand, L., Languille, M.-A., Cohen, S.X., Robinet, L., Gervais, C., Leroy, S., Bernard, D., Le Pennec, E., Josse, W., Doucet, J. and Schöder, S., European research platform IPANEMA at the SOLEIL synchrotron for ancient and historical materials. J. Synchrotron Radiat. 18 (2011) 765–772. Google ScholarPubMed
Biernacki, Ch., Celeux, G., Govaert, G. and Langrognet, F., Model-based cluster and discriminant analysis with the MIXMOD software. Comput. Stat. Data Anal. 51 (2006) 587–600. Google Scholar
Birgé, L. and Massart, P., Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4 (1998) 329–375. Google Scholar
Birgé, L. and Massart, P., Minimal penalties for gaussian model selection. Probab. Theory Related Fields 138 (2007) 33–73. Google Scholar
Blanchard, G., Schäfer, C., Rozenholc, Y. and Müller, K.R., Optimal dyadic decision trees. Mach. Learn. 66 (2007) 209–241. Google Scholar
Brunel, E., Comte, F. and Lacour, C., Adaptive estimation of the conditional density in presence of censoring. Sankhy 69 (2007) 734–763. Google Scholar
S.X. Cohen and E. Le Pennec, Conditional density estimation by penalized likelihood model selection and applications. Technical report, RR-7596. INRIA (2011). arXiv:1103.2021.
S.X. Cohen and E. Le Pennec, Conditional density estimation by penalized likelihood model selection. Submitted (2012).
S.X. Cohen and E. Le Pennec, Unsupervised segmentation of hyperspectral images with spatialized Gaussian mixture model and model selection. Submitted (2012).
de Gooijer, J. and Zerom, D., On conditional density estimation. Stat. Neerlandica 57 (2003) 159–176. Google Scholar
Donoho, D., CART and best-ortho-basis: a connection. Ann. Stat. 25 (1997) 1870–1911. Google Scholar
Efromovich, S., Conditional density estimation in a regression setting. Ann. Stat. 35 (2007) 2504–2535. Google Scholar
Efromovich, S., Oracle inequality for conditional density estimation and an actuarial example. Ann. Inst. Stat. Math. 62 (2010) 249–275. Google Scholar
Fan, J., Yao, Q. and Tong, H., Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83 (1996) 189–206. Google Scholar
Genovese, Ch. and Wasserman, L., Rates of convergence for the Gaussian mixture sieve. Ann. Stat. 28 (2000) 1105–1127. Google Scholar
Györfi, L. and Kohler, M., Nonparametric estimation of conditional distributions. IEEE Trans. Inform. Theory 53 (2007) 1872–1879. Google Scholar
Hall, P., Wolff, R. and Yao, Q., Methods for estimating a conditional distribution function. J. Amer. Stat. Assoc. 94 (1999) 154–163. Google Scholar
T. Hofmann, Probabilistic latent semantic analysis, in Proc. of Uncertainty in Artificial Intelligence (1999).
Huang, Y., Pollak, I., Do, M. and Bouman, C., Fast search for best representations in multitree dictionaries. IEEE Trans. Image Process. 15 (2006) 1779–1793. Google ScholarPubMed
Hyndman, R. and Yao, Q., Nonparametric estimation and symmetry tests for conditional density functions. J. Nonparam. Stat. 14 (2002) 259–278. Google Scholar
Hyndman, R., Bashtannyk, D. and Grunwald, G., Estimating and visualizing conditional densities. J. Comput. Graphical Stat. 5 (1996) 315–336. Google Scholar
Karaivanov, B. and Petrushev, P., Nonlinear piecewise polynomial approximation beyond besov spaces. Appl. Comput. Harmonic Anal. 15 (2003) 177–223. Google Scholar
Kolaczyk, E. and Nowak, R., Multiscale generalised linear models for nonparametric function estimation. Biometrika 92 (2005) 119–133. Google Scholar
Kolaczyk, E., Ju, J. and Gopal, S., Multiscale, multigranular statistical image segmentation. J. Amer. Stat. Assoc. 100 (2005) 1358–1369. Google Scholar
Q. Li and J. Racine, Nonparametric Econometrics: Theory and Practice. Princeton University Press (2007).
Lin, J., Divergence measures based on the Shannon entropy. IEEE Trans. Inform. Theory 37 (1991) 145–151. Google Scholar
P. Massart, Concentration inequalities and model selection, vol. 1896 of Lecture Notes in Mathematics (2007). Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour (2003), With a foreword by Jean Picard.
Maugis, C. and Michel, B., A non asymptotic penalized criterion for Gaussian mixture model selection. ESAIM: PS 15 (2012) 41–68. Google Scholar
Maugis, C. and Michel, B., Data-driven penalty calibration: a case study for Gaussian mixture model selection. ESAIM: PS 15 (2012) 320–339. Google Scholar
M. Rosenblatt, Conditional probability density and regression estimators, in Multivariate Analysis II, Proc. of Second Internat. Sympos., Dayton, Ohio, 1968. Academic Press (1969) 25–31.
L. Si and R. Jin, Adjusting mixture weights of gaussian mixture model via regularized probabilistic latent semantic analysis, in Advances in Knowledge Discovery and Data Mining (2005) 218–252.
Stone, Ch., The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Stat. 22 (1994) 118–171. Google Scholar
S. Szarek, Metric entropy of homogeneous spaces, in Quantum Probability (Gdansk 1997) (1998) 395–410.
van de Geer, S., The method of sieves and minimum contrast estimators. Math. Methods Stat. 4 (1995) 20–38. Google Scholar
A. van der Vaart and J. Wellner, Weak Convergence. Springer (1996).
van Keilegom, I. and Veraverbeke, N., Density and hazard estimation in censored regression models. Bernoulli 8 (2002) 607–625. Google Scholar
Willett, R. and Nowak, R., Multiscale poisson intensity and density estimation. IEEE Trans. Inform. Theory 53 (2007) 3171–3187. Google Scholar
Young, D. and Hunter, D., Mixtures of regressions with predictor-dependent mixing proportions. Comput. Stat. Data Anal. 54 (2010) 2253–2266. Google Scholar