Hostname: page-component-586b7cd67f-t8hqh Total loading time: 0 Render date: 2024-11-25T09:18:13.709Z Has data issue: false hasContentIssue false
  • English
  • Français

A spectral method for community detection in moderately sparse degree-corrected stochastic block models

Part of: Graph theory Probability theory on algebraic and topological structures Social and behavioral sciences: general topics Mathematical sociology Algorithms - Computer Science

Published online by Cambridge University Press:  08 September 2017

Lennart Gulikers ,
Marc Lelarge  and
Laurent Massoulié
Lennart Gulikers*
Affiliation:
Microsoft Research - INRIA Joint Centre and École Normale Supérieure
Marc Lelarge*
Affiliation:
INRIA and École Normale Supérieure
Laurent Massoulié*
Affiliation:
Microsoft Research - INRIA Joint Centre
*
* Postal address: Microsoft Research - Inria Joint Centre, Campus de l'École Polytechnique, Bâtiment Alan Turing, 1 rue Honoré d'Estienne d'Orves, 91120 Palaiseau, France.
*** Postal address: Inria Paris, 2 rue Simone Iff, CS 42112, 75589 Paris Cedex 12, France. Email address: [email protected]
* Postal address: Microsoft Research - Inria Joint Centre, Campus de l'École Polytechnique, Bâtiment Alan Turing, 1 rue Honoré d'Estienne d'Orves, 91120 Palaiseau, France.
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider community detection in degree-corrected stochastic block models. We propose a spectral clustering algorithm based on a suitably normalized adjacency matrix. We show that this algorithm consistently recovers the block membership of all but a vanishing fraction of nodes, in the regime where the lowest degree is of order log(n) or higher. Recovery succeeds even for very heterogeneous degree distributions. The algorithm does not rely on parameters as input. In particular, it does not need to know the number of communities.

Keywords

Social and information networkrandom graphdegree-corrected stochastic block modelspectral algorithmmachine learning05C80

MSC classification

Primary: 91D30: Social networks
Secondary: 05C80: Random graphs 60B20: Random matrices (probabilistic aspects; for algebraic aspects see ) 68W40: Analysis of algorithms 91C20: Clustering
Type
Research Article
Information
Advances in Applied Probability , Volume 49 , Issue 3 , September 2017 , pp. 686 - 721
Copyright
Copyright © Applied Probability Trust 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Abbe, E., Bandeira, A. S. and Hall, G. (2016). Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory 62, 471–487. CrossRefGoogle Scholar
[2] Adamic, L. A. and Glance, N. (2005). The political blogosphere and the 2004 U.S. election: divided they blog. In Proc. LinkKDD'05, ACM, New York, pp. 36–43. CrossRefGoogle Scholar
[3] Bernstein, S. (1946). The Theory of Probabilities. Gastehizdat, Moscow. Google Scholar
[4] Chaudhuri, K., Chung, F. and Tsiatas, A. (2012). Spectral clustering of graphs with general degrees in the extended planted partition model. In Proc. 25th Annual Conf. Learning Theory, pp. 35.1–35.23. Google Scholar
[5] Chung, F. and Radcliffe, M. (2011). On the spectra of general random graphs. Electron. J. Combin. 18, 215. CrossRefGoogle Scholar
[6] Chung, F., Lu, L. and Vu, V. (2004). The spectra of random graphs with given expected degrees. Internet Math. 1, 257–275. CrossRefGoogle Scholar
[7] Coja-Oghlan, A. and Lanka, A. (2010). Finding planted partitions in random graphs with general degree distributions. SIAM J. Discrete Math. 23, 1682–1714. CrossRefGoogle Scholar
[8] Dasgupta, A., Hopcroft, J. E. and McSherry, F. (2004). Spectral analysis of random graphs with skewed degree distributions. In Proc. 45th Ann. IEEE Symp. Foundations Comput. Sci., IEEE, New York, pp. 602–610. Google Scholar
[9] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2011). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84, 066106. CrossRefGoogle ScholarPubMed
[10] Feige, U. and Ofek, E. (2005). Spectral techniques applied to sparse random graphs. Random Structures Algorithms 27, 251–275. CrossRefGoogle Scholar
[11] Girvan, M. and Newman, M. E. J. (2002). Community structure in social and biological networks. Proc. Nat. Acad. Sci. USA 99, 7821–7826. CrossRefGoogle ScholarPubMed
[12] Hoff, P. D., Raftery, A. E. and Handcock, M. S. (2002). Latent space approaches to social network analysis. J. Amer. Statist. Assoc. 97, 1090–1098. CrossRefGoogle Scholar
[13] Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: first steps. Social Networks 5, 109–137. CrossRefGoogle Scholar
[14] Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press. CrossRefGoogle Scholar
[15] Jin, J. (2015). Fast community detection by SCORE. Ann. Statist. 43, 57–89. CrossRefGoogle Scholar
[16] Karrer, B. and Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Phys. Rev. E 83, 016107. CrossRefGoogle ScholarPubMed
[17] Krzakala, F. et al. (2013). Spectral redemption in clustering sparse networks. Proc. Nat. Acad. Sci. USA 110, 20935–20940. CrossRefGoogle ScholarPubMed
[18] Le, C. M. and Vershynin, R. (2015). Concentration and regularization of random graphs. Preprint. Available at https://arxiv.org/abs/1506.00669. Google Scholar
[19] Lei, J. and Rinaldo, A. (2015). Consistency of spectral clustering in stochastic block models. Ann. Statist. 43, 215–237. CrossRefGoogle Scholar
[20] Lusseau, D. et al. (2003). The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations. Behavioral Ecol. Sociobiol. 54, 396–405. CrossRefGoogle Scholar
[21] McSherry, F. (2001). Spectral partitioning of random graphs. In Proc. 42nd IEEE Symp. Foundations Comput. Sci., IEEE, Los Alamitos, CA, pp. 529–537. CrossRefGoogle Scholar
[22] Mihail, M. and Papadimitriou, C. (2002). On the eigenvalue power law. In Randomization and Approximation Techniques in Computer Science (Lecture Notes Comput. Sci. 2483), Springer, Berlin, pp. 254–262. CrossRefGoogle Scholar
[23] Mossel, E., Neeman, J. and Sly, A. (2015). Consistency thresholds for the planted bisection model. In STOC'15—Proc. 2015 ACM Symp. Theory Comput., ACM, New York, pp. 69–75. Google Scholar
[24] Newman, M. E. J. (2004). Detecting community structure in networks. Europ. Phys. J. B 38, 321–330. CrossRefGoogle Scholar
[25] Newman, M. E. J. (2010). Networks: An Introduction. Oxford University Press. CrossRefGoogle Scholar
[26] Newman, M. E. J. and Girvan, M. (2004). Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113. Google ScholarPubMed
[27] Qin, T. and Rohe, K. (2013). Regularized spectral clustering under the degree-corrected stochastic blockmodel. In Advances in Neural Information Processing Systems 26, Curran Associates, Red Hook, NY, pp. 3120–3128. Google Scholar
[28] Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39, 1878–1915. CrossRefGoogle Scholar
[29] Tomozei, D.-C. and Massoulié, L. (2014). Distributed user profiling via spectral methods. Stoch. Systems 4, 1–43. CrossRefGoogle Scholar
[30] Von Luxburg, U. (2007). A tutorial on spectral clustering. Statist. Comput. 17, 395–416. CrossRefGoogle Scholar
[31] Zachary, W. W. (1977). An information flow model for conflict and fission in small groups. J. Anthropological Res. 33, 452–473. CrossRefGoogle Scholar
[32] Zhang, X., Nadakuditi, R. and Newman, M. E. J. (2014). Spectra of random graphs with community structure and arbitrary degrees. Phys. Rev. E 89, 042816. CrossRefGoogle ScholarPubMed