Skip to main content Accessibility help
×
Home
Hostname: page-component-78dcdb465f-8p2q5 Total loading time: 0.395 Render date: 2021-04-18T00:55:13.928Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false, "newCitedByModal": true }

A spectral method for community detection in moderately sparse degree-corrected stochastic block models

Published online by Cambridge University Press:  08 September 2017

Lennart Gulikers
Affiliation:
Microsoft Research - INRIA Joint Centre and École Normale Supérieure
Marc Lelarge
Affiliation:
INRIA and École Normale Supérieure
Laurent Massoulié
Affiliation:
Microsoft Research - INRIA Joint Centre

Abstract

We consider community detection in degree-corrected stochastic block models. We propose a spectral clustering algorithm based on a suitably normalized adjacency matrix. We show that this algorithm consistently recovers the block membership of all but a vanishing fraction of nodes, in the regime where the lowest degree is of order log(n) or higher. Recovery succeeds even for very heterogeneous degree distributions. The algorithm does not rely on parameters as input. In particular, it does not need to know the number of communities.

Type
Research Article
Copyright
Copyright © Applied Probability Trust 2017 

Access options

Get access to the full version of this content by using one of the access options below.

References

[1] Abbe, E., Bandeira, A. S. and Hall, G. (2016). Exact recovery in the stochastic block model. IEEE Trans. Inf. Theory 62, 471487. CrossRefGoogle Scholar
[2] Adamic, L. A. and Glance, N. (2005). The political blogosphere and the 2004 U.S. election: divided they blog. In Proc. LinkKDD'05, ACM, New York, pp. 3643. CrossRefGoogle Scholar
[3] Bernstein, S. (1946). The Theory of Probabilities. Gastehizdat, Moscow. Google Scholar
[4] Chaudhuri, K., Chung, F. and Tsiatas, A. (2012). Spectral clustering of graphs with general degrees in the extended planted partition model. In Proc. 25th Annual Conf. Learning Theory, pp. 35.135.23. Google Scholar
[5] Chung, F. and Radcliffe, M. (2011). On the spectra of general random graphs. Electron. J. Combin. 18, 215. Google Scholar
[6] Chung, F., Lu, L. and Vu, V. (2004). The spectra of random graphs with given expected degrees. Internet Math. 1, 257275. CrossRefGoogle Scholar
[7] Coja-Oghlan, A. and Lanka, A. (2010). Finding planted partitions in random graphs with general degree distributions. SIAM J. Discrete Math. 23, 16821714. CrossRefGoogle Scholar
[8] Dasgupta, A., Hopcroft, J. E. and McSherry, F. (2004). Spectral analysis of random graphs with skewed degree distributions. In Proc. 45th Ann. IEEE Symp. Foundations Comput. Sci., IEEE, New York, pp. 602610. CrossRefGoogle Scholar
[9] Decelle, A., Krzakala, F., Moore, C. and Zdeborová, L. (2011). Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications. Phys. Rev. E 84, 066106. CrossRefGoogle ScholarPubMed
[10] Feige, U. and Ofek, E. (2005). Spectral techniques applied to sparse random graphs. Random Structures Algorithms 27, 251275. CrossRefGoogle Scholar
[11] Girvan, M. and Newman, M. E. J. (2002). Community structure in social and biological networks. Proc. Nat. Acad. Sci. USA 99, 78217826. CrossRefGoogle ScholarPubMed
[12] Hoff, P. D., Raftery, A. E. and Handcock, M. S. (2002). Latent space approaches to social network analysis. J. Amer. Statist. Assoc. 97, 10901098. CrossRefGoogle Scholar
[13] Holland, P. W., Laskey, K. B. and Leinhardt, S. (1983). Stochastic blockmodels: first steps. Social Networks 5, 109137. CrossRefGoogle Scholar
[14] Horn, R. A. and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press. CrossRefGoogle Scholar
[15] Jin, J. (2015). Fast community detection by SCORE. Ann. Statist. 43, 5789. CrossRefGoogle Scholar
[16] Karrer, B. and Newman, M. E. J. (2011). Stochastic blockmodels and community structure in networks. Phys. Rev. E 83, 016107. CrossRefGoogle ScholarPubMed
[17] Krzakala, F. et al. (2013). Spectral redemption in clustering sparse networks. Proc. Nat. Acad. Sci. USA 110, 2093520940. CrossRefGoogle ScholarPubMed
[18] Le, C. M. and Vershynin, R. (2015). Concentration and regularization of random graphs. Preprint. Available at https://arxiv.org/abs/1506.00669. Google Scholar
[19] Lei, J. and Rinaldo, A. (2015). Consistency of spectral clustering in stochastic block models. Ann. Statist. 43, 215237. CrossRefGoogle Scholar
[20] Lusseau, D. et al. (2003). The bottlenose dolphin community of Doubtful Sound features a large proportion of long-lasting associations. Behavioral Ecol. Sociobiol. 54, 396405. CrossRefGoogle Scholar
[21] McSherry, F. (2001). Spectral partitioning of random graphs. In Proc. 42nd IEEE Symp. Foundations Comput. Sci., IEEE, Los Alamitos, CA, pp. 529537. Google Scholar
[22] Mihail, M. and Papadimitriou, C. (2002). On the eigenvalue power law. In Randomization and Approximation Techniques in Computer Science (Lecture Notes Comput. Sci. 2483), Springer, Berlin, pp. 254262. CrossRefGoogle Scholar
[23] Mossel, E., Neeman, J. and Sly, A. (2015). Consistency thresholds for the planted bisection model. In STOC'15—Proc. 2015 ACM Symp. Theory Comput., ACM, New York, pp. 6975. Google Scholar
[24] Newman, M. E. J. (2004). Detecting community structure in networks. Europ. Phys. J. B 38, 321330. CrossRefGoogle Scholar
[25] Newman, M. E. J. (2010). Networks: An Introduction. Oxford University Press. CrossRefGoogle Scholar
[26] Newman, M. E. J. and Girvan, M. (2004). Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113. Google ScholarPubMed
[27] Qin, T. and Rohe, K. (2013). Regularized spectral clustering under the degree-corrected stochastic blockmodel. In Advances in Neural Information Processing Systems 26, Curran Associates, Red Hook, NY, pp. 31203128. Google Scholar
[28] Rohe, K., Chatterjee, S. and Yu, B. (2011). Spectral clustering and the high-dimensional stochastic blockmodel. Ann. Statist. 39, 18781915. CrossRefGoogle Scholar
[29] Tomozei, D.-C. and Massoulié, L. (2014). Distributed user profiling via spectral methods. Stoch. Systems 4, 143. CrossRefGoogle Scholar
[30] Von Luxburg, U. (2007). A tutorial on spectral clustering. Statist. Comput. 17, 395416. CrossRefGoogle Scholar
[31] Zachary, W. W. (1977). An information flow model for conflict and fission in small groups. J. Anthropological Res. 33, 452473. CrossRefGoogle Scholar
[32] Zhang, X., Nadakuditi, R. and Newman, M. E. J. (2014). Spectra of random graphs with community structure and arbitrary degrees. Phys. Rev. E 89, 042816. CrossRefGoogle ScholarPubMed

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 2
Total number of PDF views: 154 *
View data table for this chart

* Views captured on Cambridge Core between 08th September 2017 - 18th April 2021. This data will be updated every 24 hours.

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

A spectral method for community detection in moderately sparse degree-corrected stochastic block models
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

A spectral method for community detection in moderately sparse degree-corrected stochastic block models
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

A spectral method for community detection in moderately sparse degree-corrected stochastic block models
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *