Skip to main content
×
×
Home

A Spectral Approach to Analysing Belief Propagation for 3-Colouring

  • AMIN COJA-OGHLAN (a1), ELCHANAN MOSSEL (a2) and DAN VILENCHIK (a3)
Abstract

Belief propagation (BP) is a message-passing algorithm that computes the exact marginal distributions at every vertex of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither convergence, nor correctness in the case of convergence is guaranteed. Nonetheless, BP has gained popularity as it seems to remain effective in many cases of interest, even when the underlying graph is ‘far’ from being a tree. However, the theoretical understanding of BP (and its new relative survey propagation) when applied to CSPs is poor.

Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ‘planted’ solution; thus, we obtain the first rigorous result on BP for graph colouring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how belief propagation breaks the symmetry between the 3! possible permutations of the colour classes.

Copyright
References
Hide All
[1]Achlioptas, D. and Friedgut, E. (1999) A sharp threshold for k-colorability. Random Struct. Alg. 14 6370.
[2]Alon, N. and Kahale, N. (1997) A spectral technique for coloring random 3-colorable graphs. SIAM J. Comput. 26 17331748.
[3]Bilu, Y. and Linial, N. (2006) Lifts, discrepancy and nearly optimal spectral gap. Combinatorica 26 495519.
[4]Braunstein, A., Mézard, M., Weigt, M. and Zecchina, R. (2005) Constraint satisfaction by survey propagation. In Computational Complexity and Statistical Physics (Percus, A., Istrate, G. and Moore, C., eds), Oxford University Press.
[5]Braunstein, A., Mézard, M. and Zecchina, R. (2005) Survey propagation: An algorithm for satisfiability. Random Struct. Alg. 27 201226.
[6]Braunstein, A., Mulet, R., Pagnani, A., Weigt, M. and Zecchina, R. (2003) Polynomial iterative algorithms for coloring and analyzing random graphs. Phys. Rev. E 68 036702.
[7]Brightwell, G. R. and Winkler, P. (2002) Random colorings of a Cayley tree. In Contemporary Combinatorics, Vol. 10 of Bolyai Society Mathematical Studies, János Bolyai Math. Soc., pp. 247–276.
[8]Chung, F. and Graham, R. (2002) Sparse quasi-random graphs. Combinatorica 22 217244.
[9]Feige, U., Mossel, E. and Vilenchik, D. (2006) Complete convergence of message passing algorithms for some satisfiability problems. In Random 2006, Vol. 4110 of Lecture Notes in Computer Science, pp. 339–350.
[10]Janson, S., Łuczak, T. and Ruciński, A. (2000) Random Graphs, Wiley.
[11]Jonasson, J. (2002) Uniqueness of uniform random colorings of regular trees. Statist. Probab. Lett. 57 243248.
[12]Krzakala, F., Montanari, A., Ricci-Tersenghi, F., Semerjianc, G. and Zdeborova, L. (2007) Gibbs states and the set of solutions of random constraint satisfaction problems. Proc. Nat. Acad. Sci. 104 1031810323.
[13]Luby, M., Mitzenmacher, M., Shokrollahi, M. A. and Spielman, D. (1998) Analysis of low density parity check codes and improved designs using irregular graphs. In Proc. 30th ACM Symposium on the Theory of Computing, pp. 249–258.
[14]Luby, M., Mitzenmacher, M., Shokrollahi, M. A. and Spielman, D. (2001) Efficient erasure correcting codes. IEEE Trans. Inform. Theory 47 569584.
[15]Maneva, E., Mossel, E. and Wainwright, M. (2005) A new look at survey propagation and its generalizations. In Proc. 16th ACM–SIAM Symposium on Discrete Algorithms, pp. 1089–1098.
[16]Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Francisco, CA.
[17]Richardson, T., Shokrollahi, A. and Urbanke, R. (2001) Design of capacity-approaching irregular low-density parity check codes. IEEE Trans. Inform. Theory 47 619637.
[18]Tatikonda, S. and Jordan, M. I. (2002) Loopy belief propagation and Gibbs measures. In Uncertainty in Artificial Intelligence (UAI): Proc. 18th Conference.
[19]Weitz, D. (2006) Counting independent sets up to the tree threshold. In Proc. 38th Annual ACM Symposium on the Theory of Computing, pp. 140–149.
[20]Yamamoto, M. and Watanabe, O. (2007) Belief propagation and spectral methods. Report C–248, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Combinatorics, Probability and Computing
  • ISSN: 0963-5483
  • EISSN: 1469-2163
  • URL: /core/journals/combinatorics-probability-and-computing
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed