Skip to main content
  • Print publication year: 2017
  • Online publication date: July 2017

9 - Some applications of relative entropy in additive combinatorics



This survey looks at some recent applications of relative entropy in additive combinatorics. Specifically, we examine to what extent entropy-increment arguments can replace or even outperform more traditional energy-increment strategies or alternative approximation arguments based on the Hahn-Banach theorem.


Entropy has a long history as a tool in combinatorics. Starting with a well-known estimate for the sum of the first few binomial coefficients, some of the classical applications include Spencer's theorem that six standard deviations suffice, which states that given n finite sets, there exists a two-colouring of the elements such that all sets have discrepancy at most; a proof of the Loomis-Whitney inequality, which gives an upper bound on the volume of an n-dimensional body in Euclidean space in terms of its (n - 1)-dimensional projections; or Radhakrishnans proof [33] of Bregman's theorem on the maximum permanent of a 0/1 matrix with given row sums. For a beautiful introduction to these fascinating applications, as well as an extensive annotated bibliography, see [10].

There are other more recent results in additive combinatorics in particular where the concept of entropy has played a crucial role. Notable examples include Fox's improvement [7] of the bounds in the graph removal lemma (see also [30], which appeared in the proof-reading stages of this article); Szegedy's information-theoretic approach [41] to Sidorenko's conjecture (see also the blog post [14] by Gowers); Tao's solution [45] to the Erds discrepancy problem (see the discussion [44] on Tao's blog).

Since the above developments appear to be well captured by discussions online, we shall not cover them in any detail here. Instead we shall focus on a particular strand of recent results in additive combinatorics that could all be described as “approximation theorems” of a certain kind.

The text naturally splits into five parts. To start with we give a very brief introduction to the concept of entropy and its variants, in particular relative entropy (also known as Kullback-Leibler divergence). In Section 3, we state and prove a rather general sparse approximation theorem due to Lee [28].

Recommend this book

Email your librarian or administrator to recommend adding this book to your organisation's collection.

Surveys in Combinatorics 2017
  • Online ISBN: 9781108332699
  • Book DOI:
Please enter your name
Please enter a valid email address
Who would you like to send this to *
[1] Boaz Barak, Moritz Hardt, and Satyen Kale. The uniform hardcore lemma via approximate Bregman projections. In Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1193–1200. SIAM, Philadelphia, PA, 2009.
[2] Thomas Bloom. A quantitative improvement for Roth's theorem on arithmetic progressions. J. Lond. Math. Soc. (2), 93(3):643–663, 2016.
[3] Mei-Chu Chang. A polynomial bound in Freiman's theorem. Duke Math. J., 113(3):399–419, 2002.
[4] David Conlon, Jacob Fox, and Yufei Zhao. The Green-Tao theorem: an exposition. EMS Surv. Math. Sci., 1(2):249–282, 2014.
[5] David Conlon, Jacob Fox, and Yufei Zhao. A relative Szemerédi theorem. Geom. Funct. Anal., 25(3):733–762, 2015.
[6] Thomas Cover and Joy Thomas. Elements of information theory. Wiley Series in Telecommunications. John Wiley & Sons, Inc., New York, New York, USA, 1991.
[7] Jacob Fox. A new proof of the graph removal lemma. Ann. of Math. (2), 174(1):561–579, 2011.
[8] Ehud Friedgut. An information-theoretic proof of a hypercontractive inequality. arXiv, 1504.01506, April 2015.
[9] Alan Frieze and Ravi Kannan. Quick approximation to matrices and applications. Combinatorica, 19(2):175–220, 1999.
[10] David Galvin. Three tutorial lectures on entropy and counting. arXiv, 1406.7872, June 2014.
[11] Timothy Gowers. A new proof of Szemerédi's theorem for arithmetic progressions of length four. Geom. Funct. Anal., 8(3):529–551, 1998.
[12] Timothy Gowers. A new proof of Szemerédi's theorem. Geom. Funct. Anal., 11(3):465–588, 2001.
[13] Timothy Gowers. Decompositions, approximate structure, transference, and the Hahn-Banach theorem. Bull. Lond. Math. Soc., 42(4):573–606, 2010.
[14] Timothy Gowers. Entropy and Sidorenko's conjecture — after Szegedy. Personal blog,, November 2015.
[15] Timothy Gowers and Julia Wolf. Linear forms and quadratic uniformity for functions on Fnp. Mathematika, 57(2):215–237, 2012.
[16] Ben Green. Some constructions in the inverse spectral theory of cyclic groups. Combin. Probab. Comput., 12(2):127–138, 2003.
[17] Ben Green. Spectral structure of sets of integers. In Fourier analysis and convexity, pages 83–96. Birkhäuser Boston, Boston, MA, 2004.
[18] Ben Green. Finite field models in additive combinatorics. In Bridget S Webb, editor, Surveys in combinatorics 2005, pages 1–27. Cambridge Univ. Press, Cambridge, Cambridge, 2005.
[19] Ben Green. Roth's theorem in the primes. Ann. of Math. (2), 161(3):1609–1636, 2005.
[20] Ben Green. Montréal notes on quadratic Fourier analysis. In Additive combinatorics, pages 69–102. Amer. Math. Soc., Providence, RI, 2007.
[21] Ben Green and Tom Sanders. Boolean functions with small spectral norm. Geom. Funct. Anal., 18(1):144–162, 2008.
[22] Ben Green and Terence Tao. An inverse theorem for the Gowers U3(G) norm. Proc. Edinb. Math. Soc. (2), 51(1):73–153, 2008.
[23] Ben Green and Terence Tao. The primes contain arbitrarily long arithmetic progressions. Ann. of Math. (2), 167(2):481–547, 2008.
[24] Ben Green and Terence Tao. Linear equations in primes. Ann. of Math. (2), 171(3):1753–1850, 2010.
[25] Russell Impagliazzo. Hard-core distributions for somewhat hard problems. In FOCS ‘07. 48th Annual IEEE Symposium on Foundations of Computer Science, 2007., pages 538–545. IEEE, 1995.
[26] Russell Impagliazzo, Cristopher Moore, and Alexander Russell. An entropic proof of Chang's inequality. SIAM J. Discrete Math., 28(1):173–176, 2014.
[27] Solomon Kullback and Richard Leibler. On information and sufficiency. Ann. Math. Stat., 22(1):79–86, 1951.
[28] James Lee. Covering the large spectrum and generalized Riesz products. arXiv, 1508.07109, August 2015.
[29] Chi-Jen Lu, Shi-Chun Tsai, and Hsin-Lung Wu. Complexity of hardcore set proofs. Comput. Complexity, 20(1):145–171, 2011.
[30] Guy Moshkovitz and Asaf Shapira. A sparse regular approximation lemma. arXiv, 1610.02676, October 2016.
[31] Ryan O'Donnell. Lecture 16: The hypercontractivity theorem. Personal website,õdonnell/booleananalysis/lecture16.pdf.
[32] Sean Prendiville. Four variants of the Fourier-analytic transference principle. arXiv, 1509.09200, September 2015.
[33] Jaikumar Radhakrishnan. An entropy proof of Bregman's theorem. J. Combin. Theory Ser. A, 77(1):161–164, 1997.
[34] Omer Reingold, Luca Trevisan, Madhur Tulsiani, and Salil Vadhan. Dense subsets of pseudorandom sets. Electronic Colloquium on Computational Complexity, 45:1–33, 2008.
[35] Omer Reingold, Luca Trevisan, Madhur Tulsiani, and Salil Vadhan. New proofs of the Green-Tao-Ziegler dense model theorem: an exposition. arXiv, 0806.0381, June 2008.
[36] Sheldon Ross. A first course in probability. Macmillan Publishing Co., Inc., New York; Collier Macmillan Publishers, London, 1976.
[37] Alex Samorodnitsky. Low-degree tests at large distances. In STOC'07—Proceedings of the 39th Annual ACM Symposium on Theory of Computing, pages 506–515. ACM, New York, New York, New York, USA, 2007.
[38] Tom Sanders. On Roth's theorem on progressions. Ann. of Math. (2), 174(1):619–636, 2011.
[39] Tom Sanders. On the Bogolyubov–Ruzsa lemma. Anal. PDE, 5(3):627–655, 2012.
[40] Ilya Shkredov. On sets of large exponential sums. Dokl. Math., 74(3):860–864, 2006.
[41] Balazs Szegedy. An information theoretic approach to Sidorenko's conjecture. arXiv, 1406.6738, June 2014.
[42] Terence Tao. Moser's entropy compression argument. Personal blog,, August 2009.
[43] Terence Tao. Higher order Fourier analysis, volume 142 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2012.
[44] Terence Tao. Entropy and rare events. Personal blog,, September 2015.
[45] Terence Tao. The logarithmically averaged Chowla and Elliott conjectures for two-point correlations. Forum Math. Pi, 4:e8–36, 2016.
[46] Terence Tao and Tamar Ziegler. The primes contain arbitrarily long polynomial progressions. Acta Math., 201(2):213–305, 2008.
[47] Madhur Tulsiani and Julia Wolf. Quadratic Goldreich-Levin theorems. SIAM J. Comput., 43(2):730–766, 2014.
[48] Salil Vadhan and Jia Zheng. A uniform min-max theorem with applications in cryptography. In Advances in Cryptology – CRYPTO 2013, pages 93–110. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.
[49] Julia Wolf. Finite field models in arithmetic combinatorics – ten years on. Finite Fields Appl., 32:233–274, 2015.
[50] Jiapeng Zhang. On the query complexity for Showing Dense Model. Electronic Colloquium on Computational Complexity, 38, 2011.
[51] Yufei Zhao. An arithmetic transference proof of a relative Szemerédi theorem. arXiv, 1307.4959, July 2013.
[52] Jia Zheng. A uniform min-max theorem and characterizations of computational randomness. PhD thesis, 2014.