Skip to main content Accessibility help
  • Print publication year: 2017
  • Online publication date: July 2017

9 - Some applications of relative entropy in additive combinatorics



This survey looks at some recent applications of relative entropy in additive combinatorics. Specifically, we examine to what extent entropy-increment arguments can replace or even outperform more traditional energy-increment strategies or alternative approximation arguments based on the Hahn-Banach theorem.


Entropy has a long history as a tool in combinatorics. Starting with a well-known estimate for the sum of the first few binomial coefficients, some of the classical applications include Spencer's theorem that six standard deviations suffice, which states that given n finite sets, there exists a two-colouring of the elements such that all sets have discrepancy at most; a proof of the Loomis-Whitney inequality, which gives an upper bound on the volume of an n-dimensional body in Euclidean space in terms of its (n - 1)-dimensional projections; or Radhakrishnans proof [33] of Bregman's theorem on the maximum permanent of a 0/1 matrix with given row sums. For a beautiful introduction to these fascinating applications, as well as an extensive annotated bibliography, see [10].

There are other more recent results in additive combinatorics in particular where the concept of entropy has played a crucial role. Notable examples include Fox's improvement [7] of the bounds in the graph removal lemma (see also [30], which appeared in the proof-reading stages of this article); Szegedy's information-theoretic approach [41] to Sidorenko's conjecture (see also the blog post [14] by Gowers); Tao's solution [45] to the Erds discrepancy problem (see the discussion [44] on Tao's blog).

Since the above developments appear to be well captured by discussions online, we shall not cover them in any detail here. Instead we shall focus on a particular strand of recent results in additive combinatorics that could all be described as “approximation theorems” of a certain kind.

The text naturally splits into five parts. To start with we give a very brief introduction to the concept of entropy and its variants, in particular relative entropy (also known as Kullback-Leibler divergence). In Section 3, we state and prove a rather general sparse approximation theorem due to Lee [28].

Related content

Powered by UNSILO
[1] Boaz, Barak, Moritz, Hardt, and Satyen, Kale. The uniform hardcore lemma via approximate Bregman projections. In Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 1193–1200. SIAM, Philadelphia, PA, 2009.
[2] Thomas, Bloom. A quantitative improvement for Roth's theorem on arithmetic progressions. J. Lond. Math. Soc. (2), 93(3):643–663, 2016.
[3] Mei-Chu, Chang. A polynomial bound in Freiman's theorem. Duke Math. J., 113(3):399–419, 2002.
[4] David, Conlon, Jacob, Fox, and Yufei, Zhao. The Green-Tao theorem: an exposition. EMS Surv. Math. Sci., 1(2):249–282, 2014.
[5] David, Conlon, Jacob, Fox, and Yufei, Zhao. A relative Szemerédi theorem. Geom. Funct. Anal., 25(3):733–762, 2015.
[6] Thomas, Cover and Joy, Thomas. Elements of information theory. Wiley Series in Telecommunications. John Wiley & Sons, Inc., New York, New York, USA, 1991.
[7] Jacob, Fox. A new proof of the graph removal lemma. Ann. of Math. (2), 174(1):561–579, 2011.
[8] Ehud, Friedgut. An information-theoretic proof of a hypercontractive inequality. arXiv, 1504.01506, April 2015.
[9] Alan, Frieze and Ravi, Kannan. Quick approximation to matrices and applications. Combinatorica, 19(2):175–220, 1999.
[10] David, Galvin. Three tutorial lectures on entropy and counting. arXiv, 1406.7872, June 2014.
[11] Timothy, Gowers. A new proof of Szemerédi's theorem for arithmetic progressions of length four. Geom. Funct. Anal., 8(3):529–551, 1998.
[12] Timothy, Gowers. A new proof of Szemerédi's theorem. Geom. Funct. Anal., 11(3):465–588, 2001.
[13] Timothy, Gowers. Decompositions, approximate structure, transference, and the Hahn-Banach theorem. Bull. Lond. Math. Soc., 42(4):573–606, 2010.
[14] Timothy, Gowers. Entropy and Sidorenko's conjecture — after Szegedy. Personal blog,, November 2015.
[15] Timothy, Gowers and Julia, Wolf. Linear forms and quadratic uniformity for functions on Fnp. Mathematika, 57(2):215–237, 2012.
[16] Ben, Green. Some constructions in the inverse spectral theory of cyclic groups. Combin. Probab. Comput., 12(2):127–138, 2003.
[17] Ben, Green. Spectral structure of sets of integers. In Fourier analysis and convexity, pages 83–96. Birkhäuser Boston, Boston, MA, 2004.
[18] Ben, Green. Finite field models in additive combinatorics. In Bridget S Webb, editor, Surveys in combinatorics 2005, pages 1–27. Cambridge Univ. Press, Cambridge, Cambridge, 2005.
[19] Ben, Green. Roth's theorem in the primes. Ann. of Math. (2), 161(3):1609–1636, 2005.
[20] Ben, Green. Montréal notes on quadratic Fourier analysis. In Additive combinatorics, pages 69–102. Amer. Math. Soc., Providence, RI, 2007.
[21] Ben, Green and Tom, Sanders. Boolean functions with small spectral norm. Geom. Funct. Anal., 18(1):144–162, 2008.
[22] Ben, Green and Terence, Tao. An inverse theorem for the Gowers U3(G) norm. Proc. Edinb. Math. Soc. (2), 51(1):73–153, 2008.
[23] Ben, Green and Terence, Tao. The primes contain arbitrarily long arithmetic progressions. Ann. of Math. (2), 167(2):481–547, 2008.
[24] Ben, Green and Terence, Tao. Linear equations in primes. Ann. of Math. (2), 171(3):1753–1850, 2010.
[25] Russell, Impagliazzo. Hard-core distributions for somewhat hard problems. In FOCS ‘07. 48th Annual IEEE Symposium on Foundations of Computer Science, 2007., pages 538–545. IEEE, 1995.
[26] Russell, Impagliazzo, Cristopher, Moore, and Alexander, Russell. An entropic proof of Chang's inequality. SIAM J. Discrete Math., 28(1):173–176, 2014.
[27] Solomon, Kullback and Richard, Leibler. On information and sufficiency. Ann. Math. Stat., 22(1):79–86, 1951.
[28] James, Lee. Covering the large spectrum and generalized Riesz products. arXiv, 1508.07109, August 2015.
[29] Chi-Jen, Lu, Shi-Chun, Tsai, and Hsin-Lung, Wu. Complexity of hardcore set proofs. Comput. Complexity, 20(1):145–171, 2011.
[30] Guy, Moshkovitz and Asaf, Shapira. A sparse regular approximation lemma. arXiv, 1610.02676, October 2016.
[31] Ryan, O'Donnell. Lecture 16: The hypercontractivity theorem. Personal website,õdonnell/booleananalysis/lecture16.pdf.
[32] Sean, Prendiville. Four variants of the Fourier-analytic transference principle. arXiv, 1509.09200, September 2015.
[33] Jaikumar, Radhakrishnan. An entropy proof of Bregman's theorem. J. Combin. Theory Ser. A, 77(1):161–164, 1997.
[34] Omer, Reingold, Luca, Trevisan, Madhur, Tulsiani, and Salil, Vadhan. Dense subsets of pseudorandom sets. Electronic Colloquium on Computational Complexity, 45:1–33, 2008.
[35] Omer, Reingold, Luca, Trevisan, Madhur, Tulsiani, and Salil, Vadhan. New proofs of the Green-Tao-Ziegler dense model theorem: an exposition. arXiv, 0806.0381, June 2008.
[36] Sheldon, Ross. A first course in probability. Macmillan Publishing Co., Inc., New York; Collier Macmillan Publishers, London, 1976.
[37] Alex, Samorodnitsky. Low-degree tests at large distances. In STOC'07—Proceedings of the 39th Annual ACM Symposium on Theory of Computing, pages 506–515. ACM, New York, New York, New York, USA, 2007.
[38] Tom, Sanders. On Roth's theorem on progressions. Ann. of Math. (2), 174(1):619–636, 2011.
[39] Tom, Sanders. On the Bogolyubov–Ruzsa lemma. Anal. PDE, 5(3):627–655, 2012.
[40] Ilya, Shkredov. On sets of large exponential sums. Dokl. Math., 74(3):860–864, 2006.
[41] Balazs, Szegedy. An information theoretic approach to Sidorenko's conjecture. arXiv, 1406.6738, June 2014.
[42] Terence, Tao. Moser's entropy compression argument. Personal blog,, August 2009.
[43] Terence, Tao. Higher order Fourier analysis, volume 142 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2012.
[44] Terence, Tao. Entropy and rare events. Personal blog,, September 2015.
[45] Terence, Tao. The logarithmically averaged Chowla and Elliott conjectures for two-point correlations. Forum Math. Pi, 4:e8–36, 2016.
[46] Terence, Tao and Tamar, Ziegler. The primes contain arbitrarily long polynomial progressions. Acta Math., 201(2):213–305, 2008.
[47] Madhur, Tulsiani and Julia, Wolf. Quadratic Goldreich-Levin theorems. SIAM J. Comput., 43(2):730–766, 2014.
[48] Salil, Vadhan and Jia, Zheng. A uniform min-max theorem with applications in cryptography. In Advances in Cryptology – CRYPTO 2013, pages 93–110. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.
[49] Julia, Wolf. Finite field models in arithmetic combinatorics – ten years on. Finite Fields Appl., 32:233–274, 2015.
[50] Jiapeng, Zhang. On the query complexity for Showing Dense Model. Electronic Colloquium on Computational Complexity, 38, 2011.
[51] Yufei, Zhao. An arithmetic transference proof of a relative Szemerédi theorem. arXiv, 1307.4959, July 2013.
[52] Jia, Zheng. A uniform min-max theorem and characterizations of computational randomness. PhD thesis, 2014.