Scalar field theory is quantized canonically. We also discuss the Noether current at theclassical level and the vacuum energy at the quantum level.

In Chapter 17 we introduce the concept of an error-correcting code (ECC). We will spend time discussing what it means for a code to have low probability of error, and what is the optimum (ML or MAP) decoder. On the special case of coding for the binary symmetric channel (BSC), we showcase the evolution of our understanding of fundamental limits from pre-Shannon’s to modern finite blocklength. We also briefly review the history of ECCs. We conclude with a conceptually important proof of a weak converse (impossibility) bound for the performance of ECCs.

Chapter 33 introduces the strong data-processing inequalities (SDPIs), which are quantitative strengthening of the DPIs in Part I. As applications we show how to apply SDPI to deduce lower bounds for various estimation problems on graphs or in distributed settings. The purpose of this chapter is two-fold. First, we want to introduce general properties of the SDPI coefficients. Second, we want to show how SDPIs help prove sharp lower (impossibility) bounds on statistical estimation questions. The flavor of the statistical problems in this chapter is different from the rest of the book in that here the information about the unknown parameter θ is either more “thinly spread” across a high-dimensional vector of observations than in classical X = θ + Z type of models (see spiked Wigner and tree-coloring examples), or distributed across different terminals (as in correlation and mean estimation examples).

This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. The book introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite blocklength approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning, and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC-Bayes and variational principle, Kolmogorov’s metric entropy, strong data-processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by additional stand-alone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.

For her New Year’s resolution, Gaowen has determined to finally get organized. She puts up shelves and racks, adds new filing cabinets, cupboards, a desk, and assorted bins, and sets about putting her electronics equipment, clothes, dishes, books, keys, and everything else all in their proper places. You can think of data objects as containers to help you get organized.

This chapter deals with the renormalization group in Wilson’s spirit. General concepts,like fixed points, are illustrated with examples, such as block-variable transformations,perfect lattice actions, the Wilson–Fisher fixed points, the Callan–Symanzik equation, andvarious scenarios for running couplings.

The free electromagnetic field is quantized canonically and with the functionalintegral. We emphasize the roles of the Gauss law, helicity, and gauge fixing in thecontinuum. We also derive Planck’s formula for black-body radiation and apply it to thecosmic microwave background.

The company culture at the sports equipment manufacturer, unsurprisingly, is that most employees regularly participate in outside-of-work sports activities.

So far our discussion on information-theoretic methods has been mostly focused on statistical lower bounds (impossibility results), with matching upper bounds obtained on a case-by-case basis. In Chapter 32 we will discuss three information-theoretic upper bounds for statistical estimation under KL divergence (Yang–Barron), Hellinger (Le Cam–Birgé), and total variation (Yatracos) loss metrics. These three results apply to different loss functions and are obtained using completely different means. However, they take on exactly the same form, involving the appropriate metric entropy of the model. In particular, we will see that these methods achieve minimax optimal rates for the classical problem of density estimation under smoothness constraints.

In 1900, sponge divers near the Greek island of Antikythera came upon a Roman shipwreck bestrewn with ancient statues, jewelry, and other treasure, and an odd looking, very encrusted gear mechanism. Retrieved and then studied for decades, it took until 1974 for scientists to unravel the mystery of the mechanism’s purpose. It was a computer, designed and built in Egypt around 100 BCE, that could predict the positions of the sun, moon, and planets, and the timings of solar eclipses and Olympic Games events.

This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. The book introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite blocklength approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning, and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC-Bayes and variational principle, Kolmogorov’s metric entropy, strong data-processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by additional stand-alone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.

Chapter 29 gives an exposition of the classical large-sample asymptotics for smooth parametric models in fixed dimensions, highlighting the role of Fisher information introduced in Chapter 2. Notably, we discuss how to deduce classical lower bounds (Hammersley–Chapman–Robbins, Cramér–Rao, van Trees) from the variational characterization and the data-processing inequality (DPI) of χ2-divergence in Chapter 7.

In high-energy scattering processes, hadrons can be described as a set of partons. Thispicture is compatible with QCD, where the partons are identified as quarks, anti-quarks,and gluons. In this picture, we consider electron–positron annihilation, which can lead tohadrons or a muon–anti-muon pair. The R-ratio of the cross sections for these scenariosallows us to identify the number of colors, Nc = 3, experimentally. Next we discuss deep inelasticelectron–nucleon scattering, which leads to the concepts of the Bjorken variable,structure functions, the parton distribution function, Bjorken scaling, the Callan–Grossrelation, and the DGLAP evolution equation. The hadronic tensor takes us to the scalingfunctions, where high-energy neutrino–nucleon scattering provides further insight, inparticular a set of constraints which are expressed as sum rules.

Starting from 2-flavor QCD, isospin symmetry is employed in order to explain themultiplets of light baryons and mesons, from a constituent quark perspective. Next weinvolve the strange quark and arrive at meson mixing as well as the Gell-Mann–Okuboformula for the baryon multiplet splitting. Regarding QCD from first principles, wecomment on lattice simulation results for the hadron masses. At last we discuss the hadronspectrum in a hypothetical world with Nc=5colors.

Ethics, like other branches of philosophy, springs from seemingly simple questions. What makes honest actions right and dishonest ones wrong? Why is death a bad thing for the person who dies? Is there anything more to happiness than pleasure and freedom from pain? These are questions that naturally occur in the course of our lives, just as they naturally occurred in the lives of people who lived before us and in societies with different cultures and technologies from ours. They seem simple, yet they are ultimately perplexing. Every sensible answer one tries proves unsatisfactory upon reflection. This reflection is the beginning of philosophy. It turns seemingly simple questions into philosophical problems. And with further reflection, we plumb the depths of these problems.

This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. The book introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite blocklength approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning, and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC-Bayes and variational principle, Kolmogorov’s metric entropy, strong data-processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by additional stand-alone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.