What interest could rational agents have in acting lawfully if not for the order, stability, and other collective goods that law brings to society? Why should it otherwise matter to them that their actions are lawful? It would matter to them, of course, if acting unlawfully made them liable to punishment. But in that case their interest in acting lawfully would not come from seeing it as a good thing. It would come, rather, from seeing it as the surest way to avoid a bad thing, something they have an interest in escaping. Yet the challenge to an ethics like Kant’s that represents lawfulness as the essence of moral action is to explain what could interest rational agents in acting lawfully regardless of how the law is enforced, regardless, that is, of whether it is enforced by threats of punishment or incentives to obey. The question, then, that confronts a defender of Kant’s ethics is why a rational agent should regard an action’s being lawful as a condition of its being reasonable to do. If he cannot give an answer to this question, the charge of excessive formalism will stick.

The Higgs mechanism is introduced, first for scalar QED and then with the Higgs doublet,which takes us to the gauge bosons in the electroweak sector of the Standard Model. Nextwe discuss variants of “spontaneous symmetry breaking” patterns, which deviate from theStandard Model, in the continuum and on the lattice. Finally we consider a “smallunification” of the electroweak gauge couplings, as a toy model for the concept of GrandUnified Theories (to be address in Chapter 26).

The topological charge of smooth Yang–Mills gauge fields is discussed, describing inparticular the SU(2) instanton. This leads to the Adler–Bell–Jackiw anomaly and to θ-vacuum states, which are similar to energy bands in a crystal. Wefinally discuss the Atiyah–Singer index theorem in the continuum and more explicitly onthe lattice.

So far our discussion of channel coding was mostly following the same lines as the M-ary hypothesis testing (HT) in statistics. In Chapter 18 we introduce a key departure from this: The principal and most interesting goal in information theory is the design of the encoder mapping an input message to the channel input. Once the codebook is chosen, the problem indeed becomes that of M-ary HT and can be tackled by standard statistical tools. However, the task of choosing the encoder has no exact analogs in statistical theory (the closest being design of experiments). It turns out that the problem of choosing a good encoder will be much simplified if we adopt a suboptimal way of testing M-ary HT, based on thresholding information density.

Free fermion fields are canonically quantized, proceeding from Weyl to Dirac andMajorana fermions, and from the massless to the massive case. We discuss properties likechirality, helicity, and the fermion number, as well as the behavior under parity andcharge conjugation transformation. Fermionic statistics is applied to the cosmic neutrinobackground.

Scalar quantum field theory is introduced in the functional integral formulation,starting from classical field theory and quantum mechanics. We consider Euclidean time andrelate the system in the lattice regularization to classical statistical mechanics.

Consider the following problem: Given a stream of independent Ber(p) bits, with unknown p, we want to turn them into pure random bits, that is, independent Ber(1/2) bits. Our goal is to find a universal way to extract the most number of bits. In other words, we want to extract as many fair coin flips as possible from possibly biased coin flips, without knowing the actual bias. In 1951 von Neumann proposed the following scheme: Divide the stream into pairs of bits, output 0 if 10, output 1 if 01, otherwise do nothing and move to the next pair. Since both 01 and 10 occur with probability pq, regardless of the value of p, we obtain fair coin flips at the output. To measure the efficiency of von Neumann’s scheme, note that, on average, we have 2n bits in and 2pqn bits out. So the efficiency (rate) is pq. The question is: Can we do better? It turns out that the fundamental limit (maximal efficiency) is given by the entropy $h(p)$. In this chapter we discuss optimal randomness extractors, due to Elias and Peres respectively, and several related problems.

In Chapter 6 we start with explaining the important property of mutual information known as tensorization (or single-letterization), which allows one to maximize and minimize mutual information between two high-dimensional vectors. Next, we extend the information measures discussed in previous chapters for random variables to random processes by introducing the concepts of entropy rate (for a stochastic process) and mutual information rate (for a pair of stochastic processes).

Here is the game. You are presented with 3 doors. You can’t see what is behind them, but you have been truthfully told that behind 1 door is a fabulous luxury car, which you want. Behind the other 2 doors are goats, which you don’t want. You get to choose any 1 of the doors and receive the prize behind it.

This enthusiastic introduction to the fundamentals of information theory builds from classical Shannon theory through to modern applications in statistical learning, equipping students with a uniquely well-rounded and rigorous foundation for further study. The book introduces core topics such as data compression, channel coding, and rate-distortion theory using a unique finite blocklength approach. With over 210 end-of-part exercises and numerous examples, students are introduced to contemporary applications in statistics, machine learning, and modern communication theory. This textbook presents information-theoretic methods with applications in statistical learning and computer science, such as f-divergences, PAC-Bayes and variational principle, Kolmogorov’s metric entropy, strong data-processing inequalities, and entropic upper bounds for statistical estimation. Accompanied by additional stand-alone chapters on more specialized topics in information theory, this is the ideal introductory textbook for senior undergraduate and graduate students in electrical engineering, statistics, and computer science.

There are four fundamental optimization problems arising in information theory: I-projection, maximum likelihood, rate distortion, and capacity. In Chapter 5 we show that all these problems have convex/concave objective functions, discuss iterative algorithms for solving them, and study the capacity problem in more detail. As an application, we show that Gaussian distribution extremizes mutual information in various problems with second moment constraints.