For most of its history, science shared many characteristics of a sect. It had its own language, strict requirements for membership, and its own closely held secrets. Science can also usefully be thought of as a metaphorical mountain, existing at an altitude far above where ordinary people lived. Those who practiced it shared the common asceticism of the scientific method, while communicating with one another in the too often mysterious language of mathematics.

This description of science has been accurate for at least the 2,500 years since the time of the Pythagoreans but is now in the midst of a revolutionary change. This shift is occurring because it has become apparent to everyone that the methods of science have to be more widely used for humans to understand the world we inhabit.

One of the principal purposes of this book has been to illustrate how often it is that we can use scientific methods without the burden of complex mathematics or arcane methodologies. It is like learning how to ride a bicycle without knowing the equations of motion that describe how to balance upright on just two wheels. Stanford's Sam Savage has an evocative phrase, only slightly inaccurate, that for some tasks we can learn better through the seat of our pants than we can through the seat of our intellect. Obviously, both parts of our anatomy are often useful to master a broad range of situations.

Most of the knotty problems discussed so far were unraveled using little more than three of the essential parts of scientific investigations:

1. (1) Some carefully gathered data, combined with

2. (2) Clear thinking and

3. (3) Graphical displays that permit the results of the first two steps to be made visible.

Of course, many of the problems encountered by scientists are not susceptible to amateurs – coronary bypass surgery, the design of nuclear reactors, and genetic decoding of the Ebola virus are three that come immediately to mind; there are many others. This is not surprising. What is remarkable is how broad the range of problems is that are susceptible to illumination by thoughtful and industrious amateurs.

In the months leading up to Election Day 2012 we were torn between two very different kinds of outcome predictions. On one side were partisans, usually Republicans, telling us about the imminent defeat of President Obama. They based their prognostication on experience, inside information from “experts,” and talking heads from Fox News. On the other side, were “the Quants” represented most visibly by Nate Silver, whose predictions were based on a broad range of polls, historical data, and statistical models. The efficacy of the former method was attested to by snarky experts, armed with anecdotes and feigned fervor, who amplified the deeply held beliefs of their colleagues. The other side relied largely on the stark beauty of unadorned facts. Augmenting their bona fides was a history of success in predicting the outcomes of previous elections, and, perhaps even more convincing, was remarkable prior success, using the same methods, in predicting the outcome of a broad range of sporting events.

It would be easy to say that the apparent supporters of an anecdote-based approach to political prediction didn't really believe their own hype, but were just pretending to go along to boost their own paychecks. And perhaps that cynical conclusion was often true. But how are we to interpret the behavior of major donors who continued to pour real money into what was almost surely a rat hole of failure? And what about Mitt Romney, a man of uncommon intelligence, who appeared to believe that in January 2013, he was going to be moving into The White House? Perhaps, deep in his pragmatic and quantitative soul, he knew that the presidency was not his destiny, but I don't think so. I believe that he succumbed to that most natural of human tendencies, the triumph of hope over evidence.

We need not reach into the antics of America's right wing to find examples of humanity's frequent preference for magical thinking over empiricism; it is widespread. Renée Haynes (1906–94), a writer and historian, introduced the useful concept of a boggle threshold: “the level at which the mind boggles when faced with some new idea.”

In 1969 Bowdoin College was pathbreaking when it changed its admissions policy to make college admissions tests optional. About one-third of its accepted classes took advantage of this policy and did not submit SAT scores. I followed up on Bowdoin's class of 1999 and found that the 106 students who did not submit SAT scores did substantially worse in their first year grades at Bowdoin than did their 273 classmates who did submit SAT scores (see Figure 7.1). Would their SAT scores, had they been available to Bowdoin's admissions office, have predicted their diminished academic performance?

As it turned out, all of those students who did not submit SAT scores, actually took the test, but decided not to submit them to Bowdoin. Why? There are many plausible reasons, but one of the most likely ones was that they did not think that their test scores were high enough to be of any help in getting them into Bowdoin. Of course, under ordinary circumstances, this speculative answer is not the beginning of an investigation, but its end. The SAT scores of students who did not submit them have to be treated as missing data – at least by Bowdoin's admissions office, but not by me. Through a special data-gathering effort at the Educational Testing Service we retrieved those SAT scores and found that while the students who submitted SAT scores averaged 1323 (the sum of their verbal and quantitative scores), those who didn't submit them averaged only 1201 – more than a standard deviation lower! As it turned out, had the admissions office had access to these scores they could have predicted the lower collegiate performance of these students (see Figure 7.2).

Why would a college opt for ignorance of useful information? Again there is a long list of possible reasons, and your speculations are at least as valid as mine, so I will focus on just one: the consequences of treating missing data as missing at random (that means that the average missing score is equal to the average score that was reported, or that those who did not report their SAT scores did just as well as those who did). The average SAT score for Bowdoin's class of 1999 was observed to be 1323, but the true average, including all members of the class was 1288.

Introduction

Princeton's polymath John Tukey (1915–2000) often declared that a graph is the best, and sometimes the only, way to find what you were not looking for. Tukey was giving voice to what all data scientists now accept as gospel – statistical graphs are powerful tools for the discovery of quantitative phenomena, for their communication, and even for the efficient storage of information.

Yet, despite their ubiquitousness in modern life, graphical display is a relatively modern invention. Its origins are not shrouded in history like the invention of the wheel or of fire. There was no reason for the invention of a method to visually display data until the use of data, as evidence, was an accepted part of scientific epistemology. Thus it isn't surprising that graphs only began to appear during the eighteenth-century Enlightenment after the writings of the British empiricists John Locke (1632–1704), George Berkeley (1685–1753), and David Hume (1711–76) popularized and justified empiricism as a way of knowing things.

Graphical display did not emerge from the preempirical murk in bits and pieces. Once the epistemological ground was prepared, its birth was more like Botticelli's Venus – arising fully adult. The year 1786 is the birthday of modern statistical graphics, devised by the Scottish iconoclast William Playfair (1759–1823) who invented what was an almost entirely new way to communicate quantitative phenomena. Playfair showed the rise and fall over time of imports and exports between nations with line charts; the extent of Turkey that lay in each of three continents with the first pie chart; and the characteristics of Scotland's trade in a single year with a bar chart. Thus in a single remarkable volume, an atlas that contained not a single map, he provided spectacular versions of three of the four most important graphical forms. His work was celebrated and has subsequently been seized upon by data scientists as a crucial tool to communicate empirical findings to one another and, indeed, even to oneself.

In this section we celebrate graphical display as a tool to communicate quantitative evidence by telling four stories. In Chapter 8 we set the stage with a discussion of the empathetic mind-set required to design effective communications of any sort, although there is a modest tilting toward visual communications.

Introduction

Not all of data science requires the mastery of deep ideas; sometimes aid in thinking can come from some simple rules of thumb. We start with a couple of warm-up chapters to get us thinking about evidence. In the first, I show how the Rule of 72, long used in finance, can have much wider application. Chapter 2 examines a puzzle posed by a New York Times music critic, why are there so many piano virtuosos? By adjoining this puzzle with a parallel one in athletics I unravel both with one simple twist of my statistical wrist. In these two chapters we also meet two important statistical concepts: (1) the value of an approximate answer and (2) that the likelihood of extreme observations increases apace with the size of the sample. This latter idea – that, for example, the tallest person in a group of one hundred is likely not as tall as the tallest in a group of one thousand – although this result can be expressed explicitly with a little mathematics it can be understood intuitively without them and so be used to explain phenomena we encounter every day.

I consider the most important contribution to scientific thinking since David Hume to be Donald B. Rubin's Model for Causal Inference. Rubin's Model is the heart of this section and of this book. Although the fundamental ideas of Rubin's Model are easy to state, the deep contemplation of counterfactual conditionals can give you a headache. Yet the mastery of it changes you. In a very real sense learning this approach to causal inference is closely akin to learning how to swim or how to read. They are difficult tasks both, but once mastered you are changed forever. After learning to read or to swim, it is hard to imagine what it was like not being able to do so. In the same way, once you absorb Rubin's Model your thinking about the world will change. It will make you powerfully skeptical, pointing the way to truly find things out. In Chapter 3 I illustrate how to use this approach to assess of the causal effect of school performance on happiness as well as the opposite: the causal effect happiness has on school performance.

There have been many remarkable changes in the world over the last century, but few have surprised me as much as the transformation in public attitude toward my chosen profession, statistics – the science of uncertainty. Throughout most of my life the word boring was the most common adjective associated with the noun statistics. In the statistics courses that I have taught, stretching back almost fifty years, by far the most prevalent reason that students gave for why they were taking the course was “it's required.” This dreary reputation nevertheless gave rise to some small pleasures. Whenever I found myself on a plane, happily involved with a book, and my seatmate inquired, “What do you do?” I could reply, “I'm a statistician,” and confidently expect the conversation to come to an abrupt end, whereupon I could safely return to my book. This attitude began to change among professional scientists decades ago as the realization grew that statisticians were the scientific generalists of the modern information age. As Princeton's John Tukey, an early convert from mathematics, so memorably put it, “as a statistician, I can play in everyone's backyard.”

Statistics, as a discipline, grew out of the murk of applied probability as practiced in gambling dens to wide applicability in demography, agriculture, and the social sciences. But that was only the beginning. The rise of quantum theory made clear that even physics, that most deterministic of sciences, needed to understand uncertainty. The health professions joined in as Evidence-Based Medicine became a proper noun. Prediction models combined with exit polls let us go to sleep early with little doubt about election outcomes. Economics and finance was transformed as “quants” joined the investment teams and their success made it clear that you ignore statistical rigor in devising investment schemes at your own peril.

These triumphs, as broad and wide ranging as they were, still did not capture the public attention until Nate Silver showed up and starting predicting the outcomes of sporting events with uncanny accuracy. His success at this gave him an attentive audience for his early predictions of the outcomes of elections. Talking heads and pundits would opine, using their years of experience and deeply held beliefs, but anyone who truly cared about what would happen went to FiveThirtyEight, Silver's website, for the unvarnished truth.

A famous paradox, attributed to the Greek mathematician Zeno, involves a race between the great hero Achilles and a lowly tortoise. In view of their vastly different speeds, the tortoise was granted a substantial head start. The race began, and in a short time Achilles had reached the tortoise's starting spot. But in that short time, the tortoise had moved slightly ahead. In the second stage of the race Achilles quickly covered that short distance, but the tortoise was not stationary and he moved a little further onward. And so they continued – Achilles would reach where the tortoise had been, but the tortoise would always inch ahead, just out of his reach. From this example, the great Aristotle, concluded that, “In a race, the quickest runner can never overtake the slowest, since the pursuer must first reach the point whence the pursued started, so that the slower must always hold a lead.”

The lesson that we should take from this paradox is that when we focus only on the differences between groups, we too easily lose track of the big picture. Nowhere is this more obvious than in the current public discussions of the size of the gap in test scores between racial groups. In New Jersey the gap between the average scores of white and black students on the well-developed scale of the tests of the NAEP has shrunk by only about 25 percent over the past two decades. The conclusion drawn was that even though the change is in the right direction, it is far too slow.

But focusing on the difference blinds us to a remarkable success in education over the past twenty years. Although the direction and size of student improvements occur across many subject areas and many age groups, I will describe just one – fourth grade mathematics. The dots in Figure 12.1 represent the average scores for all available states on NAEP's fourth grade mathematics test (with the nation as a whole as well as the state of New Jersey's dots labeled for emphasis), for black students and white students in 1992 and 2011. Both racial groups made steep gains over this time period (somewhat steeper gains for blacks than for whites).

Introduction

From 1996 until 2001 I served as an elected member of the Board of Education for the Princeton Regional Schools. Toward the end of that period a variety of expansion projects were planned that required the voters pass a $61 million bond issue. Each board member was assigned to appear in several public venues to describe the projects and try to convince those in attendance of their value so that they would agree to support the bond issue. The repayment of the bond was projected to add about $500 to the annual school taxes for the average house, which would continue for the forty years of the bond. It was my misfortune to be named as the board representative to a local organization of senior citizens.

At their meeting I was treated repeatedly to the same refrain: that they had no children in the schools, that the schools were more than good enough, and that they were living on fixed incomes and any substantial increase in taxes could constitute a hardship, which would likely continue for the rest of their lives. During all of this I wisely remained silent. Then, when a pugnacious octogenarian strode to the microphone, I feared the worst. He glared out at the gathered crowd and proclaimed, “You're all idiots.” He then elaborated, “What can you add to your house for $500/year that would increase its value as much as this massive improvement to the schools? Not only that, you get the increase in your property value immediately, and you won't live long enough to pay even a small portion of the cost. You're idiots.” Then he stepped down. A large number of the gray heads in the audience turned to one another and nodded in agreement. The bond issue passed overwhelmingly.

Each year, when the real estate tax bill arrives, every homeowner is reminded how expensive public education is. Yet, when the school system works well, it is money well spent, even for those residents without children in the schools. For as surely as night follows day, real estate values march in lockstep with the reputation of the local schools. Of course, the importance of education to all of us goes well beyond the money spent on it.