Chapter 2 starts by placing experiments in the scientific process – experiments are only useful in the context of well-motivated questions, thoughtful theories, and falsifiable hypotheses. The author then turns to sampling and measurement since careful attention to these topics, despite being often neglected by experimentalists, are imperative. The remainder of Chapter 2 offers a detailed discussion of causal inference that is used to motivate an inclusive definition of “experiments.” The author views this as more than a pedantic exercise, as careful consideration of approaches to causal inference reveals the often implicit assumptions that underlie all experiments. The chapter concludes by touching on the different goals experiments may have and the basics of analysis. The chapter serves as a reminder of the underlying logic of experimentation and the type of mindset one should have when designing experiments. A central point concerns the importance of counterfactual thinking, which pushes experimentalists to think carefully about the precise comparisons needed to test a causal claim.

Chapter 6 touches on designing “good” experiments. The primary point is that regardless of changes, the fundamentals of conducing a sound experiment remain the same. A list of steps that should be taken for any design is offered.

Chapter 4 turns to experimental designs, focusing on three designs that have gained prominence in many social science applications in the last decade: audit field experiments, conjoint survey experiments, and lab-in-the-field experiments. These three designs also provide readers with examples of a type of field experiment, survey experiment, and lab experiment, respectively – the three conventional “types” of experiments employed in the social sciences. The chapter reviews the basics of each design and provides prominent examples. Importantly, it also discusses limitations and challenges of the designs, or put another way, how to think about these new designs. This chapter includes a brief overview of “public policy experiments”: It does so given the recent rise in studies of political elites, which ultimately connect to policymaking and responsiveness. The chapter makes clear that the substantive questions being explored should drive experimental design choices and not vice versa.

Chapter 1 discusses of the evolution of experiments, illustrating this development through the field of political science. The author argues that the discipline currently finds itself in a new era, parts of which apply to all of the social sciences. This new era began around 2010 and reflects the confluence of experiments achieving widespread acceptance in the discipline, technological advances, and the open science movement (these latter two dynamics have affected all of the social sciences). The era introduces many opportunities but also novel challenges. Ironically, the ease of conducting experiments today has the potential to undermine their quality. I conclude the chapter by discussing the motivation for the primer and reviewing the remainder of the book.

Chapter 5 delves into the steps that occur prior to, during, and after an experiment – including arriving at questions to explore with an experiment; documenting the steps in the process of conducting an experiment; and considering whether to replicate one’s findings after an experiment. This discussion touches on the themes of the aforementioned open science movement, offering in many instances a cautionary perspective.

Chapter 3 focuses on how to think when evaluating experiments. This includes a discussion of realism, particularly why mundane realism or resemblance to the “real world” receives far too much attention, as well as an overview of how to design experimental treatments. The chapter then turns to validity issues, offering a new way to think about external validity in assessing experiments. This includes a detailed discussion of sampling and why the onus should be more on critics of an experimental sample than on the experimentalist him/herself (i.e., to justify a sample).