THE VALUE AND CHALLENGES OF AN UNDERGRADUATE METHODS COURSE

Both pedagogical merits and classroom challenges relate to the choice of a textbook. Starting with academic value, a course in undergraduate research methods is important for demonstrating to undergraduate students how to conduct theoretically-informed scientific inquiry into the study of politics. So a methods class is important for students to understand how one goes about adding to the body of knowledge that makes general sense of the political world. From a job placement point of view, a methods class is valuable because it is one area where political science majors can pick up marketable skills. Data analytic skills are in increased demand in jobs for the government, for nonprofits, and for businesses. Introducing students to how they can apply some of these skills thereby can help them with job placement, as they point to skills they can do (such as linear regression) or software they can use (such as R, Stata, or SPSS). So the rewards for students who excel in a research methods course can be considerable.

While helping students obtain these valuable skills, instructors may find several challenges in teaching an undergrad research methods course. Anyone who is teaching about this topic will have had graduate-level methods training and thereby recognize the importance of good research design and quantitative skills. However, many undergraduate students have a hard time recognizing why they should be expected to, much less want to, take a course on this topic. In my own experience, some students react to statistics training by writing instructor evaluations that make comments to the effect of, “that is a waste of time, we should focus on political science not statistics.” The irony of these comments, of course, is that political science requires methods of scientific inquiry. Hence, the most substantial exposure students get to the rigorous study of politics often comes from the content of an undergraduate research methods course.

How challenging a methods course is to teach can depend on several factors that are completely out of the instructor’s control: Do students at the college or university have a strong mathematical background when they enroll in the course? If not, the first exposure to quantitative reasoning can be difficult. Is the course a pure elective or is it required of all majors? If it is an elective, the students are more likely to be motivated to study the material, whereas a required class may generate hostility from students who resent being forced to study such a topic. Do curriculum requirements by the department or the university mean that the course must cover certain topics? If so, the lack of flexibility may make it more difficult to tailor the course to students’ needs and abilities. Of course, there can be a perfect storm where many of these factors come together to make the course quite a challenge to teach. That said, the wide variety of textbooks allows teachers the ability to choose just what fits their needs.

QUESTIONS FACING THE TEACHER IN BOOK SELECTION

In the context of these goals and challenges, there are two main questions a political scientist can ask to help select the most ideal textbook. The first question an instructor needs to settle is what he or she hopes to accomplish with the book. If the instructor wants a book focusing strictly on quantitative analysis then either Wheelan’s, Pollock’s, or Imai’s book would be a good choice. Alternatively, if the instructor wants a book that explains to students all steps of the research process, then either Kellstedt and Whitten’s or Johnson, Reynolds, and Mycoff’s book would be more useful. Because this marks the clearest division between texts and is the first choice that should be made, this article is split into a discussion of each class of book.

Second, after an instructor chooses a general purpose of the book, he or she then must choose the book that offers the best arrangement of content for the course goals and students taking the course. On this choice, there are several considerations to consider such as topics covered and level of technical difficulty. For example, Wheelan’s Naked Statistics goes out of its way to be as non-technical as possible and only rarely presents formulas—for some classes this would be a good approach and for others it would not. As another example, Johnson, Reynolds, and Mycoff’s Political Science Research Methods includes a closing chapter that annotates a real research article—for classes that want to spend substantial time on honing writing skills that is a great innovation, but for others it might not be essential. As the books are discussed in the ensuing sections (first on books covering all aspects of research and then on statistics books), I revisit these considerations in turn.

COMPREHENSIVE RESEARCH TEXTBOOKS

I now describe the two books that seek to provide a wire-to-wire explanation of the research process. First, Kellstedt and Whitten’s Fundamentals of Political Science Research starts with an introduction to how students can think about the scientific study of politics. The second chapter describes the importance of developing interesting questions and reviewing past research. The second chapter also dedicates significant attention to causal reasoning and developing a strong theory. Chapters 3-5, respectively, describe the requisites of causal inference, the elements of good research design, and important measurement concepts. Chapters 6-11 cover the primary quantitative topics, with skills including basic probability theory, cross-tabulations, difference of means tests, and correlation coefficients. Three of the chapters focus on linear regression, building from bivariate regression, to multivariate regression, and finally to specification and diagnostic issues. Splitting linear regression up like this offers a nice pacing of a topic that is complex to most undergraduates into more manageable chunks. Chapter 11, the last of the quantitative training chapters, offers a quick overview of binary choice models (logit and probit regression) and time series analysis. Some instructors may want to skip part or all of this chapter if time (and technical goals) do not permit. For those who do want to cover one or both of these topics, however, the focus is on the theoretical motivation of these models and their interpretation. The chapter is written clearly and is easy to follow. As a strong point for this textbook, this is the only one of the five books that offers specific advice on the study of time series. Chapter 12, the final chapter, concludes by offering general advice on writing a complete research article or final report. Tips in this chapter range from what makes an effective literature review to how best to incorporate tables and figures.

Features of Kellstedt and Whitten’s book include a glossary of terms and exercises, presented at the end of each chapter. The book’s appendices include tables of chi-square critical values, t critical values (for which the last line—infinite degrees of freedom represents the Normal distribution), values for the logit link function, and values for the probit link function. Online resources for the book (http://www.cambridge.org/fpsr) include example datasets, guides for conducting in-book analyses with statistical software (SPSS, Stata, and R), as well as instructional resources. In general, for a course that wants to cover all aspects of research design, The Fundamentals of Political Science Research is an accessible option that most undergraduates should be able to understand. In terms of the quantitative content, a strength of the book is that it does not cover methods that get very little usage in contemporary research (such as Somers’ D or other ordinal measures of association), rather sticking to the most valuable topics. At 342 pages, and including a variety of resources, teachers have the option of deciding how much of a mix they prefer between quantitative work and other attributes of conducting research, as this book facilitates a combination of these two foci. Although this textbook does not offer in-depth software training, it does include some Stata code and output for illustrative purposes. For the instructor who wishes to teach students how to use Stata, a supplement would be useful, but this text would have a natural advantage of examples that correspond to the software the students are learning. (For instructors teaching students another program, the Stata examples will not serve as a distraction, as the in-text focus is on examples’ substantive meaning.) As an overall text, the book does a nice job of hitting the most important areas on each topic and covering them well, without trying to cram too much into any one topic. As such, the pacing of the book is very appropriate for a semester-length comprehensive course.

Turning to the other book that considers all attributes of research design, Johnson, Reynolds, and Mycoff’s Political Science Research Methods also offers a mixture of quantitative methods and other attributes of design. Contrasting from Kellstedt and Whitten’s book, Political Science Research Methods is considerably longer at 655 pages and puts relatively more emphasis on features beyond quantitative skills (in 10 of 15 chapters). That said, when teaching from the quantitative chapters of this book, be sure to plan ahead and split the chapters into multiple sittings. In particular, Chapter 13 (on bivariate statistics) is 100 pages long and includes a vast array of topics all together, such as cross-tabulations, ordinal measures of association, difference of means tests, ANOVA, bivariate regression, and correlation coefficients. Chapter 14 then does a 68-page follow-up that considers multivariate versions of many of these methods and introduces logistic regression. Besides splitting-up these chapters, teachers also may want to consider if they want to cover all of these topics in quantitative research or selectively focus on a few. In short, this book is the most comprehensive of the five considered in this review (with the possible exception of Imai’s quantitative-focused book), but in many cases the instructor may have to balance breadth against a potential information overload if the course tries to do too much. For courses with students who are ready to intensively study a wide range of material and cover each topic in great depth, or for instructors who are happy to pick-and-choose what to read out of a textbook, this text is a good option.

A unique feature of Johnson, Reynolds, and Mycoff’s book is that Chapters 8-10 focus on the issue of gathering original data, both from a quantitative and qualitative perspective. For political scientists who would like students to either gather some data on their own or understand the data-gathering process, these chapters respectively describe firsthand observation, gathering data from existing sources, and survey research methods. Another intriguing feature is that Chapter 15 includes an annotated example of a real research report by Becki Scola on women’s office holding in the American states. For instructors who would like to guide their students through an example of real political science research and why various parts are included in the whole of the project, this is a nice feature. Each chapter of this book includes a glossary of new terms, and the back of the book includes a comprehensive glossary as well as tables of Normal distribution p-values, t distribution critical values, chi-square critical values, and F distribution critical values. There also is a companion workbook for purchase, authored by Jason D. Mycoff, entitled Working with Political Science Research Methods: Problems and Exercises (4th ed.). This workbook is not tied to any statistical software and does not ask for extensive data analysis by the students. Rather, it focuses on asking students to write about the concepts they have studied, interpret results, and perform basic analysis with data (such as creating frequency tables and calculating means). Data sets and study resources are available at: https://edge.sagepub.com/johnson8e

BOOKS FOCUSING ON STATISTICS

I now turn to three books that focus principally on statistics. As such, these books are most appropriate in courses that focus more exclusively on quantitative methods rather than a broad look at political science research. First, even though it focuses strictly on quantitative reasoning, Wheelan’s Naked Statistics is the easiest-to-digest, least technical of all five books described in this review. As such, for the instructor who is concerned about students’ mathematical background yet wants (or has) to teach statistical reasoning, this book is an attractive option. It is a mass-marketed paperback (and quite affordable), and it explicitly avoids formulas as much as it can. With Wheelan’s book alone, a student could learn the theory behind the methods and how to interpret the results. For instructors who do want students to be able to apply formulas to real data or use statistical software to implement these methods, Naked Statistics would only be a good choice if the teacher either assigned a supplemental text (perhaps a detailed software tutorial) or felt comfortable teaching this additional information in lecture notes alone.

Wheelan’s book covers descriptive statistics, correlation coefficients, probability theory, statistical inference, polling, and linear regression analysis. The focus, again, is on the intuition behind the methods and how to interpret the results. At 304 pages, the book is easy to read and would allow for another supplementary resource. Examples in this book are not strictly political, but nor are they drawn entirely from another field—rather they cover a wide array of topics, intended to be relatable to readers’ experiences. There are no glossaries, tables, or supplementary data sets, and the occasional end-of-chapter appendices show a handful of formulas and example calculations. These chapter appendices cover the topics of variance, standard deviation, mean, correlation, margin of error on a proportion, and the t distribution (focusing on t-tests of regression coefficients). In short, this book provides the intuition of statistics with the fewest possible mathematical details and fun, engaging prose.

A second alternative, Pollock’s Essentials of Political Analysis, offers a more in-depth tour of basic statistical inference. The book starts by describing the importance of defining concepts clearly and measuring variables well in Chapter 1, and it turns to descriptive statistics at different levels of measurement in Chapter 2. In Chapter 3, Pollock explains how students should formulate a hypothesis and propose an explanation of a phenomenon—including nice illustrations of what should qualify as an “acceptable explanation” or not. This chapter also introduces the importance of graphing variables. Chapter 4 explains the logic of research design and the importance of control. After this ground material, Chapters 5-9 turn full time into statistical inference. Topics include cross-tabulations, inferences on means and proportions, difference of means tests, ordinal measures of association, correlation, linear regression, and logistic regression. The book generally lays out all of the formulas students would need to estimate these quantities, though the discussions of linear regression (Chapter 8) and logistic regression (Chapter 9) rely increasingly on intuition. That said, the intuition for the logistic regression chapter is a real highlight of the book, and does one of the clearest jobs I have seen of distinguishing probability from odds, and then odds from log-odds. All of this quite clearly sets up how logistic regression is a linear model of the log-odds with a great working example of the effect of education on voting.

The Essentials of Political Analysis includes exercises at the end of each chapter. There is no glossary, though key terms are highlighted and indexed at the end of each chapter. Important tables, such as the critical values of probability distributions, are wedged into the chapters when the text first uses them rather than collected at the end. Hence, the list of tables at the front of the book is important. For classes that will teach students how to use statistical software, there are three companion workbooks that instructors can assign: An IBM SPSS Companion to Political Analysis (now on its fifth edition), A Stata Companion to Political Analysis (third edition), or An R Companion to Political Analysis (first edition). The SPSS companion is the most refined of the three and makes for a nice introduction to the most user-friendly of the software options if instructors want to go that route. The main text is 277 pages long, so one of these supplements could easily be included in the class. The datasets for all three companions, as well as supplemental student learning materials for the main text, are all available at: http://edge.sagepub.com/Pollock

A third alternative for those considering the pure statistics route is the newest book available: Imai’s Quantitative Social Science: An Introduction. This book is distinctive from all others in two important ways: For one, the book is organized differently from many statistics texts in that each chapter focuses on a more concrete goal with many real data examples. Second, all of the real data examples are interactive for the student—R code for implementing all examples is present in-text as each concept is introduced. Thus, theory and practice are fused together. Chapter 1 offers a basic introduction to R. Chapter 2 introduces the notion of causality from the potential outcomes framework and shows how to analyze experimental and quasi-experimental data from this point of view. Chapter 3 describes measurement, showing graphs, bivariate correlation, and even a clustering algorithm. Chapter 4 introduces linear regression through the lens of statistical prediction. Chapter 5 focuses on learning from data, with examples of analyzing textual data, network data, and geospatial data. Chapters 6 and 7 dig more into the mathematics behind probability and statistical inference, and are both substantially longer than the other chapters. Chapter 7 alone is pretty substantial in that it makes up about 20% of the book on its own (the entire book being about 462 pages long). Hence, the instructor should be prepared to break-up the chapter’s material a bit more than some of the others. The concluding Chapter 8 points the reader towards what they may study next in their pursuit of data analytic skills.

Each chapter in Quantitative Social Science: An Introduction ends with several exercises, built around various example datasets with which students can practice the methods learned in R. Between these exercises and the in-text code, no software supplementary material would be needed with this book. The end of the book also includes a separate index for all of the R commands used in the book, so that readers can look up examples of specific commands as a reference. In lieu of providing readers with tables of critical values, this book instead shows users how to compute critical values quickly in R as needed. There are more than 50 exercises available online that use data from real published papers in the social sciences. The datasets and supplementary materials are all available through a dedicated website at Princeton University Press (information is also stored on GitHub: https://github.com/kosukeimai/qss). There also is an R-installable course using the “swirl” package in R for review exercises: These make for easy, interactive practice. The explanation of how to use R is quite clear in this book, but this nevertheless is the most technical of the five books reviewed here. For the instructor who does not want to teach students how to use statistical software or worries that students might not have the technical background to learn a wide variety of methods, this book may not be the best fit. It is a great book, however, for teachers who would like to maximize students’ marketable skills: walking out of this class they can say that they know how to use R, draw sophisticated graphs, estimate and make predictions from linear regression models, conduct clustering analysis (including text-as-data analysis), and conduct network analysis. The latter two are machine-learning algorithms, so those can be particularly valuable on a job market that emphasizes big data applications. This book also would help students engage with the widest possible variety of quantitative political science research.