The theory of status effects illustrates one important manner in which theories can develop. Regularities in groups faced with tasks requiring group interaction were observed over a wide variety of tasks. Status inequality in particular was obvious. Some group members had more influence and were given more opportunities to interact than others, even when expertise did not differentiate among individuals. Further such inequality seemed reciprocal: group members permitted the inequality and allowed it to stabilize. Researchers developed abstract concepts and propositions to account for these regularities and concluded that inequality arises because the task interaction causes individuals to form performance expectations for themselves and others, and once those expectations form, they affect the distribution of all behavioral components of interaction.

Logic simplifies and clarifies ordinary language and thereby improves communication by reducing or eliminating errors of interpretation. While English and other languages offer many ways of saying things, logic uses simple structures whose meanings are always clear because they are stated using only a few well-known forms. This chapter considers sentential logic in which every sentence may be either true or false but not both or of unknown truth value. Such logic simplifies communication from the uncertainty that is common in ordinary language. Negation, conjunction, disjunction and implication (or conditionals) are illustrated by truth tables. All are valuable in theory development and enable the prediction of outcomes from premises.

A theory of some finding or observation is an explanation of that finding or observation. Further, a good theory is a set of principles that are sufficient to show that the phenomenon is an instance of more general phenomena or principles. But not all explanations help us understand general phenomena because they lack some fundamental characteristics. The necessary characteristics of adequate explanations include explicit definitions and precise and limited scope, that is, they do not attempt to explain everything about a given event or action. Further, they can be tested with empirical data; they do not appeal to supernatural forces or to explanations with claims that testing is not necessary.

Mistakes and missteps in theory development and testing are inevitable but there are ways to decrease them. One way to detect potential problems is to outline the definitions, scope conditions, assumptions and finally derivations. Stripping down the theory to the simplest form helps spot definitions that are not precise or derivations that are not sound. Once done, it becomes easier to diagnose problems and think through potential solutions. Another way to find potential problems is to talk through your ideas with others—especially those you know to be skeptical. Having to explain your logic and then your potential tests with others can provide invaluable feedback and can make you rethink aspects of your formulations.

Most regression methods estimate the mean of Y given X. But it can also be useful to estimate the quantiles of Y given X. This provides more information about the relationship between X and Y.

When the outcome Y is binary or an integer, we need to modify our methods. In this chapter, we introduce logistic regression for binary data and Poisson regression for count data. These are special cases of a class of regression models called generalized linear models. Logistic regression is a special case of a more general suite of methods called classification, which are discussed in Chapter 9.

In this chapter, we explain how to estimate the prediction error of a regression model. The training error (the average of the squared residuals) under-estimates the prediction error. Instead, we use cross-validation that involves separating the data into one part for fitting the model and one part for estimating the prediction error. We can use the estimated prediction error to choose among a set of possible regression models.

In this chapter, we briefly cover a few other topics related to regression. Each topic is the subject of entire textbooks. Our goal is to give a very concise introduction to each topic. The topics include random effects and empirical Bayes, neural nets and deep learning, survival analysis, graphical models, and time series.

In this chapter, we consider nonparametric regression when we have more than one feature. First, we show how the methods in Chapter 6 can be extended to handle this case. Then, we consider additive regression, regression trees, and random forests. Another estimator based on neural nets is discussed in Chapter 12.