Principal components analysis and factor analysis are related, but distinct, methods for analyzing the structure of multivariate data. In Chapter 11, we identified linear combinations of variables that would separate known groups. If we suspect there are groups in the data or that the variables are related to one another, principal components may help us to identify those patterns.

Principal components analysis looks for a way to simplify the dimensionality of the data. If we have only two variables (e.g., length and width), we can display their relationship with a simple scatterplot of length against width that indicates if the variables are correlated with one another or if there are distinct clusters of observations. If there are three variables (add thickness for example), we can display a 3-D representation that can be rotated to look for patterns in the data. After three variables, there is no simple way to display the data. One approach is to find a way of projecting the data into a smaller number of dimensions just as a shadow is a two-dimensional projection of a three-dimensional object. That projection will result in the loss of information so we would like to find the projection that captures as much detail as possible. Principal components analysis does this by identifying the direction of maximum covariance (or correlation) in the data. The first component identifies that direction. The second component finds the next largest direction of covariance (or correlation) subject to the constraint that it must be orthogonal (at a right angle or uncorrelated) with the first dimension. If the data are highly correlated, we may be able to accurately summarize many variables in a few dimensions. Principal components analysis tells us about the structure of the data and provides us with a way of displaying the observations in a reduced number of dimensions. Sometimes that can help us to see clustering in the data that might indicate distinct artifact types (Christenson and Read, 1977; Read, 2007).

While principal components focus primarily on summarizing multivariate data, factor analysis has a different goal. Factor analysis assumes that the data are the product of unobserved (or latent) factors. The correlations between variables provide the evidence of these latent factors.