Assessing multivariate association between two random vectors x and y is an important problem in many research areas, ranging from the natural sciences (e.g., oceanography and geophysics) to the social sciences (in particular psychometrics and behaviormetrics) and to engineering. While “multivariate association” is often simply visualized as “similarity” between two random vectors, there are many different ways of measuring it. In this chapter, we provide a unifying treatment of three popular correlation analysis techniques: canonical correlation analysis (CCA), multivariate linear regression (MLR), and partial least squares (PLS). Each of these techniques transforms x and y into its respective internal representation ξ and ω. Different correlation coefficients may then be defined as functions of the diagonal cross-correlations {ki} between the internal representations ξi and ωi.

The key differences among CCA, MLR, and PLS are revealed in their invariance properties. CCA is invariant under nonsingular linear transformation of x and y, MLR is invariant under nonsingular linear transformation of y but only unitary transformation of x, and PLS is invariant under unitary transformation of x and y. Correlation coefficients then share the invariance properties of the correlation analysis technique on which they are based.

Analyzing multivariate association of complex data is further complicated by the fact that there are different types of correlation. Two scalar complex random variables x and y are called rotationally dependent if x = ky for some complex constant k.