The correlation coefficient measures the linear relation between scalar X and scalar Y. How can the linear relation between vector X and vector Y be measured?Canonical Correlation Analysis (CCA) provides a way. CCA finds a linear combination of X, and a (separate) linear combination of Y, that maximizes the correlation. The resulting maximized correlation is called a canonical correlation. More generally, CCA decomposes two sets of variables into an ordered sequence of component pairs ordered such that the first pair has maximum correlation, the second has maximum correlation subject to being uncorrelated with the first, and so on. The entire decomposition can be derived from a Singular Value Decomposition of a suitable matrix. If the dimension of the X and Y vectors is too large, overfitting becomes a problem. In this case, CCA often is computed using a few principal components of X and Y. The criterion for selecting the number of principal components is not standard. The Mutual Information Criterion (MIC) introduced in Chapter 14 is used in this chapter.

Multivariate linear regression is a method for modeling linear relations between two random vectors, say X and Y. Common reasons for using multivariate regression include (1) to predicting Y given X, (2) to testing hypotheses about the relation between X and Y, and (3) to projecting Y onto prescribed time series or spatial patterns. Special cases of multivariate regression models include Linear Inverse Models (LIMs) and Vector Autoregressive Models. Multivariate regression also is fundamental to other statistical techniques, including canonical correlation analysis, discriminant analysis, and predictable component analysis. This chapter introduces multivariate linear regression and discusses estimation, measures of association, hypothesis testing, and model selection. In climate studies, model selection often involves selecting Y as well as X. For instance, Y may be a set of principal components that need to be chosen, which is not a standard selection problem. This chapter introduces a criterion for selecting X and Y simultaneously called Mutual Information Criterion (MIC).