Multivariate Data Analysis

This chapter limits the discussion to three important elements of multivariate data analysis, namely, principal component analysis, cluster analysis, and discriminant analysis, in the context of data partitioning and pattern recognition for multiple regression.

Abstract

In the previous chapter, we introduced multivariate regression techniques involving two or more variables. Before embarking on an analysis involving large number of variables, we might want to first examine if there are any underlying data structure or patterns that we can exploit to improve and sometimes simplify the analysis. A common approach will be to graphically visualize the data cloud that is limited to three variables. Often, a fourth dimension can be added by varying the type and size of symbols, but that is our limit for graphic visualization. For high-dimensional datasets, an alternative approach is to reduce the dimensionality of the data with minimum loss of important attributes, for example, data variance. Multivariate data analysis techniques allow us to accomplish these goals. Essentially, we define a smaller number of linear combination of the original data, called principal components that allow for data visualization and pattern recognition in a reduced dimensional space. The pattern recognition or classification techniques can be either "supervised" or "unsupervised." In the unsupervised classification techniques, commonly known as cluster analysis, we partition the data into relatively "homogeneous" entities based on the characteristics of the data, without resorting to prior information. In the supervised pattern-recognition method, also known as discriminant analysis, we assign group membership to a given dataset based on a prior classification. Multivariate data analysis by itself is a vast topic, and several excellent references are available on this topic. In this chapter, we limit our discussion to three important elements of multivariate data analysis, namely, principal component analysis, cluster analysis, and discriminant analysis, in the context of data partitioning and pattern recognition for multiple regression. After introducing the concepts using a simple example, we discuss in detail the application of these techniques to the Salt Creek field data introduced in the previous chapter.