机器学习算法介绍及相关参考文献.docVIP

Linear Discriminant Analysis Linear Discriminant Analysis (LDA) is a linear inherently multi-class classification method. It was originally introduced by Fisher for two classes [9], but was later extended for multiple classes by Rao [26]. In particular, LDA computes a classification function ，where is selected as the linear projection that maximizes the Fisher-criterion ，where and are the within-class and the between class scatter matrices (see, e.g., [7]). The corresponding optimal solution for this optimization problem is given by the solution of the generalized eigenproblem or directly by computing the eigenvectors for . Since the rank of is bounded by the rank of there are non-zero eigenvalues resulting in a ()-dimensional subspace, which preserves the most discriminant information. For classification of a new sample the class label is assigned according to the result of a nearest neighbor classification. For that purpose, the Euclidean distances of the projected sample and the class centers in the LDA space are compared: . Loog et al. [19] showed that for more than two classes maximizing the Fisher criterion in Eq. (7) provides only a suboptimal solution! In particular, optimizing the Fisher criterion provides an optimal solution with respect to the Bayes error for two classes, but this can not be generalized for multiple classes. Nevertheless LDA can be applied for many practical multi-class problems. This was also confirmed by theoretical considerations by Mart′?nez and Zhu [20]. However, they showed that increasing the number of classes decreases the separability. [9] R. A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179–188, 1936. [26] C. R. Rao. The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society – Series B, 10(2):159–203, 1948. [7] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification.John Wiley Sons, 2000. [19] M. Loog, R.