WJ-CH06-模式识别-模式产生 英文版 教学文稿.ppt

* * 王杰(博士/教授/博导) 郑州大学电气工程学院 0371 wj@zzu.edu.cn 模式识别 Pattern Recognition Chapter 6 Optimal Feature Generation Optimal Feature Generation In general, feature generation is a problem-dependent task. However, there are a few general directions common in a number of applications. We focus on three such alternatives. Optimized features based on Scatter matrices (Fisher’s linear discrimination). The goal: Given an original set of m measurements , compute , by the linear transformation so that the J3 scattering matrix criterion involving Sw, Sb is maximized. AT is an matrix. The basic steps in the proof: J3 = trace{Sw-1 Sm} Syw = ATSxwA, Syb = ATSxbA, J3(A)=trace{(ATSxwA)-1 (ATSxbA)} Compute A so that J3(A) is maximum. The solution: Let B be the matrix that diagonalizes simultaneously matrices Syw, Syb , i.e: BTSywB = I , BTSybB = D where B is a ?x? matrix and D a ?x? diagonal matrix. Let C=AB an mx? matrix. If A maximizes J3(A) then The above is an eigenvalue-eigenvector problem. For an M-class problem, is of rank M-1. If ?=M-1, choose C to consist of the M-1 eigenvectors, corresponding to the non-zero eigenvalues. The above guarantees maximum J3 value. In this case: J3,x = J3,y. For a two-class problem, this results to the well known Fisher’s linear discriminant For Gaussian classes, this is the optimal Bayesian classifier, with a difference of a threshold value . If ?M-1, choose the ? eigenvectors corresponding to the ? largest eigenvectors. In this case, J3,yJ3,x, that is there is loss of information. Geometric interpretation. The vector is the projection(映射) of onto the subspace spanned by the eigenvectors of . Define The Karhunen – Loève transform minimizes the square error: The error is: It can be also shown that this is the minimum mean square error compared to any other representation of x by an ?-dimensional vector. In