06-模式识别-模式产生.ppt

Optimal Feature Generation In general, feature generation is a problem-dependent task. However, there are a few general directions common in a number of applications. We focus on three such alternatives. Optimized features based on Scatter matrices (Fisher’s linear discrimination). The goal: Given an original set of m measurements , compute , by the linear transformation so that the J3 scattering matrix criterion involving Sw, Sb is maximized. AT is an matrix. The basic steps in the proof: J3 = trace{Sw-1 Sm} Syw = ATSxwA, Syb = ATSxbA, J3(A)=trace{(ATSxwA)-1 (ATSxbA)} Compute A so that J3(A) is maximum. The solution: Let B be the matrix that diagonalizes simultaneously matrices Syw, Syb , i.e: BTSywB = I , BTSybB = D where B is a ?x? matrix and D a ?x? diagonal matrix. Let C=AB an mx? matrix. If A maximizes J3(A) then The above is an eigenvalue-eigenvector problem. For an M-class problem, is of rank M-1. If ?=M-1, choose C to consist of the M-1 eigenvectors, corresponding to the non-zero eigenvalues. The above guarantees maximum J3 value. In this case: J3,x = J3,y. For a two-class problem, this results to the well known Fisher’s linear discriminant For Gaussian classes, this is the optimal Bayesian classifier, with a difference of a threshold value . If ?M-1, choose the ? eigenvectors corresponding to the ? largest eigenvectors. In this case, J3,yJ3,x, that is there is loss of information. Geometric interpretation. The vector is the projection(映射) of onto the subspace spanned by the eigenvectors of . Principal Components(主元分析) Analysis (The Karhunen – Loève transform): The goal: Given an original set of m measurements compute for an orthogonal A, so that the elements of are optimally mutually uncorrelated(无互相关). That is Sketch of the proof(框架证明): If A is chosen so that its columns are the orthogonal eigenvectors of Rx, then whe