模式识别英文课件EstimationandLearning.docx

Chapter 5 Estimation andLearning PAGE PAGE # Outline Parametric Estimation Maximum Likelihood Estimation Unbiased Estimates Bayes Estimation Non-Parametric Estimation Histogram Estimation Parzen Window Estimation A^Neares十? Neighbour Estimation Estimation Estimation PAGE PAGE # a. Parameter estimation - assume a known form for the p.d.f. and estimate the necessary parameters (mean and variance for normal b. Density estimation ? estimate the non-parametric pdf Js from the given samples. §5.1 Parametric Estimation §5.1 Parametric Estimation PAGE PAGE # Maximum Likelihood method Parameters are viewed as unknown but fixed values Bavesian method Learning or Maximum u Poerteriori Estimation. Parameters are random variables that have their distributions A conmion assumption ? Gaussian e = (nQ / \ ? Number of Volume^ Squared dimensions of the Malialanobis = ￡ x]cova rianc e dist anc e = ￡ x] d parameters 工二E[(?x -”)厂(x_#) d(d + 1)/2 parameters o 「1 0 .5_ LI = 0 V - 0 1 .3 0 .5 .3 1 Constant density, (x - )厂工(x —)二 C - quadratic surface E - Positive semidefinite ellipsoid Principal axes: eigenvectors of X Length: 丽」A ■ eigenvalues of Z Whitening Transform A = diag^eigval （工）） ①=eigvec （工） Then: W = A -Scaling matrix-Rotation matrix- -Scaling matrix -Rotation matrix -4tUnscales^ and 4unrotate the data 倏 ~N（0J） §5? §5?1.1 Maximum Likelihood Estimation PAGE PAGE # 贸凶10) 贸凶10) Figure 5.1: Given an obsenation 船」he maximum likelihood estimate of parameter 0 is chosen co be that value which maximizes the PDF 卩（壬』0）? Parameters are fixed but unknown. D 三| ? a data set, drawn fromp(x) Notationally, we make density explicitly dependent on parameters: P(x)二 p(“ |) Assuming that the data is drawn independently (i.i.d.): N -a likelihood furictioriue^P(D\e)= Yfp(xf1 -a likelihood furictiori 77=1 To find 0 Maximize Z(0 ) w.r.t. parameters. §5? §5?1.1 Maximum Likelihood Estimation PAGE PAGE # §5? §5?1.1 Maximum Likelihood Estimation PAGE PAGE # Maximizing Z