非参数分类方法,分析归纳很到位.pptVIP

Tongji University Tongji University Pattern Recognition Lab 501 Tongji University * Pattern ClassificationPattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley Sons, 2000  Pattern Recognition Lab 501 Tongji University * Nonparametric Technique Outline Density Estimation Parzen Window Pattern Recognition Lab 501 Tongji University * Density Estimation – Overview (1) In probability and statistics, density estimation is the construction of an estimate, based on observed data, of an unobservable underlying probability density function. There are two basic approaches to perform density estimation: Parametric: the density function is assumed (i.e., Gaussian) and the parameters of the function (i.e., mean and variance) are then optimized by fitting the model to the data set Non-parametric: no functional form for the density function is assumed, and the density estimates is driven entirely by the data Pattern Recognition Lab 501 Tongji University * Density Estimation – Overview (2) Binomial distribution Pattern Recognition Lab 501 Tongji University * Density Estimation – Overview (3) When n→∞, the variance is 0, we can use k/n to estimate P. Pattern Recognition Lab 501 Tongji University * Density Estimation – Overview (4) This estimate becomes more accurate as we increase the number of sample points N and shrink the volume V Important condition Pattern Recognition Lab 501 Tongji University * Density Estimation – Overview (5) In practice the value of N (the total number of examples) is fixed In order to improve the accuracy of the estimate p(x) we could let V approach zero, but then the region would then become so small that it would enclose no examples we will have to find a compromise value for the volume V Large enough to include enough examples within Small enough to support the assumption that p(x) is constant within In conclusion, the general expression for non-parametric density estimation becomes Pattern Recognitio