模式识别prch4part1_ding.ppt

Pattern ClassificationAll materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley Sons, 2000 with the permission of the authors and the publisher;Chapter 4 (Part 1):Non-Parametric Classification (Sections 4.1-4.3);4.1 Introduction;4.2 Density Estimation;ML estimation of P = ? is reached for Therefore, the ratio k/n is a good estimate for the probability P and hence for the density function p. p(x) is continuous and that the region R is so small that p does not vary significantly within it, we can write: where x is a point within R and V the volume enclosed by R.;Combining equation (1) , (3) and (4) yields: ;Condition for convergence if we fix V, and take more training samples, k/n will converge, but then we have only obtain an estimate of the space-averaged value (P/V) of p(x). p(x) is obtained only if V approaches zero. This is the case where no samples are included in R: it is an uninteresting case! In this case, it is also an uninteresting case! ;The volume V needs to approach 0 anyway if we want to use this estimation Practically, V cannot be allowed to become arbitrarily small since the number of samples is always limited One will have to accept a certain amount of variance in the ratio k/n and a certain amount of averaging of p(x) Theoretically, if an unlimited number of samples is available, we can circumvent this difficulty To estimate the density of x, we form a sequence of regions R1, R2,…containing x: the first region contains one sample, the second two samples and so on. Let Vn be the volume of Rn, kn the number of samples falling in Rn and pn(x) be the nth estimate for p(x): pn(x) = (kn/n)/Vn (7);Three necessary conditions should apply if we want pn(x) to converge to p(x): There are two different ways of obtaining sequences of regions that satisfy these conditions: (a) Shrink an initial region where Vn = 1/?n and s