WJ-CH09-模式识别-系统评价 英文版 教学教案.ppt

* * * 王杰(博士/教授/博导) 郑州大学电气工程学院 0371 wj@zzu.edu.cn 模式识别 Pattern Recognition Chapter 9 SYSTEM EVALUATION The goal is to estimate the error probability of the designed classification system Error Counting Technique Let classes Let data points in class for testing.  the number of test points. Let Pi the probability error for class ωi The classifier is assumed to have been designed using another independent data set Assuming that the feature vectors in the test data set are independent, the probability of ki vectors from ωi being in error is SYSTEM EVALUATION Since Pi’s are not known, estimate Pi by maximizing the above binomial distribution. It turns out that Thus, count the errors and divide by the total number of test points in class Total probability of error Statistical Properties Thus the estimator is unbiased but only asymptotically consistent. Hence for small N, may not be reliable A theoretically derived estimate of a sufficient number N of the test data set is Exploiting the finite size of the data set. Resubstitution method: Use the same data for training and testing. It underestimates the error. The estimate improves for large N and large ratios. Holdout Method: Given N divide it into: N1: training points N2: test points N=N1+N2 Problem: Less data both for training and test * * * * *