WJ-CH03-模式识别-线性分类-02 英文版 幻灯片课件.ppt

王杰(博士/教授/博导) 郑州大学电气工程学院 0371 wj@zzu.edu.cn 模式识别 Pattern Recognition Chapter 3(II) Mean square error regression: Let , be jointly distributed random vectors with a joint pdf The goal: Given the value of estimate the value of . In the pattern recognition framework, given one wants to estimate the respective label . The MSE estimate of given is defined as: It turns out that: The above is known as the regression of given and it is, in general, a non-linear function of . If is Gaussian the MSE regressor is linear. SMALL in the sum of error squares sense means that is, the input xi and its corresponding class label (±1). : training pairs Pseudoinverse Matrix Define Thus Assume N=l X square and invertible. Then Pseudoinverse of X The Bias – Variance Dilemma(偏差与方差的选择,p.49) A classifier is a learning machine that tries to predict the class label y of . In practice, a finite data set D is used for its training. Let us write . Observe that: For some training sets, , the training may result to good estimates, for some others the result may be worse(训练结果有好有坏，取决于训练集). The average performance of the classifier can be tested against the MSE optimal value, in the mean squares sense, that is: where ED is the mean over all possible data sets D(可对分类器的平均性能在所有可用训练集上在MSE意义上进行评价). The above is written as: In the above, the first term is the contribution of the bias and the second term is the contribution of the variance上式第一、二项分别为偏差和方差的贡献. For a finite D, there is a trade-off between the two terms. Increasing bias it reduces variance and vice verse. This is known as the bias-variance dilemma(对有限的D,存在偏差与方差的折衷，称为偏差-方差取舍问题). Using a complex model results in low-bias but a high variance, as one changes from one training set to another. Using a s