模式识别课件prch3part1_ding.pptVIP

 Chapter 3:Maximum-Likelihood Bayesian Parameter Estimation (3.1,3.2) Introduction Maximum-Likelihood Estimation The Gaussian Case 1:unknown ? The Gaussian Case 2: unknown ? and ? Bias Appendix: ML Problem Statement 3.1 Introduction Data availability in a Bayesian framework We could design an optimal classifier if we knew: P(?i) (priors) P(x | ?i) (class-conditional densities) Unfortunately, we rarely have both complete information! Design a classifier from a training sample No problem with the estimation of prior probabilities Samples are often too small for the estimation of class-conditional densities Complexity for large dimension of feature space Parameters in ML estimation are fixed but unknown! Best parameters are obtained by maximizing the probability of obtaining the samples observed Bayesian methods view the parameters as random variables having some known prior distribution. Training data allow us to convert a distribution on this variable into a posterior probability density In either approach, we use P(?i | x)for our classification rule! 3.2 Maximum-Likelihood Estimation M-L Estimation Has good convergence properties as the sample size increase Simpler than any other alternative techniques General principle Assume we have c classes and p(x | ?j) ~ N( ?j, ?j) p(x | ?j) ? p (x | ?j, ?j) where: Use the informationprovided by the training samples to estimate D = (D1, D2, …, Dc) ? = (?1, ?2, …, ?c), each ?i (i = 1, 2, …, c) is associated with each category assume Di give no information about ?j if ij So Handle each class separately to simplify our notation Suppose that D contains n samples, x1, x2,…, xn ML estimation of ? is, by definition, the value that maximizes p(D | ?) “It is the value of ? that best agrees with the actually observed training sample” Optimal estimation Let ? = (?1, ?2, …, ?p)t and let ?? be the gradient operator We define l(?) as the log-likelihood function l(?) = ln p(D | ?) New problem st