模式识别prch3part1_ding.ppt

 ;Chapter 3:Maximum-Likelihood Bayesian Parameter Estimation (3.1,3.2);Data availability in a Bayesian framework We could design an optimal classifier if we knew: P(?i) (priors) P(x | ?i) (class-conditional densities) Unfortunately, we rarely have both complete information! Design a classifier from a training sample No problem with the estimation of prior probabilities Samples are often too small for the estimation of class-conditional densities Complexity for large dimension of feature space ;Use priori information to simplify above problem Normality of P(x | ?i) P(x | ?i) ~ N( ?i, ?i): Characterized by 2 parameters The problem is changed from estimating P(x | ?i) to estimating ?i, ?i Estimation techniques Maximum-Likelihood (ML) and the Bayesian estimations Results are nearly identical, but the approaches are conceptually different;Parameters in ML estimation are fixed but unknown! Best parameters are obtained by maximizing the probability of obtaining the samples observed Bayesian methods view the parameters as random variables having some known prior distribution. Training data allow us to convert a distribution on this variable into a posterior probability density In either approach, we use P(?i | x)for our classification rule! ;M-L Estimation Has good convergence properties as the sample size increase Simpler than any other alternative techniques General principle Assume we have c classes and p(x | ?j) ~ N( ?j, ?j) p(x | ?j) ? p (x | ?j, ?j) where:;Use the informationprovided by the training samples to estimate D = (D1, D2, …, Dc) ? = (?1, ?2, …, ?c), each ?i (i = 1, 2, …, c) is associated with each category assume Di give no information about ?j if ij So Handle each class separately to simplify our notation Suppose that D contains n samples, x1, x2,…, xn ML estimation of ? is, by definition, the value that maximizes p(D | ?) “It is the value of ? that best agrees with the actually observed training sample” ;2;Optimal estimation Let ? = (?1