web数据挖掘稿件CS五八三supervisedlearning文件材料.ppt

CS583, Bing Liu, UIC Mixture model A mixture model models the data with a number of statistical distributions. Intuitively, each distribution corresponds to a data cluster and the parameters of the distribution provide a description of the corresponding cluster. Each distribution in a mixture model is also called a mixture component. The distribution/component can be of any kind An example The figure shows a plot of the probability density function of a 1-dimensional data set (with two classes) generated by a mixture of two Gaussian distributions, one per class, whose parameters (denoted by ?i) are the mean (?i) and the standard deviation (?i), i.e., ?i = (?i, ?i). class 1 class 2 Mixture model (cont …) Let the number of mixture components (or distributions) in a mixture model be K. Let the jth distribution have the parameters ?j. Let ? be the set of parameters of all components, ? = {?1, ?2, …, ?K, ?1, ?2, …, ?K}, where ?j is the mixture weight (or mixture probability) of the mixture component j and ?j is the parameters of component j. How does the model generate documents? Document generation Due to one-to-one correspondence, each class corresponds to a mixture component. The mixture weights are class prior probabilities, i.e., ?j = Pr(cj|?). The mixture model generates each document di by: first selecting a mixture component (or class) according to class prior probabilities (i.e., mixture weights), ?j = Pr(cj|?). then having this selected mixture component (cj) generate a document di according to its parameters, with distribution Pr(di|cj; ?) or more precisely Pr(di|cj; ?j). (23) Model text documents The na?ve Bayesian classification treats each document as a “bag of words”. The generative model makes the following further assumptions: Words of a document are generated independently of context given the class label. The familiar na?ve Bayes assumption used before. The probability of a word is independent