贝叶斯分类、KNN、特征选择、评估素材.ppt

贝叶斯分类、KNN、特征选择、评估 陈翀 Ref : PengBo@NCIS@PKU, WBIA lecture 本次课大纲 文本分类Text Categorization Problem definition Na?ve Bayes Classifier K-Nearest Neighbor Classifier Evaluation Categorization Definition Given: A description of an instance, x?X, where X is the instance language or instance space. Issue: how to represent text documents. A fixed set of categories: C = {c1, c2,…, cn} Determine: The category of x: c(x)?C, where c(x) is a categorization function whose domain is X and whose range is C. We want to know how to build categorization functions (“classifiers”). Text Categorization Examples Assign labels to each document or web: Labels are most often topics such as Yahoo-categories e.g., finance, sports, newsworldasiabusiness Labels may be genres e.g., editorials movie-reviews news“ Labels may be opinion e.g., “like”, “hate”, “neutral” Labels may be domain-specific binary e.g., interesting-to-me : not-interesting-to-me” e.g., “spam” : “not-spam” e.g., “contains adult language” :“doesn’t” Classification Methods Manual classification Used by Yahoo!, Looksmart, , ODP, Medline Accurate but expensive to scale Automatic document classification Hand-coded rule-based systems Spam mail filter,… Supervised learning of a document-label assignment function No free lunch: requires hand-classified training data Note that many commercial systems use a mixture of methods Na?ve Bayes Bernoulli trial Bernoulli trial is an experiment whose outcome is random and can be either of two possible outcomes, success and failure. Binomial Distribution binomial distribution is the discrete probability distribution of the number of successes in a sequence of n independent yes/no experiments, each of which yields success with probability p. Multinomial Distribution multinomial distribution is a generalization of the binomial distribution. each trial results in one of some fixed finite number k of possible outcomes, with probabilities p1, ..., pk, there are n independent trials. We can use a r