浅探关节镜下盘状半月板损伤的治疗 ...pptVIP

CS276BText Information Retrieval, Mining, and Exploitation Lecture 5 23 January 2003 Recap Today’s topics Feature selection for text classification Measuring classification performance Nearest neighbor categorization Feature Selection: Why? Text collections have a large number of features 10,000 – 1,000,000 unique words – and more Make using a particular classifier feasible Some classifiers can’t deal with 100,000s of feat’s Reduce training time Training time for some methods is quadratic or worse in the number of features (e.g., logistic regression) Improve generalization Eliminate noise features Avoid overfitting Recap: Feature Reduction Standard ways of reducing feature space for text Stemming Laugh, laughs, laughing, laughed - laugh Stop word removal E.g., eliminate all prepositions Conversion to lower case Tokenization Break on all special characters: fire-fighter - fire, fighter Feature Selection Yang and Pedersen 1997 Comparison of different selection criteria DF – document frequency IG – information gain MI – mutual information CHI – chi square Common strategy Compute statistic for each term Keep n terms with highest value of this statistic Information Gain (Pointwise) Mutual Information Chi-Square Document Frequency Number of documents a term occurs in Is sometimes used for eliminating both very frequent and very infrequent terms How is document frequency measure different from the other 3 measures? YangPedersen: Experiments Two classification methods kNN (k nearest neighbors; more later) Linear Least Squares Fit Regression method Collections Reuters-22173 92 categories 16,000 unique terms Ohsumed: subset of medline 14,000 categories 72,000 unique terms Ltc term weighting YangPedersen: Experiments Choose feature set size Preprocess collection, discarding non-selected features / words Apply term weighting - feature vector for each document Train classifier on training set Evaluate classifier on test set Discussion You can eliminate 90% of