[工学]Data Mining - Concepts and Techniques CH03.ppt

* * * * * Data Mining: Concepts and Techniques * Dimensionality Reduction(维规约) Feature selection (i.e., attribute subset selection): Select a minimum set of features such that the probability distribution of different classes given the values for those features is as close as possible to the original distribution given the values of all features reduce # of patterns in the patterns, easier to understand Heuristic methods (due to exponential # of choices): step-wise forward selection （逐步向前选择） step-wise backward elimination （逐步向后删除） combining forward selection and backward elimination decision-tree induction （决策树归纳） * Data Mining: Concepts and Techniques * Example of Decision Tree Induction Initial attribute set: {A1, A2, A3, A4, A5, A6} A4 ? A1? A6? Class 1 Class 2 Class 1 Class 2 Reduced attribute set: {A1, A4, A6} * Data Mining: Concepts and Techniques * * Data Mining: Concepts and Techniques * Relevance Measures Quantitative relevance measure determines the classifying power of an attribute within a set of data. Methods information gain (ID3) gain ratio (C4.5) gini index ?2 contingency table statistics uncertainty coefficient * Data Mining: Concepts and Techniques * Information-Theoretic Approach Decision tree each internal node tests an attribute each branch corresponds to attribute value each leaf node assigns a classification ID3 algorithm build decision tree based on training objects with known class labels to classify testing objects rank attributes with information gain measure minimal height the least number of tests to classify an object See example * Data Mining: Concepts and Techniques * Top-Down Induction of Decision Tree Attributes = {Outlook, Temperature, Humidity, Wind} Outlook Humidity Wind sunny rain overcast yes no yes high normal no strong weak yes PlayTennis = {yes, no} * Data Mining: Concepts and Techniques * Entropy and Information Gain S contains si tuples of class Ci for i = {1, …, m} Information measures info required to classify a