Decision Trees SJSU Computer Science Department决策树州立圣何塞大学计算机科学系.pptVIP

* * * * DECISION TREES Asher Moody, CS 157B Overview Definition Motivation Algorithms ID3 Example Entropy Information Gain Applications Conclusion Decision Tree Decision trees are a fundamental technique used in data mining. Decision trees are used for classification, clustering, feature selection, and prediction. Motivation Decision trees help accurate classify data Decision trees help understand the predictive nature of the data by recognizing patterns Decision trees depict the relationships between input data and target outputs Algorithms Decision trees algorithms are greedy so once test has been selected to partition the data other options will not be explored Popular Algorithms Computer Science: ID3, C4.5, and C5.0 Statistics: Classification and Regression Trees (CART) ID3 Algorithm Given: Examples(S); Target attribute (C); Attributes (R) Initialize Root Function ID3 (S,C,R) Create a Root node for the tree IF S = empty, return a single node with value Failure; IF S = C, return a single node C; IF R = empty, return a single node with most frequent target attribute (C); ELSE BEGIN… (next slide) ID3 (cont) BEGIN Let D be the attribute with largest Gain Radio (D, S) among attributes in R; Let {dj | j = 1, 2, …, n} be the values of attribute D; Let {Sj | j = 1, 2, …, n} be the subsets of S consisting respectively of records with value dj for attribute D; Return a tree with root labeled D arcs d1, d2, …, dn going respectively to the trees; For each branch in the tree IF S = empty, add a new branch with most frequent C; ELSE ID3 (S1, C, R – {D}), ID3 (S2, C, R – {D}), …, IDC(Sn, C, R – {D}) END ID3 Return Root Example 1 Example 2 Entropy Entropy gives us a measure of how uncertain we are about the data Maximum: The measure should be maximal if all the outcomes are equally likely (uncertainty is highest when all possible events are equiprobable). where Pi is the proportion of instances in the dataset that take the ith value of the target attribute Informat