人工智能 机器学习.pptVIP

* * * * * * * * * * * * * * * * * * * * * shape circle square triangle Top-Down Decision Tree Induction Recursively build a tree top-down by divide and conquer. big, red, circle: + small, red, circle: + small, red, square: ? big, blue, circle: ? big, red, circle: + small, red, circle: + small, red, square: ? color red blue green big, red, circle: + small, red, circle: + pos small, red, square: ? neg pos big, blue, circle: ? neg neg * Decision Tree Induction Pseudocode DTree(examples, features) returns a tree If all examples are in one category, return a leaf node with that category label. Else if the set of features is empty, return a leaf node with the category label that is the most common in examples. Else pick a feature F and create a node R for it For each possible value vi of F: Let examplesi be the subset of examples that have value vi for F Add an out-going edge E to node R labeled with the value vi. If examplesi is empty then attach a leaf node to edge E labeled with the category that is the most common in examples. else call DTree(examplesi , features – {F}) and attach the resulting tree as the subtree under edge E. Return the subtree rooted at R. * Picking a Good Split Feature Goal is to have the resulting tree be as small as possible, per Occam’s razor. Finding a minimal decision tree (nodes, leaves, or depth) is an NP-hard optimization problem. Top-down divide-and-conquer method does a greedy search for a simple tree but does not guarantee to find the smallest. General lesson in ML: “Greed is good.” Want to pick a feature that creates subsets of examples that are relatively “pure” in a single class so they are “closer” to being leaf nodes. There are a variety of heuristics for picking a good test, a popular one is based on information gain that orig