人工智能与数据挖掘Lect312章节幻灯片.pptVIP

* * * * * * * * * * * AIDM * Chapter 3 Basic Data Mining Techniques 3.1 Decision Trees (For classification) * AIDM * Introduction: Classification—A Two-Step Process 1. Model construction: build a model that can describe a set of predetermined classes Preparation: Each tuple/sample is assumed to belong to a predefined class, labeled by the output attribute or class label attribute This set of examples is used for model construction: training set The model can be represented as classification rules, decision trees, or mathematical formulae Estimate accuracy of the model The known label of test sample is compared with the classified result from the model Accuracy rate is the percentage of testing set samples that are correctly classified by the model Note: Test set is independent of training set, otherwise over-fitting will occur 2. Model usage: use the model to classify future or unknown objects * AIDM * Classification Process (1): Model Construction Training Data Classification Algorithms IF rank = ‘professor’ OR years 6 THEN tenured = ‘yes’ Classifier (Model) Classification Process (2): Use the Model in Prediction Classifier Testing Data Unseen Data (Jeff, Professor, 4) Tenured? * AIDM * 1 Example (1): Training Dataset An example from Quinlan’s ID3 (1986) * AIDM * 1 Example (2): Output: A Decision Tree for “buys_computer” age? overcast student? credit rating? no yes fair excellent =30 40 no no yes yes yes 30..40 * AIDM * 2 Algorithm for Decision Tree Building Basic algorithm (a greedy algorithm) Tree is constructed in a top-down recursive divide-and-conquer manner At start, all the training examples are at the root Attributes are categorical (if continuous-valued, they are discretized in advance) Examples are partitioned recursively based on selected attributes Test attributes are selected on the basis of a heuristic or statistical measure (e.g., information gain) Conditions for stopping partitioning All samples for a given node belong to the same class Th