7class - 电科先知视听觉实验室cvlab@uestc.ppt

X = (age = 30 , income = medium, student = yes, credit_rating = fair) P(X|Ci) : P(X|buys_computer = “yes”) = 0.222 x 0.444 x 0.667 x 0.667 = 0.044 P(X|buys_computer = “no”) = 0.6 x 0.4 x 0.2 x 0.4 = 0.019 P(X|Ci)*P(Ci) : P(X|buys_computer = “yes”) * P(buys_computer = “yes”) = 0.028 P(X|buys_computer = “no”) * P(buys_computer = “no”) = 0.007 Therefore, X belongs to class (“buys_computer = yes”) * * * * * * * * * * * * * * * * * * * 第5讲 分类 分类基本概念 决策树归纳 贝叶斯分类方法 基于规则的分类 模型评估与选择 用于提高分类算法准确率的办法: 集合方法 模型评估与选择 评估度量: 怎么评估准确率? 以及要考虑其它什么度量? 利用validation test set 来评估准确性 用来评估分类器准确率方法: 保持方法Holdout method, 随机子采样 交叉验证 自助法 Bootstrap 比较分类器: 置信区域 Cost-benefit 分析与ROC 曲线 * 分类器评估度量: 混淆矩阵 Actual class\Predicted class buy_computer = yes buy_computer = no Total buy_computer = yes 6954 46 7000 buy_computer = no 412 2588 3000 Total 7366 2634 10000 Given m classes, an entry, CMi,j in a confusion matrix indicates # of tuples in class i that were labeled by the classifier as class j Confusion Matrix: Actual class\Predicted class C1 ? C1 C1 True Positives (TP) False Negatives (FN) ? C1 False Positives (FP) True Negatives (TN) 混淆矩阵例: * 分类器评估度量: 准确率, 错误率,敏感度与 Specificity Classifier Accuracy, or recognition rate: 被准确分类样本百分比 Accuracy = (TP + TN)/All Error rate: 1 – accuracy, or Error rate = (FP + FN)/All 类别平衡问题: 有些类可能很稀少, e.g. fraud, or HIV-positive 负样本重要性，与正样本大多数 Sensitivity灵敏性: 正确识别正元率 Sensitivity = TP/P Specificity特性性: 负样本识别率 Specificity = TN/N A\P C ?C C TP FN P ?C FP TN N P’ N’ All * 分类器评估度量:精度与召回率与F-measures Precision: 正确性 – 被分类器标注为正的实际为正的百分比 Recall: 完整性 – 被分类器分类为正总数百分比 Perfect score is 1.0 F measure (F1 or F-score): harmonic mean of precision and recall, F?: weighted measure of precision and recall assigns ? times as much weight to recall as to precision * Classifier Evaluation Metrics: Example * Precision = 90/230 = 39.13% Recall = 90/300 = 30.00% Actual Class\Predicted class cancer = yes cancer = no Total Recognition(%) cancer = yes 90 210 30