Chapter 8. 分类与预测-1.ppt

* * 可伸缩的决策树归纳方法 SLIQ (EDBT’96 — Mehta et al.) 为每个属性建立索引，仅仅类列表和当前使用的属性列表驻留在内存 SPRINT (VLDB’96 — J. Shafer et al.) 创建一个属性列表数据结构 PUBLIC (VLDB’98 — Rastogi Shim) 集成树分裂与树剪枝: 停止更早增长的树 RainForest (VLDB’98 — Gehrke, Ramakrishnan Ganti) 维护一个 AVC-list (attribute, value, class label) BOAT (PODS’99 — Gehrke, Ganti, Ramakrishnan Loh) 使用”自助法”创建一些较小的样本 * * * * SGI/MineSet 3.0中决策树可视化 * * Interactive Visual Mining by Perception-Based Classification (PBC) * * 主要内容 基本概念 决策树归纳 贝叶斯分类方法 基于规则的分类 支持向量机 模型评估与选择 提高分类准确率的方法 小结 * * 为什么要进行贝叶斯分类? * * 贝叶斯定理 Given training data X, posteriori probability of a hypothesis H, P(H|X), follows the Bayes theorem Informally, this can be written as posteriori = likelihood x prior/evidence Predicts X belongs to C2 iff the probability P(Ci|X) is the highest among all the P(Ck|X) for all the k classes 实际困难: 需要许多概率地初始值, 显著的计算代价 * * 朴素贝叶斯分类 设 D 是训练元组和相关联地类标号地集合。 每个元组用一个 维向量 X = (x1, x2, …, xn)表示。 假定有 m 个类C1, C2, …, Cm. 分类器将预测X属于具有最高后验概率 P(Ci|X)的类 根据贝叶斯定理 由于P(X) 是常数。因此, 仅 最大化 * * 朴素贝叶斯分类 朴素假定: 属性是条件独立的 可大大降低计算代价: 只需计算类分布 如果Ak 是分类的, P(xk|Ci) is the # of tuples in Ci having value xk for Ak divided by |Ci, D| (# of tuples of Ci in D) If Ak is continous-valued, P(xk|Ci) is usually computed based on Gaussian distribution with a mean μ and standard deviation σ and P(xk|Ci) is * * 朴素贝叶斯分类: 训练数据集 类: C1:buys_computer = ‘yes’ C2:buys_computer = ‘no’ 数据样本 X = (age =30, Income = medium, Student = yes Credit_rating = Fair) * * 朴素贝叶斯分类: 实例 P(Ci): P(buys_computer = “yes”) = 9/14 = 0.643 P(buys_computer = “no”) = 5/14= 0.357 对每类计算 P(X|Ci) P(age = “=30” | buys_computer = “yes”) = 2/9 = 0.222 P(age = “= 30” | buys_computer = “no”) = 3/5 = 0.6 P(income = “medium” | buys_computer = “yes”) = 4/9 = 0.444 P(income = “medium” | buys_computer = “no”) = 2/5 = 0.4 P(student = “yes” | buys_computer = “yes) = 6/9 = 0.667 P(student = “yes” | buys_computer = “no”) = 1/5