数据挖掘概念与技术 CHAPTER6-分类 Class Advanced.ppt

后向传播算法 多类分类 分类时设计多个类别 (i.e., 2 Classes) Method 1. One-vs.-all (OVA): 每次学习一个分类器 给定m个类, 训练m个分类其，每个类别一个 分类器 j: 把类别 j的元组定义为 positive 其他的为 negative 为分类样本 X, 所有分类器投票来集成 Method 2. All-vs.-all (AVA): 为每一对类别学习一个分类器 Given m classes, construct m(m-1)/2 binary classifiers 使用两个类别的元组训练一个分类器 为分类元组X, 每个分类器投票. X is assigned to the class with maximal vote Comparison All-vs.-all tends to be superior to one-vs.-all Problem: Binary classifier is sensitive to errors, and errors affect vote count 多类分类的Error-Correcting Codes 最初目的是在数据传输的通讯任务中通过探索数据冗余来修正误差。例: A 7-bit codeword associated with classes 1-4 半监督分类 Semi-supervised: 使用有标签和无标签数据构造分类器 Self-training: Build a classifier using the labeled data Use it to label the unlabeled data, and those with the most confident label prediction are added to the set of labeled data 重复以上过程 Adv: 容易理解; disadv: 可能增大误差 Co-training: Use two or more classifiers to teach each other 每个学习者使用元组的相互独立的特征集合来训练一个好的分类器F1 然后 f1 and f2 用来预测未知元组 X 的类别标签 Teach each other: The tuple having the most confident prediction from f1 is added to the set of labeled data for f2, vice versa Other methods, e.g., joint probability distribution of features and labels 主动学习Active Learning 获取类标签是昂贵 Active learner: query human (oracle) for labels Pool-based approach: Uses a pool of unlabeled data L: D中有标签的样本子集, U: D的一个未标记数据集 使用一个查询函数小心地从U选择1或多个元组，并咨询标签an oracle (a human annotator) The newly labeled samples are added to L, and learn a model Goal: Achieve high accuracy using as few labeled data as possible Evaluated using learning curves: Accuracy as a function of the number of instances queried (# of tuples to be queried should be small) Research issue: How to choose the data tuples to be queried? Uncertainty sampling: choose the least certain ones Reduce version space, the subset of hypotheses consistent w. the training data Reduce expected entropy over U: Find the greatest reduction in the total number of incorrect predictions 迁移学习：概念框架 Transfer learning: Extract knowledge from one