S-学习一种模式分类的新方法1S-LearningANovelApproachfor.PDFVIP

杨双红，胡包钢：S-学习：一种模式分类的新方法，2007 年全国模式识别学术会议(CCPR2007)， 北京：科学出版社，pp.10-17. （本文获得CAA 最佳论文奖）。 S-学习：一种模式分类的新方法1 杨双红 胡包钢 中国科学院自动化所 模式识别国家重点实验室 北京 100080 摘 要：提出了一种新的分类方法：S-学习。该方法采用最小二乘损失函数阶段式地逼近一个有界且单调非 增的非凸损失函数。与其他分类方法相比，该方法在理论和经验上具有以下优点：1)相比SVM，Logistic 回归，Adaboost ，岭回归等采用凸损失函数的分类器，该方法具有更好的泛化性能和对outlier 的鲁棒性； 2)相比ψ-Learning 等采用非凸损失函数的分类器，该方法在计算代价和分析性能方面有非常明显的优势； 3)该方法每一步均有解析解，计算复杂度和岭回归相当，然而泛化性能远优于岭回归，更重要的是它在核 空间内是自然稀疏的(稀疏度和SVM 相当) 因而具有更好的拓展性(scalability)；4)该方法在不增加分类器复 杂度的条件下可以稳定地增加margin ，表现出极好的渐近一致性。 关键字：机器学习、模式分类、核方法、损失函数、稀疏核学习器 S-Learning: A Novel Approach for Classification Shuang-Hong Yang and Bao-Gang Hu National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100080, China Abstract: This paper presents a new approach for classification, namely S-Learning, which uses a least square loss function within each stage to asymptotically approximate a bounded monotonically-decreasing nonconvex loss function in a stagewise manner. Several benefits are obtained by S-Learning. 1) It can achieve better generalization as well as robustness compared to methods using convex loss functions such as SVM, Logistic Regression, Adaboost, Ridge Regression, etc. 2) Compared with algorithms which use nonconvex loss functions, e.g., ψ-Learning, our approach is far more efficient to compute and also has better theoretical amenability. 3) It is of the same complexity as Ridge Regression, but in addition to the much better generalization performance, it is naturally a sparse kernel machine when the kernel trick is applied and thus has better scalability.4) It can boost the margin without boosting the complexity, leading to exciting potentials of asymptotical consistency. Keywords: