关于互信息学习准则在分类问题中的应用_ccpr2007_40.pdfVIP

胡包钢，王泳 , “关于互信息学习准则在分类问题中的应用”， 《200 年全国模式识别学术会议论文集》. 北京 : 科学出版社 , 200 : 35−45. 关于互信息学习准则在分类问题中的应用 胡包钢，王 泳 中国科学院自动化所模式识别国家重点实验室， 中国科学院北京研究生院 北京 100080 摘 要：不同于传 的基于性能为指标的机器学习方法，基于互信息准则的学习方法显示出了独到的优越 性。但是，如何理解互信息概念在分类问题中的具体内涵与应用特点仍然是不明确的。本文给出了该问题研 究的基本理论公式与方法示意图。并结合我们近期的研究工作，推导了互信息在极值情况下与分类性能关 系的相关定理，以及二值分类情况下的误差敏感度分析方程。文中给出若干简单实例，探讨了互信息概念 在分类问题应用中的基本解释与应用特点。 关键字：熵，互信息，学习准则，分类，机器学习 Applications of Mutual Information Criteria in Classification Problems Bao-Gang Hu, Yong ang National Laboratory of Pattern Recognition, Institute of Automation Beijing Graduate School Chinese Academy of Sciences, Beijing 100080, China Abstract: Diff erent with the conventional learning criteria using p erformance measures, info rmation theory based criteria p resent a unique y et benefi cial fe ature in app lications of machine learning. However, we are still f ar away f rom an in-dep th understanding of the “entropy ” type criteria, say, in relation to the conventional p erf ormance- based criteria. This p ap er p resents the basic f ormulations and schematic diagram of classifi cation learning based on information theory. ith our recent study on binary classifi cation p roblems, we derive two theorems f or the relationship s between mutual inf ormation and the conventional p erformance indexes. A set of sensitivity equations is given f or error analys is to the binary classification p roblems. e also p resent some simp ly examp les and p ersonal viewp oints towards app lyi ng mutual information criteria in comp aring with the conventional indexes. Keywords: entropy, mutual inf ormation, learning criteria, classification, machine learning 1 引言 可以认为学习准则 (也可近似理解为学习指标)的选定应该是机器学