CLOF：基于组合局部孤立点的噪声处理算法.docVIP

CLOF：基于组合局部孤立点的噪声处理算法 任义丽 ，吴俊杰 ，熊海涛 ，刘辰 北京航空航天大学经济管理学院，北京 100191　 摘要：现实世界的数据往往不可避免地包含噪声。噪声的存在会导致模型失效，进而导致不可 靠的输出，影响企业的经营管理决策。一种噪声处理方法是借鉴孤立点检测技术。LOF算法是 基于密度的典型孤立点识别算法，在实践中被广泛用于检测属性噪声。然而，LOF算法没有综 合考虑类与类、类与整体之间的信息，因而对类别噪声识别欠佳。本文在LOF算法的基础上提 出了一种基于组合局部孤立系数的类别噪声处理算法：CLOF。算法通过lofa、lof1、lof0三个 局部孤立系数值，首先在整体数据集上剔除属性噪声，其次综合利用整体和局部信息，通过三 个lof值之间的关系来识别类别噪声，并更正其类标签。在仿真数据集和UCI数据集上的实验 结果表明，CLOF算法能够同时有效识别类别噪声和属性噪声，提高分类器的分类精度，对具 有类重叠的复杂数据处理效果尤佳。 关键词：模式识别 类别噪声 属性噪声 局部孤立系数 异常点检测 中图分类号： TP181 CLOF: A Noise Removal Algorithm Based on Combined Local Outlier Factors REN Yi-li , WU Jun-jie , XIONG Hai-tao , LIU Chen School of Economics and Management,Beihang University, Beijing 100191, China Abstract: Real-world data is never perfect and often su?er from noises that may a?ect interpretations of the data, the models created from the data, and the decisions made based on the data. A common solution for handling noise is to employ outlier detection techniques. LOF is a well-known and widely used algorithm for outlier detection based on local densities of data. However, it does not perform well on removing class noises since it does not take the information of class labels into consideration. In this paper, we propose a new noise removal algorithm based on Combined Local Outlier Factors: CLOF. Speci?cally, CLOF ?rstly de?nes three local outlier factors, i.e., lofa, lof1 and lof0, and eliminates attribute noises using lofa. Then, CLOF ?nds and corrects the labels of class noises by simultaneously using the three local outlier factors. Experimental results on arti?cial and real-world UCI data sets demonstrate that CLOF can e?ectively identify class noises and attribute noises so as to improve the classi?cation performances of various classi?ers, especially for data sets with 基金项目： 高等学校博士学科点专项科研基金资助课题（20091102120014） 作者简介： 任义丽（1987-），女，硕士，主要研究方向：稀有类分析，email：renyili1987@。 通信作者：吴 俊杰（1979-），男，副教授，主要研究方向：数据挖掘，商务智能，email：wujj@。熊海涛（1982-），男，博士， 主要研究方向：稀有类分析，email：xionghtao@。 刘辰（1989-），男，本科，主要研究方