决策树你过拟合问题分析.docx

决策树过拟合问题研究 决策树过拟合问题研究 摘 要 数据库知识发现是(Knowledge Discovery in Databases，简称KDD)是当 前涉及人工智能和数据库等学科的一门相当活跃的研究领域，分类是其中的一 个重要研究方向。决策树是分类中常用的模型之一，自1966年被提出以来已经 得到了广泛的研究和应用。然而，由于种类偏见，过拟合等问题，使决策树优 化成为研究人员关注的热点。 本文基于针对可疑实例分析以及结点纯度差变化趋势两个方面分别对决策 树构造算法中的过拟合问题处理展开研究，主要工作如下： 1．综述并分析了现有决策树经典算法及主要优化算法。 2．提出了基于可疑实例影响度分析的改进的C4．5rules算法，将可疑实例 进行有效划分，并计算其全局影响度大小，使得分类规则能有效避开可疑实例 而更加正确的反应数据的真实情况。 3．针对传统决策树过拟合现象普遍且大多数预剪枝算法严重依赖领域知识 的问题，提出基于结点纯度差(PDN，Purity Distance ofNode)变化趋势的决策树优 化算法，通过跟踪相邻父子结点问的最大纯度差变化趋势，判定停止建树的时 机，可以独立于领域知识实现有效的预剪枝并很好地控制了过拟合的发生，同 时大大减小了决策树规模。 一 4．基于上述研究，实现了原型系统，从理论和实验上证明了所提出的算法 的正确性和有效性。 关键词：知识发现：分类；可疑实例；结点纯度；过拟合： Overfitting Overfitting Problem Researching on Decision Tree Abstract Knowledge Discovery in Databases(KDD)is active research domain nowadays，and it is related to few subjects such as artificial intelligence and database．Classification is important research field in KDD．Decision tree is one of the models that are often used in classification，and it has been widely researched and applied since it was proposed in 1 966． However,decision tree has some disadvantages such as variety bias，lack of anti·noise capability，etc，and optimization of decision tree has become research hotspot． The dissertation focuses suspect instances analysis and prutity distance of node two aspects，and the main achievements are follows： 1．An overview and analysis of classical and optimized decision tree algorithms is put forward． 2．The Improved C4．5rules Algorithm Based On Impact·Measurement Of The Suspect Instances，devide the suspect instances from the original data effectively and compute their impact-measurements by the information gaines of it’S attributes。that based on the forward works classification rules can avoid the suspect instances effectively and perform the ture situation of the data． 3．According to the problems that over—fitting is serious and pre-pruning depend on the field knowledge of traditional decision tree algorithm，Decision Tree Pre-Pruning Based PDN Trend algorithm i