Data Discretization Unification Kent数据离散化统一肯特.ppt

Data Discretization Unification Ruoming Jin Yuri Breitbart Chibuike Muoh Kent State University, Kent, USA Outline Motivation Problem Statement Prior Art Our Contributions Goodness Function Definition Unification Parameterized Discretization Discretization Algorithm Its Performance Motivation Motivation Possible implication of the table: If a person is between 18 and 25, the probability of procedure success is ‘much higher’ than if the person is between 45 and 55 Is that a good rule or this one is better: If a person is between 18 and 30, the probability of procedure success is ‘much higher’ than if the person is between 46 and 61 What is the best interval? Motivation Without data discretization some rules would be difficult to establish. Several existing data mining systems cannot handle continuous variables without discretization. Data discretization significantly improves the quality of the discovered knowledge. New methods of discretization needed for tables with rare events. Data discretization significantly improves the performance of data mining algorithms. Some studies reported ten fold increase in performance. However: Any discretization process generally leads to a loss of information. Minimizing such a possible loss is the mark of good discretization method. Problem Statement Problem Statement Prior Art Unsupervised Discretization – no class information is provided: Equal-width Equal-frequency Supervised Discretization – class information is provided with each attribute value: MDLP Pearson’s X2 or Wilks’ G2 statistic based methods Dougherty, Kohavi (1995) compare unsupervised and supervised methods of Holte (1993) and entropy based methods by Fayyad and Irani (1993) and conclude that supervised methods give less classification errors than the unsupervised ones and supervised methods based on entropy are better than other supervised methods . Prior Art There are several recent (2003-2006) papers introduced