CHAPTER3-数据预处理.pptVIP

* 标称数据的概念分层 用户/专家在模式级显式地指定属性的偏序 streetcitystatecountry 通过显式数据分组说明分层 {厄巴纳，香槟，芝加哥}Illinois 只说明属性集但不说明它们的偏序 系统自动产生属性偏序，根据 每个属性下不同值的数据 启发式规则：相比低层，高层概念的属性通常有较少取值 E.g., street city state country 只说明部分属性值 自动产生概念分层 Some concept hierarchies can be automatically generated based on the analysis of the number of distinct values per attribute in the given data set 含不同值最多的属性放在层次的最低层 Note: Exception—weekday, month, quarter, year country province_or_ state city street 15 distinct values 365 distinct values 3567 distinct values 674,339 distinct values * Summary Data preparation is a big issue for both warehousing and mining Data preparation includes Data cleaning and data integration Data reduction and feature selection Discretization A lot a methods have been developed but still an active area of research * References E. Rahm and H. H. Do. Data Cleaning: Problems and Current Approaches. IEEE Bulletin of the Technical Committee on Data Engineering. Vol.23, No.4 D. P. Ballou and G. K. Tayi. Enhancing data quality in data warehouse environments. Communications of ACM, 42:73-78, 1999. H.V. Jagadish et al., Special Issue on Data Reduction Techniques. Bulletin of the Technical Committee on Data Engineering, 20(4), December 1997. A. Maydanchik, Challenges of Efficient Data Cleansing (DM Review - Data Quality resource portal) D. Pyle. Data Preparation for Data Mining. Morgan Kaufmann, 1999. D. Quass. A Framework for research in Data Cleaning. (Draft 1999) V. Raman and J. Hellerstein. Potters Wheel: An Interactive Framework for Data Cleaning and Transformation, VLDB’2001. T. Redman. Data Quality: Management and Technology. Bantam Books, New York, 1992. Y. Wand and R. Wang. Anchoring data quality dimensions ontological foundations. Communications of ACM, 39:86-95, 1996. R. Wang, V. Storey, and C. Firth. A framework for analysis of data quality research. IEEE Trans. Knowledge and Data Engineering, 7:623-640, 1995. /classes/spring01/cs240b/notes/data-integration1.pdf 第一次作业 第二章：习题2.4(P53)