数据挖掘数据预处理 Data Preprocessing课件.pptVIP

Data Mining:  Concepts and Techniques — Chapter 2 —;Chapter 2: Data Preprocessing;Why Data Preprocessing?;Why Data Preprocessing?;Why Is Data Dirty?;Why Is Data Preprocessing Important?;Major Tasks in Data Preprocessing;Forms of Data Preprocessing ;Chapter 2: Data Preprocessing;Mining Data Descriptive Characteristics;Measuring the Central Tendency;Measuring the Central Tendency; Symmetric vs. Skewed Data;Measuring the Dispersion of Data; Boxplot Analysis;Measuring the Dispersion of Data (cont.);Properties of Normal Distribution Curve;Visualization of Data Dispersion: Boxplot Analysis;Histogram Analysis;Quantile Plot;Quantile-Quantile (Q-Q) Plot;Scatter plot;Loess Curve;Positively and Negatively Correlated Data; Not Correlated Data;Chapter 2: Data Preprocessing;Data Cleaning;How to Handle Missing Data?;How to Handle Noisy Data?;Simple Discretization Methods: Binning;Binning Methods for Data Smoothing;Regression;Cluster Analysis;Chapter 2: Data Preprocessing;Data Integration;Data Integration (cont.);Handling Redundancy in Data Integration;Correlation Analysis (Numerical Data);Correlation Analysis (Categorical Data);Chi-Square Calculation: An Example;Data Transformation;Data Transformation: Normalization;Data Transformation: Normalization;Chapter 2: Data Preprocessing;Data Reduction Strategies;Data Reduction Strategies;Attribute Subset Selection;Heuristic Feature Selection Methods;Example of Decision Tree Induction;Data Compression;Dimensionality Reduction:Wavelet Transformation ;DWT for Image Compression;Given N data vectors from n-dimensions, find k ≤ n orthogonal vectors (principal components) that can be best used to represent data Steps Normalize input data: Each attribute falls within the same range Compute k orthonormal (unit) vectors, i.e., principal components Each input data (vector) is a linear combination of the k principal component vectors The principal components are sorted in order of decreasing “significance” or strength Since the components are