Data Mining - University of Washington数据挖掘-华盛顿大学.ppt

5/19/99 I don’t need a title slide for a lecture Long long ago, in a galaxy far, far away… Outline Background Data mining Association Rules Classification Clustering Sequential Patterns Sequence Similarity Knowledge Discovery in Databases (KDD) What is it? Finding useful patterns in data Why do we need it? Terabytes of data Impractical to manually search for patterns Where does data mining come in? Steps of a KDD process Learn the application domain Create a target dataset Clean and preprocess data Choose type of data mining Pick an algorithm Perform data mining Interpret results Databases vs.Data warehousing Databases vs.Data warehouses Databases provide for: Queries over current data Persistent storage Atomic updates Data warehouses provide for: Storage of all data Meta data Data cleaning, integration Fast access to data Who’s interested? Databases - large amounts of data Artificial Intelligence - search, planning, machine learning Information Retrieval - searching for similar documents Image Processing - finding similar images Types of data mining Association Rules Classification Clustering Sequential Patterns Sequence Similarity Association rules What are they? Looking for common causal relationships in basket data Where are they used? Store layout Catalog design Customer segmentation Association rules example Association rules metrics For a rule a ?b support = a and b occur together in at least s% of the n baskets confidence = of all of the baskets containing a, at least c% also contain b Association rules algorithms Focus on finding support for “itemsets” The na?ve method: Combine itemsets of size k-1 that differ only on the last item to find Candidatesk Measure support of itemsets from step 1 to form large itemsetk Increase k and repeat until no new large itemsets Itemsets of size 1 Finding candidate set 2 Finding candidate set 3 Apriori algorithm An itemset cannot be a large itemset unless all of its subsets are large itemsets Reduces number of