Mining Compressed FrequentPattern Sets挖掘压缩频繁模式集.pptVIP

Progressive and Selective Merge: Computing Top-K with ad-hoc Ranking Functions Dong Xin, Jiawei Han, Kevin C.-C. Chang Department of Computer Science University of Illinois at Urbana-Champaign Road Map Introduction Sort-Merge vs. Index-Merge Proposed Solutions Experimental Results Conclusions Motivation: Smart Search in Massive Databases Search is still a problem in massive databases! Challenges on data Gigabytes of data Hundreds of dimensions Challenges on query processing Flood of results: A query may return too many answers Quality of answers: Users are often interested in only a few but high-quality results Efficiency: It is slow to process and return so many answers! Lesson from the Web Search: Ranked Queries Ranked Query: More Example Data and Query Model Data: Relation R Indices are built on attributes B-tree index on single attribute R-tree index on multiple attributes Query with ad hoc Ranking Function F Attributes from multiple indices F is lower-boundable (assume minimal k) Given a domain D One can derive the lower bound value of F(D) The lower bound value need not be tight Example Ranking Functions Typical functions All linear functions: x+y All quadratic functions: (x-100)^2+(y-100)^2 All monotone functions: x*y Some special functions Min square error: (x-y)^2 Median, min, max of input variables Road Map Introduction Sort-Merge vs. Index-Merge Proposed Solutions Experimental Results Conclusions Previous Study: Sort-Merge Our Solution: Index-Merge Search over Joint Space Sort-Merge vs. Index-Merge Road Map Introduction Sort-Merge vs. Index-Merge Proposed Solutions Problem Analysis Progressive Merge Selective Merge Experimental Results Conclusions Challenge 1: Complexity of Join Challenge 2: Empty State Efficient Search Over The Joint Space Straight forward search is expensive Assemble the complete joint space first Search over the joint space Smart search: join as necessary Progressive Merge Identify which joint states have top scores Assemble “