Selectivity Estimation For Boolean Queries培训教程文件.pptVIP

PODS 2000, 5/17/00 ATT Labs Selectivity Estimation For Boolean Queries Zhiyuan Chen (Speaker) Flip Korn Nick Koudas S.Muthukrishnan Motivation(2) Use estimates for. Query optimization. Best Filtering order. Interactive query refinement. Hard to write a query having 1- 20 answers. Ranking approach does not always work and expensive. Estimate - refine query - ... - exact answer Outline Problem Definition Related work Our approach Experiments Conclusions Problem Definition Substring predicate ?(s) is true iff string s contains ? as substring. Boolean queries: Substring predicates concatenated with AND, OR, NOT. For a string set S, a Boolean query q: Selectivity P(q) is the fraction of strings in S that satisfy q. Related Work (1) Histograms: Not suitable for strings Selectivity of adjacent substrings often differs a lot! Related Work (2) Existing work for substring queries. Conjunction-only queries: KVI96, WVI97, JNS99, JKNS99 Use Previous Approach? Set-Oriented Approach - Store Correlation Implicitly Set-Hashing Approach  A Monte Carlo technique(Cohen94,Broder98) Implementation Issues Approximate permutations: Use a set of independent hash functions. Pick the minimal hash images as signature components. Sig(A) = min{h(x)| x in A}. Signature of unions: Sig(peanut ? butter) Pair-wise min of Sig(p) and Sig(b). Algorithm Outline - No Pruning With negations. | (Peanut ? butter)? ? sandwich | = | (p ^ ? s) ? ( b ? ? s) | (Convert to DNF) = |p ? ? s| + | b ? ? s| - |p ? b ? ? s| (Eliminate disjunction by set-inclusion-exclusion) =|p| - |p ? s| + |b| - | b ? s| - |p ? b| + |p ? b ? s| (Eliminate negations) Complexity Theorem: Preprocessing: building tree and signatures O(signature length * database size) time and space. Online estimate: O(2O(L)), L is the query length. Online time only related to query length. L is small in real life. Below 1 millisecond in experiments. Experiments - Setup Data set: real ATT data - service description. 130 K strin