Hashed Samples：散列样品.pptVIP

Hashed Samples Selectivity Estimators for Set Similarity Selection Queries Set Similarity: An Application Find similar strings Decompose strings into 3-grams. Represent strings as sets of 3-grams. Compare strings by comparing their respective 3-gram sets. “Nick Koudas”: { ‘Nic’, ‘ick’, …, ‘das’ } “Nick Arkoudas”: {‘Nic’, …, ‘das’ } We can use TF/IDF similarity (or other metrics) to evaluate set similarity. Indexes For Set similarity Evaluation Current approaches use inverted lists Compute IDF of set elements (e.g., 3-grams). Create one inverted list per set element consisting of one entry per database set containing the respective elements (e.g., one entry per string containing the respective 3-gram). Use various algorithms and sorting/compression schemes for fast merging of inverted lists. Motivation Set similarity queries are very important String matching. Data cleaning. Set-valued attributes in ORDBMS. A variety of set similarity operators have been proposed (for join, selection queries) Selectivity estimation is important for query optimization The Problem Let I be a predefined set similarity measure. Let D a collection of sets. Given query set q and threshold τ, a set similarity selection query returns the answer set A: { s ? D, s.t. I(q, s) τ }. A set similarity selectivity estimation query estimates the size of A. Na?ve Solutions Random Sampling Maintain a sample S, of sets s ? D. Size of answer: |A| = |As|??|D| / |S|, where |As| = {s ? S: I(q, s) τ}. Drawbacks: Query independent. Large variance. Needs to store complete sets in the sample. Cannot handle updates. One Sample Per List Use the existing inverted index Compute one sample per inverted list. Compute independent estimates per list corresponding to the query set elements only (query specific) Report median, max, average… Drawbacks Ignores correlations between lists. Needs to store complete sets in inverted lists. Will not be better than simple random sampling. Cannot handle updates. Sample Unio