关联规则挖掘AssociationRuleMining背景简介Motivation超购物.PPT

Interestingness via Unexpectedness Need to model expectation of users (domain knowledge) Need to combine expectation of users with evidence from data (i.e., extracted patterns) + Pattern expected to be frequent - Pattern expected to be infrequent Pattern found to be frequent Pattern found to be infrequent + - Expected Patterns - + Unexpected Patterns Interestingness via Unexpectedness Web Data Domain knowledge in the form of site structure Given an itemset F = {X1, X2, …, Xk} (Xi : Web pages) L: number of links connecting the pages lfactor = L / (k ? k-1) cfactor = 1 (if graph is connected), 0 (disconnected graph) Structure evidence = cfactor ? lfactor Usage evidence Use Dempster-Shafer theory to combine domain knowledge and evidence from data P250 2、3、4、9、11 wm Immediate紧接的 Contingency table 列联表 Rule Generation for Apriori Algorithm Candidate rule is generated by merging two rules that share the same prefixin the rule consequent join(CD=AB,BD=AC)would produce the candidaterule D = ABC Prune rule D=ABC if itssubset AD=BC does not havehigh confidence Effect of Support Distribution Many real data sets have skewed support distribution Support distribution of a retail data set Effect of Support Distribution How to set the appropriate minsup threshold? If minsup is set too high, we could miss itemsets involving interesting rare items (e.g., expensive products) If minsup is set too low, it is computationally expensive and the number of itemsets is very large Using a single minimum support threshold may not be effective Multiple Minimum Support How to apply multiple minimum supports? MS(i): minimum support for item i e.g.: MS(Milk)=5%, MS(Coke) = 3%, MS(Broccoli)=0.1%, MS(Salmon)=0.5% MS({Milk, Broccoli}) = min (MS(Milk), MS(Broccoli)) = 0.1% Challenge: Support is no longer anti-monotone Suppose: Support(Milk, Coke) = 1.5% and Support(Milk, Coke, Broccoli) = 0.5% {Milk,Coke} is infrequent but {Mil