数据挖掘-Pattern_Discovery_Lecture_5.6.pdf

Session 6: Mining Colossal Patterns Mining Long Patterns: Challenges  Mining long patterns is needed in bioinformatics, social network analysis, software engineering, …  But the methods introduced so far mine only short patterns (e.g., length 10)  Challenges of mining long patterns  The curse of “downward closure” property of frequent patterns  Any sub-pattern of a frequent pattern is frequent  If {a , a , …, a } is frequent, then {a }, {a }, …, {a }, {a , a }, {a , a }, …, {a , a }, 1 2 100 1 2 100 1 2 1 3 1 100 {a , a , a }, … are all frequent! There are about 2100 such frequent itemsets! 1 2 3  No matter searching in breadth-first (e.g., Apriori) or depth-first (e.g., FPgrowth), if we still adopt the “small to large” paradigm, we have to examine so many patterns, which leads to combinatorial explosion! 2 Colossal Patterns: A Motivating Example T1 = 2 3 4 ….. 39 40  Let min-support σ= 20 40 T2 = 1 3 4 ….. 39 40      # of closed/maximal patterns of size 20: 20  : .   : .  But there is only one pattern with size close to 40 : . (i.e., long or colossal) : . T40 =1 2 3 4 …… 39  α= {41,42,…,79} of size 39 T41 = 41 42 43 ….. 79  Q: How to find it without generating