How slow is the k -means method.pptVIP

How slow is the k-means method? David Arthur Sergei Vassilvitskii Stanford University The k-means Problem Given an integer k and n data points in Rd Partition points into k clusters Choose k centers and partition points according to closest center Try to minimize φ = ∑ ||x – c(x)||2 Lloyd’s Algorithm (1982) Simply called the “k-means method” Choose k starting centers Uniformly at random usually Repeat until stable: Assign each point to the closest center Set each center to be center of mass of points assigned to it Example About k-means It always terminates Each step decreases φ At most kn configurations It can stop with arbitrarily bad clusterings About k-means Widely used because it is fast Usually far fewer than n iterations How do you formalize this? Just look at worst-case performance? k-means (Worst case # iterations) Counting number of configurations: Already showed: O(kn) Inaba et al. (SOCG 94): O(nkd) One dimension: Dasgupta (COLT 03): Ω(n) Har-Peled, Sadri (SODA 05): O(nΔ2) Δ = ratio of largest distance to smallest Our Main Result Worst case = 2Ω(√n) k-means is superpolynomial! Proof: High Level Start with configuration M with n points, which requires T iterations Add O(1) clusters, O(k) points These reset initial configuration M M stabilizes to M’ New clusters, points reset M’ to M M now has to stabilize to M’ again Now requires at least 2T iterations Proof: High Level Repeat reset construction m times: O(m2) points O(m) clusters 2m iterations Main Construction (Overview) Main Construction (Overview) Main Construction (Overview) Main Construction (Zoomed in) Main Construction (t=0) Main Construction (t=0…T) Main Construction (t=T+1)Reassigning points to clusters Main Construction (t=T+1)Reassigning points to clusters Main Construction (t=T+1)Reassigning points to clusters Main Construction (t=T+1)Recomputing centers Main Construction (t=T+2)Reassigning points to clusters Main Construction (t=T+2)Reassigning points to clusters Main Const