DENCLUE 2DENCLUE Fast Clustering based on Kernel Density.pptVIP

DENCLUE 2.0: Fast Clustering based on Kernel Density Estimation Alexander Hinneburg Martin-Luther-University Halle-Wittenberg, Germany Hans-Henning Gabriel 101tec GmbH, Halle, Germany Overview Density-based clustering and DENCLUE 1.0 Hill climbing as EM-algorithm Identification of local maxima Applications of general EM-acceleration Experiments Density-Based Clustering Assumption clusters are regions of high density in the data space , How to estimate density? parametric models mixture models non-parametric models histogram kernel density estimation Kernel Density Estimation Idea influence of a data point is modeled by a kernel density is the normalized sum of all kernels smoothing parameter h DENCLUE 1.0 Framework Clusters are defined by local maxima of the density estimate find all maxima by hill climbing Problem const. step size Problem of const. Step Size Not efficient many unnecessary small steps Not effective does not converge to a local maximumjust comes close Example New Hill Climbing Approach General approach differentiate density estimate and set to zero no solution, but can be used for iteration New DENCLUE 2.0 Hill Climbing Efficient automatically adjusted step size at no extra costs Effective converges to local maximum (proof follows) Example Proof of Convergence Cast the problem of maximizing kernel denstiy as maximizing the likelihood of a mixture model Proof of Convergence Complete likelihood is maximized by EM-Algorithm this also maximizes the original likelihood, which is the kernel density estimate When starting the EM with we do the hill climbing for Identification of local Maxima EM-Algorithm iterates until reached end point sum of k last step sizes Assumption true local maximum is in a ball of around Points with end points closerbelong to the same maximum M In case of non-unique assignmentdo a few extra EM iterations Acceleration Sparse EM update only the p% points wit