Bayesian Hierarchical Clustering - Welcome to Duke ECE Duke 贝叶斯聚类-欢迎来到杜克ECE公爵.pptVIP

Bayesian Hierarchical Clustering Paper by K. Heller and Z. Ghahramani ICML 2005 Presented by David Williams Paper Discussion Group (10.07.05) Outline Traditional Hierarchical Clustering Bayesian Hierarchical Clustering Algorithm Results Potential Application Hierarchical Clustering Given a set of data points, output is a tree Leaves are the data points Internal nodes are nested clusters Examples Evolutionary tree of living organisms Internet newsgroups Newswire documents Traditional Hierarchical Clustering Bottom-up agglomerative algorithm Begin with each data point in own cluster Iteratively merge two “closest” clusters Stop when have single cluster Closeness based on given distance measure (e.g., Euclidean distance between cluster means) Limitations No guide to choosing “correct” number of clusters, or where to prune tree Distance metric selection (especially for data such as images or sequences) How to evaluate how good result is, how to compare to other models, how to make predictions and cluster new data with existing hierarchy Bayesian Hierarchical Clustering (BHC) Basic idea: Use marginal likelihoods to decide which clusters to merge Asks what the probability is that all the data in a potential merge were generated from the same mixture component. Compare to exponentially many hypotheses at lower levels of the tree Generative model used is a Dirichlet Process Mixture Model (DPM) BHC Algorithm Overview One-pass, bottom-up method Initializes each data point in own cluster, and iteratively merges pairs of clusters Uses a statistical hypothesis test to choose which clusters to merge At each stage, algorithm considers merging all pairs of existing trees BHC Algorithm: Merging Two hypotheses compared 1. all data in the pair of trees to be merged was generated i.i.d. from the same probabilistic model with unknown parameters: (e.g., a Gaussian) 2. said data has two or more clusters in it Hypothesis H1 Probability of the data under H1: Prior over the pa