人工智能课件AI2014S_Chap07MixtureModelsandEM幻灯片.pptxVIP

Artificial Intelligence Mixture Models and EMDonghui WangAI Institute@ZJU2014.4ContentsK-means clusteringMixtures of GaussiansAn alternative view of EMThe EM algorithm in generalReferences:Bishop. “Pattern Recognition and Machine Learning”, Chapter 9. 2006.K-means clusteringK-means clusteringSuppose we have a data set {x1, x2, …, xN} in D-dimensional space and these data points have an intrinsic structure of K clusters.We use μk as a prototype associated with the kth cluster.Goal: find an assignment of data points to clusters such that some objective function.K-means clusteringResponsibilitiesPrototypes (expected value )DataK-means algorithm (batch version):Pick number of clusters kRandomly scatter k “cluster centers” in data spaceRepeat:Assign each data point to its closest cluster centerMove each cluster center to the mean of the points assigned to itExample: 5 data points and 3 clustersDistortion measure (responsibilities):K-means clusteringThe procedure of k-means algorithm:K-means clusteringThe convergence of J:K-means clusteringThe nearest prototype to xnOnline k-means algorithm (sequential k-means):K-medoids algorithm:Chooses input data points as centers;Works with an arbitrary matrix of distances between data points instead of Euclidean distance.E.g. Manhattan distance or Minkowski distanceThe applications of K-means algorithmImage segmentation:The applications of K-means algorithmImage clustering and 3D objects clustering:The applications of K-means algorithmNews video clustering and video key frame extraction:The limitation of K-means clusteringThe K-means algorithm often convergence to a local minimum.The limitation of K-means clusteringThe K-means algorithm adopts the hard assignment and doesn’t consider the data density and probabilistic distribution.The limitation of K-means clusteringThe K-means algorithm adopts the hard assignment and doesn’t consider the data density and probabilistic distribution.Mixtures of GaussiansGaussian mixture distributionDe