统计机器学习概论机器学习.ppt

The Kernel Trick Recall the SVM optimization problem The data points only appear as inner product As long as we can calculate the inner product in the feature space, we do not need the mapping explicitly Many common geometric operations (angles, distances) can be expressed by inner products Define the kernel function K by * Examples of Kernel Functions Polynomial kernel with degree d Radial basis function kernel with width s Closely related to radial basis function neural networks The feature space is infinite-dimensional Sigmoid with parameter k and q It does not satisfy the Mercer condition on all k and q * * §3.6聚类方法：K-Means Given a set of examples Dn = {z1, z2, · · · , zn} Search for K prototypes μk of disjoint subsets Sk of Dn in order to minimizewhere μk is the mean of the examples in subset Sk : We could use any distance, not just the Euclidean distance... Batch K-Means Initialization: select randomly K examples zj in Dn as initial values of each μk At each batch iteration: For each prototype μk , put in the emptied set Sk the examples of Dn that are closer to μk than to any other μj≠k. Re-compute the value of each μk as the average of the examples in Sk . The algorithm stops when no prototype moves anymore. It can be shown that the K-Means criterion will never increase. * Batch K-Means (图示1) * Batch K-Means (图示2) * Batch K-Means (图示3) * * 四、机器学习的难题与挑战注：以下部分内容引自周志华《机器学习挑战》王珏《机器学习的难题与分析》 机器学习的难题(1) * 维数灾难问题 * 维数灾难问题 * 维数灾难问题 * 维数灾难问题 * * Bayes决策：讨论 基于Bayes决策的最优分类器 Bayes决策的三个前提： 类别数确定 各类的先验概率P(Ci)已知 各类的条件概率密度函数p(x|Ci)已知 问题的转换： 基于样本估计P(Ci)和p(x|Ci) 基于样本直接确定判别函数 ?学习问题 * 三、主要统计学习方法简介 * 统计学习方法 决策树 统计推理 用数据的似然度(likelihood)和假设(Hypothesis)的概率去预测新实例的值 朴素Bayes方法(Na?ve Bayes, NB) 基于实例的学习 最近邻方法(Nearest Neighbor) 神经网络(Neural Networks) 支持向量机(Support Vector Machine) 典型聚类方法：K-Means §3.1 Decision Trees At each step, choose the feature that “reduces entropy” most. Work towards “node purity”. All the data f1 f2 Choose f2 Choose f1 Decision Trees CART (Breiman