模型推理与平均.ppt

观测数据 yj 及未观测数据 ，那么完全数据是： 完全数据的似然函数： 完全数据的对数似然函数： 2 EM算法的E步：确定Q函数 这里需要计算： 3 确定EM算法的M步，求新一轮迭代的模型参数： 4 重复以上计算，直到对数似然函数值不再有明显变化为止。 Thanks！ * Non-parametric model is sampling with replacment Parametric: assume a generating model, but what are model parameters 第八章 模型推理与平均8. Model Inference and Average 8.1 概述 模型的拟合（学习）： 回归：最小化平方和 分类：最小化交叉熵 实际上，这两种方法都是最大似然方法拟合的实例 本章的主要内容： 模型的推理 最大似然方法；用于推理的贝叶斯方法；自助法 以及这三种推理方法的关系 模型的平均和提高(improvement) Committee methods, bagging, stacking, and bunping 8.1 概述：基本概念 Statistical Inference Using data to infer the distribution that generated the data Observed data: We want to infer (or estimate or learn) F or some feature of F such as its mean. Statistical Model A set of distributions ( or a set of densities) Parametric model Non parametric model 8.1 概述：基本概念 Parametric model A set that can be parameterized by a finite number of parameters E.g. Assume the data come from a normal distribution, the model is: A parametric model takes the form: Non-parametric model A set that cannot be parameterized by a finite number of parameters E.g. Assume the data comes from 8.1 概述：基本概念 Probability density function, PDF, f(x): Cumulative density function, CDF, F(x): 8.1 概述：本章主要内容 Model Inference Maximum likelihood inference (8.2.2) EM Algorithm (8.5) Bayesian inference (8.3) Gibbs Sampling (8.6) Bootstrap (8.2.1,8.2.3,8.4) Model Averaging and improvement Bagging (8.7) Bumping (8.9) A Smoothing Example Training data, Z = {z1, z2, …, zn}, with zi = (xi , yi), xi is a one-dimensional input yi is the output N = 50 points We decide to fit a cubic spline to the data, with three knots placed at the quartiles of the X values. 8.2 The Bootstrap and Maximum Likelihood Methods The usual estimate of β, obtained by minimizing the squared error over the training set, is given by: The estimated covariance matrix of is: The standard error of a predicti