[工学]神经网络及其集成.ppt

(C) Vipin Kumar, Parallel Issues in Data Mining, VECPAR 2002 (C) Vipin Kumar, Parallel Issues in Data Mining, VECPAR 2002 Feedforward Neural Networks And Its Ensemble 例子 神经元与神经网络 神经元与神经网络 神经元的数学模型 Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) Model is an assembly of inter-connected nodes and weighted links Output node sums up each of its input value according to the weights of its links Compare output node against some threshold t General Structure of ANN Algorithm for learning ANN Initialize the weights (w0, w1, …, wk) Adjust the weights in such a way that the output of ANN is consistent with class labels of training examples Objective function: Find the weights wi’s that minimize the above objective function e.g., backpropagation algorithm (see lecture notes) Back Propagation Algorithm (BP) 神经元 Back Propagation Algorithm (BP) 记 ， 为加到单元 前输入的个数，则 ，求 对 的梯度 Back Propagation Algorithm (BP) （1）单元 是一个输出单元 Back Propagation Algorithm (BP) 由图2.7，当 为输出单元时，有 Back Propagation Algorithm (BP) 总结以上推导可写为 Back Propagation Algorithm (BP) 反向传播算法的步骤可归纳如下。 Back Propagation Algorithm (BP) Ensemble Methods Construct a set of classifiers from the training data Predict class label of previously unseen records by aggregating predictions made by multiple classifiers General Idea Bagging 基本思路：利用不同的样本集合训练分量分类器 令原始数据集为D (n个样本)，步骤如下： For i=1:L 1. 从D中独立随机抽取m个数据(mn)，形成数据集Si 2. 用Si训练得到一个分类器 End 3. 所有分类器的判决结果投票决定最终的分类判决 如果某种训练算法对于训练数据较为敏感（即训练数据的较小变化，会导致分类器的显著改变），则Bagging可减小这一特性可能造成的负面效果。 装袋算法 1：设k为自助样本集的数目。 2：for i=1 to k do 3： 生成一个大小为N的自助样本集Di。 4： 在自助样本集Di上训练一个基分类器Ci。 5：end for 6： {如果参数为真则δ(·)=1，否则δ(·)=0} Boosting An iterative procedure to adaptively change distribution of training data by focusing more on previously misclassified records Initially, all N records are assigned equal weights Unlike bagging, weights may change at the end of boosting round arcing