常见的人工神经网络.ppt

Hopfield网络的能量函数 With input Without input Hopfield 模型 Hopfield证明了异步Hopfield网络是稳定的，其中权值定义为 Whatever be the initial state of the network, the energy decreases continuously with time until the system settles down into any local minimum of the energy surface. Hopfield 网络: 联想记忆 Hopfield网络的一个主要应用 基于与数据部分相似的输入，可以回想起数据本身（attractor state） 也称作内容寻址记忆(content-addressable memory). Stored Pattern Memory Association 虞台文, Feedback Networksand Associative Memories Hopfield 网络: Associative Memories Stored Pattern Memory Association 虞台文, Feedback Networksand Associative Memories Hopfield网络的一个主要应用 基于与数据部分相似的输入，可以回想起数据本身（attractor state） 也称作内容寻址记忆(content-addressable memory). How to store patterns? =? How to store patterns? =? : Dimension of the stored pattern 权值确定: 外积(Outer Product) Vector form: Element form: Why? Satisfy the Hopfield model An example of Hopfield memory 虞台文, Feedback Networks and Associative Memories 判别规则 输入空间中 样本是空间中的一个点 权向量是一个超平面 超平面一边对应 Y=1 另一边对应 Y=-1 单层感知机学习 调整权值，减少训练集上的误差 简单的权值更新规则: 初始化 对每一个训练样本: Classify with current weights If correct, no change! If wrong: adjust the weight vector * 学习: Binary Perceptron 初始化 对每一个训练样本: Classify with current weights If correct (i.e., y=y*), no change! If wrong: adjust the weight vector by adding or subtracting the feature vector. Subtract if y* is -1. 多类判别情况 If we have multiple classes: A weight vector for each class: Score (activation) of a class y: Prediction highest score wins 学习: Multiclass Perceptron 初始化 依次处理每个样本 Predict with current weights If correct, no change! If wrong: lower score of wrong answer, raise score of right answer 感知机特性 可分性: true if some parameters get the training set perfectly correct Can represent AND, OR, NOT, etc., but not XOR 收敛性: if the training is separable, perceptron will eventually converge (binary case) Separable Non-Separable 感知机存在的问题 噪声（不可分情况）: if the data isnt separable, weights might thrash 泛化性: finds a “barely” separating solut