深学习循环神经网络.pptxVIP

第1页/共50页深学习循环神经网络第2页/共50页目录1：深度学习发展史2：从神经网络到深度学习3：循环神经网络基础4：自然语言处理基础（扩充知识）第3页/共50页1：深度学习发展史第4页/共50页深度学习发展史Computer visionNLPSpeech ……RosenblattGeoffrey Hinton HebbGeoffrey Hinton Learning model of neuronsNeural networkBack propagationDeep belief netSciencePerceptronSpeech1958200619491986201120122014SVMBoostingDecision treeKNN…DBNCNNRBMRNN…First Winter of NN机器学习第一次浪潮：机器学习第二次浪潮深度学习模型(Deep Learning)浅层学习模型(Shallow Learning)Second Winter of NN第5页/共50页深度学习（多层神经网络）神经网络神经元第6页/共50页细胞体Cell body树突dendritex1w1o线性动态系统突触synapseΣ激励函数轴突Axonxnwn来自其它神经元神经元模型神经元是构成神经网络的最基本单元(构件), 因此, 首要任务是构造人工神经元模型。SynapticterminalssynapsedendriteCell bodyAxon轴突末梢突触树突细胞体轴突第7页/共50页x1x1w1o=f(net)net = WTXyx2w1yΣ激励函数w2θaxn?wnwn??xxno 神经元模型 典型的激励函数(Activation Function)：线性函数，非线性斜面函数，阶跃函数，S型函数等。第8页/共50页x1x2w1ow2c?wn??xnInput LayerHidden LayerOutput Layer神经网络一般形式h1w11wm1x1o1v11wM1vl1vL1w1LvLNwmLxMoNwMLhL? Nonlinearity 非线性 ? Parallel Processing 并行处理? Input—Output Mapping 输入输出匹配 ? Adaptivity 自适应性第9页/共50页最简单的神经网络：Perceptrons第10页/共50页x1x2w1yw2b?wM??xMu0u0Single Layer PerceptronsRosenblatt, 1957第11页/共50页W=[1.54 1.28]b=[-0.64] W=[1.16 1.63]b=[-1.8] W=[1.66 1.11]b=[-0.823] W=[1.49 -1.39]b=[-0.743] 两类样本：白色和黑色 分类错误的样本用红色轮廓目标：分类正确所有样本，直到没有红色轮廓的样本。W=[1.66 1.11]b=[1.25] where?Single Layer Perceptrons： 迭代过程第12页/共50页Single Layer Perceptrons：局限性The objective is only to find the line that separates two linearly separable classes. As soon as the firstsolution weights vector, which separates all the data pairs corretly, is found, there will be no furtherchanges of the vector. So, the perceptron learning is not an optimization method. 线性可分问题异或问题：x1 0 0 1 1x2 0 1 0 1d 0 1 1 0Problems: it can not separate patterns when there is an overlapping of data or when classes are not linearlySeparable 不能处理线性不可分问题第13页/共50页XORANDOR011101010001Linear Separable Problemx1x2y000100010111x1x2y000101011111x1x2y000101011110第14页/共50页XOR0101Single Layer PerceptronsFor XOR problem: 1. introducing one additional neuron in a