Long-short-term-memory课件课件课件.pdf

Longshort-termmemory Π Σ A simple LSTM gate with only input, output, and forget gates. LSTMgatesmayhavemoregates.[1] Π Π Longshort-termmemory (LSTM)isarecurrentneural network (RNN)architecture(anartificialneuralnetwork) published[2] in 1997 by Sepp Hochreiter and Jürgen Schmidhuber. Like most RNNs, an LSTM network is universal inthesensethatgivenenoughnetworkunitsit cancomputeanythingaconventionalcomputercancom- pute,providedithastheproperweightmatrix,whichmay be viewed as its program. Unlike traditional RNNs, an AtypicalimplementationofanLSTMblock. LSTMnetworkiswell-suitedtolearnfromexperienceto classify, process and predict time series when there are very long time lags of unknown size between important events. ThisisoneofthemainreasonswhyLSTMout- performsalternativeRNNsand HiddenMarkovModels urearesigmoidunits ∑ ,where s issome andothersequencelearningmethodsinnumerousappli- squashingfunction,suchasthelogisticfunction. Theleft- cations. For example, LSTM achieved the best known mostoftheseunitscomputesavaluewhichiscondition- results in unsegmented connected handwriting recogni- ally fed as an input value to the block’s memory. The [3] tion, andin2009wontheICDARhandwritingcompe- otherthreeunitsserveasgatestodeterminewhenvalues tition. LSTMnetworkshavealsobeenusedforautomatic areallowedtoflowintooroutoftheblock’smemory. The speechrecognition,andwereamajorcomponentofanet- secondunitfromtheleft(onthebottomrow)isthe“in- workthatin2013achievedarecord17.7%phonemeer- putgate”. Whenitoutputsavalueclosetozero,itzeros rorrateontheclassic TIMIT naturalspeechdataset.[4]