Long-short-term-memory课件课件课件.docx

Long short-term memory ΣΠA simple LSTM gate with only input, output, and forget gates. LSTM gates may have more gates. [1] Σ Π Long short-term memory (LSTM) is a recurrent neural network (RNN) architecture (an arti?cial neural network) published [2] in 1997 by Sepp Hochreiter and Jürgen Schmidhuber. Like most RNNs, an LSTM network is universal in the sense that given enough network units it can compute anything a conventional computer can com- pute, provided it has the proper weight matrix, which may be viewed as its program. Unlike traditional RNNs, an LSTM network is well-suited to learn from experience to classify, process and predict time series when there are very long time lags of unknown size between important events. This is one of the main reasons why LSTM out- performs alternative RNNs and Hidden Markov Models and other sequence learning methods in numerous appli- cations. For example, LSTM achieved the best known results in unsegmented connected handwriting recogni- tion, [3] and in 2009 won the ICDAR handwriting compe- tition. LSTM networks have also been used for automatic speech recognition, and were a major component of a net- work that in 2013 achieved a record 17.7% phoneme er- ror rate on the classic TIMIT natural speech dataset. [4] Architecture An LSTM network is an arti?cial neural network that contains LSTM blocks instead of, or in addition to, regu- lar network units. An LSTM block may be described as a “smart” network unit that can remember a value for an arbitrary length of time. An LSTM block contains gates that determine when the input is signi?cant enough to re- member, when it should continue to remember or forget the value, and when it should output the value. A typical implementation of an LSTM block is shown to the right. The four units shown at the bottom of the ?g-  ΠΠ Π Π ∑∑ure are sigmoid units y = s( wixi) , where s is some squashing function, such as the logistic function. The left- mo