LSTM-CRF-与-CTC算法详解-(纯英版).pptxVIP

Speech recognition and NER problems with recurrent neural networksOctober 9, 2018RNN???????MemoryMemory??????RNN and CNN?RNN can also be deep.RNN faces the problem of gradient vanish and gradient explosion.The input is not only related to the current input, but also related to the previous state. LSTM?In other papers, I noticed they use to represent the forget gate.?Comparing the fully connected layers, the parameters in each LSTM neuron will be more than at lease 4 times more. So there is GRU, and you can consider using GRU when LSTM is facing a risk of overfittingIn other articles, sigmoid activation is sometimes used instead of tanh activation.It overcomes the gradient vanish problem, but still cant solve the problem of gradient explosionFurtherLike CNN, simple RNN and LSTM can be deep.In speech recognition, the device can choose to give the final result of the recognition after obtaining the entire speech. Conventional RNNS are only able to make use of previous context, and there is no reason not to exploit future context as well.BRNNs?Still it can be further, just try to combine BRNNs with LSTM, and that becomes bidirectional LSTM.Deep bidirectional RNNsAssuming the RNN has N layers in the stack, and all layers use the same hidden layer function. The hidden vector sequences are iteratively computed from n=1 to N and t=1 to T: is the output at time n of layer n.if we define the network output are and next, we just replacing each hidden sequence with forward sequence and . If LSTM is used for hidden layers and we get deep bidirectional LSTM.?????Structure learning HMMs?HMMs relying on Markov chain properties, and here is an example based on my own understanding. Assume that Y has the Markov chain properties.(In extreme cases)X: Y: start: John saw the tree. EndHere X is not the phonetic symbol corresponding to the word, it should be a way of pronunciation of the word.The task: given X, find YX: Y: start: John saw the tree. endThe P(x, y) has been g