深度学习及其应用_ 课件 0525注意力机制基础.pptxVIP

DeepLearninganditsApplicationATTENTIONMECHANISM

AttentionmechanismSJTUDeepLearningLecture.2

SJTUDeepLearningLecture.3AttentionMechanismAttentionMechanismiswidelyusedinNLP,CV,etc.andhasmanyvariants.Assigndifferentweightstoeachpartoftheinputorthemodelstructure.Extractmoresignificanthiddeninformationwithoutadditionalstorageoverhead.Makeneuralnetworksinterpretative(a?dispute?theory)

SJTUDeepLearningLecture.4AttentionMechanism

SJTUDeepLearningLecture.5AttentionMechanism

SJTUDeepLearningLecture.6Seq2SeqShortcomingWordsinonesentencesharethesameweightOnlythelasthiddenlayerofEncoderisoutputintoDecoder

SJTUDeepLearningLecture.7BottleneckProblem

SJTUDeepLearningLecture.8AttentionMechanismKyunghyunChoetal.LearningPhraseRepresentationsusingRNNEncoder–DecoderforStatisticalMachineTranslation,arXivpreprintarXiv:1406.1078?

SJTUDeepLearningLecture.9AttentionMechanism

SJTUDeepLearningLecture.10Attentionvariants

SJTUDeepLearningLecture.11Seq2SeqwithAttention

SJTUDeepLearningLecture.12Seq2SeqwithAttention

SJTUDeepLearningLecture.13GlobalAttentionAllhiddenstatelayersused

SJTUDeepLearningLecture.14LocalAttentionCalculatealignedpositionfordecodingwordAlignedpositiondecideshiddenstateswindow

SJTUDeepLearningLecture.15AttentionMechanism