大语言模型LMM简介2024 英文版 .docxVIP

Large

Language

Models

IntroductiontoLargeLanguageModels

Languagemodels

?Rememberthesimplen-gramlanguagemodel?Assignsprobabilitiestosequencesofwords

?Generate?Istrained

textbysamplingpossiblenextwordsoncountscomputedfromlotsoftext

?Largelanguagemodelsaresimilaranddifferent:?Assignsprobabilitiestosequencesofwords

?Generatetextbysamplingpossiblenextwords?Aretrainedbylearningtoguessthenextword

Largelanguagemodels

?Eventhroughpretrainedonlytopredictwords?Learnalotofusefullanguageknowledge

?Sincetrainingonalotoftext

Threearchitecturesforlargelanguagemodels

Decoders Encoders Encoder-decodersGPT,Claude, BERTfamily,Flan-T5,WhisperLlama HuBERT

Mixtral

Encoders

Manyvarieties!

?Popular:MaskedLanguageModels(MLMs)?BERTfamily

?

Trainedbypredicting

wordsonbothsides

wordsfromsurrounding

?Areusuallyfinetuned(trainedonsuperviseddata)

forclassificationtasks.

Encoder-Decoders

?Trainedtomapfromonesequencetoanother

?Verypopularfor:

?machinetranslation(mapfromonelanguagetoanother)

?speechrecognition(mapfromacousticstowords)

Large

Language

Models

IntroductiontoLargeLanguageModels

Large

Language

Models

LargeLanguageModels:Whattaskscantheydo?

Bigidea

Manytaskscanbeturnedintotasksofpredictingwords!

Thislecture:decoder-onlymodels

Alsocalled:

?CausalLLMs

?AutoregressiveLLMs?Left-to-rightLLMs

?Predictwordslefttoright

本报告来源于三个皮匠报告站（）,由用户Id:673421下载,文档Id:620742,下载日期:2025-04-01

ConditionalGeneration:Generatingtext

conditionedonprevioustext!

CompletionText

all the

LanguageModelingHead

Softmax

Unencoderlayer

logits

U U

TransformerBlocks

… …

Encoder

+ i + i + i + i + i +i + i

E E E E E E E

So long andthanksfor all the

Pre?xText

ManypracticalNLPtaskscanbecastaswordprediction!

Sentimentanalysis:“IlikeJackieChan”

1.Wegivethelanguagemodelthisstring:Thesentimento