当大型语言模型遇见编程——代码智能的崛起.pdf

DeepSeek-Coder:WhentheLargeLanguageModelMeets

Programming-TheRiseofCodeIntelligence

4

2

0

2Abstract

n

a

J

Therapiddevelopmentoflargelanguagemodelshasrevolutionizedcodeintelligencein

6

2softwaredevelopment.However,thepredominanceofclosed-sourcemodelshasrestricted

extensiveresearchanddevelopment.Toaddressthis,weintroducetheDeepSeek-Coderseries,

]

Earangeofopen-sourcecodemodelswithsizesfrom1.3Bto33B,trainedfromscratchon2

Strilliontokens.Thesemodelsarepre-trainedonahigh-qualityproject-levelcodecorpusand

.

semployafill-in-the-blanktaskwitha16Kwindowtoenhancecodegenerationandinfilling.

cOurextensiveevaluationsdemonstratethatDeepSeek-Codernotonlyachievesstate-of-the-art

[

performanceamongopen-sourcecodemodelsacrossmultiplebenchmarksbutalsosurpasses

2

vexistingclosed-sourcemodelslikeCodexandGPT-3.5.Furthermore,DeepSeek-Codermodels

6areunderapermissivelicensethatallowsforbothresearchandunrestrictedcommercialuse.

9

1

4

1

.

1

0

4

2

:

v

i

X

r

a

Figure1|ThePerformanceofDeepSeek-Coder

*Corecontributors,orderedalphabeticallybythename.

1.Introduction

Thefieldofsoftwaredevelopmenthasbeensignificantlytransformedbytheswiftadvancement

oflargelanguagemodels(OpenAI,2023;Touvronetal.,2023),whichhavebroughtabout

aneweraofcodeintelligence.Thesemodelshavethepotentialtoautomateandstreamline

manyaspectsofcoding,frombugdetectiontocodegeneration,therebyenhancingproductivity

andreducingthelikelihoodofhumanerror.However,amajorchallengeinthisfieldisthe

performancegapbetweenopen-sourcemodels(Lietal.,2023;