人工智能核心550篇论文 二维分类 1702.04008v2.pdfVIP

PublishedasaconferencepaperatICLR2017

SOFTWEIGHT-SHARINGFOR

NEURALNETWORKCOMPRESSION

KarenUllrichEdwardMeeds

UniversityofAmsterdamUniversityofAmsterdam

karen.ullrich@uva.nltmeeds@

MaxWelling

UniversityofAmsterdam

7CanadianInstituteforAdvancedResearch(CIFAR)

1welling.max@

0

2

y

aABSTRACT

M

Thesuccessofdeeplearninginnumerousapplicationdomainscreatedthede-

9siretorunandtrainthemonmobiledevices.Thishowever,conflictswiththeir

computationally,memoryandenergyintensenature,leadingtoagrowinginter-

]

Lestincompression.RecentworkbyHanetal.(2015a)proposeapipelinethat

involvesretraining,pruningandquantizationofneuralnetworkweights,obtain-

Mingstate-of-the-artcompressionrates.Inthispaper,weshowthatcompetitive

t.compressionratescanbeachievedbyusingaversionof”softweight-sharing”

a(NowlanHinton,1992).Ourmethodachievesbothquantizationandpruningin

t

sonesimple(re-)trainingprocedure.Thispointofviewalsoexposestherelation

[betweencompressionandtheminimumdescriptionlength(MDL)principle.

2

v1INTRODUCTION

8

0

0”Big