Chorus:面向长度偏斜解码的 GPU-CPU 协同大语言模型服务.pptx

Chorus:面向长度偏斜解码的 GPU-CPU 协同大语言模型服务.pptx

groups,andreleasesthecorrespondingGPUKVpagessotheGPUlanecanadmitmoreshortrequests.Tosatisfyonlineservicelevelobjectives(SLOs),CHORUScombinespressure-awareoffloadadmission,tensor-parallelAMXdecodethataggregatesCPUcomputeandmemorybandwidth,andAMXkernelsspe

您可能关注的文档

文档评论(0)

1亿VIP精品文档

相关文档