一种基于安全多方计算的快速Transformer安全推理方案.pdfVIP

计算机研究与发展DOI：10.7544/issn1000-1239.202330966

Journal

of

Computer

Research

and

Development61（5）：1218−1229，2024

一种基于安全多方计算的快速Transformer安全推理方案

11111,22

刘伟欣管晔玮霍嘉荣丁元朝郭华李博

1

（北京航空航天大学网络空间安全学院北京100191）

2

（复杂关键软件环境全国重点实验室（北京航空航天大学）北京100878）

（sy2339130lwx@buaa.edu.cn）

AFastandSecureTransformerInferenceSchemewithSecureMulti-Party

Computation

1

1

1

1

1,2

2

Liu

Weixin,

Guan

Yewei,

Huo

Jiarong,

Ding

Yuanchao,

Guo

Hua,

and

Li

Bo

1

（SchoolofCyberScienceandTechnology,BeihangUniversity,Beijing100191）

2

（StateKeyLaboratoryofComplex

CriticalSoftwareEnvironment(BeihangUniversity),Beijing100878）

AbstractTransformer

has

been

widely

used

in

many

fields

such

as

natural

language

processing

and

computer

vision,

and

has

outstanding

performance.

The

users’

data

will

be

leaked

to

the

Transformer

model

provider

during

inference.

With

the

increasing

public

attention

on

data

privacy,

the

above

data

leakage

problem

has

triggered

researchers’

study

on

secure

Transformer

inference.

Implementing

secure

Transformer

inference

with

secure

multi-party

computation

(MPC)

is

today’s

hot

topic.

Due

to

the

widely

existence

of

non-linear

functions

in

Transformer,

it

is

hard

to

use

MPC

to

implement

secure

Transformer

inference,

which

leads

to

huge

computation

and

communication

cost.

We

focus

on

Softmax

attention,

bottleneck

in

secure

Transformer

inference,

and

propose

two

kinds

of

MPC-friendly

attention

mechanism,

Softmax

f