一种基于在线蒸馏的轻量化噪声标签学习方法.pdfVIP

计算机研究与发展DOI：10.7544/issn1000-1239.202330382

Journal

of

Computer

Research

and

Development61（12）：3121−3133，2024

一种基于在线蒸馏的轻量化噪声标签学习方法

1,2,321,3

黄贻望黄雨鑫刘声

1

（铜仁学院大数据学院贵州铜仁554300）

2

（福建理工大学计算机科学与数学学院福州350001）

3

（贵州省公共大数据重点实验室（贵州大学）贵阳550025）

（hyxhh0226@163.com）

ALightweightNoiseLabelLearningMethodBasedonOnlineDistillation

1,2,3

2

1,3

Huang

Yiwang,

Huang

Yuxin,

and

Liu

Sheng

1

（SchoolofDataScience,TongrenUniversity,Tongren,Guizhou554300）

2

（SchoolofComputerScienceandMathematics,FujianUniversityofTechnology,Fuzhou350001）

3

（GuizhouProvincialKeyLaboratoryofPublicBigData(GuizhouUniversity),Guiyang550025）

AbstractTraining

deep

learning

models

with

noisy

data

containing

lossy

labels

is

a

hot

research

topic

in

machine

learning.

Studies

have

shown

that

deep

learning

model

training

is

susceptible

to

overfitting

due

to

noisy

data.

Recently,

a

method

combining

meta-learning

and

label

correction

can

make

the

model

better

adapt

to

the

noisy

data

to

mitigate

the

overfitting

phenomenon.

However,

this

meta-label

correction

method

relies

on

the

model’s

performance,

and

the

lightweight

model

does

not

have

good

generalization

performance

under

noisy

data.

To

address

this

problem,

we

propose

a

knowledge

distillation-based

meta-label

correction

learning

method

(KDMLC),

which

treats

the

meta

label

correction

model

(MLC)

composed

of

a

deep

neural

network

and

a

multilayer

perceptron

as

a

teacher

model

to

correct

the

noise

labels

and

guide

the

training

of

the

lightweight

model,

and

at

the

same

time,

KDMLC

adopts

a

two-

layer

optimization

strategy

to

train

and

enhance

the

ge