人工智能论文英文版-semantic-features：A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces.pdfVIP

semantic-features:AUser-FriendlyToolforStudyingContextualWord

EmbeddingsinInterpretableSemanticSpaces

JwalanthiRanganathan1RohanJha1KanishkaMisra1,2,⋆KyleMahowald1

1TheUniversityofTexasatAustin2ToyotaTechnologicalInstituteatChicago

{jwalanthi,rjha,kyle}@{kanishka}@

1Introduction

Theadventofdistributionalsemanticembeddings

5

2hasenabledmajorprogressinthecomputational

0understandingofwordmeaningbyenablingpre-

2cisestatisticalexplorationsofsemanticspaces(Erk,

n2009;Mikolovetal.,2013;Penningtonetal.,2014).

u

JMorerecently,theriseofLMshavemadeitpossi-

6bletostudyembeddingsofwordsincontext.Chro-

nisetal.(2023)developedamethodforprojecting

]contextualwordembeddings(CWEs)intoainter-

L

Cpretablesemanticfeaturespacedefinedbyoneof

.threedifferentsemanticnorms(Binderetal.,2016;

s

cBuchananetal.,2019;McRaeetal.,2005).This

[

isachievedbytrainingfeed-forwardmodelswhichFigure1:InteractiveDemoinUse

1mapfromCWEsfromBERTtoavectorwhose

vvaluescorrespondtofeaturenorms.1995;HovavandLevin,2008;Beavers,2011),and

9

6Ourgoalinthispaperistwofold:first,weintro-interestingfeaturespecificpreferencesthathumans

1ducesemantic-features1asanextensible,easy-demonstratewhilechoosingbetweentwodative

6

0to-uselibrarybasedonChronisetal.(2023)forconstructionsduringproduction(Bresnan,2007).

6.studyingwordembeddingsfromanyLMincontext.Ourcasestudyfocuseson