Signed-Prompt- A New Approach to Prevent Prompt Injection Attacks Against LLM-Integrated Applications信息安全资料 .docxVIP

Signed-Prompt:ANewApproachtoPreventPromptInjectionAttacksAgainstLLM-IntegratedApplications

XuchenSuo1,a)

1DepartmentofElectricalandElectronicEngineering,TheHongKongPolytechnicUniversity,HongKong,Chinaa)Correspondingauthor:xuchen.suo@connect.polyu.hk

Abstract.ThecriticalchallengeofpromptinjectionattacksinLargeLanguageModels(LLMs)integratedapplications,agrowingconcernintheArtificialIntelligence(AI)field.Suchattacks,whichmanipulateLLMsthroughnaturallanguageinputs,poseasignificantthreattothesecurityoftheseapplications.Traditionaldefensestrategies,includingoutputandinputfiltering,aswellasdelimiteruse,haveproveninadequate.ThispaperintroducestheSigned-Promptmethodasanovelsolution.Thestudyinvolvessigningsensitiveinstructionswithincommandsegmentsbyauthorizedusers,enablingtheLLMtodiscerntrustedinstructionsources.Thepaperpresentsacomprehensiveanalysisofpromptinjectionattackpatterns,followedbyadetailedexplanationoftheSigned-Promptconcept,includingitsbasicarchitectureandimplementationthroughbothpromptengineeringandfine-tuningofLLMs.ExperimentsdemonstratetheeffectivenessoftheSigned-Promptmethod,showingsubstantialresistancetovarioustypesofpromptinjectionattacks,thusvalidatingitspotentialasarobustdefensestrategyinAIsecurity.

INTRODUCTION

Inrecentyears,thefieldofArtificialIntelligence(AI)haswitnessedrapidadvancements,particularlyinthedomainofLargeLanguageModels(LLMs).Thesemodelshavebecomeincreasinglycapableofdirectlyunderstandingandrespondingtonaturallanguage,leadingtotheirwidespreadcommercialdeployment,significantlyenhancingtheinteractivityandflexibilityofassistant-likeapplications.Currently,variousAI-assistantapplicationsonthemarkethaveannouncedtheintegrationofdifferenttypesofLLMs.TheseLLM-IntegratedApplicationsplayanincreasingly