2025大语言模型推理能力榜_中文语境下_最强大脑_测评揭晓_15页_2mb.pdfVIP

EvaluatingtheReasoningCapabilitiesof

LargeLanguageModelsin

Chinese-languageContexts

*1,11211

Zhenhui(Jack)Jiang,YiLu,YifanWu,HaozheXu,ZhengyuWu,JiaxinLi

1HKUBusinessSchool,TheUniversityofHongKong,HongKong

2SchoolofManagement,XianJiaotongUniversity,P.R.China.

Abstract

WiththerapiditerationofAItechnologies,reasoningcapabilitieshavebecomeacore

indicatorformeasuringtheintelligenceleveloflargelanguagemodels(LLMs)anda

focusofresearchinbothacademiaandindustry.Thisreportaimstoestablisha

systematic,objective,andcomprehensiveevaluationframeworktoassessAI

reasoningcapabilities.Wecompared36LLMsonvarioustext-basedreasoningtasks

inChinese-languagecontextsandfoundthatGPT-o3achievedthehighestscoreinthe

basiclogicalreasoningevaluation,whileGemini2.5Flashledincontextualreasoning

evaluation.Intermsofoverallranking,Doubao1.5Pro(Thinking)securedthetop

position,closelyfollowedbyOpenAI’srecentlyreleasedGPT-5(Auto).Several

Chinese-developedLLMs—includingDoubao1.5Pro,Qwen3(Thinking),and

DeepSeek-R1—alsorankedamongtheleaders,demonstratingthestrongreasoning

performanceoffrontierChineseAItechnologies.Furtheranalysisofmodelefficiency

revealedthatmostmodelswithsuperiorreasoningcapabilitiesoftenincurredhigher

costsintermsoftokenefficiency,responsetime,andAPIusage.Notably,Doubao1.5

Pronotonlyachievedoutstandingreasoningperformancebutalsodemonstratedhigh

modelefficiency.

Keywords:LargeLanguageModel,LLM,ReasoningCapability,ModelEfficiency,

LogicReasoning,ContextualReasoning,Chinese-languageContext

Citethispaperas:

Jiang,Z.J.,Lu,Y.,Wu,Y.F.,Xu,H.Z.,WuZ.Y.,Li,J.(2025).Evaluati