英伟达Cosmos世界基础模型平台：物理人工智能研究报告-75页.pdfVIP

2025-1-7

CosmosWorldFoundationModelPlatformforPhysicalAI

NVIDIA1

Abstract

PhysicalAIneedstobetraineddigitallyfirst.Itneedsadigitaltwinofitself,thepolicymodel,anda

digitaltwinoftheworld,theworldmodel.Inthispaper,wepresenttheCosmosWorldFoundationModel

PlatformtohelpdevelopersbuildcustomizedworldmodelsfortheirPhysicalAIsetups.Weposition

aworldfoundationmodelasageneral-purposeworldmodelthatcanbefine-tunedintocustomized

worldmodelsfordownstreamapplications.Ourplatformcoversavideocurationpipeline,pre-trained

worldfoundationmodels,examplesofpost-trainingofpre-trainedworldfoundationmodels,andvideo

tokenizers.TohelpPhysicalAIbuilderssolvethemostcriticalproblemsofoursociety,wemakeour

platformopen-sourceandourmodelsopen-weightwithpermissivelicensesavailableviaNVIDIACosmos.

1.Introduction

PhysicalAIisanAIsystemequippedwithsensorsandactuators:thesensorsallowittoobservetheworld,

andtheactuatorsallowittointeractwithandmodifytheworld.Itholdsthepromiseoffreeinghuman

workersfromphysicaltasksthataredangerous,laborious,ortedious.WhileseveralfieldsofAIhaveadvanced

significantlythankstodataandcomputescalingintherecentdecade,PhysicalAIonlyinchesforward.This

islargelybecausescalingtrainingdataforPhysicalAIismuchmorechallenging,asthedesireddatamust

containsequencesofinterleavedobservationsandactions.Theseactionsperturbthephysicalworldandmay

causeseveredamagetothesystemandtheworld.ThisisespeciallytruewhentheAIisstillinitsinfancywhen

exploratoryactionsareessential.AWorldFoundationModel(WFM),adigitaltwinofthephysicalworldthata

PhysicalAIcansafelyinteractwith,hasbeenalong-sought