2024年OpenAI o1模型的预部署评估报告 英文版 (2).pdfVIP

12

USAISIandUKAISIJoint

Pre-DeploymentTest

OpenAIo1

December2024

1USAISafetyInstitute

NationalInstituteofStandardsandTechnology

2UKAISafetyInstitute

DepartmentofScienceInnovationandTechnology

Contents

1Introduction1

1.1Disclaimer...............................................1

1.1.1LimitationstoResults....................................1

2Methodology1

2.1Pre-deploymentEvaluation.....................................1

2.2EvaluatedModels...........................................2

2.3AgentDesign.............................................2

2.4TaskIterationsandCost.......................................3

2.5PresentingUncertainty........................................4

2.6Model-SamplingParameters.....................................4

ICyberCapabilitiesEvaluations