360_Mesos container在360广告系统的应用 CTO技术分享-技术开发.pdf

Mesos container在360广告系统的应用 李冬 Apr.2018 Who Am I? Where is our system? Where is our system? Why container? Service pain point  data center migration  failover  operating system  production environment  test  service expansion  resource utilization Build，Ship，and Run Any App， Anywhere Problems with service dockerization Where is my debug logs? network performance? docker registry? How to write a local file How to store persistent Registration and discovery of services We’re Docker scheduler? Docker data loading? Why Mesos? Backgroud A variety of distributed computing frameworks Problem A variety of distributed computing frameworks Resource utilization, operation and maintenance costs, data sharing Want to run multiple applications in a single cluster >>..to efficient shared hardware resources >>..to simplify scheduling logic >>..as much compatibility and scalability as possible Benefits of Mesos  Effectiveness Benefits of Mesos  Agility Mesos : Resource offers >>Quickly use the resources available in the cluster Benefits of Mesos  Expandability Mesos : Two-level scheduling >>keep mesos simple Benefits of Mesos  Modularization Mesos : Restful API >>Free to add new framework Outline  Mesos Goals  Architecture  Implementation  Results Mesos Goals High utilization of resources Support diverse framworks(current & futures) Scalability to 45,000’s of nodes(Twitter) Reliability in face of failures Mesos Architecture Implementation 20000+ lines of C++ Master failover using ZooKeeper Frameworks ported Open source in Apache Incubator Fault Tolerance Mesos master has only soft state: list of currently running frameworks and tasks Rebuild when frameworks and