Minghua Ye-使用开源软件打造类似Google的开发和生产环境.pdfVIP

GOPS2016 全球运维大会·上海站 Production Development like Google Using Open Source Technology Minghua Ye (Google) GOPS2016 全球运维大会·上海站 Scalability is vital for Cloud Royal wedding: 15M Pageview and 42k QPS Workiva: SEC reporting for Fortune 500 Spotify: 700,000 events per second GOPS2016 全球运维大会·上海站 The cornerstone of a scalable system 1. Distributed lock and storage (mutex and register) • Chubby • Zookeeper 2. Service discovery (pthread) • Etcd • SkyDNS 3. Load Balancing (scheduler) • Google network/HTTP(S) balancer • AWS Elastic loadbalancer • HAproxy / NGINX 4. Protobuf (ipc) GOPS2016 全球运维大会·上海站 Distributed Lock and Storage 1. Synchronization 2. Master Election 3. Global sequence number 4. Naming service 5. Distributed, persistent file system with strong consistency. GOPS2016 全球运维大会·上海站 Automated Service discovery 1. Autoscale 2. Auto failover 3. Zero config GOPS2016 全球运维大会·上海站 Load Balancer on Google Compute Engine 1. Google network load balancer • Support TCP / UDP • Session affinity • Websocket 2. Google SSL proxy • SSL termination • Websocket • TCP / SSL 3. Google HTTP(S) load balancer • SSL termination • Support HTTP2 • Support cloud CDN GOPS2016 全球运维大会·上海站 Customize load balancing 1. Proprietary hardware or software. 2. Open source • HAproxy • NGINX GOPS2016 全球运维大会·上海站 Protobuf Protocol buffers are a mechanism for describing extensible communication protocols, service definition and on-disk structures. 1. Backward compatibility • E.g. logs written in 2008 can still be read and used today. 2. New fields can be added