2019年Serengeti-虚拟化你的大数据应用.pptVIP

© 2009 VMware Inc. All rights reserved Serengeti - 虚拟化你的大数据应用 蔺永华 Vmware, Inc. Agenda • Today’s big data system • Why virtualize hadoop? • Serengeti introduction • Common questions about virtualization • Serengeti solution • Deep insight into Serengeti • Summary • QA Today’s Big Data System: ETL Unstructured Data (HDFS) Real Time Structured Database Big SQL Data Parallel Batch Processing Real Time Streams Real-Time Processing (s4,storm) Analytics Agenda • Today’s big data system • Why virtualize hadoop? • Serengeti introduction • Common questions about virtualization • Serengeti solution • Deep insight into Serengeti • Summary • QA Challenges To Use Hadoop in physical infrastructure Deployment • Difficult to deploy, cost several people for several days even months • Difficult to tune cluster performance Low Efficiency • Hadoop clusters are typically not 100% utilized across all hardware resources. • Difficult to share resources safely between different workload Single Point of Failure • Single point of failure for Name Node and Job tracker • No HA for Hive, HCatalog, etc. Why Virtualize Hadoop? - Get your Hadoop cluster in minutes 1/1000humanefforts, LeastHadoopoperation knowledge Fullyautomated process, 10 minutesto get a Hadoop/HBaseclusterfrom scratch Server preparation OS installation Automateby Serengeti on vSpherewith best practice Network Configuration Hadoop Installation and Configuration Manual process, costdays Why Virtualize Hadoop? - Consolidate sprawling clusters Clustersshare serverswith strongisolation • Single Hardware Infrastructure • Unified operations  Optimize • Shared Resources = higher utilization • Elastic resources = faster on-demand access Hadoop Dev Hadoop Prod HBase ClusterSprawling Single purpose clusters for various business applications lead to cluster sprawl. Cluster Consolidation  Simplify Finance Hadoop Virtualization Platform Hadoop Dev Hadoop Prod HBase ... Porta