综合论文：MesosAplatformforfinegrainedresourcesharinginthedatacenter（英文）.pdfVIP

Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi, Anthony D. Joseph, Randy Katz, Scott Shenker, Ion Stoica University of California, Berkeley Abstract Two common solutions for sharing a cluster today are We present Mesos, a platform for sharing commod- either to statically partition the cluster and run one frame- ity clusters between multiple diverse cluster computing work per partition, or to allocate a set of VMs to each frameworks, such as Hadoop and MPI. Sharing improves framework. Unfortunately, these solutions achieve nei- cluster utilization and avoids per-framework data repli- ther high utilization nor efficient data sharing. The main cation. Mesos shares resources in a fine-grained man- problem is the mismatch between the allocation granular- ner, allowing frameworks to achieve data locality by ities of these solutions and of existing frameworks. Many taking turns reading data stored on each machine. To frameworks, such as Hadoop and Dryad, employ a fine- support the sophisticated schedulers of today’s frame- grained resource sharing model, where nodes are subdi- works, Mesos introduces a distributed two-level schedul- vided into “slots” and jobs are composed of short tasks ing mechanism called resource offers. Mesos decides that are matched to slots [25, 38]. The short duration of how many resources to offer each framework, while tasks and the ability to run multiple tasks per node allow frameworks decide which resources to accept and which jobs to achieve high data locality, as each