an improved staged event driven architecture for master-worker network computing.pdfVIP

An Improved Staged Event Driven Architecture for Master-Worker Network Computing Biao Han, Zhongzhi Luan, Danfeng Zhu, Zhongxin Wu Yinan Ren, Ting Chen, Yongjian Wang School of Electronics and Information Engineering Sino-German Joint Software Institute Xi’an Jiaotong University Beihang University Xi’an, P.R. China Beijing, P.R. China panglaohu@ biao.han@ Abstract—We propose a new design for Master-Worker needs of large-scale computing. The design of it generally is network computing systems, called the Master-Worker event based on the dividing and conquering principle, such as driven architecture (MEDA). MEDA is an extension of staged Condor MW [2], MapReduce [3], Dryad [4], Hadoop [5], etc. event driven architecture and designed for the goal to meet the This model contains two types of nodes, master node and dynamic demand of Master-Worker model network computing worker node. The master node is responsible for splitting a systems, with support for high concurrency, adaptive resources large task into several relatively small jobs, then scheduling management and modular construction. In MEDA, these jobs to worker nodes for computing, and finally applications consist of a network of event-driven stages combining the results from all the worker nodes as the task connected by queues. MEDA ma