Hadoop下基于统计最优的资源调度算法.pdfVIP

第 30 卷第 2 期 Vol. 30 No.2 计算机应用研究 2013 年2 月 Application Research of Computers Feb.2013 Hadoop 下基于统计最优的资源调度算法 邓传华，范通让，高峰 (石家庄铁道大学信息科学与技术学院，石家庄的∞43) 摘 要:云计算集群中的资源存在异构和节点稳定性问题。异构资源的计算能力不同会导放较突出的作业任 务同步问题，而某个节点的不稳定状态会使运行于该节点的任务大量备份或重新计算。针对上述两问题将严重 影响集群作业的执行进度，在 Hadoop 平台下利用统计方法，提出一种资源调度算法，对计算资源较少的节点和 不稳定状态的节点进行标志并降权，让集群尽可能调度资源较好的稳定节点。实验结采表明，该算法能够在一 定程度上减少作业的周转时间，提高集群的效率和吞吐量。 关键词:云计算;资源调度; Hadoop; MapReduce 中图分类号:T凹93 文献标志码:A 文章编号: 1001-3695(2013)02-0417-03 doi: lO. 3969/j. issn. 1 ∞1-3695.2013.02.027 Resource scheduler algorithm based on statistical optimization under Hadoop DENG Chuan-hua , FAN Tong-rang , GAO Feng (衍生001 ollnformation Science Technology ,Shijiozhuang r，町dao University , Shijiαzhuang 05∞43 ， Chi=) Abstract; There are node stability problems in a heterogeneous cloud clusters. The heterogeneous resource(, differences in computing ability will lead to prominent sync issues of job task , and a unsteady node will make the task which is running in the node been backup or been recount. The two problems above will seriously affect the execution of the cluster s jobs pro- gress. According to the situation , now using the Hadoop workbench with statistical method ,this paper put forw缸d a kind of re- source scheduling algorithm which marked and dropped the less computing resources node and instability node rightly , letting cluster scheduling resources better stability node as much as possible. The experimental results show that , the algorithm can reduce the jobs cycle lime in a