改进人工鱼群算法在Hadoop作业调度算法的应用.pdfVIP

网络出版时间：2014-08-27 09:43 网络出版地址：/kcms/detail/51.1196.TP0943.076.html 优先出版 计 算 机 应 用 研 究 第31 卷 改进人工鱼群算法在Hadoop 作业调度算法的应用 吉鹏飞，齐建东，朱文飞 (北京林业大学 信息学院，北京 100083) 摘 要：作业调度算法是云计算的关键技术，也是云计算核心问题，尤其是在异构环境下资源合理分配问题成为当前研 究的热点。本文在分析Hadoop 缺省及改进的作业调度算法基础上，引入群智能算法，设计了基于改进人工鱼群算法的 Hadoop 作业调度算法。采用随机键方式对待分配任务进行编码，以任务总执行时间作为启发函数，并引入吞食行为和 跳跃行为改进人工鱼群算法，以达到进一步改善作业调度算法性能的目的。实验结果表明，改进后的人工鱼群算法作业 调度算法在异构环境下可以提高系统性能，降低运行时间。 关键词：Hadoop ；人工鱼群算法；组合优化；作业调度算法 中图分类号：TP301.6 文献标志码：A Application of improved artificial fish swarm algorithm in Hadoop scheduling algorithm JI Peng-fei , QI Jian-dong, ZHU Wen-fei (School of Information Science Technology, Beijing Forestry University, Beijing 100083, China) Abstract: As the core problem for cloud computing, especially the appropriate distribution of resources in heterogeneous environment, job scheduling algorithm has turned to be one of the current research focuses. This paper introduces swarm intelligence algorithm on the basis of the analysis on the default and improved Hadoop job scheduling algorithm, and propose a new Hadoop job scheduling algorithm based on the improved artificial fish swarm algorithm. It encodes the allocated tasks by random key approach first, with the total execution time of the tasks as heuristic information, and then improves AFSA by introducing devouring and jumping behavior, last applies improved AFSA to Hadoop j ob scheduling algorithm in order to achieve further performance. The exp