基于MapReduce模型带任务分割的平行机调度优化-控制与决策.PDF

第xx卷 第x期 控 制 与 决 策 Vol.xx No.x xxxx年 x 月 Control and Decision xxx. xxxx DOI: 10.13195/j.kzyjc.2017.1677 基于MapReduce模型带任务分割的平行机调度优化 黄基诞 , 郑斐峰 , 徐寅峰 , 刘明 (1. 东华大学旭日工商管理学院，上海200051 ；2. 同济大学经济与管理学院, 上海200092) 摘 要: MapReduce模型当前在机器调度和大数据处理方面日趋重要. 基于MapReduce模型的每个工件包 含Map和Reduce两道工序，Map工序可以分割成若干个子任务, 并在多台平行机上同时加工, Reduce工序只有在 该工件的Map工序所有子任务完成后才能启动加工, 而且Reduce工序只能在一台机器上连续加工. 结合工件的到 达时间和交货时间等约束, 以最大完工时间和总延期时间的加权和为目标函数, 构建了混合整数规划模型, 设计 了采用差分变异策略和逐维角度扰动机制的改进鲸鱼优化算法来求解模型. 利用数值仿真实验, 与经典鲸鱼优化 算法、粒子群算法及CPLEX计算结果进行对比. 实验结果表明, 所设计的算法较经典鲸鱼优化算法以及粒子群 法的效果提升显著, 验证了模型与算法改进的有效性. 关键词: 平行机调度; MapReduce; 鲸鱼优化算法; 并行处理 中图分类号: TP18 文献标志码: A Parallel machine scheduling with splitting jobs in the MapReduce system † Huang Ji-dan , Zheng Fei-feng , Xu Ying-feng , Liu Ming (1. Glorious Sun School of Business and Management, DongHua University，ShangHai 200051，China ；2. School of Economics and Management, Tongji University, ShangHai 200092 ，China) Abstract: The MapReduce model has become increasingly important in machine scheduling and big data processing. Each job consists of one map task and one reduce task. The map task can be arbitrarily split and processed on serveral machines simultaneously, while the reduce task has to be processed on a single machine and it cannot be started unless the map task has been completed. The processing of reduce task can’t be interrupted. In this paper, we consider the MapReduce problem of minimizing the weighted sum of makespan and total delay time on parallel identical machines with non-id