（毕业论文）风电场数据中心Hadoop云平台作业调度算法研究.docVIP

1308-0042修改 风电场数据中心Hadoop云平台作业调度算法研究 LUO Xianjin1 ，YUE Liming1, ZHEN Chenggang1 1.华北电力大学 控制与计算机工程学院,河北 保定 071000 1. School of Control and Computer Engineering, North China Electric Power University, Baoding 071000, China LUO Xianjin，YUE Liming，ZHEN Chenggang．Research on the job scheduling algorithm on the wind farms data center cloud platform based on hadoop. Computer Engineering and Applications Abstract: Wind data center contains status monitoring, data acquisition real-time jobs and other non-real-time jobs. The structure of C/S which has been wildly used, but it has many disadvantages, such as the imbalance of resource utilization, the high cost of management and maintenance, etc. This paper presents a data center architecture based on cloud computing platform; In order to solve the FIFO Scheduler in the open source Hadoop platform can not satisfy the real-time monitoring jobs’ requirements, the author design a dual queue job scheduler based on the existing FIFO Scheduler, which consider the deadline time and priority of the job to make schedule job decisions. Compared with the FIFO Scheduler, the experimental results show that the dual queue job scheduler can make a good performance when the cluster load is large to make the real-time jobs can be executed first, and provides a guarantee for the safe operation of the wind turbines.Key Words: hadoop; cloud platform; scheduling algorithm; cluster load 摘 要：风电场数据中心包含状态监测、数据采集等实时类作业和非实时类作业，采用C/S结构存在资源利用率不平衡、管理与维护成本高等缺点。本文设计了一种基于Hadoop云平台的数据中心架构；针对开源Hadoop平台现有FIFO调度器不能满足实时监测系统要求，在原有FIFO调度器的基础上，作者设计了一种双队列的作业调度器，综合考虑作业的截止时间和优先级来进行作业调度决策，实验结果表明，与FIFO调度器相比，双队列的作业调度器在集群负载较大时能够表现出较好的性能，保证实时类作业能够优先执行，为风电机组的安全运行提供保障。 关键词：hadoop；云平台；调度算法；集群负载 文献标志码: A 中图分类号: TP301.6引言 风能作为一种清洁的可再生能源，日益引起世界各国的关注，风力发电技术基本已趋于成熟。在大型的风电场中有几十台甚至上百台风力发电机，同时一个风力发电集团拥有多个风电场，它们分散于各地，并且多处于偏僻地区，不但给风电公司的生产运营管理带来很大困难，也给电网的调度和电网的安全运行带来很多的问题[1]。 目前，Google、Yahoo、Amazon、Facebook、百度以及阿里巴巴等都在应用云计算技术来解决它们所面临的处理海量数据的问题[2]。Hadoop（对海量数据进行分布式处理的软件框架）开源并行计算框架构建在廉价的