面向遥感数据集群处理的任务调度模型研究与应用-计算机应用技术专业毕业论文.docxVIP

Research and Application of Task Scheduling Model for The Cluster Processing of Remote Sensing Data A Dissertation Submitted to the Graduate School of Henan University in Partial Fulfillment of the Requirements for the Degree of Master of Science By Liu Chunli Supervisor: Prof.Zheng Fengbin May,2015 I I 摘 要 近年来，我国卫星遥感技术正高速发展，遥感数据量爆炸式增长，遥感数据处理任 务更加紧迫。面对如此大的遥感数据处理任务，高性能集群计算已经取得了非常不错的 效果，极大地提高了数据处理效率。在遥感数据集群处理系统中，任务调度是其非常重 要的组成部分，也是其性能提升的关键。而已有的调度算法都或多或少存在节点负载不 均、计算资源利用率低、任务吞吐量小、任务平均调度时间较长等问题。为解决上述问 题，本文在分析已有调度算法的基础上，考虑到遥感数据集群处理系统具有任务重、任 务独立、任务性质比较单一及节点性能差异大等特点，提出了一种任务调度模型。该模 型采用改进的先来先服务调度算法及节点负载均衡计算技术，能够很好地提高系统节点 利用率和任务吞吐量，有效减少任务平均调度时间。 本文的主要工作与贡献如下： 1．研究分析当前应用于遥感数据集群处理系统的已有调度算法，针对它们或多或 少存在节点利用率低、吞吐率低及任务调度时间较长等问题，提出一种基于任务和节点 的双向任务调度模型。 2．在提出该任务调度模型的基础上，首先通过分析 RMI 实现原理及其优缺点，并 根据遥感数据集群处理系统的特点，设计任务调度模型通信机制；然后根据系统任务特 点完成任务调度设计，并考虑负载均衡完成节点的负载计算与调度；最后完成任务调度 模型的总体设计。 3．将该任务调度模型应用于信息产品生产分系统中，并通过大量测试数据，表明 本文提出的任务调度模型的确可以最大程度地实现节点的负载均衡，提高系统节点利用 率和任务吞吐量，减小任务平均调度时间，系统稳定可靠。 关键词：任务调度,负载均衡, 海量遥感数据，集群处理 III III IV IV ABSTRACT In recent years, the satellite remote sensing technology is rapid development in our country, and the remote sensing data is with explosion growth, so the process of remote sensing data tasks more urgent. In the face of such a large remote sensing data processing tasks, the high -performance cluster computing has achieved very good results, and improve the efficiency of data processing greatly. Task scheduling is a very important part of the cluster of remote sensing data processing system, and it also is the key to improvement the performance of the cluster of remote sensing data processing system. However, the existing scheduling algorithms has some problems, such as the load of nodes is imbalance, the utilization of computer is not high, the throughput of tasks is very low, and the average time of task scheduling is very long. To solve the problems above, this paper proposes a task scheduling model which adopts the advanced scheduling algorithm of first come