基于Spark的电力调度数据整合模型-中国云计算.PDF

Computer Engineering and Applications 计算机工程与应用 2017 ，53（19） 65 ⦾大数据与云计算⦾ 用 基于Spark 的电力调度数据整合模型 应 1 1 2 3 4 曲朝阳 ，陈贺新 ，胡可为 ，刘耀伟 ，独健鸿 与 1 1 2 3 4 QU Zhaoyang , CHEN Hexin , HU Kewei , LIU Yaowei , DU Jianhong 程 g 1.东北电力大学 信息工程学院，吉林 吉林 132012 r 工 o 2. 国网吉林省电力有限公司 调控中心，长春 130000 . j 3. 国网吉林省电力有限公司 吉林供电公司，吉林 吉林 132000 机 a 4. 吉林市丰满发电厂，吉林 吉林 132113 e 1.College of Information Engineering, Northeast Dianli University, Jilin, Jilin 132012, China 算 c . 2.Control Center, State Grid Jilin Electric Power Co. , Ltd. , Changchun 130000, China 计 3.Jilin Power Supply Company, State Grid Jilin Electric Power Co. , Ltd. , Jilin, Jinlin 132000, China w 4.Fengman Power Plant in Jilin City, Jilin, Jinlin 132113, China ww QU Zhaoyang, CHEN Hexin, HU Kewei, et al. Power dispatch data integration model based on Spark. Computer Engineering and Applications, 2017, 53 （19）：65-70. Abstract: With the concept of data applying in the electric power industry, the construction of electric power dispatching data warehouse is foundation to support the unified data platform in the power dispatching center, in view of the multi- source data integration in the power dispatching center data warehouse there are redundant and inconsistent problems. This paper puts forward a kind of power dispatch data integration model based on Spark. First of all, the design of parallel forward maximum matching to remove redundancy algorithm, it filters the multiple system re