基于MapReduce的大规模图挖掘并行计算模型.PDFVIP

第 39 卷第 3 期 应 用 科 技 Vol.39, No.3 2012 年 6 月 Applied Science and Technology Jun. 2012 doi:10.3969/j.issn.1009-671X.201112024 基于 MapReduce 的大规模图挖掘并行计算模型 饶君, 张仁波, 东昱晓, 吴斌 北京邮电大学 计算机科学与技术学院，北京 100876 摘 要：在如何快速发现大规模网络的结构和特性问题中，网络规模及复杂度的快速增长给其分析研究带来了新的挑战. MapReduce 及其开源实现 Hadoop 给大规模图的高效处理带来了希望. 基于 MapReduce 框架的集群系统，提出了 1 种新 的计算模型用于大规模图形的 3-clique 计算，来实现图挖掘. 计算的基本步骤是：首先获取每个节点的第 1 跳信息，然 后是第 2 跳信息，最后得到所有基于该节点的 3-clique. 该计算模型可以用来计算聚集系数，并且可以用于三大通话网 络的挖掘. 实验结果证明这种计算模型具有良好的可扩展性和性能. 关键词：图挖掘；社会网络分析；MapReduce ；聚集系数；3-clique 中图分类号：TP311 文献标志码：A 文章编号：1009-671X （2012 ）03-0056-05 A parallel computing model for large-graph mining with MapReduce RAO Jun, ZHANG Renbo, DONG Yuxiao, WU Bin School of Computer Science, Beijing University of Posts and Telecommunication, Beijing 100876, China Abstract: Large-scale graphs exist everywhere. The continued exponential growth in both the size and complexity of the graphs is posing a new challenge for finding the structures and characters of a large-scale graph. An excellent promising clue for dealing with graphs with great sizes is the emerging MapReduce framework and its open-source implementation, Hadoop. The problem of 3-clique enumeration of a graph is an important operation that can help structure mining and a difficult mission for graphs with great sizes on the single computer. In this paper, we propose a parallel computing model for 3-clique enumeration based on cluster system with the help of MapReduce for large-scale graphs. The process of enumeration is firstly to extract one-leap information of the graph, then the two-leap information and fin