分布式大图处理系统中的索引设计与实现.PDF

计 算 机 研 究 与 发 展 DOI:10.7544/issn1000-1239.2016.******** Journal of Computer Research and Development 卷(期):起止页,年 分布式大图处理系统中的索引设计与实现 The Design and Implementation of Index on Distributed Big Graph Processing System Abstract With the rapid growth of technologies like web, social network, and location based service, graph data exist in a variety of applications. There are some distributed graph processing systems that perform pretty impressive when applied to offline graph data analysis tasks which often require complex computation and massive communications between vertices, leading to considerable network traffic. Google Pregel and Apache Giraph are instances of those systems. However, these graph processing systems are mostly based on Bulk Synchronized Parallel Computing(BSP) model, which means that every vertex in the graph is supposed to meet a barrier right after every single iteration during computation, which brings efficiency loss. On the other hand, Pregel-like systems only index vertices by their IDs, which means that when querying properties other than ID, the only way to access vertices that meet some certain requirements is to scan all the vertices. This may result in a high latency which is a severe problem in online querying. Thus, it is our major concern to improve Pregel-like graph processing systems so that we can obtain a better performance on querying any properties on these systems. We investigate on various index mechanisms that fit the graph data querying on distributed systems and apply one of them to the Apache Giraph Framework. The improved system supports both synchronous and asynchronous execution. Finally, we validate the superiority of our proposed mechanism by experimenting on massive synthesized graph data. Key words distributed graph