社交网站的群集存储系统的的研究与设计.pdfVIP

社交网站的群集存储系统的研究与设计 摘 要 近年来，社交网站快速发展，注册用户数目急剧增加，用户通过社 交网站共享的数据量也随之增加，对计算机存储系统的可扩展性提出 了新的挑战。基于Peer-to-Peer （P2P ）架构的应用已经成为新一代互联 网应用的重要形式，P2P 架构为应用带来了更好的扩展性、容错性和更 高的性能。基于 P2P 技术的存储系统也成为重要的研究热点，被认为 是 P2P 最具前途的应用之一。 本文讨论了一种基于 P2P 架构的高可扩展键值存储系统，该系统采 用一致性哈希算法和虚节点技术实现系统高可扩展性的同时保持了负 载平衡，为了降低数据查询延迟，每个存储节点都保存了系统全局的 路由信息，读写操作都能经过一跳到达相应的存储节点。系统采用基 于 Heartbeat 的层次化节点路由信息维护方式使得节点加入退出消息能 够快速、准确地通知到每个节点。该存储系统把每份数据分别存储到 多台服务器以提高数据的可用性，应对存储节点和网络故障造成的数 据丢失。 关键词：网络存储，分布式哈希表，P2P 网络 Research and Design of Storage Clusters for Social Networking Websites ABSTRACT Recent years have seen rapid development and prevalence of Social Networking services. New challenges have been posed to storage system design because these services usually save peta-bytes of people’s data. Peer-to-Peer (P2P) architecture has been one of the most promising architectures for content sharing applications for its inherent scalability, fault tolerance, and high performance. This paper presents the design of a highly scalable key-value storage system for building latency sensitive applications. The design of the storage system synthesizes some well known techniques to achieve high scalability as well as O(1) lookup performance. To be able to scale up to an extremely large system, the design adopts a decentralized and symmetrical architecture and uses consistent hashing for partitioning. Every node keeps a full membership of all the nodes in the system and can route a lookup in one hop. Server and network failures occur at any time and the storage system is dynamic for nodes join and leave the system continually. We design a hierarchical h