大规模数据存储-haystack-osdi10.pdfVIP

Finding a needle in Haystack: Facebook’s photo storage Doug Beaver, Sanjeev Kumar, Harry C. Li, Jason Sobel, Peter Vajgel, Facebook Inc. doug, skumar, hcli, jsobel, pv@ Abstract: This paper describes Haystack, an object stor- and thereby wastes storage capacity. Yet the more sig- age system optimized for Facebook’s Photos applica- nificant cost is that the file’s metadata must be read from tion. Facebook currently stores over 260 billion images, disk into memory in order to find the file itself. While which translates to over 20 petabytes of data. Users up- insignificant on a small scale, multiplied over billions load one billion new photos (60 terabytes) each week of photos and petabytes of data, accessing metadata is and Facebook serves over one million images per sec- the throughput bottleneck. We found this to be our key ond at peak. Haystack provides a less expensive and problem in using a network attached storage (NAS) ap- higher performing solution than our previous approach, pliance mounted over NFS. Several disk operations were which leveraged network attached storage appliances necessary to read a single photo: one (or typically more) over NFS. Our key observation is that this traditional to translate the filename to an inode number, another to design incurs an excessive number of disk operations read the inode from disk, and a final one to read the because of metadata lookups. We carefully reduce this file itself. In short, using disk IOs for metadata was the per photo metadata so that Haystack storage machines limiting factor for our read throughput. Observe that in can perform all metadata lookups in main memory. This practice this prob