基于记忆体高维相似度搜索计算方法.pdf

A Memory-Driven Computing Approach to High-Dimensional Similarity Search Krishnamurthy Viswanathan, Mijung Kim, Jun Li, Maria Gonzalez Hewlett Packard Labs HPE-2016-45 Keyword(s): High-dimensional similarity search; image search; memory-driven computing Abstract: Given a query image, how can one perform a fast search on a large repository of images and accurately retrieve images most similar to the query image? This is an example of a high-dimensional similarity search problem. High-dimensional similarity search is an important component of many analytic tasks in information retrieval, machine learning and computer vision. We demonstrate how big-memory multicore computing platforms can enable fast high-dimensional similarity search over large repositories of data, by building similarity search indexes based on Locality-Sensitive Hashing (LSH). These indexes are large and require fast random access, thus making a Memory-Driven Computing approach a natural fit for our solution. We developed our similarity search prototype on a cluster of 4 HPE Proliant DL980 servers. On a repository containing 80 million images, the prototype achieves a 12x speed-up over a highly-optimized linear scan, with an average response time of 110 ms and search accuracy of 95%, while consuming 3.7 TB for in-memory search indexes. External Posting Date: May 11, 2016 [Fulltext] Internal Posting Date: May 11, 2016 [Fulltext] 铮〤opyright 2016 Hewlett Packard Enterprise Development LP 1 A Memory-Driven Computing Approach to High-Dimensional Similarity Search Krishnamurthy Viswanathan, Mijung Kim, Jun Li, and Maria Gonzalez Hewlett