Efficient Vector Search on Disaggregated Memory with d-HNSW

Abstract: Efficient vector query processing is critical to enable AI applications at scale. Recent solutions struggle with growing vector datasets that exceed single-machine memory capacity, forcing unnecessary data movement and resource underutilization in monolithic architectures. We present d-HNSW, the first disaggregated vector similarity search engine for RDMA-based remote memory systems that achieves high performance while supporting fast data indexing with low network communication overhead. The core of d-HNSW is a novel disaggregation of the graph-based vector indexing data structure HNSW. It exploits the characteristics of greedy searching in HNSW to efficiently coordinate data transfers from the memory pool to the compute pool while serving data requests. Specifically, it leverages three ideas: (i) Representative index caching, a lightweight index constructed from a sampled subset of data, is cached in the compute pool to reduce frequent access to critical components of the hierarchical graph-based index, (ii) RDMA-friendly data layout design to reduce the networking round trips incurred by vector query and insertion and (iii) batched query-aware data loading to reduce bandwidth usage on data transfer between pools, addressing the limited cache capacity in compute nodes. We evaluate d-HNSW with extensive benchmarking datasets. The experimental results show that d-HNSW outperforms Naive d-HNSW implementation by up to 117x in latency while maintaining recall as 0.87 in dataset SIFT1M@1.