Is DiskANN also Memory Intensive?

[xiaobingxia-at]

Hello, I'm exploring DiskANN using Milvus. I was assuming that the DiskANN index takes no memory or very little memory.
But after ingesting about 10million 1536-dim vectors, the disk usage comes to ~80GB. And the memory usage comes to ~40GB.
The memory usage is far more than I expected. Is this normal?

[xiaobingxia-at]

PQCodeBudgetGBRatio
SearchCacheBudgetGBRatio
These two parameters should impact the memory usage. But which node memory? Query node or indexing node?

[xiaofan-luan]

10M 1536dim data is around 60GB, so 80GB disk usage is reasaonble.

For Memory usage, here is some part may table memory:

1. Store quantinized data into memory, based on the PQ you pick, this part usually takes 1/8 - 1/16 of your raw data
2. Cache - Our suggestion is to take 10-20% of your raw data.
3. Index build memory
4. Scalar datas - better to enable mmap on the scalar datas.

So memory can definitely reduced to 10GB-20GB but for diskANN it typically takes 1/10 - 1/5 of your raw data size.

[liliu-z]

DiskANN has three four parts: PQ, Graph, Raw Data and Cache. Graph and Raw Data are placed in disk, and the other two are in memory.
These two params affect memory usage of QueryNodes:

1. PQCodeBudgetGBRatio bigger PQ provides better recall but needs more memory
2. SearchCacheBudgetGBRatio larger cache provides better QPS but still, more memory.

[xiaofan-luan]

DiskANN has three four parts: PQ, Graph, Raw Data and Cache. Graph and Raw Data are placed in disk, and the other two are in memory. These two params affect memory usage of QueryNodes:

1. PQCodeBudgetGBRatio bigger PQ provides better recall but needs more memory
2. SearchCacheBudgetGBRatio larger cache provides better QPS but still, more memory.

What could be a recommened valuae?

[liliu-z]

DiskANN has three four parts: PQ, Graph, Raw Data and Cache. Graph and Raw Data are placed in disk, and the other two are in memory. These two params affect memory usage of QueryNodes:

1. PQCodeBudgetGBRatio bigger PQ provides better recall but needs more memory
2. SearchCacheBudgetGBRatio larger cache provides better QPS but still, more memory.

What could be a recommened valuae?

For 1536dim data, setting PQCodeBudgetGBRatio to 0.075-0.125 is usually safe to get 90-98% recall
And about SearchCacheBudgetGBRatio, as you said, 10%-20% can provide good performance improvement. The performance improvement of higher number will be marginal. But it really depends on your disk resource. Plz ensure to use NVMe SSD.