GPU_BRUTE_FORCE Index using cpu

[Rajat376]

The GPU_BRUTE_FORCE Index is using cpu when batch size (nq) is small.
Read about 'gpu_search_threshold' param but can't find it in milvus.yaml of newer version.
Is there another name for it now, or is there another way to set it?

[Presburger]

@Rajat376 Hello, where did you get this conclusion? At the index level, there shouldn't be such logic.

[yhmo]

"GPU_BRUTE_FORCE Index is using cpu when batch size (nq) is small" --------------------- We don't have such logic in source code. GPU_BRUTE_FORCE always uses GPU to compute.

How did you observe the GPU/CPU usage?
If the collection only has a few data, the search is very fast, and maybe the nvidia-smi tool could not observe the GPU actions in short milliseconds.

[Rajat376]

The collections are large over 1M
I sent a lot of load on milvus with 80 threads :
(a) when batch size is small(50) ->about 2500% cpu
(b) batch size is large(5000) -> about 500%cpu
This is a common issue for me.
The milvus search sometimes start using very high cpu(usually when load is high) and the search time increases even if gpu is not utilized completely.

[xiaofan-luan]

The collections are large over 1M I sent a lot of load on milvus with 80 threads : (a) when batch size is small(50) ->about 2500% cpu (b) batch size is large(5000) -> about 500%cpu This is a common issue for me. The milvus search sometimes start using very high cpu(usually when load is high) and the search time increases even if gpu is not utilized completely.

Because CPU will still be used under many different places, like scalar filtering, rpc....
When batch is larger, each batch share the same scalar filtering, mvcc and delete processing, so more request can be sent to GPU.
Try to use pprof and analyze the cpu usage and we can check how to improve it
@Rajat376