Helm Chart Lacks Clear Support for Multi-Node vLLM Deployment

[shohamyamin]

The current Helm chart does not explicitly support deploying vLLM across multiple vllm nodes on Kubernetes, or it's unclear how to configure it. Improved documentation or multi-node support is needed for deploying LLM that require multi-node

[ahg-g]

For reference, and if someone would like to add that to helm charts, the LeaderWorkerSet API can be used to deploy multi-node vllm on k8s: https://docs.vllm.ai/en/latest/deployment/frameworks/lws.html

Also check examples on the LWS repo: https://github.com/kubernetes-sigs/lws/tree/main/docs/examples/vllm

[YuhanLiu11]

Thanks for submitting the issue!

Multi-node deployment should be supported. If you encounter any issues running a multi-node deployment, feel free to let us know.

We will improve the documentation to clarify how to configure multi-node deployment on Kubernetes.