You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
to avoid NCCL shared memory allocation issues as described in vllm-project/vllm#6574. It might be nice to add this to the tutorial steps and expose an expedient configuration option here for anyone facing similar issues.
Thanks!
The text was updated successfully, but these errors were encountered:
I had difficulty launching this stack successfully out of the box, and had to manually increase the Docker
/dev/shm
with an additional volumeMount as:to avoid NCCL shared memory allocation issues as described in vllm-project/vllm#6574. It might be nice to add this to the tutorial steps and expose an expedient configuration option here for anyone facing similar issues.
Thanks!
The text was updated successfully, but these errors were encountered: