Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to configure /dev/shm size #44

Closed
nrober734 opened this issue Jan 30, 2025 · 2 comments
Closed

Document how to configure /dev/shm size #44

nrober734 opened this issue Jan 30, 2025 · 2 comments
Labels
documentation Improvements or additions to documentation

Comments

@nrober734
Copy link

I had difficulty launching this stack successfully out of the box, and had to manually increase the Docker /dev/shm with an additional volumeMount as:

volumeMounts:
  - name: {{ .Release.Name }}-storage
    mountPath: /data
  - name: shm
    mountPath: /dev/shm
  
volumes:
    - name: shm
      emptyDir:
        medium: Memory
        sizeLimit: 20Gi

to avoid NCCL shared memory allocation issues as described in vllm-project/vllm#6574. It might be nice to add this to the tutorial steps and expose an expedient configuration option here for anyone facing similar issues.

Thanks!

@nrober734 nrober734 changed the title Document how to adjust or else configure /dev/shm size Document how to configure /dev/shm size Jan 30, 2025
@ApostaC
Copy link
Collaborator

ApostaC commented Jan 30, 2025

Thanks for the contribution! Would you like to create a PR for this issue?

@ApostaC ApostaC added the documentation Improvements or additions to documentation label Feb 7, 2025
@ApostaC
Copy link
Collaborator

ApostaC commented Feb 11, 2025

Closing this issue as it's completed by PR #105

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

2 participants