Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running mamba seems to run single threaded on NixOs with AMD CPU #2571

Open
Corallus-Caninus opened this issue Oct 20, 2024 · 1 comment
Open

Comments

@Corallus-Caninus
Copy link

I am running cargo run --release --example mamba -- prompt "Mamba is the" on amd cpu without cuda on a fresh clone of candle. It seems to only use single threading.

Is this expected? I tried setting RAYON_NUM_THREADS but still single threaded. I installed rust toolchain using nixos and ensured I have all the required llvm dependencies, also using nixos packages.

@Corallus-Caninus
Copy link
Author

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
REDACTED 529448 94.8 15.3 12174100 11356460 pts/8 Rl+ 15:51 41:24 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.9 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:43 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.8 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:39 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.7 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:38 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.7 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:37 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.6 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:36 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.6 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:35 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.5 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:33 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.4 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:31 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.4 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:28 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.2 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:25 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 3.1 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:21 target/release/examples/mamba-trainer --prompt blah --which mamba130m
REDACTED 529448 2.9 15.3 12174100 11356460 pts/8 Sl+ 15:51 1:15 target/release/examples/mamba-trainer --prompt blah --which mamba130m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant