Windows 11 ggml_gallocr_reserve_n: failed to allocate Vulkan0 #413
Replies: 2 comments 2 replies
-
ok doing this
Helps me work without using my gpu. But i guess there is still a problem with the gpu loader with my card (all is up to date the ati driver and vulkan is installed too) |
Beta Was this translation helpful? Give feedback.
-
Can you please provide me with a link to the model that didn't stop generation output? It'll help me investigate it. Regarding the Vulkan error you saw, it doesn't necessarily mean that the context loading failed - there's a maximum memory allocation size in Vulkan, and the only way to find it currently is to test it, so this is what |
Beta Was this translation helpful? Give feedback.
-
Hello,
So i'm trying to upload some model and i tried a few on with windows.
I had an 8b one that kept showing me the error llm_load_vocab: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect
But it was loading and giving answers however it was never ending and it kept writing
You are a helpful AI assistant. <</SYS>>[/INST] Bye! <</SYS>>[/INST] Bye!
I tried to load several other that i took from hugging face after (even some 1b) and i got that error all the time
Since the 8b was like 8gb and this one 800mb i know its not that i'm out of memory
I ran those command to check
In the end i created a docker based on node22-slim and loaded that model and it seems to be working well tho a bit slow. I guess its something wrong with my llama.cpp but i'm not sure if there are ways to rebuild it. I'm open to suggestions right now
Beta Was this translation helpful? Give feedback.
All reactions