Bug: 【CANN】ggml-cann/aclnn_ops.cpp:3007: GGML_ASSERT(n_dims == src0->ne[0]) failed

[zyp2]

What happened?

按照readme中的固件驱动版本，推理时出现报错

Name and Version

最新版本

What operating system are you seeing the problem on?

No response

Relevant log output

llama_new_context_with_model: freq_scale    = 1
llama_new_context_with_model: n_ctx_per_seq (4096) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_kv_cache_init:      CANN0 KV buffer size =   132.00 MiB
llama_kv_cache_init:        CPU KV buffer size =    28.00 MiB
llama_new_context_with_model: KV self size  =  160.00 MiB, K (f16):   80.00 MiB, V (f16):   80.00 MiB
llama_new_context_with_model:        CPU  output buffer size =     0.58 MiB
llama_new_context_with_model:      CANN0 compute buffer size =  1488.00 MiB
llama_new_context_with_model:  CANN_Host compute buffer size =    16.01 MiB
llama_new_context_with_model: graph nodes  = 1606
llama_new_context_with_model: graph splits = 67 (with bs=512), 3 (with bs=1)
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
/docker_files/zyp/202411/llama.cpp-master/ggml/src/ggml-cann/aclnn_ops.cpp:3007: GGML_ASSERT(n_dims == src0->ne[0]) failed
Aborted (core dumped)

[zyp2]

910B server

[MengqingCao]

Could you provide the command to reproduce the problem? Including the model and other info.

[zyp2]

model_name：
glm-4-9b-chat
Weight conversion command：
python convert_hf_to_gguf.py --outfile /docker_files/zyp/202411/ggufs/glm4.gguf /docker_files/zyp/weight/glm-4-9b-chat
start inference command：
./build/bin/llama-cli -m /docker_files/zyp/202411/ggufs/glm4.gguf -p "请推荐一部电视剧:" -n 400 -e -ngl 33 -sm none -mg 0

[hipudding]

Not all model is support by Ascend NPU. Please check this table，This assert is that operator RoPE not support all kinds of input shapes, we are still working on it now.

[slaren]

The crash could be avoided by reporting that this operation is not supported in the supports_op function of the backend.

[hipudding]

The crash could be avoided by reporting that this operation is not supported in the supports_op function of the backend.

Yes, We will add this to supports_op function.

[zyp2]

ok,thanks

…

---- Replied Message ---- | From | ***@***.***> | | Date | 11/25/2024 14:07 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [ggerganov/llama.cpp] Bug: 【CANN】ggml-cann/aclnn_ops.cpp:3007: GGML_ASSERT(n_dims == src0->ne[0]) failed (Issue #10451) | The crash could be avoided by reporting that this operation is not supported in the supports_op function of the backend. Yes, We will add this to supports_op function. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[hipudding]

@zyp2 This bug has fixed. The performance is low because RoPE will fallback to CPU.