qwen2-vl infer report 'RuntimeError: Key and Value must have the same sequence length'

[sun11]

Reminder

• I have read the README and searched the existing issues.

System Info

• llamafactory version: 0.9.2.dev0
• Platform: Linux-6.5.0-26-generic-x86_64-with-glibc2.35
• Python version: 3.10.15
• PyTorch version: 2.2.0+cu118 (GPU)
• Transformers version: 4.46.3
• Datasets version: 3.1.0
• Accelerate version: 1.0.1
• PEFT version: 0.12.0
• TRL version: 0.9.6
• GPU type: NVIDIA GeForce RTX 4090
• DeepSpeed version: 0.15.4
• Bitsandbytes version: 0.44.1

Reproduction

FORCE_TORCHRUN=1 llamafactory-cli train examples/predict.yaml
训练qwen2-vl-7b模型一切正常，推断时报错:
'RuntimeError: Key and Value must have the same sequence length'

回退到commit:bcb4fb353ed810477b7813c6904b33c51efb7f81后推断qwen2-vl-7b正常，即fix mrope修改前

Expected behavior

No response

Others

No response