Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

对Qwen2-VL的继续预训练异常，报No dot product attention support for the provided inputs! #414

Open
yyli2022 opened this issue Jan 5, 2025 · 0 comments
Open

对Qwen2-VL的继续预训练异常，报No dot product attention support for the provided inputs! #414

yyli2022 opened this issue Jan 5, 2025 · 0 comments

Comments

@yyli2022
Copy link

yyli2022 commented Jan 5, 2025

预训练异常，前辈们有什么解决方案呀？感谢~

命令如下：
sh run_mcore_qwen.sh
dsw
7B
1
32
1e-5
1e-6
2048
2048
bf16
2
2
1
true
false
true
false
100000
/app_data/images_dock_0104/wds
/app_data/images_dock_0104/wds
/app_data/qwen/Qwen2-VL-7B-Instruct-tp2pp2-0105
20000
200
/app_data/output/mcore_qwen2vl_pretrain
image
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@yyli2022