-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[multimodal] llava-1.5-7b-hf doesn't work on mmmu_val
#2360
Comments
Hi! We'll take a look at this. If I recall correctly this is due to an inconsistency in the input formats for this model as compared to other HF |
thanks for the quick reply! it doesn't seem to be just llava-v1.5-7b however. I have some issues with Idefics2-8b as well. Versions:
Command:
Traceback:
|
Thanks @BabyChouSr , this is helpful-- in our testing we found idefics2 would run and avoid this error if setting ( @baberabb also making you aware of this thread in case you hadn't seen it!) |
for |
Reproduction:
Error:
I tried using vllm but they have also have an issue with the number of image tokens = 4 * 576 = 2304 != the number of image placeholders being 2305.
The text was updated successfully, but these errors were encountered: