Do NOT add Ovis2.5 model to transfromers package

[hfassold]

Please do NOT add Ovis2.5 model to official transformers package.

Otherwise, we will have the same problem as with Ovis2. Ovis2 runs with transformers 4.49, but not with transformers 4.55 as your implementation clashes with the 'official' one from the transformers team. The clash occurs because they use the same id for thr model architecture

[thisisiron]

Could you explain what the problem is?

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 549, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1346, in from_pretrained
    config_class = get_class_from_dynamic_module(
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 616, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module, force_reload=force_download)
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 311, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/iron/.cache/huggingface/modules/transformers_modules/AIDC_hyphen_AI/Ovis2_hyphen_2B/63d303670a24612b2d0dd368490fcd7f75989d73/configuration_ovis.py", line 14, in <module>
    AutoConfig.register("aimv2", AIMv2Config)
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1401, in register
    CONFIG_MAPPING.register(model_type, config, exist_ok=exist_ok)
  File "/home/iron/venv_vlmeval/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1081, in register
    raise ValueError(f"'{key}' is already used by a Transformers config, pick another name.")
ValueError: 'aimv2' is already used by a Transformers config, pick another name.

If the issue is the "aimv2" collision shown in the traceback, then this is not caused by Ovis2.
It happens because "aimv2" was added to the 'transformers' library, which now conflicts with the custom registration.

To fix this, you can update the Hugging Face code by changing the registration to:

AutoConfig.register("aimv2", AIMv2Config, exist_ok=True)

This resolves the conflict.

[hfassold]

Thanks. I think this is exactly the problem.

How to apply this fix ? Do I have to modify the source code of the 'transformers' python package and build it then from source ?
That's not practical, won't do that.

[thisisiron]

In the traceback above, you can see that the following path exists(check your Hugging Face .cache directory):

/home/iron/.cache/huggingface/modules/transformers_modules/AIDC_hyphen_AI/Ovis2_hyphen_2B/63d303670a24612b2d0dd368490fcd7f75989d73/configuration_ovis.py

Inside this file, you will find a line like:

AutoConfig.register("aimv2", AIMv2Config)

To fix the error, simply modify it to:

AutoConfig.register("aimv2", AIMv2Config, exist_ok=True)

This allows the registration to proceed even if "aimv2" is already registered in the Transformers library.
You don't need to modify the Transformers library itself.