Add support for Idefics 3

[stelterlab]

Model description

Please add support for HuggingFaceM4/Idefics3-8B-Llama3 in tgi:

Idefics3 is an open multimodal model that accepts arbitrary sequences of image and text inputs and produces text outputs. The model can answer questions about images, describe visual content, create stories grounded on multiple images, or simply behave as a pure language model without visual inputs.

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

Well, the necessary changes for the transformers library are just waiting for a review for the PR:

huggingface/transformers#32473

as the time of writing this model request.

As model/finetune and transformers lib is made by the same famous company I would assume there should be no big problems. ;-)

[ErikKaum]

Hi @stelterlab 👋

We have a PR in the making, no big problems indeed ;) but we are a bit constrained on bandwidth at the moment, so it's not moving as fast as we'd like

[efenocchi]

Hi @ErikKaum I noticed a bug, could you check my last comment in the PR?

[ErikKaum]

Hi @efenocchi I'm unfortunately not super well versed in the transformers library. I'd consider reaching out to the people in the conversation you have in the repo 👍