Missing files when saving vision models

[Ammar-Alnagar]

when saving a finetuned multimodal llm using :

True: model.save_pretrained_merged
True: model.push_to_hub_merged(

the missing files are the following :
1-tokenizer.json
2-tokenizer_config.json
3-special_toekns_map.json
4-model.safetensors.index
5-preprocessor_config.json
6-chat_template.json

[Ammar-Alnagar]

and whenever i try loading them i get the following error : ValueError: Supplied state dict for vision_model.global_transformer.layers.0.mlp.fc1.weight does not contain bitsandbytes__* and possibly other quantized_stats components.

[ThomasDh-C]

Same issue here with those files not being created for either of those commands

[Ammar-Alnagar]

the files issues is semi-solved , only file is missing right now is model.safetensors.index but the error when loading is still there

[ThomasDh-C]

This function currently:

1. Saves the tokenizer config to a folder
2. Saves the model config to a folder
3. Pushes that folder to Hub
4. Downloads only safetensor files for original model, applies lora to those and pushes each fixed safetensor independently

This means that all the other files for the original model aren't pushed to the Hub (or save to local folder). Copying over these files from the original model fixes the issue.

[danielhanchen]

Oh hey - much apologies on the delay @ThomasDh-C @Ammar-Alnagar - I was trying to refactor everything to make saving much simpler - it looks like I might have forgotten to download all other files from the original repo - I'll recheck and fix it! HNY as well!

[Ammar-Alnagar]

@danielhanchen it's totally alright, i think the major issue here is the one with loading, most probably due to dynamic quantization.