Add script to write Llama's HF-formatted config.json for vLLM

[MartinGleize]

What does this PR do? Please describe:
Add script to write HF-format config.json for Llama. This config.json is then used to load fairseq2 Llama checkpoints directly in vLLM.

Does your PR introduce any breaking changes? If yes, please list them:
No

Check list:

• Was the content of this PR discussed and approved via a GitHub issue? (no need for typos or documentation improvements)
• Did you read the contributor guideline?
• Did you make sure that your PR does only one thing instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• Did you update the CHANGELOG? (no need for typos, documentation, or minor internal changes)

[cbalioglu]

LGTM, just left two nit comments.

[cbalioglu]

I think this is fine, but would be great if we could also have a way to automatically dump this config.json in LLM finetuning recipes.

[MartinGleize]

Agreed!

[cbalioglu]

At least the content of the PR looks good to me. Assuming that the generated config.json works with vLLM, I think it is good to g.

[uralik]

looking great! @MartinGleize We might need to test this when we have experiment that started from some other experiment checkpoint (using resume from ckpt dir argument). In this case the model name might be just "checkpoint_step_N", and smth could be not retrieved as planned into the final config. We have a bunch of such examples on our side, can help with testing in this branch if needed.

[cbalioglu]

looking great! @MartinGleize We might need to test this when we have experiment that started from some other experiment checkpoint (using resume from ckpt dir argument). In this case the model name might be just "checkpoint_step_N", and smth could be not retrieved as planned into the final config. We have a bunch of such examples on our side, can help with testing in this branch if needed.

Let me merge AssetCard.flatten() tomorrow morning. It has been ready for a long time. I was aiming to merge it with recipe refactoring, but can do it separately for this PR.

[cbalioglu]

Ok, looks like a bug in the reference implementation: pytorch/executorch#7265

[uralik]

Ok, looks like a bug in the reference implementation: pytorch/executorch#7265

also discussed here (and not resolved looks like): meta-llama/llama-models#241

[cbalioglu]

flatten(). fyi.

card = card.flatten()

will produce an asset card with all base metadata merged into one.

[cbalioglu]

@MartinGleize Let me know if you need a hand with making RoPE scaling factor configurable. I believe it is a good nit exercise to work on, but happy to help if you need bandwidth for other parts of the project.

[MartinGleize]

@MartinGleize Let me know if you need a hand with making RoPE scaling factor configurable. I believe it is a good nit exercise to work on, but happy to help if you need bandwidth for other parts of the project.

Should be no problem for me!

[cbalioglu]

Nice, looks good to me.

[MartinGleize]

I'm still wondering about tie_word_embeddings=true in the config.json (was trying to get rid of the last use of "is_llama_3_2"). Our impl and the ref impl don't have any mention of this or code (as far as I can see), but on HG they indeed have this setting to true in their config. Am I missing something or should we hard-code it to false on our side? That seems pretty important.

[cbalioglu]

We definitely do not use tied weights for LLaMA. I am not sure how HG came up with that setting as I don't see any weight tying in the reference implementation either. I believe we have to set it to False, since our checkpoints will have different weights for the embeddings and final projection layer.