[QEff. Finetuning]: Adding FinetuningPipeline (finetune_experiemental.py) and related code

[quic-swatia]

1. Added FinetuningPipeline (finetune_experiemental.py) which integrates all the components added for HF-trainer and enable running fine tuning through it.
2. Added files to handle PEFT and training config.
3. Made changes in the config_manager and callbacks files.
4. Added unit tests for the FinetuningPipeline (test_finetune.py)
5. Updated tests in test_callback and test_config_manager based on above changes.

[quic-akuruvil]

Please update the documentation also, with sample commands for the user, on how to use the Finetune Api, with and without the sample config file

[quic-akuruvil]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

[quic-swatia]

Please update the documentation also, with sample commands for the user, on how to use the Finetune Api, with and without the sample config file

It's already merged through Tanisha's #747 PR.

[quic-swatia]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

Tanisha observed this error due to incorrect accelerate package version. Error got resolved for her with the correct version. Inside the docker since accelerate is already installed, she didn't observe it.

[quic-akuruvil]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

Tanisha observed this error due to incorrect accelerate package version. Error got resolved for her with the correct version. Inside the docker since accelerate is already installed, she didn't observe it.

Is losses converging now? Let's check that for at least one of the test configs(llama-1b +gsm8k) and match the metrics for train and eval loop.

[quic-akuruvil]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

Tanisha observed this error due to incorrect accelerate package version. Error got resolved for her with the correct version. Inside the docker since accelerate is already installed, she didn't observe it.

Is losses converging now? Let's check that for at least one of the test configs and match the metrics for train and eval loop.

Please include the metrics values here for reference as well.

[quic-swatia]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

Tanisha observed this error due to incorrect accelerate package version. Error got resolved for her with the correct version. Inside the docker since accelerate is already installed, she didn't observe it.

Is losses converging now? Let's check that for at least one of the test configs and match the metrics for train and eval loop.

Please include the metrics values here for reference as well.

Yes, the loss is converging. Following are the metrics of finetuning "meta-llama/Llama-3.2-1B for 5 epochs on single SOC: {"eval_loss":1.0224987268447876,"eval_runtime":484.8933,"eval_samples_per_second":2.72,"eval_steps_per_second":2.72,"eval_entropy":0.9871161538059735,"eval_num_tokens":6525025.0,"eval_mean_token_accuracy":0.7452040632806826,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365}
{"train_runtime":32856.1501,"train_samples_per_second":1.137,"train_steps_per_second":1.137,"total_flos":3.8132170931712e+16,"train_loss":1.0178058738101043,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365}

Training loss at the start of training :1.5146,

[quic-akuruvil]

Also when we tested earlier, the training pipeline was failing due to model and inputs being at different devices. Both model and inputs need to be in QAIC. Is that fixed, tested and verified here.

Tanisha observed this error due to incorrect accelerate package version. Error got resolved for her with the correct version. Inside the docker since accelerate is already installed, she didn't observe it.

Is losses converging now? Let's check that for at least one of the test configs and match the metrics for train and eval loop.

Please include the metrics values here for reference as well.

Yes, the loss is converging. Following are the metrics of finetuning "meta-llama/Llama-3.2-1B for 5 epochs on single SOC: {"eval_loss":1.0224987268447876,"eval_runtime":484.8933,"eval_samples_per_second":2.72,"eval_steps_per_second":2.72,"eval_entropy":0.9871161538059735,"eval_num_tokens":6525025.0,"eval_mean_token_accuracy":0.7452040632806826,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365} {"train_runtime":32856.1501,"train_samples_per_second":1.137,"train_steps_per_second":1.137,"total_flos":3.8132170931712e+16,"train_loss":1.0178058738101043,"epoch":5.0,"num_input_tokens_seen":6525025,"global_step":37365}

Training loss at the start of training :1.5146,

@quic-swatia This looks good. Training entropy is not logged, please add that. And check for multi soc convergence as well.