[QEff. Finetuning] Adding finetune_experiemental.py and related files#731
[QEff. Finetuning] Adding finetune_experiemental.py and related files#731quic-swatia wants to merge 5 commits intoquic:ft_experimentalfrom
Conversation
Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com>
quic-swatia
requested review from
ochougul,
quic-amitraj,
quic-hemagnih and
quic-rishinr
as code owners
quic-swatia
changed the title
quic-akuruvil
left a comment
quic-akuruvil
left a comment
There was a problem hiding this comment.
Check and verify the functionality of python -m Qefficient.finetune_experimental.py with the new stack.
Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com>
quic-swatia
force-pushed
the
HF-Trainer-main
branch
from
8235e30 to
d0d3251
Compare
quic-swatia
force-pushed
the
HF-Trainer-main
branch
2 times, most recently
from
0043eaa to
d0d3251
Compare
Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com>
Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com>
quic-swatia
force-pushed
the
HF-Trainer-main
branch
from
b0ea868 to
a30e5a1
Compare
quic-akuruvil
left a comment
quic-akuruvil
left a comment
There was a problem hiding this comment.
--enble_pp, pipeline parallelism support is also missing. Please add that too.
| try: | ||
| import torch_qaic # noqa: F401 | ||
| except ImportError as e: | ||
| logger.log_rank_zero( |
There was a problem hiding this comment.
Here we are passing and not blocking. It will be fine here but if user has provided device="qaic" and torch_qaic is loaded then we need to break the execution. This has to happen either here or inside ConfigManager. I believe this kind of validation and all other validations about config should reside inside ConfigManager.
CC: @tchawada
|
Please check and need to add trl library also in requirements, if not already present. |
…th recetly merged PRs. Made changes in the test files as well accordinlgy. Signed-off-by: Swati Allabadi <sallabad@qti.qualcomm.com>
Its's working with this PR. |
PP enablement will be done iteratively in the subsequent PR. |
| # callback_config.callbacks is a dictionary of callback configurations | ||
| for callback_name, callback_kwargs in callback_config["callbacks"].items(): | ||
| try: | ||
| callback_instance = create_callbacks(callback_name, **callback_kwargs) |
There was a problem hiding this comment.
Currently it is not handling case in which callback_kwargs is None, can you see it once.
| Execute the complete fine-tuning pipeline. | ||
| """ | ||
| # Validate configuration | ||
| self.config_manager.validate_config() |
| try: | ||
| import torch_qaic # noqa: F401 | ||
| except ImportError as e: | ||
| logger.log_rank_zero( |
4 participants
Uh oh!
There was an error while loading. Please reload this page.