We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
如题
qlora只是降低训练的时候显存占用是吧?训完还是一个未量化的模型?训完我merge lora后还是一个未量化的模型,比lora训出来的模型要掉一些点数。 所以我理解qlora和lora一样只是穷人玩法,用qlora训完再量化是比lora训完量化指标要低的。如果想最优还是全参数微调再量化。
No response
The text was updated successfully, but these errors were encountered:
是的
Sorry, something went wrong.
No branches or pull requests
Reminder
System Info
如题
Reproduction
qlora只是降低训练的时候显存占用是吧?训完还是一个未量化的模型?训完我merge lora后还是一个未量化的模型,比lora训出来的模型要掉一些点数。
所以我理解qlora和lora一样只是穷人玩法,用qlora训完再量化是比lora训完量化指标要低的。如果想最优还是全参数微调再量化。
Expected behavior
No response
Others
No response
The text was updated successfully, but these errors were encountered: