Skip to content

Commit

Permalink
Bug fixes (#1473)
Browse files Browse the repository at this point in the history 
* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors

* Update cross_entropy_loss.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Throw error when inferencing longer than max_popsition_embeddings (#1236)

* Throw error when inferencing longer than max_popsition_embeddings without rope scaling

* Update llama.py

---------

Co-authored-by: Daniel Han <[email protected]>

* CLI now handles user input strings for dtype correctly (#1235)

Co-authored-by: root <[email protected]>

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update flex_attention.py

* Update loader.py

* Update loader.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* triton_cast

* Update utils.py

* Qwen 2.5 Coder

* Fix/export mistral (#1281)

* Enhance install_python_non_blocking to handle protobuf installation and process management

* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"

This reverts commit f09974b.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"

This reverts commit 9fc1307.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Update __init__.py

---------

Co-authored-by: Daniel Han <[email protected]>

* DOC Update - Update README.md with os.environ in example (#1269)

* Update README.md with os.environ in example

Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup. 
As currently the  unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials

* Update README.md

---------

Co-authored-by: Daniel Han <[email protected]>

* fix/get_chat_template (#1246)

* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to

* Remove type hinting

* Update chat_templates.py

---------

Co-authored-by: Daniel Han <[email protected]>

* fix/sft-trainer (#1276)

* Add patch for SFTTrainer to maintain backward compatibility with TRL changes

* Update trainer.py

* Update trainer.py

* Refactor trainer patch to maintain backward compatibility with TRL changes

* Update trainer.py

* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch

---------

Co-authored-by: Daniel Han <[email protected]>

* Update __init__.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update tokenizer_utils.py

* Update llama.py

* Fix #853

* fix/sfttrainer-compatibility (#1293)

* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance

* Update trainer.py

* Update trainer.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Update rms_layernorm.py

* Update rms_layernorm.py

* Gemma

* Update rms_layernorm.py

* Update gemma2.py

* Cut Cross Entropy

* Update llama.py

* Cut Cross Entropy

* Update llama.py

* Update llama.py

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* patch_fast_lora

* vision

* Update fast_lora.py

* Update _utils.py

* Update _utils.py

* Vision

* Update trainer.py

* Update save.py

* FastBaseVisionModel

* Update loader_utils.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update _utils.py

* tokenizer_name

* Update loader.py

* Update vision.py

* Update save.py

* Update save.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update _utils.py

* Update loader.py

* kwargs

* logits

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* error

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* Update llama.py

* Update vision.py

* Update loader.py

* Old torch versions

* Update loader.py

* Update loader.py

* prints

* recheck

* Update loader.py

* Update loader.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Feat/kto (#1316)

* Add PatchKTOTrainer and update model imports

* Update dpo.py

* Update __init__.py

* Delete unsloth/models/kto.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Fix orpo/dpo trainer  (#1286)

* change the colab notebook for dpo zephyr and orpo

* use original tokenizer

* Update README.md

* Update README.md

---------

Co-authored-by: Daniel Han <[email protected]>

* skip modules

* Update vision.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Fix llama.cpp

* Update save.py

* Update save.py

* Update vision.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update _utils.py

* Update save.py

* Update save.py

* Update mapper.py

* modules

* Fix vision model tokenizer padding side. (#1384)

* Dynamic quants (#1379)

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

* Update _utils.py

* Update llama.py

* CE Loss

* Update cross_entropy_loss.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors

* Update cross_entropy_loss.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Throw error when inferencing longer than max_popsition_embeddings (#1236)

* Throw error when inferencing longer than max_popsition_embeddings without rope scaling

* Update llama.py

---------

Co-authored-by: Daniel Han <[email protected]>

* CLI now handles user input strings for dtype correctly (#1235)

Co-authored-by: root <[email protected]>

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update flex_attention.py

* Update loader.py

* Update loader.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* triton_cast

* Update utils.py

* Qwen 2.5 Coder

* Fix/export mistral (#1281)

* Enhance install_python_non_blocking to handle protobuf installation and process management

* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"

This reverts commit f09974b.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"

This reverts commit 9fc1307.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Update __init__.py

---------

Co-authored-by: Daniel Han <[email protected]>

* DOC Update - Update README.md with os.environ in example (#1269)

* Update README.md with os.environ in example

Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup. 
As currently the  unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials

* Update README.md

---------

Co-authored-by: Daniel Han <[email protected]>

* fix/get_chat_template (#1246)

* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to

* Remove type hinting

* Update chat_templates.py

---------

Co-authored-by: Daniel Han <[email protected]>

* fix/sft-trainer (#1276)

* Add patch for SFTTrainer to maintain backward compatibility with TRL changes

* Update trainer.py

* Update trainer.py

* Refactor trainer patch to maintain backward compatibility with TRL changes

* Update trainer.py

* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch

---------

Co-authored-by: Daniel Han <[email protected]>

* Update __init__.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update tokenizer_utils.py

* Update llama.py

* Fix #853

* fix/sfttrainer-compatibility (#1293)

* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance

* Update trainer.py

* Update trainer.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Update rms_layernorm.py

* Update rms_layernorm.py

* Gemma

* Update rms_layernorm.py

* Update gemma2.py

* Cut Cross Entropy

* Update llama.py

* Cut Cross Entropy

* Update llama.py

* Update llama.py

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* patch_fast_lora

* vision

* Update fast_lora.py

* Update _utils.py

* Update _utils.py

* Vision

* Update trainer.py

* Update save.py

* FastBaseVisionModel

* Update loader_utils.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update _utils.py

* tokenizer_name

* Update loader.py

* Update vision.py

* Update save.py

* Update save.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update _utils.py

* Update loader.py

* kwargs

* logits

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* error

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* Update llama.py

* Update vision.py

* Update loader.py

* Old torch versions

* Update loader.py

* Update loader.py

* prints

* recheck

* Update loader.py

* Update loader.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Feat/kto (#1316)

* Add PatchKTOTrainer and update model imports

* Update dpo.py

* Update __init__.py

* Delete unsloth/models/kto.py

---------

Co-authored-by: Daniel Han <[email protected]>

* Fix orpo/dpo trainer  (#1286)

* change the colab notebook for dpo zephyr and orpo

* use original tokenizer

* Update README.md

* Update README.md

---------

Co-authored-by: Daniel Han <[email protected]>

* skip modules

* Update vision.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Fix llama.cpp

* Update save.py

* Update save.py

* Update vision.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update save.py

* Update _utils.py

* Update save.py

* Update save.py

* Update mapper.py

* modules

---------

Co-authored-by: Edd <[email protected]>
Co-authored-by: Datta Nimmaturi <[email protected]>
Co-authored-by: Edwin Fennell <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Uday Girish Maradana <[email protected]>
Co-authored-by: cell-dame <[email protected]>

* Update README.md

Unsloth Dynamic 4-bit Quantization Update

* Fix vision model tokenizer padding side.

* Update vision.py

---------

Co-authored-by: Daniel Han <[email protected]>
Co-authored-by: Edd <[email protected]>
Co-authored-by: Datta Nimmaturi <[email protected]>
Co-authored-by: Edwin Fennell <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Uday Girish Maradana <[email protected]>
Co-authored-by: cell-dame <[email protected]>
Co-authored-by: Michael Han <[email protected]>

* Add citation section to README.md (#1377)

* Add citation section to README.md

* Update README.md

---------

Co-authored-by: Daniel Han <[email protected]>

* Granite support (#1218)

* [WIP] Support for Granite

* Fixup inference

* Cleanup flex attention

* remove sliding window

* Use torch.add for residual multiplier

* Llama 3.3

* Update llama.py

* Update llama.py

* fullgraph

* Fix loader.py to work on Windows (#1453)

* Update README.md

Llama 3.3 + Reddit

* Update README.md

Apple ML Cross Entropy

* Update README.md

Removing double citation

* Fix loader.py to work on Windows

---------

Co-authored-by: Michael Han <[email protected]>

* Update save.py warning message (#1425)

* Update README.md

Llama 3.3 + Reddit

* Update README.md

Apple ML Cross Entropy

* Update README.md

Removing double citation

* Update save.py warning message

---------

Co-authored-by: Michael Han <[email protected]>

* Change _fix_chat_template in case a template has both endif and endfor (#1388)

* Update llama and derivatives to pass position embeddings explicitly for transformers v4.47+ (#1442)

* Update save.py

* Update llama.py

* Update mistral.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Temp fix

* Update _utils.py

* Update _utils.py

* Update pyproject.toml

* Name Error Bug Fix - import from packaging.version import Version (#1468)

* Version

* Update pyproject.toml

* Update pyproject.toml

* Version

* Update pyproject.toml

* Update pyproject.toml

* dependencies

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update mistral.py

* Update pyproject.toml

* Update pyproject.toml

* Update pyproject.toml

* Update granite.py

* Update cohere.py

* Triton windows

* Update gemma2.py

* Update pyproject.toml

* Update _utils.py

* Update pyproject.toml

* Residual & LoRA

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Bug fix

* Update loader.py

* Update loader.py

* Update loader.py

* Update _utils.py

* Update loader.py

---------

Co-authored-by: Edd <[email protected]>
Co-authored-by: Datta Nimmaturi <[email protected]>
Co-authored-by: Edwin Fennell <[email protected]>
Co-authored-by: root <[email protected]>
Co-authored-by: Uday Girish Maradana <[email protected]>
Co-authored-by: cell-dame <[email protected]>
Co-authored-by: Zewen Shen <[email protected]>
Co-authored-by: Michael Han <[email protected]>
Co-authored-by: Scott Phillips <[email protected]>
Co-authored-by: qingy1337 <[email protected]>
Co-authored-by: Giulia Baldini <[email protected]>
Co-authored-by: Yonghye Kwon <[email protected]>
  • Loading branch information
@danielhanchen @Erland366
@Datta0 @Rabbidon @udaygirish @dame-cell @ZewenShen @shimmyshimmer @fundthmcalculus @qingy1337 @giuliabaldini @developer0hye
13 people authored Dec 26, 2024
1 parent a240783 commit 4fbbd7e
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 1 deletion.
22 changes: 21 additions & 1 deletion unsloth/models/_utils.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

__version__ = "2024.12.9"
__version__ = "2024.12.10"

__all__ = [
"prepare_model_for_kbit_training",
Expand Down Expand Up @@ -1093,6 +1093,22 @@ def patch_gradient_accumulation_fix(Trainer):
"if self.model_accepts_loss_kwargs:",
"if False:",
)

# Fix when num_items_in_batch is nothing
# https://github.com/huggingface/transformers/pull/35207
function = re.sub(
r"else:\n"\
r"([\s]{4,})self\.accelerator\.backward\(loss, \*\*kwargs\)\n"\
r"(.+?)if num_items_in_batch is None\:\n"\
r"(.+?)return loss\.detach\(\) \/ self\.args\.gradient_accumulation_steps",

"else:\n"\
"\2if num_items_in_batch is None:\n"\
"\3loss /= self.args.gradient_accumulation_steps\n"\
"\1self.accelerator.backward(loss, **kwargs)",

function,
)

exec(function, globals())
Trainer.training_step = _unsloth_training_step
Expand Down Expand Up @@ -1130,6 +1146,8 @@ def unsloth_compile_transformers(
fuse_lm_head = True,
gradient_checkpointing = True,
manual_replacements = True,
fast_lora_forwards = True,
fast_residual_stream = True,
epilogue_fusion = True,
max_autotune = False,
shape_padding = True,
Expand Down Expand Up @@ -1174,6 +1192,8 @@ def unsloth_compile_transformers(
fuse_lm_head = fuse_lm_head,
gradient_checkpointing = gradient_checkpointing,
manual_replacements = manual_replacements,
fast_lora_forwards = fast_lora_forwards,
fast_residual_stream = fast_residual_stream,
epilogue_fusion = epilogue_fusion,
max_autotune = max_autotune,
shape_padding = shape_padding,
Expand Down
2 changes: 2 additions & 0 deletions unsloth/models/loader.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,8 @@ def from_pretrained(
fuse_lm_head = True,
gradient_checkpointing = True,
manual_replacements = True,
fast_lora_forwards = False,
fast_residual_stream = False,
epilogue_fusion = True,
max_autotune = False,
shape_padding = True,
Expand Down

0 comments on commit 4fbbd7e

Please sign in to comment.