[2024-06-19] LMFlow v0.0.8

Major new features since v0.0.4

• Support conversation templates
• Support new optimization algorithms, e.g. LISA
• Update requirements to support latest models
• Fix bugs in qlora/lora scripts
• Fix tokenization parallelism bug
• Improve script interfaces

What's Changed

• README refactor by @shizhediao in #607
• Improve interface of finetuning scripts by @research4pan in #611
• resize banner by @shizhediao in #612
• Doc Reformat by @shizhediao in #614
• Dev update transformers by @yaoguany in #616
• Added QLoRA support for Decoder transformers with tune_strategy "Normal" by @TensorBlast in #613
• announce long context support by @shizhediao in #621
• fix deepspeed zero3 config bugs by @yaoguany in #622
• Update version.py by @hendrydong in #624
• update qr code by @shizhediao in #625
• FIX BUG: trust_remote_code flag didn't take effect by @conght in #633
• Add explanations about supported CUDA versions by @research4pan in #634
• Added citation for RAFT by @shizhediao in #635
• Update qrcode by @shizhediao in #636
• [Features] Support multi_modal training by @lianqing11 in #628
• Add scripts to convert raw file to text-only json by @research4pan in #638
• speculative decoding by @wheresmyhair in #630
• [Feature] Speculative Inference by @wheresmyhair in #640
• add readme for speculative decoding by @wheresmyhair in #641
• update news about speculative decoding by @shizhediao in #642
• update llama flash attention by @yaoguany in #646
• update qrcode by @shizhediao in #647
• [FIX] Fix multi-modal training by @lianqing11 in #648
• Fix: --disable_group_texts 1 keep short samples by @research4pan in #649
• Support all types with --disable_group_texts 1 by @research4pan in #650
• Fix model downloading for CPU-only servers by @research4pan in #651
• add block size to fingerprint by @RolandMinrui in #653
• update qrcode by @shizhediao in #654
• Update QR code for wechat by @research4pan in #656
• update qrcode by @shizhediao in #657
• update qr code for wechat by @research4pan in #667
• Update version of datasets dependency by @research4pan in #668
• Add flash attention install for A6000 by @research4pan in #669
• Update hf_decoder_model.py by @yaoguany in #670
• fix bugs in llama flash attention by @yaoguany in #681
• code exection class and test cases by @Bob17293729 in #674
• Update README to reflect changes in v0.0.6 by @research4pan in #696
• fix bugs in requirements.txt since previous one can cause errors by @xu1868 in #697
• fix merge lora bug by @Dominic789654 in #698
• Upgrade transformers deps to support mistral by @research4pan in #700
• add lisa code and lisa args by @Dominic789654 in #701
• add GPU memory check script by @Dominic789654 in #702
• Support multi-gpu inference by @research4pan in #699
• src/lmflow/args.py typo fix by @wheresmyhair in #703
• add more info when fail to import flash attn by @wheresmyhair in #704
• Add script to finetune llama-2 with lisa by @research4pan in #705
• add LISA args introduction by @Dominic789654 in #706
• add lisa bibtex by @Dominic789654 in #708
• Rpan add news by @research4pan in #709
• Fix typo in README.md by @research4pan in #710
• Update lisa code by @Dominic789654 in #711
• Fix recent bad commits, which lead to OOM in 7B by @research4pan in #712
• Add more memory saving options for LISA script by @research4pan in #716
• Add stable version 0.0.7 by @research4pan in #717
• Export --lisa_interval_steps by @research4pan in #718
• Update documentation.yaml to fix github check failure when submitting PRs by @wheresmyhair in #723
• support more model by @Dominic789654 in #724
• Update README.md to temporary allow latest fix by @research4pan in #725
• Rpan dev lisa by @research4pan in #715
• ShareGPT format support by @wheresmyhair in #720
• Add latest support for flash-attention from hf by @research4pan in #727
• Fix LISA init OOM caused by deepspeed by @research4pan in #728
• Improve installation documentation in README.md by @research4pan in #733
• Add temporary dataset document by @research4pan in #734
• Improve README and requirement.txt by @research4pan in #735
• documentation update and action fix by @wheresmyhair in #736
• documentation page style update and readme dataset update by @wheresmyhair in #737
• Add explanation for LISA with model-parallelism by @research4pan in #738
• typo fix and disable html logo by @wheresmyhair in #739
• Update dependency for transformers by @research4pan in #740
• Rpan lisa more models by @research4pan in #742
• Fix: typo in --lisa_layers_attribute by @research4pan in #743
• Add inferencer by @research4pan in #744
• Improve LISA layers_attribute mapping logics by @research4pan in #746
• Change --disable_group_texts default to True by @research4pan in #747
• readme (zh) update by @wheresmyhair in #748
• readme (zh) typo fix by @wheresmyhair in #749
• Change version to 0.0.7 in version.py by @research4pan in #750
• readme (ko) update by @wheresmyhair in #751
• Change lmflow.org to a temporary ip by @research4pan in #753
• Add more clarification for data server problem by @research4pan in #755
• Remove flash-attn in requirements.txt by @research4pan in #757
• ko and zh readme update by @wheresmyhair in #758
• ko readme update by @wheresmyhair in #761
• [New feature] Integrate DPO by @gzliyu in #762
• readme jp update by @wheresmyhair in #763
• Recover data server by @research4pan in #764
• remove data server address notice by @wheresmyhair in #765
• readme update by @wheresmyhair in #766
• hindi readme update by @wheresmyhair in #769
• conversation dataset (sharegpt format) custom template dev by @wheresmyhair in #767
• custom template document update by @wheresmyhair in #770
• conversation template doc typo fix by @wheresmyhair in #771
• template info update by @wheresmyhair in #772
• add support for llama-3 template by @wheresmyhair in #773
• run llama 3 with lisa by @wheresmyhair in #776
• Remove redundant statements in setup.py by @research4pan in #779
• Add contributor support by @research4pan in #780
• Contrib README typo fix by @wheresmyhair in #782
• add chatml conversation template by @wheresmyhair in #781
• Custom conversation template improvement and document update by @wheresmyhair in #783
• Update examples for Full-param SFT and LISA by @research4pan in #784
• Fixes & updates on lora, qlora scripts and hf_decoder_model by @wheresmyhair in #787
• Add phi3 conversation template support by @wheresmyhair in #788
• Add trust_remote_code option to finetune shells by @wheresmyhair in #789
• Finetune shell typo fix by @wheresmyhair in #790
• Remove lora qlora aggregated shell by @wheresmyhair in #792
• README update by @wheresmyhair in #793
• README update, adding conversation template examples by @wheresmyhair in #794
• Add DPO support by @wheresmyhair in #797
• [New feature] Integrate DPO by @gzliyu in #759
• README update, remove lora save aggregate shell by @wheresmyhair in #800
• Merge LoRA and base model by @wheresmyhair in #802
• README zh update by @wheresmyhair in #803
• README kr update by @wheresmyhair in #804
• README es update by @wheresmyhair in #807
• README jp update by @wheresmyhair in #805
• README hindi update by @wheresmyhair in #808
• Fix eval_dataset number log. by @uApiv in #809
• DeepSeek conversation template support by @wheresmyhair in #810
• Add DeepSeek template and template register by @wheresmyhair in #814
• Change conversation template file structure by @wheresmyhair in #817
• Yizhenjia template update by @wheresmyhair in #819
• add lisa-diffusion project by @shaoshitong in #818
• Add chatglm3 template by @wheresmyhair in #820
• Add chatglm3 template by @wheresmyhair in #822
• Support yi and yi1.5 template by @wheresmyhair in #824
• Yizhenjia template update by @wheresmyhair in #826
• Add zephyr template by @wheresmyhair in #828
• Customized conversation template guide update by @wheresmyhair in #829
• Guide update by @wheresmyhair in #830
• Add finetuning doc by @wheresmyhair in #832
• Doc dataset page update by @wheresmyhair in #835
• Add paired conversation dataset description by @wheresmyhair in #838
• Add supported models table by @wheresmyhair in #839
• Readme update by @wheresmyhair in #843
• Reward modeling support by @wheresmyhair in #836
• [Bug fix] Tokenization multiprocessing fix by @wheresmyhair in #845
• [Bug fix] Blocking function args missing fix by @wheresmyhair in #846
• [Bug fix] Fix tokenizer multiprocessing in reward model by @wheresmyhair in #847
• Long context summarize demo by @HALIS-sh in #844
• [Model Support] Qwen2 update by @wheresmyhair in #849
• [Usability] Add preset lora target modules by @wheresmyhair in #850
• [Model] hf model modification and inheritance change by @wheresmyhair in #852
• Add langchain chatbot by @YanxinLu in #848
• Add multi node README by @research4pan in #853
• Usability update by @wheresmyhair in #857

New Contributors

• @TensorBlast made their first contribution in #613
• @conght made their first contribution in #633
• @wheresmyhair made their first contribution in #630
• @RolandMinrui made their first contribution in #653
• @Bob17293729 made their first contribution in #674
• @xu1868 made their first contribution in #697
• @gzliyu made their first contribution in #762
• @uApiv made their first contribution in #809
• @shaoshitong made their first contribution in #818
• @HALIS-sh made their first contribution in #844
• @YanxinLu made their first contribution in #848

Full Changelog: v0.0.4...v0.0.8