Skip to content

Commit

Permalink
Merge pull request #32 from eltociear/patch-1
Browse files Browse the repository at this point in the history
Update README.md
  • Loading branch information
frankaging authored Apr 6, 2024
2 parents 61b98b2 + 545d5f5 commit e12c279
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ Note that Llama-2 models can follow instructions zero-shot. We encourge people t
**Usage and License Notices**: Our chat-model is intended and licensed for research use only. The model is CC BY NC 4.0 (allowing only non-commercial use) should not be used outside of research purposes.


## Why you should use ReFT as opppose to PEFT?
## Why you should use ReFT as oppose to PEFT?

There are various benefits such as saving memory and storage. In addition to that, ReFT is more interpretable and extensible than PEFT. The interventions we are learning is simply a causal abstraction of the task you are training without touching any model weights. The intervention site search space is large, and can be at any token position which is more flexibile. We showcase ReFT performance on various benchmarks against popular PEFT such as LoRA and its newer variants (e.g., DoRA) in our paper.

Expand Down

0 comments on commit e12c279

Please sign in to comment.