CPT Tuner

[tsachiblau]

Hey,

This pull request introduces Context-Aware Prompt Tuning (CPT), a new and effective technique that builds on In-Context Learning (ICL) and Prompt Tuning (PT) with enhancements through adversarial optimization. CPT allows for better generalization and stability on various classification tasks.

The approach is based on a research paper, which will soon be available. The core idea of CPT is demonstrated and implemented in the following repository:
https://github.com/tsachiblau/CPT.

We are submitting this pull request to integrate the CPT method into the PEFT library, allowing users to experiment with this novel method. Thank you for reviewing this contribution!

The paper is attached
Context_aware_Prompt_Tuning__Advancing_In_Context_Learning_with_Adversarial_Methods_PEFT.pdf

Thanks,
Tsachi

[BenjaminBossan]

Hi, thanks for creating this PR to add this new method to PEFT.

I did not have time yet to do a review, but I wanted to alert you that with the provided information, you could be de-anonymized as paper author. Not sure if that's a big deal for the submission process, but just wanted to let you know.

[tsachiblau]

I plan to upload it to arXiv anyway, so that works for me. Thanks for letting me know.