Add BoRA to PEFT

[cxmscb]

Feature request

https://openreview.net/forum?id=eUMJXFgMjM

https://anonymous.4open.science/r/BoRA/glue_roberta/mypeft/bora.py

Your contribution

N/A

[BenjaminBossan]

Thanks @cxmscb for the proposal.

IIUC, the paper is not accepted yet -- could there be changes to the method in the future? If yes, I would like to wait until it's clear that nothing changes anymore.

I only skimmed the paper thus far and I wondered if it could be implemented as a "LoRA variant", which is a concept in PEFT that allows to re-use most of the LoRA code instead of having to add a completely new PEFT method (see #2895 as an example).

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.