[WIP] Add LoRA multihead attention module#1324

Merged

BenjaminBossan merged 61 commits intohuggingface:mainfrom

BenjaminBossan:feat-add-lora-multihead-attention

Jan 8, 2025

Commits on Jan 8, 2024

Extend test coverage a bit
BenjaminBossan
committed

Commits on Jan 12, 2024

Apply LoRA also to the out_proj of MHA
BenjaminBossan
committed

Commits on Feb 7, 2024

Commits on Feb 26, 2024

Commits on Mar 11, 2024

Commits on Mar 26, 2024

Commits on May 21, 2024

Merge branch 'main' into feat-add-lora-multihead-attention
BenjaminBossan
committed

Commits on Jul 26, 2024

Commits on Sep 3, 2024

Merge branch 'main' into feat-add-lora-multihead-attention
BenjaminBossan
committed

Commits on Sep 12, 2024

Merge branch 'main' into feat-add-lora-multihead-attention
BenjaminBossan
committed

Commits on Sep 18, 2024

MAINT: Give stale bot permissions for PRs too (#2064)
BenjaminBossan
committed
ENH BOFT don't save boft_P buffer (#2050)

sywangyi
authored and
BenjaminBossan
committed
FIX Command line args in PiSSA preprocess (#2053)

keakon
authored and
BenjaminBossan
committed
MNT Update deprecated evaluation_strategy (#1664)

muellerzr
authored and
BenjaminBossan
committed
ENH Multi adapters in same batch: modules_to_save (#1990)

saeid93
authored and
BenjaminBossan
committed
FIX Bug that prevents BOFT from loading 2 adapters (#2068)
BenjaminBossan
committed
TST Skip some quantization tests on XPU (#2074)

faaany
authored and
BenjaminBossan
committed
Improve test coverage for initialization of MHA
BenjaminBossan
committed

Commits on Oct 14, 2024

Merge branch 'main' into feat-add-lora-multihead-attention
BenjaminBossan
committed

Commits on Oct 21, 2024

Fix bug with unloading multihead attention layer
BenjaminBossan
committed

Commits on Oct 22, 2024

Fix bug in unloading
BenjaminBossan
committed

Commits on Nov 1, 2024

Commits on Nov 26, 2024

Commits on Dec 9, 2024

Commits on Jan 8, 2025