Add support for Quantization-Aware Low-Rank Adaptation (QALoRA)#2571

Open

gapsong wants to merge 8 commits intohuggingface:mainfrom

gapsong:qa_lora

Commits on Jun 4, 2025

Add support for Quantization-Aware Low-Rank Adaptation (QALoRA) in LoraLayer and GPTQLoraLinear
gapsong
committed

Commits on Jun 9, 2025

Refactor QALoraLinearVariant methods to raise NotImplementedError for unsupported operations.
gapsong
committed
Fix dtype handling and use foward in GPTQLoraLinear forward method
gapsong
committed

Commits on Jun 11, 2025

Enhance QALoRA support by refining group size handling and error management in variants
gapsong
committed

Commits on Jun 14, 2025

Enhance GPTQ and QALoRA implementations with improved error handling and refactor training tests
gapsong
committed

Commits on Jun 16, 2025

Refactor QALoRA implementation by removing unused memory printing function, enhancing gradient checks in tests, and improving adapter handling in model training.
gapsong
committed

Commits on Jun 17, 2025