Skip to content

Commit 73f8bf0

Browse files
jpablomchYuan0320
andauthored
Update README to mention new heuristics and point to NNCF Impl. (#48)
Co-authored-by: Yuan Jinjie <[email protected]>
1 parent 4ebfc5b commit 73f8bf0

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

SQFT/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ Specifically, the highlights of SQFT include:
1212
Please refer to our paper for more details.
1313

1414
## News
15+
- **[2025.07.02]** SQFT's NLS algorithm is now supported in [OpenVINO NNCF](https://github.com/openvinotoolkit/nncf) for quantization-aware tuning to improve accuracy on downstream Tasks, including new heuristics. More details can be found [here](https://github.com/openvinotoolkit/nncf/tree/v2.17.0/examples/llm_compression/torch/downstream_qat_with_nls).
1516
- **[2024.09.24]** **SQFT** paper has been released ([link](https://arxiv.org/pdf/2410.03750)) and **accepted at EMNLP 2024 Findings**. :books:
1617
- **[2024.09.24]** Release the code for **SQFT**. :tada:
1718

@@ -384,6 +385,15 @@ lm_eval --model hf \
384385

385386
</details>
386387

388+
## New Heuristic
389+
390+
In addition to the super simple heuristic using the median config, we have designed new heuristics based on training loss and activation counts during training. Specifically, we have:
391+
392+
- **Loss-based Sub-adapter**: This heuristic selects sub-adapters based on the loss information recorded during training.
393+
- **Activation-based Sub-adapter**: This heuristic selects sub-adapters based on the most frequently activated configurations during training.
394+
395+
For more details, please refer to the script [here](https://github.com/openvinotoolkit/nncf/blob/v2.17.0/examples/llm_compression/torch/downstream_qat_with_nls/main.py).
396+
387397
## Released Fine-tuned Models 🤗
388398

389399
- Mistral-7B-v0.3

0 commit comments

Comments
 (0)