Dear authors,
first of all thanks for the continued support of this awesome repo!
I'm currently trying it out but I'm running into trouble with QQQ.
Mainly, adapting basic_usage_wikitext2-py found at
https://github.com/ModelCloud/GPTQModel/blob/main/examples/quantization/basic_usage_wikitext2.py
to work with QQQ using llama3-8b yields models which score very bad on the evaluation (ppl >400).
Could you please provide an example script for running QQQ, ideally something similar to what is already provided on your README?
Thanks a lot!