[Question] Exllamv2 kernel may have accuracy issue for group_size 16

[wenhuach21]

It seems gptqmodel version and autogptq version both have accuracy issue. I tested from autoround, but I am not quite sure, you could have a test from your side

[Qubitium]

@wenhuach21 We have https://github.com/ModelCloud/GPTQModel/blob/main/tests/test_kernel_output.py and https://github.com/ModelCloud/GPTQModel/blob/main/tests/test_kernel_output_ipex.py to test and compare the qualities between different kernels. I will check if group size 16 is covered.

[Qubitium]

For now main has disabled Exllama V2 kernel selection for group_size == 16 until we can get to the bottom of this.