Skip to content

Issues: vllm-project/llm-compressor

Beta
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

44 Open 200 Closed
44 Open 200 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Feature] Log/info/Save/Restore quantization steps enhancement New feature or request
#1410 opened May 4, 2025 by mratsim
[AWQ] Insane memory requirement: over 900GB for 32B model bug Something isn't working
#1409 opened May 4, 2025 by mratsim
AWQ Qwen3-235B-A22B and Qwen3-30B-A3B bug Something isn't working
#1406 opened May 1, 2025 by ehartford
@brian-dellabetta
9
Error when computing device_map for Mistral-small-3.1-24B-Instruct-2503 bug Something isn't working
#1403 opened Apr 30, 2025 by VAmblardPEReN
For multimodal models, such as QwenVL2.5, is the SmoothQuantModifier necessary when performing W8A8 quantization?
#1394 opened Apr 28, 2025 by weirdo2310
@kylesayrs
2
For FP8 Fused MoE layers, only per-tensor scalesfor weights and activations are supporte? bug Something isn't working
#1393 opened Apr 28, 2025 by shuxiaobo
@brian-dellabetta
1
AWQ -- Clean up forward passes with kwargs using inspect.bind enhancement New feature or request good first issue A good first issue for users wanting to contribute
#1385 opened Apr 25, 2025 by brian-dellabetta
2
1
Load the model to CPU but quantize using the GPU enhancement New feature or request
#1383 opened Apr 25, 2025 by sgsdxzy
@kylesayrs
1
Is there any way quant model on multi nodes? enhancement New feature or request
#1382 opened Apr 25, 2025 by shuxiaobo
1
How to run AWQ-W4Afp8 quantization?
#1368 opened Apr 22, 2025 by wanzhenchn
@brian-dellabetta
2
Potentially deprecated path bug Something isn't working
#1361 opened Apr 18, 2025 by eldarkurtic
@dsikka
2
Running vllm after oneshot causes rerun of oneshot bug Something isn't working
#1358 opened Apr 16, 2025 by brian-dellabetta
problem cause when doing w8a8kvfp8 for qwen2.5-vl
#1354 opened Apr 16, 2025 by coolKeen
@kylesayrs
5
Extend e2e tests to add asym support for W8A8-Int8 enhancement New feature or request good first issue A good first issue for users wanting to contribute
#1344 opened Apr 10, 2025 by dsikka
1
how to Quant DeepseekV2 bug Something isn't working
#1320 opened Apr 3, 2025 by liusy58
Recipe.model_dump() output is not valid input for Recipe.model_validate(...) bug Something isn't working good first issue A good first issue for users wanting to contribute
#1319 opened Apr 2, 2025 by rahul-tuli
[Gemma3] - FP8 Dynamic KeyError: 'vision_model.encoder.layers.0.mlp.fc1.weight_scale' bug Something isn't working
#1306 opened Apr 1, 2025 by m4r1k
@brian-dellabetta
3
[Gemma3] - oneshot doesn't output preprocessor_config.json & processor_config.json bug Something isn't working
#1305 opened Apr 1, 2025 by m4r1k
@brian-dellabetta
1
Can't quantize kv cache: observer = self.k_observers[layer_idx] liste index out of range bug Something isn't working
#1295 opened Mar 28, 2025 by DreamGenX
1
@brian-dellabetta
4
Q Scaling factor for FP8 KV Cache ?
#1294 opened Mar 28, 2025 by Syst3m1cAn0maly
@brian-dellabetta
7
Has anyone successfully quantinize Deepseek-R1 to w8a8? question Further information is requested
#1274 opened Mar 21, 2025 by taishan1994
3
Hope support tensor parallel enhancement New feature or request
#1249 opened Mar 13, 2025 by Arcmoon-Hu
@brian-dellabetta
1
Support gemma3 enhancement New feature or request good first issue A good first issue for users wanting to contribute
#1248 opened Mar 13, 2025 by zf-inworld
use cpu memory 1.4TB, but use gpu memory 300MB bug Something isn't working
#1240 opened Mar 11, 2025 by mmdbhs
@dsikka
2
Cuda out of memory while loading the quantized model bug Something isn't working vllm Using vLLM
#1234 opened Mar 7, 2025 by Bakht-Ullah
@kylesayrs
10
ProTip! Updated in the last three days: updated:>2025-05-01.