[Bugfix] Refactor QuantizationMixin to use resolved config #1912
+94
−31
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Brian Dellabetta <[email protected]>
|
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed. |
Signed-off-by: Brian Dellabetta <[email protected]>
brian-dellabetta
commented
Oct 9, 2025
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
kylesayrs
reviewed
Oct 10, 2025
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
Signed-off-by: Brian Dellabetta <[email protected]>
brian-dellabetta
requested review from
HDCharles,
dsikka,
fynnsu,
rahul-tuli and
shanjiaz
zhanglei1172
pushed a commit
to zhanglei1172/llm-compressor
that referenced
this pull request
Oct 17, 2025
…ect#1912) SUMMARY: Fixes vllm-project#1906 This refactors QuantizationMixin to not update any pydantic fields during validation. Rather than modifying them in order to make them the source of truth, this adds properties `resolved_config` & `resolved_targets` that all modifiers should instead use as source of truth. These are resolved once, when needed, and not serialized, which should fix the bug in vllm-project#1906 TEST PLAN: Added a `test_resolved_targets` unit test --------- Signed-off-by: Brian Dellabetta <[email protected]> Signed-off-by: LeiZhang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY:
Fixes #1906
This refactors QuantizationMixin to not update any pydantic fields during validation. Rather than modifying them in order to make them the source of truth, this adds properties
resolved_config&resolved_targetsthat all modifiers should instead use as source of truth. These are resolved once, when needed, and not serialized, which should fix the bug in #1906TEST PLAN:
Added a
test_resolved_targetsunit test