Skip to content

[v1]Support automatic discovery of registered kernels. #9509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Nov 26, 2025
Merged

[v1]Support automatic discovery of registered kernels. #9509

merged 11 commits into from
Nov 26, 2025
+228 −36

Conversation

@frozenleaves
Copy link
Contributor

@frozenleaves frozenleaves commented Nov 18, 2025 •
edited
Loading

What does this PR do?

  1. 完善了kernel的自动注册和自动发现机制，现在可以通过apply_available_kernels接口调用所有可用kernels，无需逐个调用
  2. 针对已支持的模型中，不支持swiglu融合算子的MLP层做了排除

@frozenleaves frozenleaves changed the title Main 1118 Temporarily support npu fused options on v0, powered by v1 kernels Nov 18, 2025
@frozenleaves frozenleaves marked this pull request as ready for review November 20, 2025 11:38
@frozenleaves frozenleaves changed the title Temporarily support npu fused options on v0, powered by v1 kernels [v1]Support automatic discovery of registered kernels. Nov 20, 2025
hiyouga
hiyouga reviewed Nov 20, 2025
View reviewed changes
src/llamafactory/v1/plugins/model_plugins/kernels/mlp/npu_swiglu.py Outdated
# Bind function as an instance method to preserve `self` semantics
# and replace the original forward
replace_modules.append(module.__class__.__name__)
if module.__class__.__name__ in ("Glm4MLP", "Glm4vTextMLP", "Phi3MLP"):
Copy link
Owner

@hiyouga hiyouga Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A mapping dict is better

Copy link
Contributor Author

@frozenleaves frozenleaves Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, let me modify it.

Copy link
Contributor Author

@frozenleaves frozenleaves Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix

@frozenleaves frozenleaves requested a review from hiyouga November 22, 2025 01:50
@frozenleaves frozenleaves mentioned this pull request Nov 26, 2025
Merged
hiyouga
hiyouga approved these changes Nov 26, 2025
View reviewed changes
@hiyouga hiyouga merged commit f17efde into hiyouga:main Nov 26, 2025
16 checks passed
@hiyouga hiyouga added the solved This problem has been already solved label Nov 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hiyouga hiyouga hiyouga approved these changes

Assignees

No one assigned

Labels

solved This problem has been already solved

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@frozenleaves @hiyouga