Skip to content

[GPU]qwen3 moe support #32048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
WeldonWangwang wants to merge 88 commits into from
Closed

[GPU]qwen3 moe support #32048

WeldonWangwang wants to merge 88 commits into from

Conversation

@WeldonWangwang
Copy link
Contributor

@WeldonWangwang WeldonWangwang commented Sep 11, 2025 •
edited
Loading

Details:

  • Rebase PR30448 to releases/2025/3

luo-cheng2021 and others added 30 commits April 8, 2025 09:29
@luo-cheng2021
add perf
aa10cbe
@luo-cheng2021
perf for sdpa/pa
3a80081
@luo-cheng2021
add git ignore
d209f03
@luo-cheng2021
insert if for moe expert
45bf16e
@luo-cheng2021
add moeexpert support
795e323
@luo-cheng2021
fix moexpert precision is always f32
d7f2602
@luo-cheng2021
add moeexpert support for gpu
7f4b901
@luo-cheng2021
opt: 1, simplify subgraph inside moeexpert; 2, only compute skip flag…
76a7d5b 
… when expert_no == 0 for gpu
@luo-cheng2021
opt: remove nonzero->split from subgraph into moeexpert for gpu
6079269
@riverlijunjie @luo-cheng2021
Support Qwen3 rms kernel for input with dynamic padding
c385c8f
@riverlijunjie @luo-cheng2021
Add test case
ada754a
@luo-cheng2021
WA: moe_expert wait all inputs ready
38ded44
@luo-cheng2021
fix incorrect output shape computation
f84303e
@luo-cheng2021
add fast path for expert mask computation if no padding
df0ca20
@riverlijunjie @usstq
qwen3 moe compile model opt, from 150s to 70s in LNL (openvinotoolkit#66
00e7d9a 
)

* Build Subgraph in parallel to improve compile_model performance

* SharedOpOptimization optimizes attribute visit

---------

Co-authored-by: Tingqian Li <[email protected]>
@ceciliapeng2011
Move FuseMoeExpert2 ahead of CommonOptimizations to decrease compilin…
019262b 
…g time.
@luo-cheng2021
not use subgraph for moeexpert
994c094
@luo-cheng2021
fix scale/zp layout; first expert should not be inplace
22c93ee
@luo-cheng2021
merge all experts into one op
7c872a0
@riverlijunjie
Optimize gather and index_add performance
75b9683
@luo-cheng2021
fix out_of_resource error on lunarlake
2d8eb4e
@riverlijunjie
Move weigts from usm_host to usm_device memory
06e436c
@riverlijunjie
Add ITT for MoE
02f2331
@riverlijunjie
Optimize BMG first token due to index_add kernel
b6b5f1d
@luo-cheng2021
opt: merge all experts into one for batch1
9383141
@luo-cheng2021
opt: cl code for mlp_*
c7ef4ea
@luo-cheng2021
change weight back to ba
76e6ed7
@luo-cheng2021
small tune for lunarlake
8471f6b
@luo-cheng2021
fuse onehot into moe
818ba1b
@luo-cheng2021
not wait gpu for batch1
eed40eb
luo-cheng2021 and others added 17 commits May 20, 2025 16:00
@luo-cheng2021
apply review comments
ff0c74f
@riverlijunjie
Some reviewer comments
18312f2
@riverlijunjie
Remove CM kernel and move to the following PR
e06f8de
@luo-cheng2021
apply review comments
f43d457
@peterchen-intel
Merge branch 'master' into gpu/qwen3_moe_cm
0c90c5a
@luo-cheng2021
Merge remote-tracking branch 'upstream/master' into gpu/qwen2_moe_cm
255992b
@luo-cheng2021
apply review comments
4662b8f
@riverlijunjie
Fix CI issues
0d0ecca 
1. usm_host input issue.
2. lru_cache issue, onednn kerenl cache corruption
3. use usm_host for moe internal buffer
@luo-cheng2021
transformation support more weight datatype
41fa911
@luo-cheng2021
Merge remote-tracking branch 'upstream/master' into gpu/qwen3_moe_cm
e7e909a
@luo-cheng2021
apply review comments
73071de
@luo-cheng2021
Merge remote-tracking branch 'upstream/master' into gpu/qwen3_moe_cm
2181617
@luo-cheng2021
fuse router+expert0 first to avoid fusing expert success but router f…
ed49b4e 
…ailed.
@peterchen-intel
Merge branch 'master' into gpu/qwen3_moe_cm
ad36133
@luo-cheng2021
solve merge conflicts
37ca973
@riverlijunjie
Merge branch 'main' into gpu/qwen3_moe_cm
cba1763
@riverlijunjie
Handle reviewer's comments
1ed2a11
@github-actions github-actions bot added category: IE Tests OpenVINO Test: plugins and common category: GPU OpenVINO GPU plugin category: transformations OpenVINO Runtime library - Transformations category: docs OpenVINO documentation labels Sep 11, 2025
@WeldonWangwang WeldonWangwang changed the title Gpu/qwen3 moe cm [GPU]qwen3 moe support Sep 11, 2025
@SearchSavior
Copy link

SearchSavior commented Sep 11, 2025

@WeldonWangwang @peterchen-intel Very exciting progress on this PR!

@savvadesogle
Copy link

savvadesogle commented Sep 25, 2025

Will this support Qwen3 80b next?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@peterchen-intel peterchen-intel Awaiting requested review from peterchen-intel

Assignees

No one assigned

Labels

category: docs OpenVINO documentation category: GPU OpenVINO GPU plugin category: IE Tests OpenVINO Test: plugins and common category: transformations OpenVINO Runtime library - Transformations do not merge do_not_review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@WeldonWangwang @SearchSavior @savvadesogle @peterchen-intel @luo-cheng2021 @riverlijunjie @ceciliapeng2011 @usstq