Skip to content

Johnny1882 attention replacement #288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 75 commits into from
Apr 20, 2025
Merged

Johnny1882 attention replacement #288

merged 75 commits into from
Apr 20, 2025

Conversation

Aaron-Zhao123
Copy link
Collaborator

No description provided.

Johnny1882 and others added 30 commits February 25, 2025 23:39
@Johnny1882
add mla transform pass framework
9195360
@Johnny1882
use bert instead of dummy model
5ac6036
@Johnny1882
integrating bert and bert2mla transform in mase
5e61e44
@Johnny1882
finish simple conversion from bertsdpaselfattention to mla, not yet t…
cea2b79 
…ested
@Johnny1882
finish mla transform pass implementation, todo: test
9e466ef
@Johnny1882
improve test script
0b8b4be
@Johnny1882
trying to solve conversion error
9d4bd67
@Johnny1882
add transform pass from gpt2 to gqa, improve framework's compactability
6801842
@Johnny1882
finish debugging mgqa transform pass evaluation and training
ad5d5af
@Johnny1882
rename
a341c54
@Johnny1882
restore MLA code
e7ea5dc
@Johnny1882
fix module inconsistency problem
02cd945
@Johnny1882
complete all other module weight loading
25c1803
MLA completed first iteration
9faac9a
testing file debug
19faf85
@Johnny1882
replace spdaattention instead
535df89
@zzx04
fc_before_test
855baa2
@Johnny1882
.
69952c8
@Johnny1882
add inference test file
aa4054f
@Johnny1882
gpt2 to sdpa test passed
5bacc00
fc_v2_after_test
463794d
debug
d458ea7
@Johnny1882
debug
83d1f5e
@Johnny1882
Merge branch 'mgqa-transform-pass' of https://github.com/Johnny1882/mase
27438f5 
 into mgqa-transform-pass

merge ethan change
Merge remote branch 'origin/mgqa-transform-pass' with strategy ours (…
75a092e 
…ignoring remote changes)
Save changes before switching branch
3877711
@Johnny1882
Merge branch 'mgqa-transform-pass' of https://github.com/Johnny1882/mase
c9956a2 
 into mgqa-transform-pass
@Johnny1882
refine test file
de9e7b7
fc_v3_debugged
caff05f
@Johnny1882
fix mgqa kvhead shape mismatch problem
acc1e77
Yuxuan E Chen and others added 25 commits March 26, 2025 02:58
memory+speed
d36cb85
gpt to mla memory test
aba191c
@zzx04
Update fc_transform_pass.py
a11b6e2
llama to MLA
7388961
llama-mla transofrm passed
47de5b1
clean
aa4db6c
@Johnny1882
merge fc pass into attention_transform_helper
c1761c7
@Johnny1882
black reformat
69202cd
@Johnny1882
Merge remote-tracking branch 'origin/llama-to-mla-transform' into att…
84192c4 
…ention-replacement
@Johnny1882
organize code and framework format test cases
f039478
@Johnny1882
reformat code and framework, properly format tests
2473710
@zzx04
Final fc_transform_pass.py
724fa7d
@zzx04
reformat fc_transform_pass.py
b9f5f36
@zzx04
reformat fc_transform_pass.py
f0102be
black reformat
ff9e5d3
@Johnny1882
reformat code and structure
fb54272
@Johnny1882
change kernel to dummy to pass mase test
e331a4e
@Johnny1882
restore kernel.py
bdafccf
@Johnny1882
final code reformat
c99a9db
@Aaron-Zhao123
Merge branch 'attention-replacement' of https://github.com/Johnny1882…
79692d5 
…/mase into Johnny1882-attention-replacement
@Aaron-Zhao123
fix various redundant code blocks
4c4360c
@Aaron-Zhao123
file restructure
86913f5
@Aaron-Zhao123
formatting
f945426
@Aaron-Zhao123
fix tuple bug
99cc11b
@Aaron-Zhao123
remove Johnny's tests...
ce5961b
@Aaron-Zhao123 Aaron-Zhao123 mentioned this pull request Apr 19, 2025
Closed
jianyicheng
jianyicheng approved these changes Apr 20, 2025
View reviewed changes
Copy link
Collaborator

@jianyicheng jianyicheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing to MASE! I am generally happy with the PR. It would be helpful to add more comments in a future PR to describe your implementation for each class.

@Aaron-Zhao123 Aaron-Zhao123 merged commit 5c1beb1 into main Apr 20, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jianyicheng jianyicheng jianyicheng approved these changes

@JeffreyWong20 JeffreyWong20 Awaiting requested review from JeffreyWong20

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Aaron-Zhao123 @jianyicheng @Johnny1882 @zzx04