Fetch from nvidia Megatron-LM #5
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…xSingleTransformer layer Co-authored-by: Lifu Zhang <[email protected]>
Fix for Enabling CUDA Graph for MMDiT and fluxSingleTransformer layer See merge request ADLR/megatron-lm!2984
Merge branch 'ko3n1g/ci/codeless-builds' into 'main' See merge request ADLR/megatron-lm!3109
chore: Update codeowners See merge request ADLR/megatron-lm!3079
Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: William Dykas <[email protected]>
Switch seq parallel utility See merge request ADLR/megatron-lm!3011
…N8G_mcore_tp1_pp1_resume_torch_dist_dist_optimizer`
ci: Broken test `gpt3_345m_nightly_dgx_a100_1N8G_mcore_tp1_pp1_resume_torch_dist_dist_optimizer` See merge request ADLR/megatron-lm!3110
…lopt Co-authored-by: Chenhan Yu <[email protected]> Co-authored-by: Chen-Han Yu <[email protected]>
Removed deprecated real quant configs in Modelopt See merge request ADLR/megatron-lm!2934
ci: Fix publish notify job See merge request ADLR/megatron-lm!3117
ci: Upload pipeline telemetrics See merge request ADLR/megatron-lm!3106
Fix `post_training/test_get_gpt_modelopt_spec_interface` See merge request ADLR/megatron-lm!3118
Co-authored-by: Shanmugam Ramasamy <[email protected]>
Remove legacy bert tests See merge request ADLR/megatron-lm!3023
Co-authored-by: Ali Taghibakhshi <[email protected]> Co-authored-by: Mcore Bot <[email protected]>
Alit/config mamba head See merge request ADLR/megatron-lm!2601
Update CODEOWNERS to make modelopt review only for QAT. See merge request ADLR/megatron-lm!3125
Run nemo2 tests instead of nemo1 See merge request ADLR/megatron-lm!3119
…attn for dynamic batching. Co-authored-by: Shanmugam Ramasamy <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: Vijay Korthikanti <[email protected]> Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: root <[email protected]>
Integrating paged attention feature of flash_attn for dynamic batching. See merge request ADLR/megatron-lm!2955
Co-authored-by: Mcore Bot <[email protected]> Co-authored-by: yaoyu-33 <[email protected]> Co-authored-by: Chenhan Yu <[email protected]>
add l2 norm in torch_norm.py for LLAMA-4 support See merge request ADLR/megatron-lm!2960
Co-authored-by: Santosh Bhavani <[email protected]>
Updated setup instructions in README.md See merge request ADLR/megatron-lm!3210
…robatched inference is on
Disable cudagraphs when pipeline parallel microbatched inference is on See merge request ADLR/megatron-lm!3151
Co-authored-by: oliver könig <[email protected]> Co-authored-by: Mcore Bot <[email protected]>
Inference functional test: 580M Minitron See merge request ADLR/megatron-lm!2812
…ron" This reverts commit f8c8c9c.
Co-authored-by: oliver könig <[email protected]> Co-authored-by: Mcore Bot <[email protected]>
…hanges during inference
Invalidate cached SSM tensors if batch size changes during inference See merge request ADLR/megatron-lm!3277
ci: Move unit test logic to file See merge request ADLR/megatron-lm!3291
…'serialization_format'
Adapt _write_item call to new signature with 'serialization_format' See merge request ADLR/megatron-lm!3243
Co-authored-by: Russell Hewett <[email protected]>
Add in-process restart See merge request ADLR/megatron-lm!2711
This reverts commit d87ba91.
ci: Run on multiple clusters See merge request ADLR/megatron-lm!3292
ci: Allow specific TE-ref See merge request ADLR/megatron-lm!3302
ci(fix): Write logs to log_dir See merge request ADLR/megatron-lm!3299
Address dist checkpointing PyT 24.08 failure See merge request ADLR/megatron-lm!3253
ci(hotfix): Downstream pipeline See merge request ADLR/megatron-lm!3307
…nal argparse flag to clear GPU... Co-authored-by: Szymon Migacz <[email protected]>
MR feedback: added units for arguments, optional argparse flag to clear GPU... See merge request ADLR/megatron-lm!3308
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.