Fetch from nvidia Megatron-LM #5
Open
+180,528
−20,389
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
test: Update nightly values See merge request ADLR/megatron-lm!2618
Co-authored-by: Mikołaj Błaż <[email protected]>
Reuse global metadata for first saves See merge request ADLR/megatron-lm!2517
Co-authored-by: Selvaraj Anandaraj <[email protected]>
Added CP support for partial DistOpt See merge request ADLR/megatron-lm!2497
…VRx implementation
Bring in-job restart up-to-date with latest NVRx implementation See merge request ADLR/megatron-lm!2560
Co-authored-by: Dennis Liu <[email protected]>
Uneven Virtual Pipeline Parallelism See merge request ADLR/megatron-lm!1961
Co-authored-by: Zijie Yan <[email protected]>
Add aux loss free routing. Closes #356 See merge request ADLR/megatron-lm!2026
…el load Co-authored-by: Mikołaj Błaż <[email protected]>
…into 'main' Broadcast sharded objects during fully parallel load See merge request ADLR/megatron-lm!2417
Co-authored-by: Ryan Wolf <[email protected]>
Support CP + EP with DP last rank ordering See merge request ADLR/megatron-lm!2586
ci: update nightly values See merge request ADLR/megatron-lm!2635
Ensure CPU tensors are cloned See merge request ADLR/megatron-lm!2604
Co-authored-by: Oliver Koenig <[email protected]>
ci: Release results See merge request ADLR/megatron-lm!2636
Cudagraphable RNG and cudagraph memory fixes See merge request ADLR/megatron-lm!2503
… TensorRT Model Optimizer
Support MCore MambaModel quantization through TensorRT Model Optimizer See merge request ADLR/megatron-lm!2527
Disable the FP8 transpose cache when using torch FSDP2 See merge request ADLR/megatron-lm!2508
Port multimodal inference to MCore API See merge request ADLR/megatron-lm!2445
Co-authored-by: Selvaraj Anandaraj <[email protected]>
Added option for parallel cross entropy See merge request ADLR/megatron-lm!2707
Fix multi-rank inference See merge request ADLR/megatron-lm!2776
Fix RNG tracker for inference See merge request ADLR/megatron-lm!2781
…ontiguous in `prepare_input_tensors_for_wgrad_compute` Co-authored-by: Jennifer Chen <[email protected]>
NeMo SFT QAT fix: Make `all_gathered_input` contiguous in `prepare_input_tensors_for_wgrad_compute` See merge request ADLR/megatron-lm!2822
Only materialize logits for the last token during inference See merge request ADLR/megatron-lm!2624
Co-authored-by: Mcore Bot <[email protected]>
barebones radio g support See merge request ADLR/megatron-lm!2622
build: Better caching See merge request ADLR/megatron-lm!2818
chore: Benchmark for PyTorch 24.10 (Mcore 0.11.0) See merge request ADLR/megatron-lm!2788
build: Bisect depedencies See merge request ADLR/megatron-lm!2779
Configurable FSDP modules See merge request ADLR/megatron-lm!2765
Co-authored-by: Rahul Kandu <[email protected]> Co-authored-by: Rahul Kandu <[email protected]>
Workload Inspector on-demand profiling feature See merge request ADLR/megatron-lm!2714
Change Mamba textgen server to use MCore inference See merge request ADLR/megatron-lm!2621
ci: Publish analytics See merge request ADLR/megatron-lm!2839
ci: Small improvements to release tests See merge request ADLR/megatron-lm!2835
ci: Upload statistics only for MRs See merge request ADLR/megatron-lm!2842
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.