Repository Analysis

NVIDIA/Megatron-LM

Ongoing research training transformer models at scale

11.6 Low AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of NVIDIA/Megatron-LM, a Python project with 17,055 GitHub stars. SynthScan v2.0 examined 947,395 lines of code across 2811 source files, recording 7877 pattern matches distributed across 25 syntactic categories. The overall adjusted score of 11.6 places this repository in the Low AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

11.6
Adjusted Score
11.6
Raw Score
100%
Time Factor
2026-07-14
Last Push
17.1K
Stars
Python
Language
947.4K
Lines of Code
2.8K
Files
7.9K
Pattern Hits
2026-07-14
Scan Date
0.07
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 30HIGH 163MEDIUM 961LOW 6723

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 7877 distinct pattern matches across 25 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers4353 hits · 4218 pts
SeverityFileLineSnippetContext
LOWpretrain_vlm.py227def train_valid_test_datasets_provider(train_val_test_num_samples):CODE
LOWpretrain_vlm.py267def _preprocess_data_for_llava(data):CODE
LOWpretrain_vlm.py465def llava_position_embedding_ranks(pp_ranks):CODE
LOWtrain_rl.py339def train_valid_test_datasets_provider(train_val_test_num_samples):CODE
LOWpretrain_gpt.py184def _build_cached_logits_loss_func(CODE
LOWpretrain_gpt.py372def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig:CODE
LOWpretrain_gpt.py439def train_valid_test_datasets_provider(train_val_test_num_samples, vp_stage=None):CODE
LOWmodel_provider.py62def count_parameters_in_layer(model, layer_name):CODE
LOWpretrain_hybrid.py184def _build_cached_logits_loss_func(CODE
LOWpretrain_hybrid.py359def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig:CODE
LOWpretrain_hybrid.py403def train_valid_test_datasets_provider(train_val_test_num_samples, vp_stage=None):CODE
LOWgpt_builders.py108def _get_transformer_layer_spec(use_te, config):CODE
LOWtasks/finetune_utils.py53def _cross_entropy_forward_step(batch, model):CODE
LOWtasks/finetune_utils.py95def _build_infinite_size_dataloader(dataloader):CODE
LOWtasks/finetune_utils.py106def _build_train_valid_dataloaders(train_dataset, valid_dataset,CODE
LOWtasks/data_utils.py35def build_tokens_types_paddings_from_text(text_a, text_b,CODE
LOWtasks/data_utils.py49def build_tokens_types_paddings_from_ids(text_a_ids, text_b_ids, max_seq_length,CODE
LOWtasks/eval_utils.py65def calculate_correct_answers(name, model, dataloader,CODE
LOWtasks/eval_utils.py111 def correct_answers_forward_step(batch, model):CODE
LOWtools/run_vlm_text_generation.py161def generate_and_write_samples(model):CODE
LOWtools/prepare_cache.py54def _normalize_prepare_cache_args(args: Any) -> None:CODE
LOWtools/prepare_cache.py65def _validate_prepare_cache_args(args: Any) -> None:CODE
LOWtools/prepare_cache.py82def _disable_cache_load_only_flags(args: Any) -> Dict[str, bool]:CODE
LOWtools/prepare_cache.py102def _print_effective_configuration(CODE
LOWtools/prepare_cache.py120def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig:CODE
LOWtools/run_inference_performance_test.py49def add_inference_benchmarking_args(parser):CODE
LOWtools/build_sequences_per_dataset.py60def build_sequences_per_dataset(args):CODE
LOWtools/run_dynamic_text_generation_server.py21def add_text_generation_server_args(parser: argparse.ArgumentParser):CODE
LOWtools/run_dynamic_text_generation_server.py37async def run_text_generation_server(CODE
LOWtools/checkpoint/saver_hf_llava.py176 def receive_vision_projection(self):CODE
LOWtools/checkpoint/saver_hf_llava.py352 def save_state_dict_to_hf_checkpoint(self):CODE
LOWtools/checkpoint/schema_core.py10def get_core_transformer_block_key(model_key):CODE
LOWtools/checkpoint/schema_hf.py173def get_language_model_schema(CODE
LOWtools/checkpoint/loader_base.py30 def _maybe_parse_additional_megatron_args(self, margs, checkpoint_args):CODE
LOWtools/checkpoint/loader_base.py90 def _maybe_ensure_additional_required_arguments(self):CODE
LOWtools/checkpoint/loader_base.py107 def ensure_required_arguments(self):CODE
LOWtools/checkpoint/loader_base.py193 def get_models_for_pipeline_stage(count, dtype):CODE
LOWtools/checkpoint/loader_base.py425 def build_checkpoint_metadata(self, true_vocab_size):CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py404def initialize_ssm_layer_params(CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py152def parse_hybrid_layer_pattern(pattern):CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py183def build_layer_index_mapping(layer_types, direction):CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py316def validate_pattern_gpt_compatible(layer_types, direction):CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py361def validate_source_args_gpt_compatible(source_args, direction):CODE
LOWtools/checkpoint/dist_checkpoint_io.py69def resolve_checkpoint_subdir(load_dir):CODE
LOWtools/checkpoint/dist_checkpoint_io.py113def ensure_single_rank_process_group():CODE
LOWtools/checkpoint/dist_checkpoint_io.py159def load_dist_checkpoint_full(load_dir):CODE
LOWtools/checkpoint/dist_checkpoint_io.py221def save_dist_checkpoint_full(CODE
LOWtools/checkpoint/dist_checkpoint_io.py260def write_latest_iteration_marker(save_dir, iteration):CODE
LOWtools/checkpoint/loader_mixtral_hf.py28def load_args_from_checkpoint(args):CODE
LOWtools/checkpoint/loader_mixtral_hf.py63def verify_transformers_version():CODE
LOWtools/checkpoint/saver_llava.py134 def _maybe_parse_additional_megatron_args(self, margs):CODE
LOWtools/checkpoint/saver_llava.py320 def receive_vision_projection(self):CODE
LOWtools/checkpoint/checkpoint_inspector.py288def save_checkpoint_with_pickle_protocol(state_dict, output_dir, pickle_protocol=4):CODE
LOWtools/checkpoint/checkpoint_inspector.py292 def transform_object_override(write_item, obj):CODE
LOWtools/checkpoint/checkpoint_inspector.py831def convert_torch_dist_to_fsdp_dtensor(CODE
LOWtools/checkpoint/saver_base.py31 def _maybe_parse_additional_megatron_args(self, margs):CODE
LOWtools/checkpoint/saver_base.py38 def insert_megatron_path_and_check_te(self):CODE
LOWtools/checkpoint/saver_base.py277 def receive_checkpoint_metadata(self):CODE
LOWtools/checkpoint/saver_base.py341 def save_local_models_to_checkpoint(self):CODE
LOWtools/checkpoint/loader_llava.py54 def _maybe_parse_additional_megatron_args(self, margs, checkpoint_args):CODE
4293 more matches not shown…
Decorative Section Separators755 hits · 2630 pts
SeverityFileLineSnippetContext
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py400# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py402# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py132# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py134# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py225# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py227# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py486# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py488# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py530# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py532# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py627# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py629# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py698# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py700# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py722# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/gpt_hybrid_conversion.py724# ---------------------------------------------------------------------------COMMENT
MEDIUMtools/checkpoint/saver_llava.py225 #-----------COMMENT
MEDIUMtools/checkpoint/saver_llava.py186 #-----------COMMENT
MEDIUMtools/checkpoint/saver_base.py377 #-----------COMMENT
MEDIUMtools/checkpoint/saver_base.py420 # --------------COMMENT
MEDIUMtools/checkpoint/saver_base.py432 # ------------------COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py360 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py362 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py386 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py388 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py410 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py412 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py433 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py435 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py474 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py476 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py496 # ================================================================COMMENT
MEDIUM…s/common_pile_dataset/create_common_pile_ci_dataset.py498 # ================================================================COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh24# ── Parse KEY=VALUE positional args ───────────────────────────────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh75# ── Read model_config.yaml ────────────────────────────────────────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh108# ── Build model args (substituting ${CHECKPOINT_LOAD_PATH}) ───────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh131# ── Make image-bundled extras (mamba-ssm) visible to the cog venv ─────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh171# ── Launch the inference server in the background ─────────────────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh228# ── Wait for server readiness ─────────────────────────────────────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh248# ── Benchmark sweep ───────────────────────────────────────────────────────────COMMENT
MEDIUM…ts/performance_tests/shell_test_utils/run_perf_test.sh267# ── Baseline comparison or recording ──────────────────────────────────────────COMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py313 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py315 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py439 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py441 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_optimizer_param_scheduler.py302# ── get_canonical_lr_for_logging tests ──────────────────────────────────────COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py64# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py66# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py858# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py860# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py1305# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py1307# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py1592# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py1594# ===========================================================================COMMENT
MEDIUMtests/unit_tests/test_argument_utils.py681# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_argument_utils.py683# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/tokenizers/test_tokenizer.py641# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/tokenizers/test_tokenizer.py643# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/tokenizers/test_tokenizer.py473# ------------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/tokenizers/test_tokenizer.py475# ------------------------------------------------------------------------COMMENT
695 more matches not shown…
Unused Imports860 hits · 810 pts
SeverityFileLineSnippetContext
LOWtrain_rl.py4CODE
LOWmamba_builders.py14CODE
LOWmamba_builders.py15CODE
LOWpretrain_gpt.py26CODE
LOWpretrain_gpt.py64CODE
LOWpretrain_hybrid.py25CODE
LOWpretrain_hybrid.py64CODE
LOWgpt_builders.py20CODE
LOWtools/merge_datasets.py3CODE
LOWtools/preprocess_data_nmt.py6CODE
LOWtools/run_inference_performance_test.py37CODE
LOWtools/run_inference_performance_test.py41CODE
LOWtools/run_inference_performance_test.py43CODE
LOWtools/run_text_generation_server.py6CODE
LOWtools/linter.py11CODE
LOWtools/preprocess_mmdata.py12CODE
LOWtools/checkpoint/saver_core.py3CODE
LOWtools/checkpoint/saver_core.py5CODE
LOWtools/checkpoint/saver_core.py7CODE
LOWtools/checkpoint/saver_core.py8CODE
LOWtools/checkpoint/remap_gpt_dsa_to_mamba.py38CODE
LOWtools/checkpoint/remap_gpt_dsa_to_mamba.py39CODE
LOWtools/checkpoint/remap_gpt_dsa_to_mamba.py108CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py122CODE
LOWtools/checkpoint/dist_checkpoint_io.py20CODE
LOWtools/checkpoint/dist_checkpoint_io.py31CODE
LOWtools/checkpoint/utils.py3CODE
LOWtools/checkpoint/hybrid_conversion.py10CODE
LOWtools/checkpoint/loader_mixtral_hf.py3CODE
LOWtools/checkpoint/loader_mixtral_hf.py167CODE
LOWtools/checkpoint/saver_llava.py2CODE
LOWtools/checkpoint/saver_llava.py4CODE
LOWtools/checkpoint/saver_llava.py173CODE
LOWtools/checkpoint/saver_llava.py173CODE
LOWtools/checkpoint/checkpoint_inspector.py37CODE
LOWtools/checkpoint/loader_core.py3CODE
LOWtools/checkpoint/loader_core.py4CODE
LOWtools/checkpoint/loader_core.py5CODE
LOWtools/checkpoint/loader_core.py6CODE
LOWtools/checkpoint/loader_core.py7CODE
LOWtools/checkpoint/saver_base.py2CODE
LOWtools/checkpoint/loader_llava.py3CODE
LOWtools/checkpoint/loader_llava.py7CODE
LOWtools/bert_embedding/embed.py3CODE
LOWtools/bert_embedding/embed.py8CODE
LOWtools/bert_embedding/embed.py15CODE
LOWtools/bert_embedding/embed.py16CODE
LOWtools/bert_embedding/embed.py19CODE
LOWtools/bert_embedding/embed.py21CODE
LOWtools/bert_embedding/embed.py21CODE
LOWtools/bert_embedding/__init__.py3CODE
LOWtools/bert_embedding/__init__.py3CODE
LOW…formance_tests/shell_test_utils/compare_to_baseline.py23CODE
LOWtests/unit_tests/test_fp8_param.py3CODE
LOWtests/unit_tests/test_fp8_param.py29CODE
LOWtests/unit_tests/test_fp8_param.py39CODE
LOWtests/unit_tests/test_inference.py3CODE
LOWtests/unit_tests/test_basic.py2CODE
LOWtests/unit_tests/conftest.py4CODE
LOWtests/unit_tests/test_emerging_optimizers.py11CODE
800 more matches not shown…
Cross-File Repetition111 hits · 555 pts
SeverityFileLineSnippetContext
HIGHtrain_rl.py0forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt modelSTRING
HIGHpretrain_hybrid.py0forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt modelSTRING
HIGHmegatron/elastification/pretrain_hybrid_flex.py0forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt modelSTRING
HIGHpretrain_gpt.py0loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): STRING
HIGHpretrain_hybrid.py0loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): STRING
HIGHmegatron/elastification/pretrain_hybrid_flex.py0loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): STRING
HIGHpretrain_gpt.py0build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samplesSTRING
HIGHpretrain_hybrid.py0build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samplesSTRING
HIGHexamples/post_training/modelopt/finetune.py0build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samplesSTRING
HIGHexamples/t5/pretrain_t5.py0build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samplesSTRING
HIGHmegatron/elastification/pretrain_hybrid_flex.py0build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samplesSTRING
HIGHtools/run_text_generation_server.py0utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when STRING
HIGHexamples/inference/advanced/gpt_static_inference.py0utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when STRING
HIGH…amples/inference/advanced/simple_t5_batch_inference.py0utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when STRING
HIGHtools/checkpoint/saver_hf_llava.py0required top-level function that creates the saver and calls its .save().STRING
HIGHtools/checkpoint/saver_core.py0required top-level function that creates the saver and calls its .save().STRING
HIGHtools/checkpoint/saver_llava.py0required top-level function that creates the saver and calls its .save().STRING
HIGHtools/checkpoint/loader_base.py0orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: STRING
HIGHtools/checkpoint/loader_core.py0orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: STRING
HIGHtools/checkpoint/loader_llava.py0orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: STRING
HIGHtools/checkpoint/loader_base.py0parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args.STRING
HIGHtools/checkpoint/saver_base.py0parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args.STRING
HIGHtools/checkpoint/loader_llava.py0parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args.STRING
HIGHtools/checkpoint/loader_base.py0construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv.STRING
HIGHtools/checkpoint/loader_core.py0construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv.STRING
HIGHtools/checkpoint/saver_base.py0construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv.STRING
HIGHtools/checkpoint/loader_llava.py0construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv.STRING
HIGHtests/unit_tests/test_hyper_comm_grid.py0set up distributed environment for the entire test class.STRING
HIGH…it_tests/pipeline_parallel/test_bridge_communicator.py0set up distributed environment for the entire test class.STRING
HIGH…sts/pipeline_parallel/test_multimodule_communicator.py0set up distributed environment for the entire test class.STRING
HIGH…it_tests/pipeline_parallel/test_bridge_communicator.py0destroy all tracked grids and bridge communicator pgs.STRING
HIGH…_tests/pipeline_parallel/test_multimodule_schedules.py0destroy all tracked grids and bridge communicator pgs.STRING
HIGHtests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py0destroy all tracked grids and bridge communicator pgs.STRING
HIGHtests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py0return a transformerconfig for the vision projection mlp.STRING
HIGHexamples/mimo/configs/llava_vlm.py0return a transformerconfig for the vision projection mlp.STRING
HIGHexamples/mimo/configs/llava_avlm.py0return a transformerconfig for the vision projection mlp.STRING
HIGH…unit_tests/dist_checkpointing/models/test_gpt_model.py0test model loading with different vocab size (caused by tp padding).STRING
HIGH…nit_tests/dist_checkpointing/models/test_bert_model.py0test model loading with different vocab size (caused by tp padding).STRING
HIGHtests/unit_tests/dist_checkpointing/models/common.py0test model loading with different vocab size (caused by tp padding).STRING
HIGH…ultimodal/evaluation/evaluate_video_phys_game_bench.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_realworldqa.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_ai2d.py0merge input files to a format compatible with the evaluator.STRING
HIGH…es/multimodal/evaluation/evaluate_video_motionbench.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_spdocvqa.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_textvqa.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_chartqa.py0merge input files to a format compatible with the evaluator.STRING
HIGH…amples/multimodal/evaluation/evaluate_video_mvbench.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_mathvista.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_vqav2.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_ocrbench.py0merge input files to a format compatible with the evaluator.STRING
HIGHexamples/multimodal/evaluation/evaluate_infovqa.py0merge input files to a format compatible with the evaluator.STRING
HIGHmegatron/core/utils.py0check if minimum version of `transformer-engine` is installed.STRING
HIGH…/distributed/fsdp/src/megatron_fsdp/mixed_precision.py0check if minimum version of `transformer-engine` is installed.STRING
HIGH…atron/core/distributed/fsdp/src/megatron_fsdp/utils.py0check if minimum version of `transformer-engine` is installed.STRING
HIGHmegatron/core/ssm/mamba_mixer.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
HIGHmegatron/core/transformer/attention.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
HIGHmegatron/core/transformer/transformer_layer.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
HIGHmegatron/core/transformer/mlp.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
HIGHmegatron/core/transformer/multi_token_prediction.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
HIGHmegatron/core/transformer/moe/moe_layer.py0args: name (str | none): module instance name passed top-down from its paranet moduleSTRING
51 more matches not shown…
Deep Nesting571 hits · 482 pts
SeverityFileLineSnippetContext
LOWgpt_builders.py25CODE
LOWtasks/finetune_utils.py147CODE
LOWtasks/eval_utils.py65CODE
LOW.gitlab/scripts/check_imports.py66CODE
LOW.gitlab/scripts/check_imports.py123CODE
LOWtools/preprocess_data.py310CODE
LOWtools/run_vlm_text_generation.py88CODE
LOWtools/checkpoint/saver_hf_llava.py69CODE
LOWtools/checkpoint/loader_base.py254CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py631CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py726CODE
LOWtools/checkpoint/dist_checkpoint_io.py131CODE
LOWtools/checkpoint/hybrid_conversion.py59CODE
LOWtools/checkpoint/hybrid_conversion.py114CODE
LOWtools/checkpoint/hybrid_conversion.py170CODE
LOWtools/checkpoint/hybrid_conversion.py222CODE
LOWtools/checkpoint/saver_llava.py183CODE
LOWtools/checkpoint/checkpoint_inspector.py74CODE
LOWtools/checkpoint/checkpoint_inspector.py335CODE
LOWtools/checkpoint/checkpoint_inspector.py951CODE
LOWtools/checkpoint/saver_base.py365CODE
LOWtools/checkpoint/loader_llava.py161CODE
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py63CODE
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py109CODE
LOWtools/moe_routing/analyze_routing_predictability.py54CODE
LOWtools/moe_routing/analyze_routing_concentration.py70CODE
LOW…formance_tests/shell_test_utils/compare_to_baseline.py36CODE
LOW…shell_test_utils/determinism/print_nsys_leaderboard.py85CODE
LOWtests/unit_tests/test_fp8_param.py215CODE
LOWtests/unit_tests/test_optimizer.py844CODE
LOWtests/unit_tests/test_utils.py366CODE
LOWtests/unit_tests/test_imports.py63CODE
LOWtests/unit_tests/test_layer_wise_optimizer.py254CODE
LOWtests/unit_tests/test_layer_wise_optimizer.py724CODE
LOWtests/unit_tests/ssm/ops/test_ssd_bmm.py68CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py291CODE
LOW…t_tests/tools/checkpoint/test_gpt_hybrid_conversion.py461CODE
LOWtests/unit_tests/extension/test_kitchen_sdpa.py60CODE
LOWtests/unit_tests/determinism/utils.py119CODE
LOW…ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py1050CODE
LOW…ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py1125CODE
LOW…ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py1128CODE
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py177CODE
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py914CODE
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py1084CODE
LOWtests/unit_tests/transformer/test_cuda_graphs.py506CODE
LOWtests/unit_tests/transformer/test_cuda_graphs.py698CODE
LOWtests/unit_tests/transformer/test_cuda_graphs.py1368CODE
LOWtests/unit_tests/transformer/test_transformer_block.py405CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py2057CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3185CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3219CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3305CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3433CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3470CODE
LOW…mental_attention_variant/test_attention_variant_dsa.py3708CODE
LOW…erimental_attention_variant/dsa_native_parity_utils.py416CODE
LOWtests/unit_tests/training/config/test_container_base.py248CODE
LOW…it_tests/pipeline_parallel/test_bridge_communicator.py411CODE
LOW…sts/pipeline_parallel/test_multimodule_communicator.py170CODE
511 more matches not shown…
Hallucination Indicators30 hits · 415 pts
SeverityFileLineSnippetContext
CRITICALtools/checkpoint/loader_mixtral_hf.py69 model.embedding.word_embeddings.weight.data.copy_(CODE
CRITICALtools/checkpoint/loader_mixtral_hf.py74 model.decoder.final_layernorm.weight.data.copy_(hf_model.model.norm.weight)CODE
CRITICALtools/checkpoint/loader_mixtral_hf.py103 layer.mlp.router.weight.data.copy_(hf_layer.block_sparse_moe.gate.weight)CODE
CRITICALtools/checkpoint/loader_mixtral_hf.py127 layer.self_attention.linear_qkv.layer_norm_weight.data.copy_(hf_layer.input_layernorm.weight)CODE
CRITICALtools/checkpoint/saver_llava.py199 model.vision_model.conv1.weight.data.copy_(vit_embeddings_msg["conv1 weight"])CODE
CRITICALtools/checkpoint/saver_llava.py201 model.vision_model.conv1.bias.data.copy_(vit_embeddings_msg["conv1 bias"])CODE
CRITICALtools/checkpoint/saver_llava.py202 model.vision_model.position_embeddings.weight.data.copy_(vit_embeddings_msg["position embeddings"])CODE
CRITICALtools/checkpoint/saver_llava.py208 model.vision_model.embedder.weight.data.copy_(embedder_weight[tp_rank])CODE
CRITICALtools/checkpoint/saver_llava.py210 model.vision_model.embedder.bias.data.copy_(embedder_bias[tp_rank])CODE
CRITICALtools/checkpoint/saver_llava.py214 model.vision_model.ln_pre.weight.data.copy_(vit_embeddings_msg["ln pre weight"])CODE
CRITICALtools/checkpoint/saver_llava.py215 model.vision_model.ln_pre.bias.data.copy_(vit_embeddings_msg["ln pre bias"])CODE
CRITICALtools/checkpoint/saver_llava.py218 model.vision_model.ln_post.weight.data.copy_(vit_embeddings_msg["ln post weight"])CODE
CRITICALtools/checkpoint/saver_llava.py219 model.vision_model.ln_post.bias.data.copy_(vit_embeddings_msg["ln post bias"])CODE
CRITICALtools/checkpoint/saver_llava.py343 model.vision_projection.encoder.linear_fc1.weight.data.copy_(CODE
CRITICALtools/checkpoint/saver_llava.py345 model.vision_projection.encoder.linear_fc2.weight.data.copy_(CODE
CRITICALtools/checkpoint/saver_llava.py348 model.vision_projection.encoder.linear_fc1.layer_norm_weight.data.copy_(CODE
CRITICALtools/checkpoint/saver_llava.py351 model.vision_projection.encoder.linear_fc1.layer_norm_bias.data.copy_(CODE
CRITICALtools/checkpoint/saver_llava.py354 model.vision_projection.encoder.linear_fc1.bias.data.copy_(CODE
CRITICALtools/checkpoint/saver_llava.py356 model.vision_projection.encoder.linear_fc2.bias.data.copy_(vision_projection_l1_bias)CODE
CRITICALtests/unit_tests/transformer/moe/test_routers.py115 assert self.sequential_mlp.router.weight.grad.abs().sum() == 0CODE
CRITICALtests/unit_tests/transformer/moe/test_routers.py121 assert self.sequential_mlp.router.weight.grad.abs().sum() > 0CODE
CRITICALtests/unit_tests/transformer/moe/test_routers.py126 self.sequential_mlp.router.weight.grad.fill_(0)CODE
CRITICALtests/unit_tests/transformer/moe/test_routers.py129 assert self.sequential_mlp.router.weight.grad.abs().sum() > 0CODE
CRITICALtests/unit_tests/models/test_gpt_model.py87 assert self.gpt_model.embedding.word_embeddings.weight.std().cpu().item() == approx(CODE
CRITICALtests/unit_tests/models/test_gpt_model.py90 assert self.gpt_model.embedding.word_embeddings.weight.mean().cpu().item() == approx(CODE
CRITICALmegatron/core/parallel_state.py2169 and torch.distributed.distributed_c10d._world.pg_map.get(_DATA_PARALLEL_GROUP_GLOO, None)CODE
CRITICALmegatron/core/parallel_state.py2178 and torch.distributed.distributed_c10d._world.pg_map.get(CODE
CRITICALmegatron/core/parallel_state.py2217 and torch.distributed.distributed_c10d._world.pg_map.get(CODE
CRITICALmegatron/core/parallel_state.py2231 and torch.distributed.distributed_c10d._world.pg_map.get(CODE
CRITICALmegatron/core/transformer/attention.py1965 self.linear_qkv.weight.main_param.data.copy_(CODE
Self-Referential Comments130 hits · 380 pts
SeverityFileLineSnippetContext
MEDIUMgpt_builders.py42 # Define the decoder block specCOMMENT
MEDIUMgpt_builders.py54 # Define the decoder layer specCOMMENT
MEDIUMgpt_builders.py67 # Define the decoder block specCOMMENT
MEDIUM.gitlab/stages/05.publish.yml149 # Define the full refspec for the branchCOMMENT
MEDIUMtools/run_inference_performance_test.py113 # Create a list of valid token IDsCOMMENT
MEDIUMtools/common_pile_dataset/setup_common_pile_dataset.sh87# Create a virtual environment to avoid system package conflictsCOMMENT
MEDIUMtools/common_pile_dataset/setup_common_pile_dataset.sh122# Create the output directoryCOMMENT
MEDIUMtests/unit_tests/test_optimizer.py1060 # Create a new state_dict with all params set to 3.COMMENT
MEDIUMtests/unit_tests/test_optimizer.py1202 # Create a simple model for testingCOMMENT
MEDIUMtests/unit_tests/test_optimizer.py1284 # Create a simple modelCOMMENT
MEDIUMtests/unit_tests/test_utils.py509 # Create a straggler_detector with enabled set to false.COMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py637 # Create a process groupCOMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py640 # Create a tensor for communication testCOMMENT
MEDIUMtests/unit_tests/test_hyper_comm_grid.py741 # Create a unique tensor based on rankCOMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py87 # Create a simple linear model for testingCOMMENT
MEDIUMtests/unit_tests/test_emerging_optimizers.py657 # Create a model with QKV-like parameterCOMMENT
MEDIUMtests/unit_tests/test_training.py189 # Create a mock state_dict with gradients (use deterministic values for reproducibility).COMMENT
MEDIUMtests/unit_tests/ssm/ops/test_ssm_kernel.py83 # Create the Mixer instance directlyCOMMENT
MEDIUMtests/unit_tests/fusions/test_torch_softmax.py197 # Create a padding maskCOMMENT
MEDIUM…/unit_tests/post_training/test_modelopt_module_spec.py249 # Define the expected signatureCOMMENT
MEDIUM…/unit_tests/post_training/test_modelopt_module_spec.py288 # Define the expected signatureCOMMENT
MEDIUM…tests/distributed/test_torch_fully_sharded_parallel.py56 # Create a dummy model and configs.COMMENT
MEDIUM…tests/distributed/test_torch_fully_sharded_parallel.py63 # Create the sharded model.COMMENT
MEDIUM…tests/distributed/test_torch_fully_sharded_parallel.py84 # Create a dummy model and configs.COMMENT
MEDIUM…tests/distributed/test_torch_fully_sharded_parallel.py90 # Create a custom process group (using the default world for testing)COMMENT
MEDIUM…tests/distributed/test_torch_fully_sharded_parallel.py93 # Create the sharded model with explicit process groupCOMMENT
MEDIUM…ests/distributed/mfsdp_v1/test_mfsdp_uneven_dtensor.py600 # Create a manual uneven sharding along dim 1 with possible zero-length local on some ranksCOMMENT
MEDIUM…tests/transformer/test_transformer_block_custom_pgs.py259 # Create a transformer block with default process groupsCOMMENT
MEDIUM…tests/transformer/test_transformer_block_custom_pgs.py285 # Create a transformer block with custom process groupsCOMMENT
MEDIUM…tests/transformer/test_transformer_block_custom_pgs.py737 # Create a single transformer blockCOMMENT
MEDIUMtests/unit_tests/transformer/test_cuda_graphs.py777 # Create the CUDA graphs - this is where the is_last_layer logic is testedCOMMENT
MEDIUMtests/unit_tests/transformer/test_cuda_graphs.py1024 # Create a mapping of sample_keys to indicesCOMMENT
MEDIUMtests/unit_tests/transformer/test_transformer_block.py545 # Create a new build_layers method that uses interleaved attentionCOMMENT
MEDIUM…ts/unit_tests/transformer/moe/test_token_dispatcher.py174 # Create the answer.COMMENT
MEDIUMtests/unit_tests/transformer/moe/test_shared_experts.py386 # Create a dummy input tensor.COMMENT
MEDIUMtests/unit_tests/transformer/moe/test_aux_loss.py267 # Create a new config with updated parametersCOMMENT
MEDIUMtests/unit_tests/transformer/moe/test_aux_loss.py270 # Create the router with the updated configCOMMENT
MEDIUM…s/unit_tests/training/config/test_instantiate_utils.py334 # Create a mock that raises an error when used with functools.partialCOMMENT
MEDIUMtests/unit_tests/training/config/test_yaml_utils.py215 # Create a mock torch dtypeCOMMENT
MEDIUMtests/unit_tests/training/config/test_yaml_utils.py262 # Create a mock GenerationConfigCOMMENT
MEDIUMtests/unit_tests/utils/test_experimental_log_once.py23 # Define a fresh function with the decorator so it has its own closure state.COMMENT
MEDIUMtests/unit_tests/models/test_gpt_model.py237 # Define the expected signatureCOMMENT
MEDIUMtests/unit_tests/models/test_fastconformer_model.py45 # Create a parameter with the target dtype so ``next(self.parameters()).dtype``COMMENT
MEDIUM…s/unit_tests/models/mimo/test_mimo_audio_submodules.py212 # Create a time arrayCOMMENT
MEDIUM…s/unit_tests/models/mimo/test_mimo_audio_submodules.py215 # Create a simple sine wave at 440 Hz (A4)COMMENT
MEDIUM…nit_tests/models/mimo/test_mimo_embedding_alignment.py22 # Create a minimal MimoModelConfigCOMMENT
MEDIUM…nit_tests/models/mimo/test_mimo_embedding_alignment.py73 # Create a simple batchCOMMENT
MEDIUM…nit_tests/models/mimo/test_mimo_embedding_alignment.py249 # Create a test case with 2 batches:COMMENT
MEDIUM…nit_tests/models/mimo/test_mimo_embedding_alignment.py285 # Create the unflattened embeddings that would come from a vision encoderCOMMENT
MEDIUMtests/unit_tests/models/mimo/test_mimo_submodules.py92 # Create the main module specCOMMENT
MEDIUMtests/unit_tests/models/mimo/test_mimo_submodules.py351 # Create a data batch without imagesCOMMENT
MEDIUMtests/unit_tests/dist_checkpointing/test_fp8.py33 # Create a quantizer for FP8 conversionCOMMENT
MEDIUM…s/unit_tests/dist_checkpointing/test_fully_parallel.py596 # Create a mock that will do what it's supposed to do,COMMENT
MEDIUM…/unit_tests/inference/contexts/test_dynamic_context.py318 # Initialize all variablesCOMMENT
MEDIUM…/unit_tests/inference/contexts/test_dynamic_context.py1092 # Create an active_requests_mask where requests 0, 2, and 4 are finished (0),COMMENT
MEDIUM…/unit_tests/inference/contexts/test_dynamic_context.py1182 # Create an active_requests_mask where all requests are finishedCOMMENT
MEDIUM…ts/unit_tests/inference/engines/test_dynamic_engine.py2157 # Create a deterministic mock forward pass that returns logitsCOMMENT
MEDIUM…ts/unit_tests/inference/engines/test_dynamic_engine.py1727 # Create a request with length 513COMMENT
MEDIUMtests/unit_tests/data/test_builder.py94 # Define the class here to avoid pytest warningsCOMMENT
MEDIUMtests/unit_tests/rl/test_rl_utils.py763 # Create a realistic GPTModel as used in RL trainingCOMMENT
70 more matches not shown…
Excessive Try-Catch Wrapping212 hits · 264 pts
SeverityFileLineSnippetContext
LOWtasks/finetune_utils.py61 except Exception:CODE
LOWtasks/eval_utils.py114 except Exception:CODE
MEDIUMtasks/eval_utils.py111def correct_answers_forward_step(batch, model):CODE
LOW.gitlab/scripts/check_imports.py117 except Exception:CODE
LOWtools/check_copyright.py22 except Exception as e:CODE
MEDIUMtools/check_copyright.py23 print(f"Error reading {file_path}: {e}")CODE
MEDIUMtools/text_generation_cli.py21 print(f"Error {response.status_code}: {response.json()['message']}")CODE
LOWtools/checkpoint/saver_hf_llava.py400 except Exception as e:CODE
LOWtools/checkpoint/loader_base.py78 except Exception as e:CODE
MEDIUMtools/checkpoint/loader_base.py79 print(f"Error validating Megatron arguments: {e}")CODE
LOWtools/checkpoint/saver_core.py70 except Exception as e:CODE
LOWtools/checkpoint/gpt_hybrid_conversion.py384 except Exception:CODE
LOWtools/checkpoint/dist_checkpoint_io.py215 except Exception:CODE
LOWtools/checkpoint/hybrid_conversion.py322 except Exception:CODE
LOWtools/checkpoint/loader_mixtral_hf.py340 except Exception:CODE
MEDIUMtools/checkpoint/loader_mixtral_hf.py337def load_checkpoint(queue, args):CODE
LOWtools/checkpoint/saver_llava.py404 except Exception as e:CODE
LOWtools/checkpoint/loader_core.py90 except Exception as e:CODE
LOWtools/checkpoint/loader_llava.py360 except Exception as e:CODE
LOWtools/bert_embedding/embed.py217 except Exception:CODE
LOWtools/bert_embedding/embed.py223 except Exception:CODE
LOWtests/unit_tests/conftest.py48 except Exception:CODE
LOWtests/unit_tests/conftest.py96 except Exception as e:CODE
LOWtests/unit_tests/test_hyper_comm_grid.py531 except Exception as e:CODE
LOWtests/unit_tests/test_api_backwards_compat_setup.py106 except Exception as e:CODE
LOWtests/unit_tests/test_api_backwards_compat_setup.py134 except Exception as e:STRING
LOWtests/unit_tests/test_imports.py58 except Exception:CODE
LOWtests/unit_tests/test_imports.py77 except Exception:CODE
LOWtests/unit_tests/test_utilities.py114 except Exception:CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py650except Exception:CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py19except Exception:CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py152 except Exception:CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py178 except Exception:CODE
LOWtests/unit_tests/tokenizers/test_tokenizer.py320 except Exception:CODE
LOW…ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py1142 except Exception:CODE
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py74 except Exception as e:CODE
LOWtests/unit_tests/transformer/moe/test_aux_loss.py38except Exception: # pragma: no cover - defensiveCODE
LOWtests/unit_tests/transformer/moe/test_routers.py25except Exception: # pragma: no cover - defensiveCODE
LOWtests/unit_tests/models/test_hybrid_moe_model.py427 except Exception:CODE
LOW…s/unit_tests/models/mimo/test_mimo_audio_submodules.py123 except Exception as e:CODE
LOW…s/unit_tests/models/mimo/test_mimo_audio_submodules.py197 except Exception as e:CODE
LOWtests/unit_tests/models/mimo/test_mimo_submodules.py55 except Exception as e:CODE
LOWtests/unit_tests/models/mimo/test_mimo_submodules.py105 except Exception as e:CODE
LOWtests/unit_tests/models/mimo/test_mimo_submodules.py229 except Exception as e:CODE
LOWtests/unit_tests/models/mimo/test_mimo_submodules.py284 except Exception as e:CODE
MEDIUMtests/unit_tests/models/mimo/test_mimo_model.py148def setup_method(self, method):CODE
MEDIUMtests/unit_tests/models/mimo/test_mimo_model.py164def teardown_method(self, method):CODE
MEDIUMtests/unit_tests/models/mimo/test_mimo_model.py609def setup_method(self, method):CODE
MEDIUMtests/unit_tests/models/mimo/test_mimo_model.py623def teardown_method(self, method):CODE
LOWtests/unit_tests/models/mimo/test_mimo_model.py151 except Exception:CODE
LOWtests/unit_tests/models/mimo/test_mimo_model.py167 except Exception:CODE
LOWtests/unit_tests/models/mimo/test_mimo_model.py612 except Exception:CODE
LOWtests/unit_tests/models/mimo/test_mimo_model.py626 except Exception:CODE
LOWtests/unit_tests/dist_checkpointing/test_fp8.py24 except Exception as e:CODE
LOW…sts/unit_tests/dist_checkpointing/test_safe_globals.py166 except Exception as exc:CODE
MEDIUM…sts/unit_tests/dist_checkpointing/test_safe_globals.py163def load(i):CODE
LOWtests/unit_tests/dist_checkpointing/test_async_save.py30 except Exception as e:CODE
LOWtests/unit_tests/resharding/test_model_swap.py34except Exception:CODE
LOWtests/unit_tests/resharding/test_model_swap.py44except Exception:CODE
LOWtests/unit_tests/resharding/test_model_swap.py365 except Exception:CODE
152 more matches not shown…
Modern Structural Boilerplate195 hits · 196 pts
SeverityFileLineSnippetContext
LOWtools/trigger_internal_ci.py48logger = logging.getLogger(__name__)CODE
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py33logger = logging.getLogger(__name__)CODE
LOW…sts/unit_tests/distributed/mfsdp_v1/test_annotation.py45def _setup_nvtx_recording(monkeypatch: pytest.MonkeyPatch, events: list[NvtxEvent]) -> None:CODE
LOW…ts/unit_tests/distributed/mfsdp_v2/test_fully_shard.py22logger = logging.getLogger(__name__)CODE
LOW…sts/unit_tests/distributed/mfsdp_v2/test_cuda_graph.py18logger = logging.getLogger(__name__)CODE
LOWtests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py53logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/recipe_parser.py13logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/check_status_of_main.py18logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/launch_jet_workload.py30logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/launch_jet_workload.py33logger = logging.getLogger(__name__)CODE
LOW…/test_utils/python_scripts/launch_nemo_run_workload.py16logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/notify.py19logger = logging.getLogger(__name__)CODE
LOW…st_utils/python_scripts/download_unit_tests_dataset.py18logger = logging.getLogger(__name__)CODE
LOW…/test_utils/python_scripts/compare_golden_values_kl.py63logger = logging.getLogger(__name__)CODE
LOW…test_utils/python_scripts/download_coverage_results.py15logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/auto_reminder_github.py23logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/wait_for_resources.py19logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/dashboard.py15logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/approve_merge_gate.py27logger = logging.getLogger(__name__)CODE
LOWtests/test_utils/python_scripts/swap_pr_labels.py19logger = logging.getLogger(__name__)CODE
LOW…ts/test_utils/python_scripts/download_golden_values.py19logger = logging.getLogger(__name__)CODE
LOW…l_tests/python_test_utils/compute_golden_statistics.py43logger = logging.getLogger(__name__)CODE
LOW…t_utils/test_pretraining_resume_checkpoint_pipeline.py11logger = logging.getLogger(__name__)CODE
LOW…s/python_test_utils/test_inference_regular_pipeline.py14logger = logging.getLogger(__name__)CODE
LOW…python_test_utils/test_pretraining_regular_pipeline.py11logger = logging.getLogger(__name__)CODE
LOWtests/functional_tests/python_test_utils/common.py20logger = logging.getLogger(__name__)CODE
LOW…nal_tests/python_test_utils/test_grpo_training_loop.py11logger = logging.getLogger(__name__)CODE
LOW…n_test_utils/get_test_results_from_tensorboard_logs.py13logger = logging.getLogger(__name__)CODE
LOWexamples/mimo/training/distributed.py15__all__ = ["initialize_distributed", "print_rank_0", "shutdown_distributed"]CODE
LOWscripts/check_api_backwards_compatibility.py26logger = logging.getLogger(__name__)CODE
LOWmegatron/post_training/checkpointing.py22logger = logging.getLogger(__name__)CODE
LOWmegatron/core/fault_injector.py31__all__ = ["FaultInjectorConfig", "setup_fault_injection", "maybe_raise_workload_exception"]CODE
LOWmegatron/core/fault_injector.py42logger = logging.getLogger(__name__)CODE
LOWmegatron/core/parallel_state.py19logger = logging.getLogger(__name__)CODE
LOWmegatron/core/full_cuda_graph.py12logger = logging.getLogger(__name__)CODE
LOWmegatron/core/__init__.py30__all__ = [CODE
LOWmegatron/core/msc_utils.py6logger = logging.getLogger(__name__)CODE
LOWmegatron/core/msc_utils.py67__all__ = ['MultiStorageClientFeature', 'open_file']CODE
LOWmegatron/core/optimizer_param_scheduler.py16logger = logging.getLogger(__name__)CODE
LOWmegatron/core/nccl_allocator.py15logger = logging.getLogger(__name__)CODE
LOWmegatron/core/utils.py54logger = logging.getLogger(__name__)CODE
LOWmegatron/core/config_logger.py126__all__ = ['has_config_logger_enabled', 'log_config_to_disk']CODE
LOWmegatron/core/num_microbatches_calculator.py9logger = logging.getLogger(__name__)CODE
LOWmegatron/core/timers.py32logger = logging.getLogger(__name__)CODE
LOWmegatron/core/rerun_state_machine.py32logger = logging.getLogger(__name__)CODE
LOWmegatron/core/rerun_state_machine.py243 def set_mode(self, mode: RerunMode) -> None:CODE
LOWmegatron/core/rerun_state_machine.py1393def _set_rerun_state_machine(rerun_state_machine) -> None:CODE
LOWmegatron/core/ssm/mamba_mixer.py97logger = logging.getLogger(__name__)CODE
LOWmegatron/core/ssm/gated_delta_net.py56logger = logging.getLogger(__name__)CODE
LOWmegatron/core/ssm/ops/__init__.py14__all__ = ["mamba_chunk_scan_combined_varlen", "causal_conv1d_varlen_fn"]CODE
LOWmegatron/core/tokenizers/megatron_tokenizer.py34logger = logging.getLogger(__name__)CODE
LOWmegatron/core/tokenizers/utils/build_tokenizer.py23logger = logging.getLogger(__name__)CODE
LOWmegatron/core/tokenizers/text/parsers/__init__.py12__all__ = ["PARSER_MAPPING"]CODE
LOW…ore/tokenizers/text/parsers/qwen3_coder_tool_parser.py14logger = logging.getLogger(__name__)CODE
LOW…ore/tokenizers/text/libraries/huggingface_tokenizer.py17logger = logging.getLogger(__name__)CODE
LOW…n/core/tokenizers/text/libraries/tiktoken_tokenizer.py18logger = logging.getLogger(__name__)CODE
LOW…n/core/tokenizers/text/libraries/tiktoken_tokenizer.py28__all__ = ["TikTokenTokenizer", "reload_mergeable_ranks"]CODE
LOW…ore/tokenizers/text/libraries/megatron_hf_tokenizer.py20logger = logging.getLogger(__name__)CODE
LOWmegatron/core/tensor_parallel/__init__.py49__all__ = [CODE
LOWmegatron/core/post_training/modelopt/layers.py17logger = logging.getLogger(__name__)CODE
135 more matches not shown…
Redundant / Tautological Comments127 hits · 194 pts
SeverityFileLineSnippetContext
LOWtrain_rl.py62 # Check if fp8_model_init supports preserve_high_precision_init_valCOMMENT
LOWtools/check_copyright.py20 # Check if the expected header is at the start of the fileCOMMENT
LOWtools/common_pile_dataset/setup_common_pile_dataset.sh50# Check if create_common_pile_ci_dataset.py was scp'd alongside this scriptCOMMENT
LOWdocker/common/install_source_wheels.sh25# Check if required arguments are providedCOMMENT
LOWtests/unit_tests/test_optimizer.py45 # Check if FP8 block scaling is available.COMMENT
LOWtests/unit_tests/test_utils.py512 # Check if configuration was success.COMMENT
LOWtests/unit_tests/test_utils.py515 # Check if the instance is in disabled state.COMMENT
LOWtests/unit_tests/test_utils.py519 # Check if all ranks have straggler detector enabled.COMMENT
LOWtests/unit_tests/conftest.py83 # Check if data directory exists and has contentCOMMENT
LOWtests/unit_tests/fusions/test_torch_softmax.py145 # Check if output is a valid probability distributionCOMMENT
LOWtests/unit_tests/transformer/test_cuda_graphs.py572 # Check if cuda graph is correctly setting is first/last layerCOMMENT
LOWtests/unit_tests/transformer/test_attention.py416 # Check if output and bias have the correct shapeCOMMENT
LOWtests/unit_tests/transformer/test_attention.py598 # Check if the output is closeCOMMENT
LOW…/unit_tests/transformer/test_multi_latent_attention.py1552 # Check if the output is the sameCOMMENT
LOW…it_tests/transformer/moe/test_moe_layer_discrepancy.py69 # Check if parameters are the sameCOMMENT
LOW…it_tests/transformer/moe/test_moe_layer_discrepancy.py75 # Check if input is the same across all ranksCOMMENT
LOW…it_tests/transformer/moe/test_moe_layer_discrepancy.py150 # Check if output is the same across all ranksCOMMENT
LOW…it_tests/transformer/moe/test_moe_layer_discrepancy.py216 # Check if output is the same across all ranksCOMMENT
LOWtests/unit_tests/transformer/moe/test_moe_layer.py164 # Check if the moe layer is interleaved correctlyCOMMENT
LOWtests/unit_tests/models/test_llava_model.py753 # Check if output shape is as expectedCOMMENT
LOW…_tests/dist_checkpointing/test_layer_wise_optimizer.py147 # Check if optimizer is ChainedOptimizer (expected for standard setup)COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py1075 # Assign blocks to the requests (one block per request)COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py1151 # Assign blocks to the requests:COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py3446 # Check if any request was evicted during this stepCOMMENT
LOW…ts/test_utils/python_scripts/download_golden_values.py282 # Check if we should skip based on only_failing flagCOMMENT
LOW…nal_tests/test_cases/common/ckpt_converter/__main__.py895 # Print results.COMMENT
LOW…unctional_tests/shell_test_utils/run_batch_ci_tests.sh173 # Check if file is empty (job still running or not started)COMMENT
LOWtests/functional_tests/shell_test_utils/run_ci_test.sh11# Set umask to 0002 to allow group read/write permissionsCOMMENT
LOWtests/functional_tests/shell_test_utils/run_ci_test.sh267 ## Loop over the list of model configs in the params file and run each one in sequence, collectingCOMMENT
LOWdocs/conf.py43# Check if we should skip autodoc generationCOMMENT
LOWexamples/post_training/modelopt/finetune.py251 # Check if this is OpenAI chat data?COMMENT
LOWexamples/mimo/utils/logging.py40 # Print output projectionsCOMMENT
LOWexamples/gptoss/02_train.sh55# Check if checkpoint path existsCOMMENT
LOWexamples/gptoss/02_train.sh62# Check if tensorboard logs path existsCOMMENT
LOWexamples/multimodal/dataset_helpers.py121 # Check if all samples fit in the knapsack capacity.COMMENT
LOWexamples/inference/utils.py235 # Check if we have any prompts (from command line or JSONL)COMMENT
LOWexamples/inference/advanced/gpt_dynamic_inference.py263 # Check if all requests are finished.COMMENT
LOWexamples/rl/benchmark_refit.py216 # Print resultsCOMMENT
LOWexamples/rl/benchmark_refit.py312 # Print resultsCOMMENT
LOWexamples/rl/environments/countdown/countdown.py34 # Check if all numbers in equation are availableCOMMENT
LOWscripts/check_api_backwards_compatibility.py240 # Check if this breakage kind should be ignored globally (not a signature change)COMMENT
LOWscripts/check_api_backwards_compatibility.py253 # Check if this is a breakage kind we ignore for __init__ methodsCOMMENT
LOWscripts/check_api_backwards_compatibility.py263 # Check if it's a child of a filtered objectCOMMENT
LOWscripts/check_api_backwards_compatibility.py350 # Print resultsCOMMENT
LOW.github/actions/check-nvidia-sso-membership/action.yml94 # Check if SSO file is availableCOMMENT
LOW.github/actions/check-nvidia-sso-membership/action.yml103 # Check if username exists as a key in the JSON objectCOMMENT
LOWmegatron/post_training/model_builder.py286 # Set num_layers to 0 for base model in offline modeCOMMENT
LOWmegatron/core/parallel_state.py836 # Set NCCL_COLLNET_ENABLE to 1 to enable SHARP for the dp group.COMMENT
LOWmegatron/core/parallel_state.py1275 # Set NCCL_COLLNET_ENABLE to 1 to enable SHARP for the dp_replica group.COMMENT
LOWmegatron/core/parallel_state.py1304 # Set NCCL_COLLNET_ENABLE to 0 to restrict SHARP application to the dp_replica group.COMMENT
LOWmegatron/core/fp4_utils.py14# Check if Transformer Engine is installedCOMMENT
LOWmegatron/core/fp4_utils.py25# Check if Transformer Engine has class for fp4 tensors.COMMENT
LOWmegatron/core/fp8_utils.py23# Check if Transformer Engine is installedCOMMENT
LOWmegatron/core/fp8_utils.py40# Check if Transformer Engine has class for fp8 tensors.COMMENT
LOWmegatron/core/fp8_utils.py55# Check if Transformer Engine has MXFP8Tensor classCOMMENT
LOWmegatron/core/fp8_utils.py653 # Check if fp8_model_init supports setting recipeCOMMENT
LOWmegatron/core/fp8_utils.py658 # Check if fp8_model_init supports preserve_high_precision_init_valCOMMENT
LOWmegatron/core/utils.py3010 # Check if any deprecated key is present in kwargsCOMMENT
LOWmegatron/core/ssm/ops/causal_conv1d_triton.py243 # Check if input is 2D, temporarily treat as 3D for uniform processingCOMMENT
LOWmegatron/core/tokenizers/megatron_tokenizer.py67 # Check if metadata file existsCOMMENT
67 more matches not shown…
Docstring Block Structure37 hits · 185 pts
SeverityFileLineSnippetContext
HIGHtools/checkpoint/remap_gpt_dsa_to_mamba.py46Return the HybridModel state-dict key corresponding to *key* from GPTModel. Args: key: A key from the GPTMoSTRING
HIGH…ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py756 Run a small deterministic training loop using a mocked hybrid Mamba+MoE model and optimizer. This helpeSTRING
HIGHtests/test_utils/python_scripts/launch_jet_workload.py105Block until a downstream JET pipeline reaches a terminal status. jetclient's own ``pipeline.wait`` polls the GitLabSTRING
HIGHmegatron/core/process_groups_config.py668Get process group collection for a specific module. Args: module_name: Name of the module. STRING
HIGHmegatron/core/hyper_comm_grid.py209Create a process group based on a list of dimension names Note: The unique key used to store the process group STRING
HIGHmegatron/core/utils.py120Validates the request to the experimental function. Args: func (Callable): Callee max_lSTRING
HIGHmegatron/core/utils.py186Validates the request to the experimental function. Args: func (Callable): Callee max_lSTRING
HIGHmegatron/core/utils.py218Pass-through to callee attribute if experimental flag is enabled. Args: super (supeSTRING
HIGHmegatron/core/timers.py385Returns the output string with logged timer values according to configured options. Args: names (LiSTRING
HIGH…ore/tokenizers/text/libraries/huggingface_tokenizer.py189 Adds a dictionary of special tokens (eos, pad, cls...). If special tokens are NOT in the vocabulary, thSTRING
HIGHmegatron/core/fusions/fused_bias_swiglu.py210Implementation of biased SwiGLU that handles different input shapes. This function reshapes the input if necessary,STRING
HIGHmegatron/core/fusions/fused_bias_geglu.py154Implementation of biased GEGLU that handles different input shapes. This function reshapes the input if necessary, STRING
HIGHmegatron/core/optimizer/optimizer.py538Filter and reorder state_dict parameter groups to match current optimizer groups. Keys used for matching align wSTRING
HIGH…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py4605 Release the specified parameter bucket, freeing its associated buffer storage. This function marks or STRING
HIGH…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py5016 Creates a distributed tensor (DTensor) from a local tensor with support for Megatron-FSDP and Tensor Parallel sSTRING
HIGH…e/distributed/fsdp/src/megatron_fsdp/uneven_dtensor.py140 Validates the chunk metadata of an uneven DTensor to ensure correctness and boundary coverage. Notes: - `gSTRING
HIGH…e/distributed/fsdp/src/megatron_fsdp/uneven_dtensor.py257 Gather a DTensor with potentially uneven sharding across ranks into a full tensor. This function handles DTensSTRING
HIGH…core/distributed/fsdp/src/megatron_fsdp/fully_shard.py455 Fully shard the optimizer for Megatron-FSDP. This is an in-place operation on the optimizer instance, which modSTRING
HIGH…tron/core/datasets/blended_megatron_dataset_builder.py497Build the DistributedDataset Return None if and only if the underlying dataset class is not built on the currenSTRING
HIGHmegatron/core/datasets/object_storage_utils.py136Ascertain whether the object at the given S3 path exists in S3 Args: client (S3Client): The S3 client STRING
HIGHmegatron/core/datasets/indexed_dataset.py88Get the size of the dtype/code in bytes Args: key (Union[int, Type[numpy.number]]): The dtype or coSTRING
HIGHmegatron/core/datasets/indexed_dataset.py798Return from the dataset Args: idx (Union[int, numpy.integer, slice]): The index or index slice intoSTRING
HIGH…ron/core/pipeline_parallel/multimodule_communicator.py496Compute the total number of pipeline stages across a multi-module chain. Interprets ``topology`` as a directed STRING
HIGHmegatron/core/models/hybrid/hybrid_layer_allocation.py161Count layers by type across the full hybrid pattern (main + MTP). Parses the pattern to extract main and MTP componSTRING
HIGHmegatron/core/models/hybrid/hybrid_layer_allocation.py200Parse a unified hybrid pattern string into main and MTP components. The pattern uses "/" as a separator between theSTRING
HIGHmegatron/core/models/hybrid/hybrid_layer_allocation.py301Validate and convert a single pipeline segment pattern to a layer type list. This is used after the main pattern haSTRING
HIGHmegatron/core/models/hybrid/hybrid_layer_allocation.py339Select and validate the pipeline segment for the given PP rank and VP stage. When the main pattern contains '|' pipSTRING
HIGHmegatron/core/dist_checkpointing/validation.py294Raises or logs an error in case missing or unexpected keys are non-empty. Args: missing_keys (Set[str]): miSTRING
HIGHmegatron/core/dist_checkpointing/validation.py372Validate if the ShardedTensors and ShardedObjects from multiple processes define correct sharding. Local ShardedTenSTRING
HIGH…tron/core/dist_checkpointing/strategies/async_utils.py669Finalizes all available calls. This method must be called on all ranks. Args: blocking (boSTRING
HIGH…ng/nvshmem_copy_service/memory/tensor_pointer_utils.py19 Extract the data pointer from a tensor. Args: tensor: Can be torch.Tensor, CuPy array, or STRING
HIGHmegatron/core/inference/shards_spec.py101Parse + validate the ``--inference-shards`` string. Args: spec_str: Raw CLI value, e.g. ``"tp=2,dp=1+tp=1,dSTRING
HIGHmegatron/core/export/trtllm/trtllm_layers.py85Helper function to rename model layer names to TRTLLM Layer names We go through each layer (keys) in the model STRING
HIGHmegatron/training/arguments.py172Validate model config arguments from heterogeneous config. This function takes model arguments and validates them bSTRING
HIGHmegatron/training/config/instantiate_utils.py128Instantiate an object or callable from a config object. This function takes a configuration object (dictionary, lisSTRING
HIGHmegatron/training/config/instantiate_utils.py232Recursively instantiates a node within a configuration structure. This function handles the instantiation of indiviSTRING
HIGHmegatron/rl/rl_utils.py1291Pad trajectories and extract the generation masks. Args: rollouts: Rollouts to extract trajectories from. STRING
Structural Annotation Overuse73 hits · 132 pts
SeverityFileLineSnippetContext
LOWtools/checkpoint/saver_hf_llava.py17 # Step 1: Reshape back to (num_head, 3*head_dim, -1)COMMENT
LOWtools/checkpoint/saver_hf_llava.py20 # Step 2: Slice along the head_dim dimension to get q, k, vCOMMENT
LOWtools/checkpoint/saver_hf_llava.py25 # Step 3: Reshape each back to (num_head * head_dim, -1)COMMENT
LOWtools/common_pile_dataset/README.md113### Step 1: Vocabulary FilesCOMMENT
LOWtools/common_pile_dataset/README.md119### Step 2: Download Raw TextCOMMENT
LOWtools/common_pile_dataset/README.md132### Step 3: GPT PreprocessingCOMMENT
LOWtools/common_pile_dataset/README.md137### Step 4: BERT PreprocessingCOMMENT
LOWtools/common_pile_dataset/README.md150### Step 5: T5 PreprocessingCOMMENT
LOWtools/common_pile_dataset/README.md155### Step 6: VerificationCOMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py361 # Step 1: Get vocabulary filesCOMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py387 # Step 2: Get raw text dataCOMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py411 # Step 3: Preprocess for GPT (GPT2BPETokenizer)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py434 # Step 4: Preprocess for BERT (BertWordPieceLowerCase + split-sentences)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py475 # Step 5: Preprocess for T5 (BertWordPieceCase)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py497 # Step 6: Clean up and verifyCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2038 # Step 1: Forward pass for all 3 requestsCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2047 # Step 2: Forward pass where req 10 finishes, req 11 continues. Req 999 is NOT scheduled.COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2136 # Step 1: All 3 requests are active, process forward passCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2145 # Step 2: Both decode requests finish, chunked prefill NOT scheduled this step.COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2069 # Step 3: Add the next chunk. It should sit exactly at the boundary (index 1) and inherit the state.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py3657 # Step 1: prefill for request 0 — should NOT count as a spec step.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py3667 # Step 2: decode for request 0 — should count spec tokens.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py2753 # Step 1: Prefill. Processes the 4 prompt tokens.COMMENT
LOW…l_tests/python_test_utils/compute_golden_statistics.py11 # Step 1: Run batch tests (from megatron-rl directory):STRING
LOW…l_tests/python_test_utils/compute_golden_statistics.py15 # Step 2: Wait for jobs to complete, then compute statistics:STRING
LOWdocs/developer/submit.md15## Step 1: Mark PR as "Ready for Review"COMMENT
LOWdocs/developer/submit.md24## Step 2: Final Review (`megatron/core` only)COMMENT
LOWdocs/developer/submit.md32## Step 3: ApprovedCOMMENT
LOWdocs/developer/submit.md37## Step 4: MergeCOMMENT
LOWdocs/user-guide/data-loading.md39### Step 1: Consolidate dataset filesCOMMENT
LOWdocs/user-guide/data-loading.md53### Step 2: Pre-build the dataset cacheCOMMENT
LOWdocs/user-guide/data-loading.md77### Step 3: Optionally pre-build per-dataset metadataCOMMENT
LOWdocs/user-guide/data-loading.md87### Step 4: Launch training with optimized data loadingCOMMENT
LOWexamples/gptoss/README.md3## Step 0: Install DependenciesCOMMENT
LOWexamples/gptoss/README.md57## Step 1: Convert HuggingFace to Megatron (Optional - skip if you already have a Megatron checkpoint)COMMENT
LOWexamples/gptoss/README.md65## Step 2: Train from ScratchCOMMENT
LOWexamples/gptoss/README.md147## Step 3: Convert Megatron to HuggingFaceCOMMENT
LOW.github/pull_request_template.md33#### Step 1: Mark PR as "Ready for Review"COMMENT
LOW.github/pull_request_template.md42#### Step 2: Final ReviewCOMMENT
LOW.github/pull_request_template.md48#### Step 3: ApprovedCOMMENT
LOW.github/scripts/oncall_manager.py135 # Step 1: Add new oncall first (include current members to avoid removing anyone yet)COMMENT
LOW.github/scripts/oncall_manager.py142 # Step 2: Now set the usergroup to contain only the new oncallCOMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1871 # Step 3: Split parameter groups to meet communication segmentation requirements.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1916 # Step 4: Generate the groups of collective buckets, where each group aggregatesCOMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1751 # Step 0: Register new FSDP unit modules.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1784 # Step 1: Group the parameters according to their execution order and attributes.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1822 # Step 2: Bucket the parameters based on the guide bucket size.COMMENT
LOWmegatron/core/transformer/moe/README.md152### Step 1: Find the feasible parallel mapping under the memory capacity of the GPUCOMMENT
LOWmegatron/core/transformer/moe/README.md170### Step 2: Select Optimal Parallelism StrategyCOMMENT
LOWmegatron/core/transformer/moe/README.md226### Step 3: Enable Performance Features Based on Profiling BottlenecksCOMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py265 # Step 1: Find maximum value in the row for numerical stabilityCOMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py277 # Step 2: Compute sum of exp(x - max_val)COMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py293 # Step 3: Compute final log_softmax values: x - max_val - log_sum_expCOMMENT
LOWmegatron/core/pipeline_parallel/bridge_communicator.py437 # Step 1: broadcast its shape so receivers can allocateCOMMENT
LOWmegatron/core/pipeline_parallel/bridge_communicator.py443 # Step 2: broadcast the actual tensorCOMMENT
LOW…n/core/dist_checkpointing/strategies/fully_parallel.py238 # Step 3: load part of the checkpoint.COMMENT
LOW…n/core/dist_checkpointing/strategies/fully_parallel.py264 # Step 4: exchange data between ranksCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py232 # Step 1: Segment tasks (break large tasks into chunks)COMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py242 # Step 2: Pack tasks into workload groupsCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py249 # Step 3: Schedule workloads to iterationsCOMMENT
13 more matches not shown…
Over-Commented Block142 hits · 128 pts
SeverityFileLineSnippetContext
LOW.gitlab/scripts/check_imports.py1# Copyright (c) 2025, NVIDIA CORPORATION.COMMENT
LOWtools/bisect.sh1#!/usr/bin/env bashCOMMENT
LOWtools/trigger_internal_ci.py1#!/usr/bin/env python3COMMENT
LOWtools/checkpoint/convert.py1# Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.COMMENT
LOWtools/checkpoint/convert.py21# full model weights, nothing split.COMMENT
LOWtools/checkpoint/convert.py41# consumed_train_samplesCOMMENT
LOWtools/checkpoint/convert.py61# "mlp l0 bias"COMMENT
LOWtools/checkpoint/gpt_hybrid_conversion.py221 else:COMMENT
LOWtools/checkpoint/gpt_hybrid_conversion.py241#COMMENT
LOWtools/common_pile_dataset/setup_common_pile_dataset.sh1#!/bin/bashCOMMENT
LOW…t_cases/gpt/gpt_583m_perf_gb200_4gpu/model_config.yaml1# Inference perf test: 583M mcore-mistral checkpoint, TP=1 PP=1 DP=4 (4 GPUs).COMMENT
LOW…rid/hybrid_nanov3_3b_perf_gb200_4gpu/model_config.yaml1# Inference perf test: nanov3 3B hybrid MoE checkpoint, TP=1 PP=1 EP=4 (4 GPUs).COMMENT
LOW…ts/performance_tests/shell_test_utils/run_perf_test.sh1#!/usr/bin/env bashCOMMENT
LOW…ts/performance_tests/shell_test_utils/run_perf_test.sh121# Override TP/PP/EP from config (the args file ships defaults; config wins).COMMENT
LOWtests/unit_tests/test_imports.py1# Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW…s/checkpoint/test_gpt_hybrid_conversion_parallelism.py41# tiny synthetic DCP checkpoint and round-trips it through the converter onCOMMENT
LOWtests/unit_tests/determinism/configs.py121 # mamba-attn-mlp covers Mamba + attention + MLP paths. pure-mamba isCOMMENT
LOWtests/unit_tests/determinism/configs.py181 "TP": "tensor_model_parallel_size",COMMENT
LOW…unit_tests/transformer/test_fsdp_dtensor_checkpoint.py1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW…s/unit_tests/elastification/test_hybrid_flex_router.py81 tensor_model_parallel_size=1, pipeline_model_parallel_size=1COMMENT
LOW…s/unit_tests/pipeline_parallel/test_pipeline_layout.py361COMMENT
LOW…s/unit_tests/dist_checkpointing/test_fully_parallel.py521 # main replica is in DP group with ranks [0, 2, 4, 6],COMMENT
LOWtests/unit_tests/inference/test_hybrid_moe.py61# Combinatorial sweep: unordered combinations with repetition of ALL_STATESCOMMENT
LOWtests/test_utils/python_scripts/recipe_parser.py21# cadence remains the trigger axis.COMMENT
LOWtests/test_utils/python_scripts/test_oncall_manager.py1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOWtests/test_utils/recipes/h100/moe.yaml101 - environment: [lts]COMMENT
LOWtests/test_utils/recipes/h100/moe.yaml221 # Super important mr, mr-github tests that run for both DEV and LTS per mr, mr-github #COMMENT
LOWtests/test_utils/recipes/h100/gpt.yaml461 # - test_case: [gpt3_mcore_te_tp2_pp1_fsdp2_resume_torch_dist]COMMENT
LOWtests/test_utils/recipes/h100/gpt.yaml521 platforms: [dgx_h100]COMMENT
LOWtests/test_utils/recipes/gb200/moe.yaml101 platforms: [dgx_gb200]COMMENT
LOWtests/test_utils/recipes/gb200/moe.yaml201 # - test_case: [gpt3_mcore_te_tp2_pp1_frozen_resume_torch_dist_te_8experts2parallel_dist_optimizer]COMMENT
LOW…unctional_tests/test_cases/common/moe_perf/__main__.py401COMMENT
LOW…unctional_tests/test_cases/common/moe_perf/__main__.py421 # test_moe_layer_performance(case, debug_mode=True)COMMENT
LOW…unctional_tests/shell_test_utils/run_batch_ci_tests.sh1#!/bin/bashCOMMENT
LOWdocs/autodoc2_docstrings_parser.py1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOWdocs/conf.py1# Copyright (c) 2025-2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOWexamples/megatron_fsdp/sbatch_mfsdp_deepseek_v3.sh201# Note: Update SBATCH parameters below according to your cluster configurationCOMMENT
LOWexamples/megatron_fsdp/train_llama3_8b_fsdp_h100_fp8.sh121 --ckpt-format fsdp_dtensorCOMMENT
LOWexamples/megatron_fsdp/sbatch_checkpoint_convert.sh21SLURM_LOGS="${OUTPUT_PATH}/slurm_logs"COMMENT
LOWexamples/mimo/train.py121 # iterator exhausted on all ranksCOMMENT
LOWexamples/inference/run_inference_server.sh1#!/bin/bashCOMMENT
LOWexamples/inference/run_offline_inference.sh1#!/bin/bashCOMMENT
LOW.github/workflows/sync-team-usergroups.yml1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/cicd-approve-test-queue.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/release-docs.yml1# Copyright (c) 2025, NVIDIA CORPORATION.COMMENT
LOW.github/workflows/community-bot.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/nightly-sync-main-to-dev.yml1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/release.yaml1# Copyright (c) 2019-2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/_claude-fix-attempt.yml1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/_claude-fix-attempt.yml21#COMMENT
LOW.github/workflows/_claude-fix-attempt.yml41# branch. If an earlier attempt already created that commit, fixed code verifiesCOMMENT
LOW.github/workflows/community-request-assignee.yml1# Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.COMMENT
LOW.github/workflows/claude-fix.yml1# Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/claude-fix.yml21#COMMENT
LOW.github/workflows/claude-fix.yml41# publishes it from a fresh runner, ensures CI exists for the exact SHA, andCOMMENT
LOW.github/workflows/close-inactive-issue-pr.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/oncall-assign.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/oncall-rotation.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/cherry-pick-release-commit.yml1# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.COMMENT
LOW.github/workflows/release-nightly-docs.yml1# Copyright (c) 2026, NVIDIA CORPORATION.COMMENT
82 more matches not shown…
AI Structural Patterns145 hits · 126 pts
SeverityFileLineSnippetContext
LOWtools/bert_embedding/huggingface.py51CODE
LOWtests/unit_tests/test_layer_wise_optimizer.py149CODE
LOW…s/checkpoint/test_gpt_hybrid_conversion_parallelism.py69CODE
LOWtests/unit_tests/distributed/mfsdp_v1/utils.py39CODE
LOW…/unit_tests/transformer/test_multi_token_prediction.py227CODE
LOW…er/experimental_attention_variant/test_absorbed_mla.py83CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py1092CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py1239CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py1594CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py2582CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py2672CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py2759CODE
LOW…perimental_attention_variant/test_dsa_native_parity.py2910CODE
LOW…ts/unit_tests/transformer/moe/test_token_dispatcher.py37CODE
LOWtests/unit_tests/transformer/moe/test_paged_stashing.py59CODE
LOW…s/elastification/test_inject_flextron_forward_logic.py33CODE
LOW…it_tests/pipeline_parallel/test_bridge_communicator.py163CODE
LOWtests/unit_tests/dist_checkpointing/utils.py294CODE
LOWtests/unit_tests/resharding/test_planner.py26CODE
LOWtests/unit_tests/inference/test_wandb_logging.py46CODE
LOWtests/unit_tests/inference/coordinator_test_utils.py16CODE
LOW…ests/inference/contexts/test_dynamic_prefix_caching.py50CODE
LOW…eration_controllers/test_text_generation_controller.py58CODE
LOW…eration_controllers/test_text_generation_controller.py1971CODE
LOW…sts/unit_tests/inference/engines/test_static_engine.py38CODE
LOW…sts/unit_tests/inference/engines/test_static_engine.py322CODE
LOWtests/unit_tests/data/test_bin_reader.py97CODE
LOWtests/test_utils/python_scripts/recipe_parser.py363CODE
LOW…/academic_paper_scripts/detxoify_lm/perspective_api.py88CODE
LOWexamples/post_training/modelopt/utils.py54CODE
LOWexamples/post_training/modelopt/utils.py132CODE
LOWexamples/mimo/training/builder.py114CODE
LOWexamples/mimo/data/mock.py165CODE
LOWexamples/mimo/data/mock.py41CODE
LOWexamples/multimodal/model.py18CODE
LOW…ples/multimodal/model_converter/vision_model_tester.py73CODE
LOWmegatron/core/parallel_state.py547CODE
LOWmegatron/core/ssm/mamba_mixer.py161CODE
LOWmegatron/core/ssm/mamba_mixer.py771CODE
LOWmegatron/core/ssm/mamba_context_parallel.py329CODE
LOWmegatron/core/ssm/mamba_context_parallel.py356CODE
LOWmegatron/core/ssm/mamba_context_parallel.py199CODE
LOWmegatron/core/ssm/gated_delta_net.py77CODE
LOWmegatron/core/ssm/ops/ssd_combined.py23CODE
LOWmegatron/core/ssm/ops/ssd_combined.py163CODE
LOW…ore/tokenizers/text/libraries/huggingface_tokenizer.py26CODE
LOW…n/core/tokenizers/text/libraries/tiktoken_tokenizer.py270CODE
LOWmegatron/core/fusions/fused_softmax.py176CODE
LOWmegatron/core/tensor_parallel/layers.py827CODE
LOWmegatron/core/tensor_parallel/mappings.py57CODE
LOWmegatron/core/tensor_parallel/mappings.py81CODE
LOWmegatron/core/tensor_parallel/mappings.py100CODE
LOWmegatron/core/tensor_parallel/mappings.py115CODE
LOWmegatron/core/tensor_parallel/mappings.py592CODE
LOWmegatron/core/tensor_parallel/mappings.py621CODE
LOWmegatron/core/post_training/modelopt/layers.py108CODE
LOWmegatron/core/optimizer/emerging_optimizers.py163CODE
LOWmegatron/core/optimizer/emerging_optimizers.py325CODE
LOWmegatron/core/optimizer/__init__.py451CODE
LOWmegatron/core/optimizer/distrib_optimizer.py238CODE
85 more matches not shown…
Verbosity Indicators49 hits · 90 pts
SeverityFileLineSnippetContext
LOWtools/checkpoint/saver_hf_llava.py17 # Step 1: Reshape back to (num_head, 3*head_dim, -1)COMMENT
LOWtools/checkpoint/saver_hf_llava.py20 # Step 2: Slice along the head_dim dimension to get q, k, vCOMMENT
LOWtools/checkpoint/saver_hf_llava.py25 # Step 3: Reshape each back to (num_head * head_dim, -1)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py361 # Step 1: Get vocabulary filesCOMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py387 # Step 2: Get raw text dataCOMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py411 # Step 3: Preprocess for GPT (GPT2BPETokenizer)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py434 # Step 4: Preprocess for BERT (BertWordPieceLowerCase + split-sentences)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py475 # Step 5: Preprocess for T5 (BertWordPieceCase)COMMENT
LOW…s/common_pile_dataset/create_common_pile_ci_dataset.py497 # Step 6: Clean up and verifyCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2038 # Step 1: Forward pass for all 3 requestsCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2047 # Step 2: Forward pass where req 10 finishes, req 11 continues. Req 999 is NOT scheduled.COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2136 # Step 1: All 3 requests are active, process forward passCOMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2145 # Step 2: Both decode requests finish, chunked prefill NOT scheduled this step.COMMENT
LOW…/unit_tests/inference/contexts/test_dynamic_context.py2069 # Step 3: Add the next chunk. It should sit exactly at the boundary (index 1) and inherit the state.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py3657 # Step 1: prefill for request 0 — should NOT count as a spec step.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py3667 # Step 2: decode for request 0 — should count spec tokens.COMMENT
LOW…ts/unit_tests/inference/engines/test_dynamic_engine.py2753 # Step 1: Prefill. Processes the 4 prompt tokens.COMMENT
LOW…l_tests/python_test_utils/compute_golden_statistics.py11 # Step 1: Run batch tests (from megatron-rl directory):STRING
LOW…l_tests/python_test_utils/compute_golden_statistics.py15 # Step 2: Wait for jobs to complete, then compute statistics:STRING
LOW.github/scripts/oncall_manager.py135 # Step 1: Add new oncall first (include current members to avoid removing anyone yet)COMMENT
LOW.github/scripts/oncall_manager.py142 # Step 2: Now set the usergroup to contain only the new oncallCOMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1871 # Step 3: Split parameter groups to meet communication segmentation requirements.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1916 # Step 4: Generate the groups of collective buckets, where each group aggregatesCOMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1751 # Step 0: Register new FSDP unit modules.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1784 # Step 1: Group the parameters according to their execution order and attributes.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1822 # Step 2: Bucket the parameters based on the guide bucket size.COMMENT
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py4468 # If use double buffer, we need to check if the next bucketCOMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py265 # Step 1: Find maximum value in the row for numerical stabilityCOMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py277 # Step 2: Compute sum of exp(x - max_val)COMMENT
LOW…e/transformer/custom_layers/batch_invariant_kernels.py293 # Step 3: Compute final log_softmax values: x - max_val - log_sum_expCOMMENT
LOWmegatron/core/pipeline_parallel/bridge_communicator.py437 # Step 1: broadcast its shape so receivers can allocateCOMMENT
LOWmegatron/core/pipeline_parallel/bridge_communicator.py443 # Step 2: broadcast the actual tensorCOMMENT
LOW…n/core/dist_checkpointing/strategies/fully_parallel.py238 # Step 3: load part of the checkpoint.COMMENT
LOW…n/core/dist_checkpointing/strategies/fully_parallel.py264 # Step 4: exchange data between ranksCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py232 # Step 1: Segment tasks (break large tasks into chunks)COMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py242 # Step 2: Pack tasks into workload groupsCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py249 # Step 3: Schedule workloads to iterationsCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py258 # Step 4: Prepare iteration schedulesCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py264 # Step 5: Build GPU execution plansCOMMENT
LOW…gatron/core/resharding/nvshmem_copy_service/service.py273 # Step 6: Create double-buffered eventsCOMMENT
LOW…harding/nvshmem_copy_service/core/pipeline_executor.py153 # Step 1: Pack NEXT iteration (async)COMMENT
LOW…harding/nvshmem_copy_service/core/pipeline_executor.py165 # Step 2: Unpack PRIOR iteration (async)COMMENT
LOW…harding/nvshmem_copy_service/core/pipeline_executor.py180 # Step 3: Send CURRENT iterationCOMMENT
LOW…harding/nvshmem_copy_service/core/pipeline_executor.py223 # Step 5: Wait for async pack to complete (double-buffer safety)COMMENT
LOW…shmem_copy_service/planning/communication_scheduler.py39 # Step 1: Collect all batches across all PE pairsCOMMENT
LOW…shmem_copy_service/planning/communication_scheduler.py44 # Step 2: Assign batches to iterations using greedy conflict-free algorithmCOMMENT
LOW…shmem_copy_service/planning/communication_scheduler.py49 # Step 3: Exchange detailed workload summaries (Task IDs/Sizes)COMMENT
LOW…shmem_copy_service/planning/communication_scheduler.py55 # Step 4: Build schedule map for this PECOMMENT
LOW…e/communication/torch_symm_triton/fused_collectives.py151 # Step 1: - reduce-scatter + residual add for this token + collect sq sumCOMMENT
AI Slop Vocabulary35 hits · 80 pts
SeverityFileLineSnippetContext
LOWtasks/finetune_utils.py120 # shuffling so we can just use a simple infinite loop.COMMENT
MEDIUMtests/unit_tests/test_inference.py107 # we are replicating what lm-eval-harness::TemplateLM::_encode_pair doesCOMMENT
MEDIUM…s/checkpoint/test_gpt_hybrid_conversion_parallelism.py43# harness launched pytest. When that default PG is multi-rank (e.g. Megatron'sCOMMENT
MEDIUM…ests/determinism/correctness/test_transformer_layer.py210 # stress harness relies on the connection count the launcherCOMMENT
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py236 # DP: Only relevant when using HSDP, where we need the flattened DP group for data parallelism. (Otherwise, just pasCOMMENT
LOW…t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py238 # DP-Shard-CP: Only required if using CP. Otherwise, just pass dp_shard to FSDP.COMMENT
MEDIUMtests/unit_tests/transformer/test_utils.py331 """Test a comprehensive scenario with multiple configurations."""STRING
MEDIUMtests/unit_tests/training/models/test_base.py171 """from_dict() reconstructs configs from serialized dicts, handles nested dataclasses, and is robust to unknown keysSTRING
LOW…s/unit_tests/dist_checkpointing/models/test_mlp_glu.py129 # Load happens in-place, so we can just use the same tensorsCOMMENT
MEDIUM…/unit_tests/inference/contexts/test_dynamic_context.py2534 # 6. Verify seamless append (no legacy offset math needed)COMMENT
MEDIUM…cipes/h100/gpt-dynamic-inference-with-coordinator.yaml92 # skills/run-performance-tests/SKILL.md for the harness it runs under.COMMENT
MEDIUM…inference_server_smoke_tp1_pp1_dp8_583m/serve_smoke.py43 # the JET harness expects at ``logs/*/*/attempt_0/*/std*.log``) while stillCOMMENT
LOW…sts/functional_tests/shell_test_utils/_run_training.sh85 # If value is "true", just use the keyCOMMENT
MEDIUM.github/workflows/claude_review.yml105 # Strict review: comprehensive Megatron-LM focused analysisCOMMENT
LOWmegatron/core/optimizer_param_scheduler.py235 # If the learning rate is constant, just return the initial value.COMMENT
LOWmegatron/core/timers.py476 # polutes the runs list, so we just add each as a scalarCOMMENT
MEDIUMmegatron/core/ssm/triton_cache_manager.py69 # use temp dir to be robust against program interruptionsCOMMENT
LOW…ore/tokenizers/text/parsers/qwen3_coder_tool_parser.py16# These map to vLLM types but we just use dictionaries for nowCOMMENT
LOWmegatron/core/tensor_parallel/random.py108 # if not using cuda graphs, just use the builtin pytorch functionCOMMENT
LOWmegatron/core/tensor_parallel/random.py189 # already graphable, just return it.COMMENT
LOWmegatron/core/tensor_parallel/random.py195 # already non-graphable, just return it.COMMENT
MEDIUM…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1926 # Set aggregate buckets by FSDP units, i.e. buckets pertaining to the sameCOMMENT
MEDIUM…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py2557 # to leverage NCCL UBR for high-precision gradient reduction withCOMMENT
MEDIUM…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py3516 # for a seamless user experience and coverage for ZeRO-1 and ZeRO-2?COMMENT
MEDIUM…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py4737 # TODO(@cspades): Clean up this logic in conjunction withCOMMENT
LOW…atron/core/distributed/fsdp/src/megatron_fsdp/utils.py199 # if not using cuda graphs, just use the builtin pytorch functionCOMMENT
LOWmegatron/core/transformer/transformer_config.py2438 # so just set both if either is specified.COMMENT
LOWmegatron/core/transformer/transformer_layer.py906 # elements in bias_chunks are the same for all chunks, so we can just use the first oneCOMMENT
MEDIUMmegatron/core/transformer/mlp.py207 # Weight resharding across TP sizes will have aforementioned problems.COMMENT
MEDIUMmegatron/core/datasets/indexed_dataset.py6# Essentially re-written in entiretyCOMMENT
MEDIUM…e/models/common/embeddings/language_model_embedding.py145 # the original tensor from being garbage collected. Clone to facilitate GC.COMMENT
LOWmegatron/core/models/bert/bert_model.py193 # For local layer spec we just use b1ssCOMMENT
LOWmegatron/core/extensions/transformer_engine.py689 # TODO should we ditch normalization config and just use spec to choose LayerNorm vs RMSNorm?COMMENT
LOW…tron/core/dist_checkpointing/strategies/async_utils.py479 # to simply call `sync_all_async_calls` to check if other ranks complete the writingCOMMENT
MEDIUM…/core/inference/data_parallel_inference_coordinator.py386 # Todo [Siddharth]: Make this more robust to handle invalid messages.COMMENT
Cross-Language Confusion14 hits · 55 pts
SeverityFileLineSnippetContext
HIGH…core/transformer/heterogeneous/heterogeneous_config.py177 "n_heads_in_group": null,STRING
HIGH…er/experimental_attention_variant/dsa_cudnn_kernels.py1695 """KL(target || predict) over selected top-K entries.STRING
HIGH…er/experimental_attention_variant/dsa_cudnn_kernels.py1747 """KL(target || predict) over the **full** KV axis, averaged over ``(B, S_q)``.STRING
HIGH…mer/experimental_attention_variant/dsa_indexer_loss.py25 """Return elementwise ``KL(target || predict)`` contributions."""STRING
HIGH…mer/experimental_attention_variant/dsa_indexer_loss.py35 """Return ``KL(target || predict)`` reduced over the key dimension."""STRING
HIGH…mer/experimental_attention_variant/dsa_indexer_loss.py43 """Return ``KL(target || predict)`` reduced over every dimension."""STRING
HIGH…e/pipeline_parallel/fine_grained_activation_offload.py547 self.push(chunk)CODE
HIGH…e/pipeline_parallel/fine_grained_activation_offload.py742 self.push(cur_chunk)CODE
HIGHmegatron/core/inference/inference_request.py94 Each block hash is computed as SHA-256(parent_digest || block_bytes), whereSTRING
HIGHmegatron/core/inference/unified_memory.py113 if (device != prev_device && device >= 0) cudaSetDevice(device);CODE
HIGHmegatron/core/inference/unified_memory.py149 if (device != prev_device && prev_device >= 0) cudaSetDevice(prev_device);CODE
HIGHmegatron/core/inference/contexts/dynamic_context.py4052 # : [ XX | XX | 16 XX | 12 72 24 88 XX ] (XX = undefined)COMMENT
HIGHmegatron/core/inference/moe/vllm_fused_moe.py518 `input` are undefined).STRING
HIGHmegatron/training/arguments.py3287 "n_heads_in_group": null,STRING
Fake / Example Data17 hits · 19 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/transformer/test_full_cuda_graph.py35 dummy_data = torch.ones(1, 4)CODE
LOWtests/unit_tests/transformer/test_full_cuda_graph.py40 return model(dummy_data), loss_funcCODE
LOWtests/unit_tests/pipeline_parallel/test_schedules.py528 dummy_data = torch.ones(1, 4)CODE
LOWtests/unit_tests/pipeline_parallel/test_schedules.py533 return model(dummy_data), loss_funcCODE
LOW…sts/unit_tests/dist_checkpointing/test_safe_globals.py28 dummy_obj = Namespace(dummy_value=0)CODE
LOWexamples/multimodal/text_generation_mistral_clip.sh7INPUT_IMAGE_PATH="placeholder"CODE
LOWexamples/multimodal/text_generation_mistral_clip.sh8GROUNDTRUTH_PATH="placeholder"CODE
LOW…imodal/nvlm/run_text_generation_yi_34b_internvit_6b.sh7INPUT_IMAGE_PATH="placeholder"CODE
LOW…imodal/nvlm/run_text_generation_yi_34b_internvit_6b.sh8GROUNDTRUTH_PATH="placeholder"CODE
LOW…ultimodal/nvlm/run_text_generation_qwen25_7b_siglip.sh8INPUT_IMAGE_PATH="placeholder"CODE
LOW…ultimodal/nvlm/run_text_generation_qwen25_7b_siglip.sh9GROUNDTRUTH_PATH="placeholder"CODE
LOW…al/nvlm/run_text_generation_qwen20_72b_internvit_6b.sh8INPUT_IMAGE_PATH="placeholder"CODE
LOW…al/nvlm/run_text_generation_qwen20_72b_internvit_6b.sh9GROUNDTRUTH_PATH="placeholder"CODE
LOW…/nvlm/run_text_generation_qwen25_7b_internvit_video.sh8INPUT_IMAGE_PATH="placeholder"CODE
LOW…/nvlm/run_text_generation_qwen25_7b_internvit_video.sh9GROUNDTRUTH_PATH="placeholder"CODE
LOW…al/llama_3p1_nemotron_nano_vl_8b_v1/text_generation.sh7INPUT_IMAGE_PATH="placeholder"CODE
LOW…al/llama_3p1_nemotron_nano_vl_8b_v1/text_generation.sh8GROUNDTRUTH_PATH="placeholder"CODE
Modern AI Meta-Vocabulary4 hits · 12 pts
SeverityFileLineSnippetContext
MEDIUMtests/functional_tests/shell_test_utils/run_ci_test.sh360 # orchestration's resumable retrigger.COMMENT
MEDIUM…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py1837 # We may share the embedding model weight and the final output layer,COMMENT
MEDIUMmegatron/core/inference/apis/async_llm.py214 # SIGINT or out-of-band orchestration).COMMENT
MEDIUM…t_generation_controllers/text_generation_controller.py2327 # Pick the context window that we need to pass through the network.COMMENT
Dead Code5 hits · 10 pts
SeverityFileLineSnippetContext
MEDIUMtools/checkpoint/loader_llava.py52CODE
MEDIUM…nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py183CODE
MEDIUM…nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py184CODE
MEDIUM…nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py185CODE
MEDIUM…nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py186CODE
Synthetic Comment Markers1 hit · 5 pts
SeverityFileLineSnippetContext
HIGH.github/workflows/claude-copy-to-main.yml118 --body "🤖 **This PR was auto-generated by Claude** via the Claude copy workflow.\n\nCherry-picked from CODE
Example Usage Blocks3 hits · 4 pts
SeverityFileLineSnippetContext
LOWtools/bisect.sh5# Usage:COMMENT
LOWtools/common_pile_dataset/setup_common_pile_dataset.sh9# Usage:COMMENT
LOW…unctional_tests/shell_test_utils/run_batch_ci_tests.sh5# Usage:COMMENT
TODO Padding3 hits · 3 pts
SeverityFileLineSnippetContext
LOW…/inference/test_data_parallel_inference_coordinator.py122 self.ep_consensus_interval = 1 # TODO: add tests for interval > 1 (e.g. verify pausing is delayed by at most N CODE
LOW…ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py5081 # TODO: Add validation checks for the legality of DTensor.COMMENT
LOWmegatron/core/transformer/multi_token_prediction.py1417 # TODO: implement block-based recompute for MTPCOMMENT
Slop Phrases2 hits · 2 pts
SeverityFileLineSnippetContext
MEDIUMmegatron/core/process_groups_config.py660 """Check if this rank has a language model.STRING
MEDIUMmegatron/core/process_groups_config.py663 True if this rank has a language model, False otherwise.STRING
Overly Generic Function Names3 hits · 2 pts
SeverityFileLineSnippetContext
LOW…s/unit_tests/training/config/test_instantiate_utils.py52def test_function(arg1=None, arg2=None, **kwargs):CODE
LOWtests/unit_tests/training/config/test_yaml_utils.py45def test_function():CODE
LOWmegatron/core/utils.py2713 def my_function():STRING