Ongoing research training transformer models at scale
This report summarizes scanner evidence for NVIDIA/Megatron-LM, a Python project with 18,111 GitHub stars. SynthScan examined 1,235,355 lines of code across 3396 source files, recording 13966 pattern matches distributed across 26 syntactic categories. The legacy adjusted pattern signal is 14.8 (Moderate pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 10775 finding(s).
Are imports declared and package/API names verified? 1183 finding(s).
Are wrappers, guards, and repeated structures necessary? 11214 finding(s).
Which changes need focused human review? 13966 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 13966 distinct pattern matches across 26 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pretrain_vlm.py | 248 | def train_valid_test_datasets_provider(train_val_test_num_samples): | CODE |
| LOW | pretrain_vlm.py | 289 | def _preprocess_data_for_llava(data): | CODE |
| LOW | pretrain_vlm.py | 487 | def llava_position_embedding_ranks(pp_ranks): | CODE |
| LOW | train_rl.py | 339 | def train_valid_test_datasets_provider(train_val_test_num_samples): | CODE |
| LOW | pretrain_gpt.py | 208 | def _build_cached_logits_loss_func( | CODE |
| LOW | pretrain_gpt.py | 420 | def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig: | CODE |
| LOW | pretrain_gpt.py | 491 | def train_valid_test_datasets_provider(train_val_test_num_samples, vp_stage=None): | CODE |
| LOW | model_provider.py | 66 | def count_parameters_in_layer(model, layer_name): | CODE |
| LOW | pretrain_hybrid.py | 231 | def _build_cached_logits_loss_func( | CODE |
| LOW | pretrain_hybrid.py | 383 | def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig: | CODE |
| LOW | pretrain_hybrid.py | 430 | def train_valid_test_datasets_provider(train_val_test_num_samples, vp_stage=None): | CODE |
| LOW | gpt_builders.py | 114 | def _get_transformer_layer_spec(use_te, config): | CODE |
| LOW | tasks/finetune_utils.py | 54 | def _cross_entropy_forward_step(batch, model): | CODE |
| LOW | tasks/finetune_utils.py | 96 | def _build_infinite_size_dataloader(dataloader): | CODE |
| LOW | tasks/finetune_utils.py | 107 | def _build_train_valid_dataloaders(train_dataset, valid_dataset, | CODE |
| LOW | tasks/data_utils.py | 35 | def build_tokens_types_paddings_from_text(text_a, text_b, | CODE |
| LOW | tasks/data_utils.py | 49 | def build_tokens_types_paddings_from_ids(text_a_ids, text_b_ids, max_seq_length, | CODE |
| LOW | tasks/eval_utils.py | 65 | def calculate_correct_answers(name, model, dataloader, | CODE |
| LOW | tasks/eval_utils.py | 111 | def correct_answers_forward_step(batch, model): | CODE |
| LOW | tools/run_vlm_text_generation.py | 163 | def generate_and_write_samples(model): | CODE |
| LOW | tools/prepare_cache.py | 54 | def _normalize_prepare_cache_args(args: Any) -> None: | CODE |
| LOW | tools/prepare_cache.py | 65 | def _validate_prepare_cache_args(args: Any) -> None: | CODE |
| LOW | tools/prepare_cache.py | 82 | def _disable_cache_load_only_flags(args: Any) -> Dict[str, bool]: | CODE |
| LOW | tools/prepare_cache.py | 102 | def _print_effective_configuration( | CODE |
| LOW | tools/prepare_cache.py | 120 | def core_gpt_dataset_config_from_args(args: Any) -> GPTDatasetConfig: | CODE |
| LOW | tools/check_process_group_usage.py | 150 | def _binding_seen_by_nested_scopes(events): | CODE |
| LOW | tools/check_golden_values.py | 80 | def compares_deterministically(model_config_text: str) -> bool: | CODE |
| LOW | tools/check_golden_values.py | 92 | def find_legacy_precision_metrics(golden_values: Any) -> list[str]: | CODE |
| LOW | tools/run_inference_performance_test.py | 48 | def add_inference_benchmarking_args(parser): | CODE |
| LOW | tools/check_kernel_determinism_coverage.py | 76 | def strip_comments_and_strings(text: str) -> str: | CODE |
| LOW | tools/build_sequences_per_dataset.py | 60 | def build_sequences_per_dataset(args): | CODE |
| LOW | tools/run_dynamic_text_generation_server.py | 69 | def add_text_generation_server_args(parser: argparse.ArgumentParser): | CODE |
| LOW | tools/run_dynamic_text_generation_server.py | 144 | def parse_args_and_detect_vlm( | CODE |
| LOW | tools/run_dynamic_text_generation_server.py | 194 | def _build_engine_for_vlm_or_gpt(is_vlm: bool) -> DynamicInferenceEngine: | CODE |
| LOW | tools/run_dynamic_text_generation_server.py | 304 | async def run_text_generation_server( | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1190 | def _metadata_same_layout_is_extra_state_key(fqn: str) -> bool: | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1200 | def _metadata_same_layout_is_common_state_key(fqn: str) -> bool: | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1206 | def _metadata_same_layout_validate_common_state_storage( | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1237 | def _metadata_same_layout_path(fqn: str) -> tuple[str | int, ...]: | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1241 | def _metadata_same_layout_tensor_dtype( | CODE |
| LOW⚡ | tools/checkpoint/weighted_merge.py | 1250 | def _metadata_same_layout_tensor_layout( | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 411 | def _schedule_minus_sqrt_decay(x: float) -> float: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 444 | def _schedule_to_merge_coefficients( | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 533 | def _warn_manual_weight_policy(weights: Iterable[float], *, normalize: bool) -> None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 555 | def _discover_checkpoint_iterations(checkpoint_root: str | Path) -> list[int]: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 572 | def _filter_checkpoints_by_interval( | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 593 | def _select_checkpoints_in_window( | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 626 | def _checkpoint_paths_for_iterations( | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 633 | def _validate_min_checkpoints(num_checkpoints: int, min_checkpoints: int | None) -> None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 646 | def _read_latest_checkpointed_iteration(path: Path) -> int | str: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 703 | def _write_latest_checkpointed_iteration(checkpoint_dir: str | Path, iteration: int) -> None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 785 | def _directory_size_for_accounting(path: str | Path, byte_accounting: str) -> int: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 800 | def _distributed_memory_peaks(host_peak_bytes: int) -> tuple[int, int, int, int]: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 847 | def _merge_state_dict_containers(target: Any, source: Any) -> Any: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 921 | def _direct_dcp_save_uses_no_dist() -> bool: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 942 | def _checkpoint_iteration_from_dir(checkpoint_dir: Path) -> int | None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 1000 | def _prepare_temporary_output_dir(output_dir: Path) -> Path: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 1018 | def _require_publishable_checkpoint_dir(checkpoint_dir: Path) -> None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 1047 | def _best_effort_fsync_checkpoint_metadata(checkpoint_dir: Path) -> None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 1055 | def _publish_temporary_output_dir(temporary_dir: Path, output_dir: Path) -> None: | CODE |
| 8946 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tools/checkpoint/gpt_hybrid_conversion.py | 400 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tools/checkpoint/gpt_hybrid_conversion.py | 402 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 132 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 134 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 225 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 227 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 486 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 488 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 530 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 532 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 627 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 629 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 698 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 700 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 722 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tools/checkpoint/gpt_hybrid_conversion.py | 724 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tools/checkpoint/saver_llava.py | 225 | #----------- | COMMENT |
| MEDIUM | tools/checkpoint/saver_llava.py | 186 | #----------- | COMMENT |
| MEDIUM | tools/checkpoint/saver_base.py | 378 | #----------- | COMMENT |
| MEDIUM | tools/checkpoint/saver_base.py | 421 | # -------------- | COMMENT |
| MEDIUM | tools/checkpoint/saver_base.py | 433 | # ------------------ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 360 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 362 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 386 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 388 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 410 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 412 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 433 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 435 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 474 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 476 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 496 | # ================================================================ | COMMENT |
| MEDIUM⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 498 | # ================================================================ | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 24 | # ── Parse KEY=VALUE positional args ─────────────────────────────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 77 | # ── Read model_config.yaml ──────────────────────────────────────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 119 | # ── Build model args (substituting ${CHECKPOINT_LOAD_PATH}) ─────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 143 | # ── Make image-bundled extras (mamba-ssm) visible to the cog venv ───────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 183 | # ── Launch the inference server in the background ───────────────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 263 | # ── Wait for server readiness ───────────────────────────────────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 283 | # ── Benchmark sweep ─────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 303 | # ── Baseline comparison or recording ────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/unit_tests/test_fp8_param.py | 693 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | tests/unit_tests/test_fp8_param.py | 695 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_hyper_comm_grid.py | 314 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_hyper_comm_grid.py | 316 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_hyper_comm_grid.py | 440 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_hyper_comm_grid.py | 442 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | tests/unit_tests/test_optimizer_param_scheduler.py | 302 | # ── get_canonical_lr_for_logging tests ────────────────────────────────────── | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 72 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 74 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1302 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1304 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1749 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1751 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1988 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 1990 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 2159 | # =========================================================================== | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_emerging_optimizers.py | 2161 | # =========================================================================== | COMMENT |
| MEDIUM | tests/unit_tests/test_emerging_optimizers.py | 498 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/unit_tests/test_emerging_optimizers.py | 500 | # --------------------------------------------------------------------------- | COMMENT |
| 1140 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | train_rl.py | 4 | CODE | |
| LOW | mamba_builders.py | 14 | CODE | |
| LOW | mamba_builders.py | 15 | CODE | |
| LOW | pretrain_gpt.py | 24 | CODE | |
| LOW | pretrain_gpt.py | 73 | CODE | |
| LOW | pretrain_hybrid.py | 24 | CODE | |
| LOW | pretrain_hybrid.py | 73 | CODE | |
| LOW | gpt_builders.py | 20 | CODE | |
| LOW | tools/merge_datasets.py | 3 | CODE | |
| LOW | tools/preprocess_data_nmt.py | 6 | CODE | |
| LOW | tools/run_inference_performance_test.py | 34 | CODE | |
| LOW | tools/run_inference_performance_test.py | 38 | CODE | |
| LOW | tools/run_inference_performance_test.py | 40 | CODE | |
| LOW | tools/check_kernel_determinism_coverage.py | 22 | CODE | |
| LOW | tools/run_text_generation_server.py | 6 | CODE | |
| LOW | tools/linter.py | 11 | CODE | |
| LOW | tools/preprocess_mmdata.py | 12 | CODE | |
| LOW | tools/checkpoint/weighted_merge.py | 15 | CODE | |
| LOW | tools/checkpoint/saver_core.py | 3 | CODE | |
| LOW | tools/checkpoint/saver_core.py | 5 | CODE | |
| LOW | tools/checkpoint/saver_core.py | 7 | CODE | |
| LOW | tools/checkpoint/saver_core.py | 8 | CODE | |
| LOW | tools/checkpoint/remap_gpt_dsa_to_mamba.py | 38 | CODE | |
| LOW | tools/checkpoint/remap_gpt_dsa_to_mamba.py | 39 | CODE | |
| LOW | tools/checkpoint/remap_gpt_dsa_to_mamba.py | 108 | CODE | |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 122 | CODE | |
| LOW | tools/checkpoint/dist_checkpoint_io.py | 20 | CODE | |
| LOW | tools/checkpoint/dist_checkpoint_io.py | 31 | CODE | |
| LOW | tools/checkpoint/utils.py | 3 | CODE | |
| LOW | tools/checkpoint/hybrid_conversion.py | 10 | CODE | |
| LOW | tools/checkpoint/loader_mixtral_hf.py | 3 | CODE | |
| LOW | tools/checkpoint/loader_mixtral_hf.py | 168 | CODE | |
| LOW | tools/checkpoint/saver_llava.py | 2 | CODE | |
| LOW | tools/checkpoint/saver_llava.py | 4 | CODE | |
| LOW | tools/checkpoint/saver_llava.py | 173 | CODE | |
| LOW | tools/checkpoint/saver_llava.py | 173 | CODE | |
| LOW | tools/checkpoint/checkpoint_inspector.py | 38 | CODE | |
| LOW | tools/checkpoint/loader_core.py | 3 | CODE | |
| LOW | tools/checkpoint/loader_core.py | 4 | CODE | |
| LOW | tools/checkpoint/loader_core.py | 5 | CODE | |
| LOW | tools/checkpoint/loader_core.py | 6 | CODE | |
| LOW | tools/checkpoint/loader_core.py | 7 | CODE | |
| LOW | tools/checkpoint/saver_base.py | 2 | CODE | |
| LOW | tools/checkpoint/loader_llava.py | 3 | CODE | |
| LOW | tools/checkpoint/loader_llava.py | 7 | CODE | |
| LOW | tools/bert_embedding/embed.py | 3 | CODE | |
| LOW | tools/bert_embedding/embed.py | 8 | CODE | |
| LOW | tools/bert_embedding/embed.py | 15 | CODE | |
| LOW | tools/bert_embedding/embed.py | 16 | CODE | |
| LOW | tools/bert_embedding/embed.py | 19 | CODE | |
| LOW | tools/bert_embedding/embed.py | 21 | CODE | |
| LOW | tools/bert_embedding/embed.py | 21 | CODE | |
| LOW | tools/bert_embedding/__init__.py | 3 | CODE | |
| LOW | tools/bert_embedding/__init__.py | 3 | CODE | |
| LOW | …formance_tests/shell_test_utils/compare_to_baseline.py | 23 | CODE | |
| LOW | tests/unit_tests/test_fp8_param.py | 3 | CODE | |
| LOW | tests/unit_tests/test_fp8_param.py | 33 | CODE | |
| LOW | tests/unit_tests/test_fp8_param.py | 49 | CODE | |
| LOW | tests/unit_tests/test_inference.py | 3 | CODE | |
| LOW | tests/unit_tests/test_basic.py | 2 | CODE | |
| 1043 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | train_rl.py | 0 | forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt model | STRING |
| HIGH | pretrain_hybrid.py | 0 | forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt model | STRING |
| HIGH | megatron/elastification/pretrain_hybrid_flex.py | 0 | forward training step. args: data_iterator : input data iterator model (hybridmodel): the gpt model | STRING |
| HIGH | pretrain_gpt.py | 0 | loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): | STRING |
| HIGH | pretrain_hybrid.py | 0 | loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): | STRING |
| HIGH | megatron/elastification/pretrain_hybrid_flex.py | 0 | loss function. args: loss_mask (torch.tensor): used to mask out some portions of the loss output_tensor (torch.tensor): | STRING |
| HIGH | pretrain_gpt.py | 0 | build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samples | STRING |
| HIGH | pretrain_hybrid.py | 0 | build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samples | STRING |
| HIGH | examples/post_training/modelopt/finetune.py | 0 | build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samples | STRING |
| HIGH | examples/t5/pretrain_t5.py | 0 | build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samples | STRING |
| HIGH | megatron/elastification/pretrain_hybrid_flex.py | 0 | build the train test and validation datasets. args: train_val_test_num_samples : a list containing the number of samples | STRING |
| HIGH | tools/run_text_generation_server.py | 0 | utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when | STRING |
| HIGH | examples/inference/advanced/gpt_static_inference.py | 0 | utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when | STRING |
| HIGH | …amples/inference/advanced/simple_t5_batch_inference.py | 0 | utility to get the relevant backend for running inference this function will automatically chose the trtllmbackend when | STRING |
| HIGH | tools/checkpoint/saver_hf_llava.py | 0 | required top-level function that creates the saver and calls its .save(). | STRING |
| HIGH | tools/checkpoint/saver_core.py | 0 | required top-level function that creates the saver and calls its .save(). | STRING |
| HIGH | tools/checkpoint/saver_llava.py | 0 | required top-level function that creates the saver and calls its .save(). | STRING |
| HIGH | tools/checkpoint/loader_base.py | 0 | orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: | STRING |
| HIGH | tools/checkpoint/loader_core.py | 0 | orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: | STRING |
| HIGH | tools/checkpoint/loader_llava.py | 0 | orchestrates loading a megatron checkpoint and sending model parameters over a given multiprocessing queue. args: args: | STRING |
| HIGH | tools/checkpoint/loader_base.py | 0 | parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args. | STRING |
| HIGH | tools/checkpoint/saver_base.py | 0 | parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args. | STRING |
| HIGH | tools/checkpoint/loader_llava.py | 0 | parse megatron arguments by forcibly overwriting sys.argv. populates self.margs and self.checkpoint_args. | STRING |
| HIGH | tools/checkpoint/loader_base.py | 0 | construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv. | STRING |
| HIGH | tools/checkpoint/loader_core.py | 0 | construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv. | STRING |
| HIGH | tools/checkpoint/saver_base.py | 0 | construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv. | STRING |
| HIGH | tools/checkpoint/loader_llava.py | 0 | construct a sys.argv list for megatron's argument parser. this centralizes the hack of overwriting sys.argv. | STRING |
| HIGH | tests/unit_tests/test_hyper_comm_grid.py | 0 | set up distributed environment for the entire test class. | STRING |
| HIGH | …it_tests/pipeline_parallel/test_bridge_communicator.py | 0 | set up distributed environment for the entire test class. | STRING |
| HIGH | …sts/pipeline_parallel/test_multimodule_communicator.py | 0 | set up distributed environment for the entire test class. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_overlap.py | 0 | model with direct parameters and multiple child fsdpmodules. | STRING |
| HIGH | …ts/unit_tests/distributed/mfsdp_v2/test_fully_shard.py | 0 | model with direct parameters and multiple child fsdpmodules. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_context.py | 0 | model with direct parameters and multiple child fsdpmodules. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_memory.py | 0 | model with direct parameters and multiple child fsdpmodules. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_overlap.py | 0 | run through every child layer with a root-owned bias. | STRING |
| HIGH | …ts/unit_tests/distributed/mfsdp_v2/test_fully_shard.py | 0 | run through every child layer with a root-owned bias. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_context.py | 0 | run through every child layer with a root-owned bias. | STRING |
| HIGH | tests/unit_tests/distributed/mfsdp_v2/test_memory.py | 0 | run through every child layer with a root-owned bias. | STRING |
| HIGH | …sts/unit_tests/distributed/mfsdp_v2/test_cuda_graph.py | 0 | run through every child layer with a root-owned bias. | STRING |
| HIGH | …it_tests/pipeline_parallel/test_bridge_communicator.py | 0 | destroy all tracked grids and bridge communicator pgs. | STRING |
| HIGH | …_tests/pipeline_parallel/test_multimodule_schedules.py | 0 | destroy all tracked grids and bridge communicator pgs. | STRING |
| HIGH | tests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py | 0 | destroy all tracked grids and bridge communicator pgs. | STRING |
| HIGH | tests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py | 0 | return a transformerconfig for the vision projection mlp. | STRING |
| HIGH | examples/mimo/configs/llava_vlm.py | 0 | return a transformerconfig for the vision projection mlp. | STRING |
| HIGH | examples/mimo/configs/llava_avlm.py | 0 | return a transformerconfig for the vision projection mlp. | STRING |
| HIGH | …unit_tests/dist_checkpointing/models/test_gpt_model.py | 0 | test model loading with different vocab size (caused by tp padding). | STRING |
| HIGH | …nit_tests/dist_checkpointing/models/test_bert_model.py | 0 | test model loading with different vocab size (caused by tp padding). | STRING |
| HIGH | tests/unit_tests/dist_checkpointing/models/common.py | 0 | test model loading with different vocab size (caused by tp padding). | STRING |
| HIGH | …ultimodal/evaluation/evaluate_video_phys_game_bench.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_realworldqa.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_ai2d.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | …es/multimodal/evaluation/evaluate_video_motionbench.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_spdocvqa.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_textvqa.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_chartqa.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | …amples/multimodal/evaluation/evaluate_video_mvbench.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_mathvista.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_vqav2.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_ocrbench.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| HIGH | examples/multimodal/evaluation/evaluate_infovqa.py | 0 | merge input files to a format compatible with the evaluator. | STRING |
| 73 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pretrain_gpt.py | 301 | CODE | |
| LOW | pretrain_gpt.py | 491 | CODE | |
| LOW | gpt_builders.py | 25 | CODE | |
| LOW | tasks/finetune_utils.py | 148 | CODE | |
| LOW | tasks/eval_utils.py | 65 | CODE | |
| LOW | .gitlab/scripts/check_imports.py | 66 | CODE | |
| LOW | .gitlab/scripts/check_imports.py | 123 | CODE | |
| LOW | tools/preprocess_data.py | 310 | CODE | |
| LOW | tools/run_vlm_text_generation.py | 90 | CODE | |
| LOW | tools/check_process_group_usage.py | 265 | CODE | |
| LOW | tools/run_dynamic_text_generation_server.py | 304 | CODE | |
| LOW | tools/checkpoint/weighted_merge.py | 1271 | CODE | |
| LOW | tools/checkpoint/weighted_merge.py | 1532 | CODE | |
| LOW | tools/checkpoint/saver_hf_llava.py | 69 | CODE | |
| LOW | tools/checkpoint/loader_base.py | 258 | CODE | |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 631 | CODE | |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 726 | CODE | |
| LOW | tools/checkpoint/dist_checkpoint_io.py | 129 | CODE | |
| LOW | tools/checkpoint/hybrid_conversion.py | 59 | CODE | |
| LOW | tools/checkpoint/hybrid_conversion.py | 114 | CODE | |
| LOW | tools/checkpoint/hybrid_conversion.py | 170 | CODE | |
| LOW | tools/checkpoint/hybrid_conversion.py | 222 | CODE | |
| LOW | tools/checkpoint/saver_llava.py | 183 | CODE | |
| LOW | tools/checkpoint/checkpoint_inspector.py | 80 | CODE | |
| LOW | tools/checkpoint/checkpoint_inspector.py | 341 | CODE | |
| LOW | tools/checkpoint/checkpoint_inspector.py | 1311 | CODE | |
| LOW | tools/checkpoint/saver_base.py | 366 | CODE | |
| LOW | tools/checkpoint/loader_llava.py | 161 | CODE | |
| LOW | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 63 | CODE | |
| LOW | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 109 | CODE | |
| LOW | tools/moe_routing/analyze_routing_predictability.py | 54 | CODE | |
| LOW | tools/moe_routing/analyze_routing_concentration.py | 70 | CODE | |
| LOW | …formance_tests/shell_test_utils/compare_to_baseline.py | 37 | CODE | |
| LOW | …formance_tests/shell_test_utils/compare_to_baseline.py | 75 | CODE | |
| LOW | …shell_test_utils/determinism/print_nsys_leaderboard.py | 85 | CODE | |
| LOW | tests/unit_tests/test_fp8_param.py | 240 | CODE | |
| LOW | tests/unit_tests/test_optimizer.py | 956 | CODE | |
| LOW | tests/unit_tests/test_utils.py | 385 | CODE | |
| LOW | tests/unit_tests/test_imports.py | 63 | CODE | |
| LOW | tests/unit_tests/test_tensor_metrics.py | 1099 | CODE | |
| LOW | tests/unit_tests/test_layer_wise_optimizer.py | 321 | CODE | |
| LOW | tests/unit_tests/test_layer_wise_optimizer.py | 874 | CODE | |
| LOW | tests/unit_tests/test_sequence_packing.py | 177 | CODE | |
| LOW | tests/unit_tests/test_sequence_packing.py | 348 | CODE | |
| LOW | tests/unit_tests/testmon_cache.py | 257 | CODE | |
| LOW | tests/unit_tests/ssm/test_gdp_dynamic_inference.py | 1105 | CODE | |
| LOW | tests/unit_tests/ssm/test_wide_shortcut_parallel.py | 183 | CODE | |
| LOW | tests/unit_tests/ssm/test_wide_shortcut_parallel.py | 264 | CODE | |
| LOW | tests/unit_tests/ssm/test_wide_shortcut_parallel.py | 352 | CODE | |
| LOW | tests/unit_tests/ssm/ops/mamba2/test_ssd_bmm.py | 68 | CODE | |
| LOW | …/unit_tests/tokenizers/test_explicit_assistant_loss.py | 74 | CODE | |
| LOW | tests/unit_tests/tokenizers/test_tokenizer.py | 297 | CODE | |
| LOW | tests/unit_tests/fusions/test_gated_norm.py | 103 | CODE | |
| LOW | tests/unit_tests/fusions/test_gated_norm.py | 187 | CODE | |
| LOW | …t_tests/tools/checkpoint/test_gpt_hybrid_conversion.py | 461 | CODE | |
| LOW | tests/unit_tests/extension/test_kitchen_sdpa.py | 67 | CODE | |
| LOW | tests/unit_tests/determinism/utils.py | 119 | CODE | |
| LOW | …sts/unit_tests/determinism/kernels/test_moe_kernels.py | 336 | CODE | |
| LOW | tests/unit_tests/determinism/kernels/harness.py | 73 | CODE | |
| LOW | tests/unit_tests/determinism/kernels/harness.py | 131 | CODE | |
| 701 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | train_rl.py | 287 | except: | CODE |
| HIGH | tools/run_inference_performance_test.py | 109 | except NotImplementedError as e: | CODE |
| HIGH | tools/run_text_generation_server.py | 191 | except ValueError as ve: | CODE |
| HIGH | tools/checkpoint/weighted_merge.py | 763 | except FileNotFoundError: | CODE |
| HIGH | tools/checkpoint/weighted_merge.py | 750 | except FileNotFoundError: | CODE |
| HIGH | tools/checkpoint/dist_checkpoint_io.py | 213 | except Exception: | CODE |
| HIGH | tools/checkpoint/saver_base.py | 195 | except ImportError: | CODE |
| HIGH | tools/checkpoint/loader_llava.py | 298 | except: | CODE |
| HIGH | tools/checkpoint/loader_llava.py | 303 | except: | CODE |
| HIGH | …shell_test_utils/determinism/print_nsys_leaderboard.py | 40 | except ValueError: | CODE |
| HIGH | …ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py | 1148 | except Exception: | CODE |
| HIGH | tests/unit_tests/transformer/test_utils.py | 397 | except (TypeError, ValueError): | CODE |
| HIGH | tests/unit_tests/training/config/test_yaml_utils.py | 78 | except ValueError: | CODE |
| HIGH | tests/unit_tests/models/mimo/test_mimo_model.py | 163 | except Exception: | CODE |
| HIGH | tests/unit_tests/models/mimo/test_mimo_model.py | 179 | except Exception: | CODE |
| HIGH | tests/unit_tests/models/mimo/test_mimo_model.py | 933 | except Exception: | CODE |
| HIGH | tests/unit_tests/models/mimo/test_mimo_model.py | 947 | except Exception: | CODE |
| HIGH | …/inference/test_data_parallel_inference_coordinator.py | 352 | except (asyncio.TimeoutError, Exception): | CODE |
| HIGH | …/inference/test_data_parallel_inference_coordinator.py | 373 | except (asyncio.TimeoutError, asyncio.CancelledError, Exception): | CODE |
| HIGH | …s/unit_tests/inference/test_common_inference_params.py | 45 | except ValueError: | CODE |
| HIGH | tests/unit_tests/data/test_builder.py | 178 | except AssertionError: | CODE |
| HIGH | tests/test_utils/python_scripts/swap_pr_labels.py | 262 | except Exception: | CODE |
| HIGH | tests/test_utils/python_scripts/swap_pr_labels.py | 271 | except Exception: | CODE |
| HIGH | …l_tests/python_test_utils/compute_golden_statistics.py | 581 | except (ValueError, TypeError): | CODE |
| HIGH | examples/mimo/model_providers/llava_vlm.py | 78 | except (ModuleNotFoundError, AssertionError): | CODE |
| HIGH | examples/mimo/model_providers/llava_vlm.py | 145 | except (ModuleNotFoundError, AssertionError): | CODE |
| HIGH | examples/mimo/model_providers/llava_avlm.py | 77 | except (ModuleNotFoundError, AssertionError): | CODE |
| HIGH | examples/mimo/model_providers/llava_avlm.py | 173 | except (ModuleNotFoundError, AssertionError): | CODE |
| HIGH | scripts/check_api_backwards_compatibility.py | 218 | except Exception: | CODE |
| HIGH | .github/scripts/dco_gate.py | 145 | except GateError: | CODE |
| HIGH | megatron/core/package_info.py | 29 | except (_subprocess.CalledProcessError, OSError): | CODE |
| HIGH | megatron/core/parallel_state.py | 2594 | except Exception: # finalize must never block teardown | CODE |
| HIGH | megatron/core/fp4_utils.py | 24 | except (ImportError, ModuleNotFoundError): | CODE |
| HIGH | megatron/core/fp8_utils.py | 29 | except (ImportError, ModuleNotFoundError): | CODE |
| HIGH | megatron/core/msc_utils.py | 105 | except RuntimeError: | CODE |
| HIGH | …n/core/tokenizers/text/libraries/tiktoken_tokenizer.py | 12 | except ImportError: | CODE |
| HIGH | megatron/core/fusions/fused_mhc_kernels.py | 70 | except ImportError: | CODE |
| HIGH | megatron/core/fusions/fused_mhc_kernels.py | 83 | except ImportError: | CODE |
| HIGH | megatron/core/fusions/fused_mhc_kernels.py | 68 | except ImportError: | CODE |
| HIGH | megatron/core/distributed/__init__.py | 5 | except ImportError: | CODE |
| HIGH | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 219 | except RuntimeError: | CODE |
| HIGH | megatron/core/transformer/cuda_graphs.py | 242 | except ImportError: | CODE |
| HIGH | megatron/core/transformer/cuda_graphs.py | 2124 | except StopIteration: | CODE |
| HIGH | megatron/core/transformer/attention.py | 82 | except ImportError as e: | CODE |
| HIGH | …e/transformer/custom_layers/batch_invariant_kernels.py | 1760 | except ImportError: | CODE |
| HIGH | …e/transformer/custom_layers/batch_invariant_kernels.py | 1937 | except ImportError: | CODE |
| HIGH | …e/transformer/custom_layers/batch_invariant_kernels.py | 1952 | except (ImportError, AttributeError): | CODE |
| HIGH | …e/transformer/custom_layers/batch_invariant_kernels.py | 1777 | except ImportError: | CODE |
| HIGH | …e/transformer/custom_layers/batch_invariant_kernels.py | 1787 | except ImportError: | CODE |
| HIGH | megatron/core/datasets/object_storage_utils.py | 11 | except ModuleNotFoundError: | CODE |
| HIGH | megatron/core/datasets/megatron_dataset.py | 90 | except (AttributeError, NotImplementedError): | CODE |
| HIGH | megatron/core/datasets/megatron_dataset.py | 94 | except (AttributeError, NotImplementedError): | CODE |
| HIGH | megatron/core/datasets/indexed_dataset.py | 27 | except ModuleNotFoundError: | CODE |
| HIGH | megatron/core/extensions/transformer_engine.py | 3871 | except ImportError: | CODE |
| HIGH | …ron/core/resharding/copy_services/nixl_copy_service.py | 275 | except Exception: | CODE |
| HIGH | megatron/core/resharding/nvshmem_copy_service/compat.py | 27 | except ImportError: | CODE |
| HIGH | megatron/core/tuning/table.py | 57 | except metadata.PackageNotFoundError: | CODE |
| HIGH | megatron/core/inference/apis/_llm_base.py | 322 | except Exception: | CODE |
| HIGH | …re/inference/disaggregation/inference_state_handoff.py | 1097 | except Exception: | CODE |
| HIGH | …ore/inference/disaggregation/transfer_backends/nixl.py | 644 | except Exception: | CODE |
| 30 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL⚡ | tools/checkpoint/loader_mixtral_hf.py | 69 | model.embedding.word_embeddings.weight.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/loader_mixtral_hf.py | 74 | model.decoder.final_layernorm.weight.data.copy_(hf_model.model.norm.weight) | CODE |
| CRITICAL | tools/checkpoint/loader_mixtral_hf.py | 103 | layer.mlp.router.weight.data.copy_(hf_layer.block_sparse_moe.gate.weight) | CODE |
| CRITICAL | tools/checkpoint/loader_mixtral_hf.py | 127 | layer.self_attention.linear_qkv.layer_norm_weight.data.copy_(hf_layer.input_layernorm.weight) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 199 | model.vision_model.conv1.weight.data.copy_(vit_embeddings_msg["conv1 weight"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 201 | model.vision_model.conv1.bias.data.copy_(vit_embeddings_msg["conv1 bias"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 202 | model.vision_model.position_embeddings.weight.data.copy_(vit_embeddings_msg["position embeddings"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 208 | model.vision_model.embedder.weight.data.copy_(embedder_weight[tp_rank]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 210 | model.vision_model.embedder.bias.data.copy_(embedder_bias[tp_rank]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 214 | model.vision_model.ln_pre.weight.data.copy_(vit_embeddings_msg["ln pre weight"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 215 | model.vision_model.ln_pre.bias.data.copy_(vit_embeddings_msg["ln pre bias"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 218 | model.vision_model.ln_post.weight.data.copy_(vit_embeddings_msg["ln post weight"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 219 | model.vision_model.ln_post.bias.data.copy_(vit_embeddings_msg["ln post bias"]) | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 343 | model.vision_projection.encoder.linear_fc1.weight.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 345 | model.vision_projection.encoder.linear_fc2.weight.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 348 | model.vision_projection.encoder.linear_fc1.layer_norm_weight.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 351 | model.vision_projection.encoder.linear_fc1.layer_norm_bias.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 354 | model.vision_projection.encoder.linear_fc1.bias.data.copy_( | CODE |
| CRITICAL⚡ | tools/checkpoint/saver_llava.py | 356 | model.vision_projection.encoder.linear_fc2.bias.data.copy_(vision_projection_l1_bias) | CODE |
| CRITICAL⚡ | tests/unit_tests/transformer/moe/test_routers.py | 779 | assert self.sequential_mlp.router.weight.grad.abs().sum() == 0 | CODE |
| CRITICAL⚡ | tests/unit_tests/transformer/moe/test_routers.py | 785 | assert self.sequential_mlp.router.weight.grad.abs().sum() > 0 | CODE |
| CRITICAL⚡ | tests/unit_tests/transformer/moe/test_routers.py | 790 | self.sequential_mlp.router.weight.grad.fill_(0) | CODE |
| CRITICAL⚡ | tests/unit_tests/transformer/moe/test_routers.py | 793 | assert self.sequential_mlp.router.weight.grad.abs().sum() > 0 | CODE |
| CRITICAL⚡ | tests/unit_tests/models/test_gpt_model.py | 156 | assert self.gpt_model.embedding.word_embeddings.weight.std().cpu().item() == approx( | CODE |
| CRITICAL⚡ | tests/unit_tests/models/test_gpt_model.py | 159 | assert self.gpt_model.embedding.word_embeddings.weight.mean().cpu().item() == approx( | CODE |
| CRITICAL | …ests/inference/engines/request_lifecycle_test_utils.py | 253 | for module in env.engine.controller.inference_wrapped_model.model.modules(): | CODE |
| CRITICAL | …ests/inference/engines/request_lifecycle_test_utils.py | 274 | for module in env.engine.controller.inference_wrapped_model.model.modules(): | CODE |
| CRITICAL | megatron/core/parallel_state.py | 2678 | and torch.distributed.distributed_c10d._world.pg_map.get(_DATA_PARALLEL_GROUP_GLOO, None) | CODE |
| CRITICAL | megatron/core/parallel_state.py | 2687 | and torch.distributed.distributed_c10d._world.pg_map.get( | CODE |
| CRITICAL | megatron/core/parallel_state.py | 2741 | and torch.distributed.distributed_c10d._world.pg_map.get( | CODE |
| CRITICAL | megatron/core/parallel_state.py | 2755 | and torch.distributed.distributed_c10d._world.pg_map.get( | CODE |
| CRITICAL | megatron/core/transformer/attention.py | 2094 | self.linear_qkv.weight.main_param.data.copy_( | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | gpt_builders.py | 42 | # Define the decoder block spec | COMMENT |
| MEDIUM | gpt_builders.py | 54 | # Define the decoder layer spec | COMMENT |
| MEDIUM | gpt_builders.py | 67 | # Define the decoder block spec | COMMENT |
| MEDIUM | .gitlab/stages/05.publish.yml | 149 | # Define the full refspec for the branch | COMMENT |
| MEDIUM | tools/preprocess_data_fast.py | 54 | # Create an IndexedDatasetBuilder | COMMENT |
| MEDIUM | tools/run_inference_performance_test.py | 112 | # Create a list of valid token IDs | COMMENT |
| MEDIUM | tools/common_pile_dataset/setup_common_pile_dataset.sh | 87 | # Create a virtual environment to avoid system package conflicts | COMMENT |
| MEDIUM | tools/common_pile_dataset/setup_common_pile_dataset.sh | 122 | # Create the output directory | COMMENT |
| MEDIUM | tests/unit_tests/test_optimizer.py | 1172 | # Create a new state_dict with all params set to 3. | COMMENT |
| MEDIUM | tests/unit_tests/test_optimizer.py | 1384 | # Create a simple model for testing | COMMENT |
| MEDIUM | tests/unit_tests/test_optimizer.py | 1466 | # Create a simple model | COMMENT |
| MEDIUM⚡ | tests/unit_tests/test_utils.py | 528 | # Create a straggler_detector with enabled set to false. | COMMENT |
| MEDIUM | tests/unit_tests/test_hyper_comm_grid.py | 633 | # Create a process group | COMMENT |
| MEDIUM | tests/unit_tests/test_hyper_comm_grid.py | 636 | # Create a tensor for communication test | COMMENT |
| MEDIUM | tests/unit_tests/test_hyper_comm_grid.py | 737 | # Create a unique tensor based on rank | COMMENT |
| MEDIUM | tests/unit_tests/test_emerging_optimizers.py | 439 | # Create a simple linear model for testing | COMMENT |
| MEDIUM | tests/unit_tests/test_emerging_optimizers.py | 1101 | # Create a model with QKV-like parameter | COMMENT |
| MEDIUM | tests/unit_tests/test_training.py | 303 | # Create a mock state_dict with gradients (use deterministic values for reproducibility). | COMMENT |
| MEDIUM | tests/unit_tests/ssm/ops/mamba2/test_ssm_kernel.py | 84 | # Create the Mixer instance directly | COMMENT |
| MEDIUM | tests/unit_tests/fusions/test_torch_softmax.py | 197 | # Create a padding mask | COMMENT |
| MEDIUM | …/unit_tests/post_training/test_modelopt_module_spec.py | 278 | # Define the expected signature | COMMENT |
| MEDIUM | …/unit_tests/post_training/test_modelopt_module_spec.py | 317 | # Define the expected signature | COMMENT |
| MEDIUM | …tests/distributed/test_torch_fully_sharded_parallel.py | 78 | # Create a dummy model and configs. | COMMENT |
| MEDIUM | …tests/distributed/test_torch_fully_sharded_parallel.py | 85 | # Create the sharded model. | COMMENT |
| MEDIUM⚡ | …tests/distributed/test_torch_fully_sharded_parallel.py | 106 | # Create a dummy model and configs. | COMMENT |
| MEDIUM⚡ | …tests/distributed/test_torch_fully_sharded_parallel.py | 112 | # Create a custom process group (using the default world for testing) | COMMENT |
| MEDIUM⚡ | …tests/distributed/test_torch_fully_sharded_parallel.py | 115 | # Create the sharded model with explicit process group | COMMENT |
| MEDIUM | …ests/distributed/mfsdp_v1/test_mfsdp_uneven_dtensor.py | 600 | # Create a manual uneven sharding along dim 1 with possible zero-length local on some ranks | COMMENT |
| MEDIUM | …tests/transformer/test_transformer_block_custom_pgs.py | 266 | # Create a transformer block with default process groups | COMMENT |
| MEDIUM | …tests/transformer/test_transformer_block_custom_pgs.py | 292 | # Create a transformer block with custom process groups | COMMENT |
| MEDIUM | …tests/transformer/test_transformer_block_custom_pgs.py | 744 | # Create a single transformer block | COMMENT |
| MEDIUM | tests/unit_tests/transformer/test_cuda_graphs.py | 1048 | # Create the CUDA graphs - this is where the is_last_layer logic is tested | COMMENT |
| MEDIUM | tests/unit_tests/transformer/test_cuda_graphs.py | 1299 | # Create a mapping of sample_keys to indices | COMMENT |
| MEDIUM⚡ | tests/unit_tests/transformer/test_transformer_block.py | 701 | # Create a new build_layers method that uses interleaved attention | COMMENT |
| MEDIUM | …ts/unit_tests/transformer/moe/test_token_dispatcher.py | 298 | # Create the answer. | COMMENT |
| MEDIUM | tests/unit_tests/transformer/moe/test_shared_experts.py | 435 | # Create a dummy input tensor. | COMMENT |
| MEDIUM | tests/unit_tests/transformer/moe/test_aux_loss.py | 267 | # Create a new config with updated parameters | COMMENT |
| MEDIUM | tests/unit_tests/transformer/moe/test_aux_loss.py | 270 | # Create the router with the updated config | COMMENT |
| MEDIUM⚡ | …s/unit_tests/training/config/test_instantiate_utils.py | 334 | # Create a mock that raises an error when used with functools.partial | COMMENT |
| MEDIUM | tests/unit_tests/training/config/test_yaml_utils.py | 215 | # Create a mock torch dtype | COMMENT |
| MEDIUM | tests/unit_tests/training/config/test_yaml_utils.py | 262 | # Create a mock GenerationConfig | COMMENT |
| MEDIUM | tests/unit_tests/utils/test_experimental_log_once.py | 28 | # Define a fresh function with the decorator so it has its own closure state. | COMMENT |
| MEDIUM | tests/unit_tests/models/test_gpt_model.py | 318 | # Define the expected signature | COMMENT |
| MEDIUM | tests/unit_tests/models/test_fastconformer_model.py | 45 | # Create a parameter with the target dtype so ``next(self.parameters()).dtype`` | COMMENT |
| MEDIUM | …s/unit_tests/models/mimo/test_mimo_audio_submodules.py | 212 | # Create a time array | COMMENT |
| MEDIUM | …s/unit_tests/models/mimo/test_mimo_audio_submodules.py | 215 | # Create a simple sine wave at 440 Hz (A4) | COMMENT |
| MEDIUM | …nit_tests/models/mimo/test_mimo_embedding_alignment.py | 22 | # Create a minimal MimoModelConfig | COMMENT |
| MEDIUM | …nit_tests/models/mimo/test_mimo_embedding_alignment.py | 73 | # Create a simple batch | COMMENT |
| MEDIUM | …nit_tests/models/mimo/test_mimo_embedding_alignment.py | 343 | # Create a test case with 2 batches: | COMMENT |
| MEDIUM | …nit_tests/models/mimo/test_mimo_embedding_alignment.py | 379 | # Create the unflattened embeddings that would come from a vision encoder | COMMENT |
| MEDIUM | tests/unit_tests/models/mimo/test_mimo_submodules.py | 92 | # Create the main module spec | COMMENT |
| MEDIUM | tests/unit_tests/models/mimo/test_mimo_submodules.py | 351 | # Create a data batch without images | COMMENT |
| MEDIUM | …tests/dist_checkpointing/test_save_tokenizer_assets.py | 69 | # Create a fake tokenizer model file | COMMENT |
| MEDIUM | …tests/dist_checkpointing/test_save_tokenizer_assets.py | 263 | # Create a fake tokenizer model file | COMMENT |
| MEDIUM | …tests/dist_checkpointing/test_save_tokenizer_assets.py | 329 | # Create a fake tokenizer file | COMMENT |
| MEDIUM | tests/unit_tests/dist_checkpointing/test_fp8.py | 33 | # Create a quantizer for FP8 conversion | COMMENT |
| MEDIUM | …s/unit_tests/dist_checkpointing/test_fully_parallel.py | 597 | # Create a mock that will do what it's supposed to do, | COMMENT |
| MEDIUM | …/unit_tests/inference/contexts/test_dynamic_context.py | 602 | # Initialize all variables | COMMENT |
| MEDIUM | …/unit_tests/inference/contexts/test_dynamic_context.py | 1599 | # Create an active_requests_mask where requests 0, 2, and 4 are finished (0), | COMMENT |
| MEDIUM | …/unit_tests/inference/contexts/test_dynamic_context.py | 1689 | # Create an active_requests_mask where all requests are finished | COMMENT |
| 80 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tasks/finetune_utils.py | 62 | except Exception: | CODE |
| LOW | tasks/eval_utils.py | 114 | except Exception: | CODE |
| MEDIUM | tasks/eval_utils.py | 111 | def correct_answers_forward_step(batch, model): | CODE |
| LOW | .gitlab/scripts/check_imports.py | 117 | except Exception: | CODE |
| LOW⚡ | tools/check_copyright.py | 22 | except Exception as e: | CODE |
| MEDIUM⚡ | tools/check_copyright.py | 23 | print(f"Error reading {file_path}: {e}") | CODE |
| MEDIUM | tools/text_generation_cli.py | 21 | print(f"Error {response.status_code}: {response.json()['message']}") | CODE |
| MEDIUM | tools/checkpoint/weighted_merge.py | 927 | def _git_revision() -> str | None: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 745 | except Exception: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 760 | except Exception: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 914 | except Exception as exc: | CODE |
| LOW | tools/checkpoint/weighted_merge.py | 936 | except Exception: | CODE |
| LOW | tools/checkpoint/saver_hf_llava.py | 400 | except Exception as e: | CODE |
| LOW | tools/checkpoint/loader_base.py | 78 | except Exception as e: | CODE |
| MEDIUM | tools/checkpoint/loader_base.py | 79 | print(f"Error validating Megatron arguments: {e}") | CODE |
| LOW | tools/checkpoint/saver_core.py | 70 | except Exception as e: | CODE |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 384 | except Exception: | CODE |
| LOW | tools/checkpoint/dist_checkpoint_io.py | 213 | except Exception: | CODE |
| LOW | tools/checkpoint/hybrid_conversion.py | 322 | except Exception: | CODE |
| LOW | tools/checkpoint/loader_mixtral_hf.py | 344 | except Exception: | CODE |
| MEDIUM | tools/checkpoint/loader_mixtral_hf.py | 341 | def load_checkpoint(queue, args): | CODE |
| LOW | tools/checkpoint/saver_llava.py | 404 | except Exception as e: | CODE |
| LOW | tools/checkpoint/loader_core.py | 90 | except Exception as e: | CODE |
| LOW | tools/checkpoint/loader_llava.py | 360 | except Exception as e: | CODE |
| LOW | tools/bert_embedding/embed.py | 217 | except Exception: | CODE |
| LOW | tools/bert_embedding/embed.py | 223 | except Exception: | CODE |
| LOW | tests/unit_tests/conftest.py | 108 | except Exception: | CODE |
| LOW | tests/unit_tests/conftest.py | 156 | except Exception as e: | CODE |
| LOW | tests/unit_tests/test_hyper_comm_grid.py | 529 | except Exception as e: | CODE |
| LOW | tests/unit_tests/test_api_backwards_compat_setup.py | 106 | except Exception as e: | CODE |
| LOW | tests/unit_tests/test_api_backwards_compat_setup.py | 134 | except Exception as e: | STRING |
| LOW | tests/unit_tests/test_imports.py | 58 | except Exception: | CODE |
| LOW | tests/unit_tests/test_imports.py | 77 | except Exception: | CODE |
| LOW | tests/unit_tests/test_utilities.py | 304 | except Exception: | CODE |
| LOW | tests/unit_tests/ssm/test_gated_delta_net.py | 1492 | except Exception: | CODE |
| LOW⚡ | tests/unit_tests/tokenizers/test_tokenizer.py | 1107 | except Exception: | CODE |
| LOW | tests/unit_tests/tokenizers/test_tokenizer.py | 22 | except Exception: | CODE |
| LOW | tests/unit_tests/tokenizers/test_tokenizer.py | 159 | except Exception: | CODE |
| LOW | tests/unit_tests/tokenizers/test_tokenizer.py | 185 | except Exception: | CODE |
| LOW | tests/unit_tests/tokenizers/test_tokenizer.py | 326 | except Exception: | CODE |
| LOW | …istributed/mfsdp_v1/test_mfsdp_fused_mla_checkpoint.py | 353 | except Exception: | CODE |
| LOW | …ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py | 1148 | except Exception: | CODE |
| LOW | …t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py | 174 | except Exception as e: | CODE |
| LOW | …distributed/mfsdp_v1/test_mfsdp_nonuniform_sharding.py | 73 | except Exception as e: | CODE |
| LOW | …sformer/moe/test_moe_single_grouped_weight_numerics.py | 591 | except Exception: | CODE |
| LOW | …sformer/moe/test_moe_single_grouped_weight_numerics.py | 624 | except Exception: | CODE |
| LOW | …sformer/moe/test_moe_single_grouped_weight_numerics.py | 651 | except Exception: | CODE |
| LOW | …sformer/moe/test_moe_single_grouped_weight_numerics.py | 698 | except Exception: | CODE |
| MEDIUM | …sformer/moe/test_moe_single_grouped_weight_numerics.py | 110 | def teardown_method(self, method): | CODE |
| LOW | tests/unit_tests/transformer/moe/test_aux_loss.py | 38 | except Exception: # pragma: no cover - defensive | CODE |
| LOW | tests/unit_tests/transformer/moe/test_routers.py | 55 | except Exception: # pragma: no cover - defensive | CODE |
| LOW | tests/unit_tests/models/test_hybrid_moe_model.py | 483 | except Exception: | CODE |
| LOW | …s/unit_tests/models/mimo/test_mimo_audio_submodules.py | 123 | except Exception as e: | CODE |
| LOW | …s/unit_tests/models/mimo/test_mimo_audio_submodules.py | 197 | except Exception as e: | CODE |
| LOW | tests/unit_tests/models/mimo/test_mimo_submodules.py | 55 | except Exception as e: | CODE |
| LOW | tests/unit_tests/models/mimo/test_mimo_submodules.py | 105 | except Exception as e: | CODE |
| LOW | tests/unit_tests/models/mimo/test_mimo_submodules.py | 229 | except Exception as e: | CODE |
| LOW | tests/unit_tests/models/mimo/test_mimo_submodules.py | 284 | except Exception as e: | CODE |
| MEDIUM | tests/unit_tests/models/mimo/test_mimo_model.py | 160 | def setup_method(self, method): | CODE |
| MEDIUM | tests/unit_tests/models/mimo/test_mimo_model.py | 176 | def teardown_method(self, method): | CODE |
| 241 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tools/checkpoint/remap_gpt_dsa_to_mamba.py | 46 | Return the HybridModel state-dict key corresponding to *key* from GPTModel. Args: key: A key from the GPTMo | STRING |
| HIGH | …ted/mfsdp_v1/test_mcore_fully_sharded_data_parallel.py | 756 | Run a small deterministic training loop using a mocked hybrid Mamba+MoE model and optimizer. This helpe | STRING |
| HIGH | tests/test_utils/python_scripts/launch_jet_workload.py | 66 | Block until a downstream JET pipeline reaches a terminal status. jetclient's own ``pipeline.wait`` polls the GitLab | STRING |
| HIGH | examples/multimodal/data_loading/image_processing.py | 1780 | Create an image tiling strategy based on the provided arguments. This function encapsulates the logic for crea | STRING |
| HIGH | megatron/core/process_groups_config.py | 826 | Get process group collection for a specific module. Args: module_name: Name of the module. | STRING |
| HIGH | megatron/core/hyper_comm_grid.py | 209 | Create a process group based on a list of dimension names Note: The unique key used to store the process group | STRING |
| HIGH | megatron/core/utils.py | 124 | Validates the request to the experimental function. Args: func (Callable): Callee max_l | STRING |
| HIGH | megatron/core/utils.py | 190 | Validates the request to the experimental function. Args: func (Callable): Callee max_l | STRING |
| HIGH | megatron/core/utils.py | 222 | Pass-through to callee attribute if experimental flag is enabled. Args: super (supe | STRING |
| HIGH | megatron/core/timers.py | 385 | Returns the output string with logged timer values according to configured options. Args: names (Li | STRING |
| HIGH | megatron/core/ssm/causal_conv1d.py | 111 | Apply causal Conv1d to a contiguous context-parallel shard. Args: x: Input tensor of shape ``[B, T, D]``. T | STRING |
| HIGH | …ore/tokenizers/text/libraries/huggingface_tokenizer.py | 238 | Adds a dictionary of special tokens (eos, pad, cls...). If special tokens are NOT in the vocabulary, th | STRING |
| HIGH | megatron/core/fusions/fused_bias_swiglu.py | 464 | Implementation of biased SwiGLU that handles different input shapes. This function reshapes the input if necessary, | STRING |
| HIGH | megatron/core/fusions/fused_bias_geglu.py | 154 | Implementation of biased GEGLU that handles different input shapes. This function reshapes the input if necessary, | STRING |
| HIGH | megatron/core/optimizer/optimizer.py | 628 | Pair each current param_group with its saved counterpart by identifier tuple. Construction order isn't part of | STRING |
| HIGH | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 4954 | Release the specified parameter bucket, freeing its associated buffer storage. This function marks or | STRING |
| HIGH | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 5378 | Creates a distributed tensor (DTensor) from a local tensor with support for Megatron-FSDP and Tensor Parallel s | STRING |
| HIGH | …e/distributed/fsdp/src/megatron_fsdp/uneven_dtensor.py | 140 | Validates the chunk metadata of an uneven DTensor to ensure correctness and boundary coverage. Notes: - `g | STRING |
| HIGH | …e/distributed/fsdp/src/megatron_fsdp/uneven_dtensor.py | 257 | Gather a DTensor with potentially uneven sharding across ranks into a full tensor. This function handles DTens | STRING |
| HIGH | …core/distributed/fsdp/src/megatron_fsdp/fully_shard.py | 501 | Fully shard the optimizer for Megatron-FSDP. This is an in-place operation on the optimizer instance, which mod | STRING |
| HIGH | megatron/core/transformer/fsdp_dtensor_checkpoint.py | 945 | Report model weights that an ``fsdp_dtensor`` load cannot supply. Torch DCP loads with ``allow_partial_load=True`` | STRING |
| HIGH | megatron/core/transformer/moe/moe_utils.py | 991 | Compute the normalized score distribution over all experts. Args: logits: Router logits with experts in the | STRING |
| HIGH | …tron/core/datasets/blended_megatron_dataset_builder.py | 510 | Build the DistributedDataset Return None if and only if the underlying dataset class is not built on the curren | STRING |
| HIGH | megatron/core/datasets/object_storage_utils.py | 136 | Ascertain whether the object at the given S3 path exists in S3 Args: client (S3Client): The S3 client | STRING |
| HIGH | megatron/core/datasets/indexed_dataset.py | 95 | Get the size of the dtype/code in bytes Args: key (Union[int, Type[numpy.number]]): The dtype or co | STRING |
| HIGH | megatron/core/datasets/indexed_dataset.py | 791 | Return from the dataset Args: idx (Union[int, numpy.integer, slice]): The index or index slice into | STRING |
| HIGH | …ron/core/pipeline_parallel/multimodule_communicator.py | 586 | Compute the total number of pipeline stages across a multi-module chain. Interprets ``topology`` as a directed | STRING |
| HIGH | megatron/core/models/hybrid/hybrid_layer_allocation.py | 139 | Count layers by type across the full hybrid pattern (main + MTP). Parses the pattern to extract main and MTP compon | STRING |
| HIGH | megatron/core/models/hybrid/hybrid_layer_allocation.py | 178 | Parse a unified hybrid pattern string into main and MTP components. The pattern uses "/" as a separator between the | STRING |
| HIGH | megatron/core/models/hybrid/hybrid_layer_allocation.py | 281 | Validate and convert a single pipeline segment pattern to layer configs. This is used after the main pattern has be | STRING |
| HIGH | megatron/core/models/hybrid/hybrid_layer_allocation.py | 318 | Select and validate the pipeline segment for the given PP rank and VP stage. When the main pattern contains '|' pip | STRING |
| HIGH | megatron/core/models/hybrid/shortcut_block.py | 64 | Group physical layers into their registered shortcut-block hierarchy. Grouping updates the layer names through the | STRING |
| HIGH | megatron/core/models/hybrid/layers/utils.py | 69 | Create a layer-specific config from a normalized stack-level config. Args: config: Normalized stack-level c | STRING |
| HIGH | megatron/core/models/hybrid/layers/utils.py | 90 | Return the canonical symbol for a layer config. Args: layer_config: Layer config whose hybrid symbol should | STRING |
| HIGH | …tron/core/dist_checkpointing/gpt_checkpoint_interop.py | 101 | Derive hybrid->GPT layer index maps from a hybrid layer pattern. Args: hybrid_layer_pattern: the run's unif | STRING |
| HIGH | megatron/core/dist_checkpointing/validation.py | 288 | Raises or logs an error in case missing or unexpected keys are non-empty. Args: missing_keys (Set[str]): mi | STRING |
| HIGH | megatron/core/dist_checkpointing/validation.py | 366 | Validate if the ShardedTensors and ShardedObjects from multiple processes define correct sharding. Local ShardedTen | STRING |
| HIGH | …ng/nvshmem_copy_service/memory/tensor_pointer_utils.py | 19 | Extract the data pointer from a tensor. Args: tensor: Can be torch.Tensor, CuPy array, or | STRING |
| HIGH | megatron/core/inference/inference_request.py | 886 | Populate generated text by decoding the complete generated token stream. Args: tokenizer: Tokenizer | STRING |
| HIGH | megatron/core/inference/headers.py | 102 | Resolve an integer wire value to its category header member. Args: value (int): The integer header value re | STRING |
| HIGH | megatron/core/inference/shards_spec.py | 101 | Parse + validate the ``--inference-shards`` string. Args: spec_str: Raw CLI value, e.g. ``"tp=2,dp=1+tp=1,d | STRING |
| HIGH | megatron/core/inference/quantization/utils.py | 65 | Resolve the canonical MXFP8 storage required by a grouped-MoE backend. Args: inference_grouped_gemm_backend | STRING |
| HIGH | megatron/core/inference/moe/__init__.py | 23 | Convert a config value to a grouped-GEMM backend. Args: value: Backend name or an already-converted | STRING |
| HIGH | megatron/core/export/trtllm/trtllm_layers.py | 85 | Helper function to rename model layer names to TRTLLM Layer names We go through each layer (keys) in the model | STRING |
| HIGH | megatron/training/arguments.py | 197 | Validate model config arguments from heterogeneous config. This function takes model arguments and validates them b | STRING |
| HIGH | megatron/training/callbacks.py | 312 | Return list of callbacks registered for an event. Args: event_name: Name of the event. Ret | STRING |
| HIGH | megatron/training/tensor_metrics/definitions.py | 71 | Compute one additive sum-of-squares contribution per input tensor. Tensors the fused multi-tensor L2 kernel acc | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 182 | Return the relation for an axis. Args: axis: Abstract rank axis identifier. Returns: | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 214 | Return a copy with one axis assigned a new placement. Args: axis: Existing abstract rank axis ident | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 491 | Convert selected tensors to a potential batch of mergeable contributions. Args: values: Selected ob | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 516 | Combine per-tensor contributions for one result. Args: values: Prepared tensors corresponding to `c | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 598 | Advance one completed logical-reduction stage. Args: values: Completed branch reductions in request | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 724 | Prepare and run a metric to completion for observed tensors. Args: metric: Metric implementation to | STRING |
| HIGH | megatron/training/tensor_metrics/core.py | 789 | Complete metric steps that may have been started earlier. Together with `prepare`, this separation lets short-l | STRING |
| HIGH | megatron/training/config/instantiate_utils.py | 128 | Instantiate an object or callable from a config object. This function takes a configuration object (dictionary, lis | STRING |
| HIGH | megatron/training/config/instantiate_utils.py | 232 | Recursively instantiates a node within a configuration structure. This function handles the instantiation of indivi | STRING |
| HIGH | megatron/training/utils/checkpoint_utils.py | 130 | Read the run configuration from a YAML file (rank 0 only). Reads the file on rank 0 and broadcasts the result to ot | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/check_golden_values.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | tools/check_kernel_determinism_coverage.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | tools/trigger_internal_ci.py | 48 | logger = logging.getLogger(__name__) | CODE |
| LOW | …t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW | …distributed/mfsdp_v1/test_mfsdp_nonuniform_sharding.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/unit_tests/distributed/mfsdp_v2/test_mcore_adapter.py | 42 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ts/unit_tests/distributed/mfsdp_v2/test_fully_shard.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/unit_tests/distributed/mfsdp_v2/test_memory.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | …sts/unit_tests/distributed/mfsdp_v2/test_annotation.py | 70 | def _setup_nvtx_recording(monkeypatch: pytest.MonkeyPatch, events: list[NvtxEvent]) -> None: | CODE |
| LOW | …sts/unit_tests/distributed/mfsdp_v2/test_cuda_graph.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/unit_tests/resharding/test_model_swap.py | 253 | def set_hidden_states(self, hidden_states: torch.Tensor) -> None: | CODE |
| LOW | tests/test_utils/python_scripts/recipe_parser.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/check_status_of_main.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | tests/test_utils/python_scripts/launch_jet_workload.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | tests/test_utils/python_scripts/launch_jet_workload.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/test_utils/python_scripts/launch_nemo_run_workload.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/notify.py | 37 | logger = logging.getLogger(__name__) | CODE |
| LOW | …st_utils/python_scripts/download_unit_tests_dataset.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/test_utils/python_scripts/compare_golden_values_kl.py | 67 | logger = logging.getLogger(__name__) | CODE |
| LOW | …test_utils/python_scripts/download_coverage_results.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/auto_reminder_github.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/wait_for_resources.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/dashboard.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/approve_merge_gate.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_utils/python_scripts/swap_pr_labels.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ts/test_utils/python_scripts/download_golden_values.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | …l_tests/python_test_utils/compute_golden_statistics.py | 43 | logger = logging.getLogger(__name__) | CODE |
| LOW | …t_utils/test_pretraining_resume_checkpoint_pipeline.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/python_test_utils/test_inference_regular_pipeline.py | 14 | logger = logging.getLogger(__name__) | CODE |
| LOW | …python_test_utils/test_pretraining_regular_pipeline.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/functional_tests/python_test_utils/common.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | …nal_tests/python_test_utils/test_grpo_training_loop.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | …n_test_utils/get_test_results_from_tensorboard_logs.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | examples/mimo/pretrain_mimo.py | 53 | def _set_stock_parallel_args(args: argparse.Namespace) -> None: | CODE |
| LOW | examples/mimo/training/encoder_prefetch.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | examples/mimo/training/distributed.py | 15 | __all__ = ["initialize_distributed", "print_rank_0", "shutdown_distributed"] | CODE |
| LOW | scripts/check_api_backwards_compatibility.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | .github/scripts/re_review_reminder.py | 42 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/post_training/model_builder.py | 98 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/post_training/checkpointing.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | megatron/core/fault_injector.py | 31 | __all__ = ["FaultInjectorConfig", "setup_fault_injection", "maybe_raise_workload_exception"] | CODE |
| LOW⚡ | megatron/core/fault_injector.py | 42 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/parallel_state.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/full_cuda_graph.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/__init__.py | 30 | __all__ = [ | CODE |
| LOW | megatron/core/_rank_utils.py | 81 | def set_default_log_ranks(ranks: Iterable[int]) -> None: | CODE |
| LOW | megatron/core/msc_utils.py | 6 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/msc_utils.py | 111 | __all__ = ['MultiStorageClientFeature', 'maybe_msc'] | CODE |
| LOW | megatron/core/optimizer_param_scheduler.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/nccl_allocator.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/utils.py | 57 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/config_logger.py | 126 | __all__ = ['has_config_logger_enabled', 'log_config_to_disk'] | CODE |
| LOW | megatron/core/num_microbatches_calculator.py | 9 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/timers.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/rerun_state_machine.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/rerun_state_machine.py | 243 | def set_mode(self, mode: RerunMode) -> None: | CODE |
| LOW | megatron/core/rerun_state_machine.py | 1393 | def _set_rerun_state_machine(rerun_state_machine) -> None: | CODE |
| LOW | megatron/core/ssm/mamba_mixer.py | 111 | logger = logging.getLogger(__name__) | CODE |
| LOW | megatron/core/ssm/gated_delta_product.py | 126 | logger = logging.getLogger(__name__) | CODE |
| 219 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pretrain_hybrid.py | 501 | # The agent sets NVRX_LAUNCH_TIME on EVERY cohort, cycle 0 included, so this override must be | COMMENT |
| LOW | pretrain_hybrid.py | 521 | (_NVRX_CYCLE not in ('', '0') and _NVRX_CYCLE.isdigit()) or _NVRX_CYCLE_START is not None | COMMENT |
| LOW | tasks/finetune_utils.py | 121 | # shuffling so we can just use a simple infinite loop. | COMMENT |
| LOW | .gitlab/scripts/check_imports.py | 1 | # Copyright (c) 2025, NVIDIA CORPORATION. | COMMENT |
| LOW | tools/bisect.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | tools/trigger_internal_ci.py | 1 | #!/usr/bin/env python3 | COMMENT |
| LOW | tools/checkpoint/convert.py | 1 | # Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved. | COMMENT |
| LOW | tools/checkpoint/convert.py | 21 | # full model weights, nothing split. | COMMENT |
| LOW | tools/checkpoint/convert.py | 41 | # consumed_train_samples | COMMENT |
| LOW | tools/checkpoint/convert.py | 61 | # "mlp l0 bias" | COMMENT |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 221 | else: | COMMENT |
| LOW | tools/checkpoint/gpt_hybrid_conversion.py | 241 | # | COMMENT |
| LOW | tools/checkpoint/checkpoint_inspector.py | 1221 | COMMENT | |
| LOW | tools/common_pile_dataset/setup_common_pile_dataset.sh | 1 | #!/bin/bash | COMMENT |
| LOW | docker/common/install_nccl.sh | 1 | # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. | COMMENT |
| LOW | …t_cases/gpt/gpt_583m_perf_gb200_4gpu/model_config.yaml | 1 | # Inference perf test: 583M mcore-mistral checkpoint, TP=1 PP=1 DP=4 (4 GPUs). | COMMENT |
| LOW | …_cases/gpt/gpt_583m_perf_async_sched/model_config.yaml | 1 | # Inference perf test: 583M mcore-mistral checkpoint, TP=1 PP=1 DP=8 (8 GPUs), | COMMENT |
| LOW | …gpt_583m_perf_async_sched_gb200_4gpu/model_config.yaml | 1 | # Inference perf test: 583M mcore-mistral checkpoint, TP=1 PP=1 DP=4 (4 GPUs), | COMMENT |
| LOW | …rid/hybrid_nanov3_3b_perf_gb200_4gpu/model_config.yaml | 1 | # Inference perf test: nanov3 3B hybrid MoE checkpoint, TP=1 PP=1 EP=4 (4 GPUs). | COMMENT |
| LOW | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | …ts/performance_tests/shell_test_utils/run_perf_test.sh | 141 | ) | COMMENT |
| LOW | tests/unit_tests/test_imports.py | 1 | # Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved. | COMMENT |
| LOW | tests/unit_tests/testmon_mandatory_tests.yaml | 1 | # Unit tests that must always run when a source directory changes, regardless of | COMMENT |
| LOW | tests/unit_tests/ssm/test_gdp_dynamic_inference.py | 501 | COMMENT | |
| LOW | tests/unit_tests/ssm/test_gdp_dynamic_inference.py | 841 | assert torch.equal(graph_conv, ref_conv) | COMMENT |
| LOW | tests/unit_tests/tokenizers/test_text_parsers.py | 141 | COMMENT | |
| LOW | tests/unit_tests/tokenizers/test_text_parsers.py | 241 | json.loads(raw, parse_constant=_bare) | COMMENT |
| LOW | …s/checkpoint/test_gpt_hybrid_conversion_parallelism.py | 41 | COMMENT | |
| LOW | tests/unit_tests/determinism/configs.py | 121 | # mamba-attn-mlp covers Mamba + attention + MLP paths. pure-mamba is | COMMENT |
| LOW | tests/unit_tests/determinism/configs.py | 181 | "TP": "tensor_model_parallel_size", | COMMENT |
| LOW | …sts/distributed/test_grad_sync_with_expert_parallel.py | 341 | with_context_parallel=True, partial_data_parallel=True | COMMENT |
| LOW | tests/unit_tests/distributed/mfsdp_v2/conftest.py | 61 | # communicator resources from accumulating across tests. | COMMENT |
| LOW | tests/unit_tests/distributed/mfsdp_v2/test_overlap.py | 101 | COMMENT | |
| LOW | …unit_tests/transformer/test_fsdp_dtensor_checkpoint.py | 1 | # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. | COMMENT |
| LOW | tests/unit_tests/transformer/test_mhc_block_manager.py | 401 | # ============================================================================ | COMMENT |
| LOW | …s/unit_tests/elastification/test_hybrid_flex_router.py | 81 | tensor_model_parallel_size=1, pipeline_model_parallel_size=1 | COMMENT |
| LOW | tests/unit_tests/training/test_freeze_all_layers.py | 101 | assert chunk.router.frozen_expert_bias is True | COMMENT |
| LOW | …s/unit_tests/dist_checkpointing/test_fully_parallel.py | 521 | 'a': [(0, 0)], # saved by rank 0 obviously | COMMENT |
| LOW | …unit_tests/generalized_tensor_parallel/test_gtp_dcp.py | 1141 | # Gated-delta-product (GDP) in_proj: gather+split under GTP_remat | COMMENT |
| LOW | …/unit_tests/inference/test_mtp_cuda_graph_inference.py | 1261 | # through `_run_test`, which seeds `random.seed(...)` so every EP rank builds an IDENTICAL | COMMENT |
| LOW | …ests/inference/contexts/test_dynamic_prefix_caching.py | 1381 | # | COMMENT |
| LOW | …nit_tests/inference/engines/test_gdp_cuda_graph_e2e.py | 101 | # Batch-invariant CUDA-graph buckets align their token counts to the | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 961 | num_speculative_tokens: int = 0 | COMMENT |
| LOW | …es/test_dynamic_engine_parallel_and_chunked_prefill.py | 281 | num_expected = request.sampling_params.num_tokens_to_generate | COMMENT |
| LOW | …es/test_dynamic_engine_parallel_and_chunked_prefill.py | 641 | COMMENT | |
| LOW | …es/test_dynamic_engine_parallel_and_chunked_prefill.py | 1241 | # not supported"), but transformer_config validation actually PERMITS the combination -- | COMMENT |
| LOW | …es/test_dynamic_engine_parallel_and_chunked_prefill.py | 1741 | # Generated tokens are too blunt on their own: a zero-filled carry | COMMENT |
| LOW | tests/test_utils/python_scripts/recipe_parser.py | 21 | # cadence remains the trigger axis. | COMMENT |
| LOW | tests/test_utils/python_scripts/test_oncall_manager.py | 1 | # Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved. | COMMENT |
| LOW | tests/test_utils/recipes/h100/moe.yaml | 121 | # Weekly tests: Run both DEV and LTS unless something is flaky # | COMMENT |
| LOW | tests/test_utils/recipes/h100/moe.yaml | 261 | # Super important mr, mr-github tests that run for both DEV and LTS per mr, mr-github # | COMMENT |
| LOW | …cipes/h100/gpt-dynamic-inference-with-coordinator.yaml | 101 | # Throughput coverage for this 583M dynamic-batching DP=8 config now lives at | COMMENT |
| LOW | tests/test_utils/recipes/h100/gpt.yaml | 461 | - environment: [dev] | COMMENT |
| LOW | tests/test_utils/recipes/gb200/moe.yaml | 101 | scope: [nightly] | COMMENT |
| LOW | tests/test_utils/recipes/gb200/moe.yaml | 201 | - environment: [dev] | COMMENT |
| LOW | …unctional_tests/test_cases/common/moe_perf/__main__.py | 401 | COMMENT | |
| LOW | …unctional_tests/test_cases/common/moe_perf/__main__.py | 421 | # test_moe_layer_performance(case, debug_mode=True) | COMMENT |
| LOW | …epseek_proxy_mfsdp_v2_ep2_ep_overlap/model_config.yaml | 1 | # DeepSeek-style proxy: 8 layers — 3 dense, rest MoE with expert parallelism 2. | COMMENT |
| LOW | …eepseek_proxy_mfsdp_v2_ep2_hfsdp_mb2/model_config.yaml | 1 | # DeepSeek-style proxy: 8 layers - 3 dense, rest MoE with expert parallelism 2. | COMMENT |
| LOW | …_pp1_ep4_16B_mtp_sp_block_cudagraphs/model_config.yaml | 1 | # Inference functional test: MTP + sequence parallelism + block-scope CUDA graphs. | COMMENT |
| 187 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | train_rl.py | 63 | # Check if fp8_model_init supports preserve_high_precision_init_val | COMMENT |
| LOW⚡ | tools/check_copyright.py | 20 | # Check if the expected header is at the start of the file | COMMENT |
| LOW | tools/common_pile_dataset/setup_common_pile_dataset.sh | 50 | # Check if create_common_pile_ci_dataset.py was scp'd alongside this script | COMMENT |
| LOW | docker/common/install_source_wheels.sh | 25 | # Check if required arguments are provided | COMMENT |
| LOW | tests/unit_tests/test_optimizer.py | 50 | # Check if FP8 block scaling is available. | COMMENT |
| LOW⚡ | tests/unit_tests/test_utils.py | 531 | # Check if configuration was success. | COMMENT |
| LOW⚡ | tests/unit_tests/test_utils.py | 534 | # Check if the instance is in disabled state. | COMMENT |
| LOW⚡ | tests/unit_tests/test_utils.py | 538 | # Check if all ranks have straggler detector enabled. | COMMENT |
| LOW | tests/unit_tests/conftest.py | 143 | # Check if data directory exists and has content | COMMENT |
| LOW | tests/unit_tests/fusions/test_torch_softmax.py | 145 | # Check if output is a valid probability distribution | COMMENT |
| LOW | tests/unit_tests/transformer/test_cuda_graphs.py | 843 | # Check if cuda graph is correctly setting is first/last layer | COMMENT |
| LOW | tests/unit_tests/transformer/test_attention.py | 419 | # Check if output and bias have the correct shape | COMMENT |
| LOW | tests/unit_tests/transformer/test_attention.py | 601 | # Check if the output is close | COMMENT |
| LOW | …/unit_tests/transformer/test_multi_latent_attention.py | 1608 | # Check if the output is the same | COMMENT |
| LOW | …it_tests/transformer/moe/test_moe_layer_discrepancy.py | 69 | # Check if parameters are the same | COMMENT |
| LOW | …it_tests/transformer/moe/test_moe_layer_discrepancy.py | 75 | # Check if input is the same across all ranks | COMMENT |
| LOW | …it_tests/transformer/moe/test_moe_layer_discrepancy.py | 150 | # Check if output is the same across all ranks | COMMENT |
| LOW | …it_tests/transformer/moe/test_moe_layer_discrepancy.py | 216 | # Check if output is the same across all ranks | COMMENT |
| LOW | tests/unit_tests/transformer/moe/test_moe_layer.py | 164 | # Check if the moe layer is interleaved correctly | COMMENT |
| LOW | tests/unit_tests/models/test_llava_model.py | 990 | # Check if output shape is as expected | COMMENT |
| LOW | …_tests/dist_checkpointing/test_layer_wise_optimizer.py | 144 | # Check if optimizer is ChainedOptimizer (expected for standard setup) | COMMENT |
| LOW | …/unit_tests/inference/contexts/test_dynamic_context.py | 1582 | # Assign blocks to the requests (one block per request) | COMMENT |
| LOW | …/unit_tests/inference/contexts/test_dynamic_context.py | 1658 | # Assign blocks to the requests: | COMMENT |
| LOW | …ts/test_utils/python_scripts/download_golden_values.py | 288 | # Check if we should skip based on only_failing flag | COMMENT |
| LOW | …nal_tests/test_cases/common/ckpt_converter/__main__.py | 898 | # Print results. | COMMENT |
| LOW | …unctional_tests/shell_test_utils/run_batch_ci_tests.sh | 173 | # Check if file is empty (job still running or not started) | COMMENT |
| LOW | tests/functional_tests/shell_test_utils/run_ci_test.sh | 16 | # Set umask to 0002 to allow group read/write permissions | COMMENT |
| LOW | tests/functional_tests/shell_test_utils/run_ci_test.sh | 276 | ## Loop over the list of model configs in the params file and run each one in sequence, collecting | COMMENT |
| LOW | docs/conf.py | 43 | # Check if we should skip autodoc generation | COMMENT |
| LOW | examples/post_training/modelopt/finetune.py | 369 | # Check if this is OpenAI chat data? | COMMENT |
| LOW | examples/mimo/utils/logging.py | 40 | # Print output projections | COMMENT |
| LOW | examples/gptoss/02_train.sh | 55 | # Check if checkpoint path exists | COMMENT |
| LOW | examples/gptoss/02_train.sh | 62 | # Check if tensorboard logs path exists | COMMENT |
| LOW | examples/multimodal/dataset_helpers.py | 142 | # Check if all samples fit in the knapsack capacity. | COMMENT |
| LOW | examples/multimodal/data_loading/knapsacks.py | 34 | # Check if all samples fit in the knapsack capacity. | COMMENT |
| LOW | examples/multimodal/data_loading/knapsacks.py | 118 | # Check if any sample exceeds max_capacity | COMMENT |
| LOW | examples/multimodal/data_loading/image_processing.py | 1388 | # Check if we need to degrade (only if degradation is enabled) | COMMENT |
| LOW | …amples/multimodal/data_loading/cookers/conversation.py | 437 | # Check if all media files are retrieved | COMMENT |
| LOW | …amples/multimodal/data_loading/cookers/conversation.py | 501 | # Check if all media files are retrieved | COMMENT |
| LOW | examples/inference/utils.py | 253 | # Check if we have any prompts (from command line or JSONL) | COMMENT |
| LOW | examples/inference/advanced/gpt_dynamic_inference.py | 274 | # Check if all requests are finished. | COMMENT |
| LOW | examples/rl/environments/countdown/countdown.py | 34 | # Check if all numbers in equation are available | COMMENT |
| LOW | scripts/check_api_backwards_compatibility.py | 240 | # Check if this breakage kind should be ignored globally (not a signature change) | COMMENT |
| LOW | scripts/check_api_backwards_compatibility.py | 253 | # Check if this is a breakage kind we ignore for __init__ methods | COMMENT |
| LOW | scripts/check_api_backwards_compatibility.py | 263 | # Check if it's a child of a filtered object | COMMENT |
| LOW | scripts/check_api_backwards_compatibility.py | 350 | # Print results | COMMENT |
| LOW | .github/actions/check-nvidia-sso-membership/action.yml | 73 | # Check if SSO file is available | COMMENT |
| LOW | .github/actions/check-nvidia-sso-membership/action.yml | 82 | # Check if username exists as a key in the JSON object | COMMENT |
| LOW | megatron/post_training/model_builder.py | 354 | # Set num_layers to 0 for base model in offline mode | COMMENT |
| LOW | megatron/core/parallel_state.py | 988 | # Set NCCL_COLLNET_ENABLE to 1 to enable SHARP for the dp group. | COMMENT |
| LOW | megatron/core/parallel_state.py | 1540 | # Set NCCL_COLLNET_ENABLE to 1 to enable SHARP for the dp_replica group. | COMMENT |
| LOW | megatron/core/parallel_state.py | 1569 | # Set NCCL_COLLNET_ENABLE to 0 to restrict SHARP application to the dp_replica group. | COMMENT |
| LOW | megatron/core/fp4_utils.py | 18 | # Check if Transformer Engine is installed | COMMENT |
| LOW | megatron/core/fp4_utils.py | 29 | # Check if Transformer Engine has class for fp4 tensors. | COMMENT |
| LOW | megatron/core/fp8_utils.py | 23 | # Check if Transformer Engine is installed | COMMENT |
| LOW | megatron/core/fp8_utils.py | 41 | # Check if Transformer Engine has class for fp8 tensors. | COMMENT |
| LOW | megatron/core/fp8_utils.py | 56 | # Check if Transformer Engine has MXFP8Tensor class | COMMENT |
| LOW | megatron/core/fp8_utils.py | 889 | # Check if fp8_model_init supports setting recipe | COMMENT |
| LOW | megatron/core/fp8_utils.py | 894 | # Check if fp8_model_init supports preserve_high_precision_init_val | COMMENT |
| LOW | megatron/core/utils.py | 3276 | # Check if any deprecated key is present in kwargs | COMMENT |
| 69 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 17 | # Step 1: Reshape back to (num_head, 3*head_dim, -1) | COMMENT |
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 20 | # Step 2: Slice along the head_dim dimension to get q, k, v | COMMENT |
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 25 | # Step 3: Reshape each back to (num_head * head_dim, -1) | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 113 | ### Step 1: Vocabulary Files | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 119 | ### Step 2: Download Raw Text | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 132 | ### Step 3: GPT Preprocessing | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 137 | ### Step 4: BERT Preprocessing | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 150 | ### Step 5: T5 Preprocessing | COMMENT |
| LOW | tools/common_pile_dataset/README.md | 155 | ### Step 6: Verification | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 361 | # Step 1: Get vocabulary files | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 387 | # Step 2: Get raw text data | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 411 | # Step 3: Preprocess for GPT (GPT2BPETokenizer) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 434 | # Step 4: Preprocess for BERT (BertWordPieceLowerCase + split-sentences) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 475 | # Step 5: Preprocess for T5 (BertWordPieceCase) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 497 | # Step 6: Clean up and verify | COMMENT |
| LOW⚡ | …/unit_tests/transformer/test_multi_latent_attention.py | 2305 | # Step 1: MXFP8 GEMM. Same as TE Linear's forward under fp8_autocast. | COMMENT |
| LOW⚡ | …/unit_tests/transformer/test_multi_latent_attention.py | 2331 | # Step 3: Quantize. Same quantizer + format as _QuantizeKVForFusedAttn uses for K/V. | COMMENT |
| LOW | …/unit_tests/transformer/test_multi_latent_attention.py | 2318 | # Step 2: RoPE. fused_apply_mla_rope_for_q expects (S, B, NH, Q_HEAD_DIM) when | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2625 | # Step 1: Forward pass for all 3 requests | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2634 | # Step 2: Forward pass where req 10 finishes, req 11 continues. Req 999 is NOT scheduled. | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2723 | # Step 1: All 3 requests are active, process forward pass | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2732 | # Step 2: Both decode requests finish, chunked prefill NOT scheduled this step. | COMMENT |
| LOW | …/unit_tests/inference/contexts/test_dynamic_context.py | 2656 | # Step 3: Add the next chunk. It should sit exactly at the boundary (index 1) and inherit the state. | COMMENT |
| LOW | …nference/contexts/test_dynamic_context_mtp_kv_cache.py | 1377 | # Step 1: the batch is one over the cap, so the spare holder is force-paused. | COMMENT |
| LOW | …nference/contexts/test_dynamic_context_mtp_kv_cache.py | 1388 | # Step 2: retire one filler so the paused row fits back in the active window. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4434 | # Step 1: schedule and launch the first prefill chunk as the async primer. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4438 | # Step 2: resolve the first chunk and launch the second chunk. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4505 | # Step 5: resolve that batch and launch the next decode/prefill batch. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4513 | # Step 6: resolve request 1's final decode and launch request 2's final | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4655 | # Step 1: A's full prompt plus B's first token fill the 256-token budget. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4662 | # Step 2: B's final one-token chunk. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4668 | # Step 3: resolve B's prefill output. | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4456 | # Step 3: resolve request 1's second chunk and launch the batch containing | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4479 | # Step 4: resolve that mixed batch and launch request 1's decode with | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4537 | # Step 7: resolve request 2's final prefill and launch request 3's final | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4549 | # Step 8: resolve request 3's final prefill output. | COMMENT |
| LOW⚡ | …l_tests/python_test_utils/compute_golden_statistics.py | 11 | # Step 1: Run batch tests (from megatron-rl directory): | STRING |
| LOW⚡ | …l_tests/python_test_utils/compute_golden_statistics.py | 15 | # Step 2: Wait for jobs to complete, then compute statistics: | STRING |
| LOW⚡ | docs/developer/submit.md | 15 | ## Step 1: Mark PR as "Ready for Review" | COMMENT |
| LOW⚡ | docs/developer/submit.md | 24 | ## Step 2: Final Review (`megatron/core` only) | COMMENT |
| LOW⚡ | docs/developer/submit.md | 32 | ## Step 3: Approved | COMMENT |
| LOW⚡ | docs/developer/submit.md | 37 | ## Step 4: Merge | COMMENT |
| LOW | docs/user-guide/data-loading.md | 39 | ### Step 1: Consolidate dataset files | COMMENT |
| LOW | docs/user-guide/data-loading.md | 53 | ### Step 2: Pre-build the dataset cache | COMMENT |
| LOW | docs/user-guide/data-loading.md | 77 | ### Step 3: Optionally pre-build per-dataset metadata | COMMENT |
| LOW | docs/user-guide/data-loading.md | 87 | ### Step 4: Launch training with optimized data loading | COMMENT |
| LOW | examples/gptoss/README.md | 3 | ## Step 0: Install Dependencies | COMMENT |
| LOW | examples/gptoss/README.md | 57 | ## Step 1: Convert HuggingFace to Megatron (Optional - skip if you already have a Megatron checkpoint) | COMMENT |
| LOW | examples/gptoss/README.md | 65 | ## Step 2: Train from Scratch | COMMENT |
| LOW | examples/gptoss/README.md | 147 | ## Step 3: Convert Megatron to HuggingFace | COMMENT |
| LOW⚡ | examples/multimodal/data_loading/image_processing.py | 1109 | # Step 2: Check if total tokens is within budget | COMMENT |
| LOW⚡ | examples/multimodal/data_loading/image_processing.py | 1116 | # Step 3: We're over budget, need to scale down | COMMENT |
| LOW | examples/multimodal/data_loading/image_processing.py | 1098 | # Step 1: Process each media with current token budget | COMMENT |
| LOW⚡ | .github/pull_request_template.md | 34 | #### Step 1: Mark PR as "Ready for Review" | COMMENT |
| LOW⚡ | .github/pull_request_template.md | 43 | #### Step 2: Final Review | COMMENT |
| LOW⚡ | .github/pull_request_template.md | 49 | #### Step 3: Approved | COMMENT |
| LOW⚡ | .github/scripts/oncall_manager.py | 135 | # Step 1: Add new oncall first (include current members to avoid removing anyone yet) | COMMENT |
| LOW⚡ | .github/scripts/oncall_manager.py | 142 | # Step 2: Now set the usergroup to contain only the new oncall | COMMENT |
| LOW⚡ | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1935 | # Step 3: Split parameter groups to meet communication segmentation requirements. | COMMENT |
| LOW⚡ | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1989 | # Step 4: Generate the groups of collective buckets, where each group aggregates | COMMENT |
| 36 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/checkpoint/weighted_merge.py | 1621 | CODE | |
| LOW | tools/bert_embedding/huggingface.py | 51 | CODE | |
| LOW | tests/unit_tests/test_layer_wise_optimizer.py | 210 | CODE | |
| LOW | …s/checkpoint/test_gpt_hybrid_conversion_parallelism.py | 71 | CODE | |
| LOW | tests/unit_tests/distributed/mfsdp_v1/utils.py | 39 | CODE | |
| LOW | …/unit_tests/transformer/test_multi_token_prediction.py | 908 | CODE | |
| LOW | …er/experimental_attention_variant/test_absorbed_mla.py | 83 | CODE | |
| LOW | …mental_attention_variant/test_dsv4_hybrid_attention.py | 58 | CODE | |
| LOW | …mental_attention_variant/test_attention_variant_csa.py | 565 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 1092 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 1239 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 1594 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 2582 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 2672 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 2759 | CODE | |
| LOW | …perimental_attention_variant/test_dsa_native_parity.py | 2910 | CODE | |
| LOW | …ts/unit_tests/transformer/moe/test_token_dispatcher.py | 79 | CODE | |
| LOW | tests/unit_tests/transformer/moe/test_paged_stashing.py | 169 | CODE | |
| LOW | …nit_tests/transformer/moe/test_moe_recompute_memory.py | 28 | CODE | |
| LOW | …s/elastification/test_inject_flextron_forward_logic.py | 33 | CODE | |
| LOW | …it_tests/pipeline_parallel/test_bridge_communicator.py | 164 | CODE | |
| LOW | tests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py | 524 | CODE | |
| LOW | tests/unit_tests/models/mimo/test_mimo_1f1b_schedule.py | 801 | CODE | |
| LOW | tests/unit_tests/dist_checkpointing/utils.py | 250 | CODE | |
| LOW | tests/unit_tests/dist_checkpointing/utils.py | 422 | CODE | |
| LOW | tests/unit_tests/resharding/test_planner.py | 34 | CODE | |
| LOW | …s/inference/test_dynamic_prefix_caching_coordinator.py | 211 | CODE | |
| LOW | tests/unit_tests/inference/test_wandb_logging.py | 52 | CODE | |
| LOW | tests/unit_tests/inference/coordinator_test_utils.py | 19 | CODE | |
| LOW | …ests/inference/contexts/test_dynamic_prefix_caching.py | 76 | CODE | |
| LOW | …eration_controllers/test_text_generation_controller.py | 74 | CODE | |
| LOW | …eration_controllers/test_text_generation_controller.py | 3897 | CODE | |
| LOW | …generation_controllers/test_mtp_kv_cache_controller.py | 42 | CODE | |
| LOW | …generation_controllers/test_mtp_kv_cache_controller.py | 1121 | CODE | |
| LOW | …ts/inference/engines/test_hybrid_prefix_caching_e2e.py | 235 | CODE | |
| LOW | …sts/unit_tests/inference/engines/test_static_engine.py | 34 | CODE | |
| LOW | …sts/unit_tests/inference/engines/test_static_engine.py | 324 | CODE | |
| LOW | tests/unit_tests/data/test_bin_reader.py | 99 | CODE | |
| LOW | tests/test_utils/python_scripts/recipe_parser.py | 365 | CODE | |
| LOW | …/academic_paper_scripts/detxoify_lm/perspective_api.py | 88 | CODE | |
| LOW | examples/post_training/modelopt/utils.py | 76 | CODE | |
| LOW | examples/post_training/modelopt/utils.py | 154 | CODE | |
| LOW | examples/mimo/training/builder.py | 125 | CODE | |
| LOW | examples/mimo/data/mock.py | 165 | CODE | |
| LOW | examples/mimo/data/mock.py | 41 | CODE | |
| LOW | examples/multimodal/model.py | 30 | CODE | |
| LOW | examples/multimodal/mimo_checkpoint_model.py | 139 | CODE | |
| LOW | examples/multimodal/image_processing.py | 152 | CODE | |
| LOW | …ples/multimodal/model_converter/vision_model_tester.py | 77 | CODE | |
| LOW | examples/multimodal/data_loading/image_processing.py | 661 | CODE | |
| LOW | megatron/core/parallel_state.py | 622 | CODE | |
| LOW | megatron/core/ssm/mamba_mixer.py | 175 | CODE | |
| LOW | megatron/core/ssm/gated_delta_product.py | 212 | CODE | |
| LOW | megatron/core/ssm/mamba_layer.py | 185 | CODE | |
| LOW | megatron/core/ssm/mamba_context_parallel.py | 330 | CODE | |
| LOW | megatron/core/ssm/mamba_context_parallel.py | 357 | CODE | |
| LOW | megatron/core/ssm/gated_delta_net/common.py | 113 | CODE | |
| LOW | megatron/core/ssm/ops/gdp/chunk_h.py | 234 | CODE | |
| LOW | megatron/core/ssm/ops/gdp/chunk.py | 39 | CODE | |
| LOW | megatron/core/ssm/ops/gdp/fused_recurrent.py | 187 | CODE | |
| 140 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 17 | # Step 1: Reshape back to (num_head, 3*head_dim, -1) | COMMENT |
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 20 | # Step 2: Slice along the head_dim dimension to get q, k, v | COMMENT |
| LOW⚡ | tools/checkpoint/saver_hf_llava.py | 25 | # Step 3: Reshape each back to (num_head * head_dim, -1) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 361 | # Step 1: Get vocabulary files | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 387 | # Step 2: Get raw text data | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 411 | # Step 3: Preprocess for GPT (GPT2BPETokenizer) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 434 | # Step 4: Preprocess for BERT (BertWordPieceLowerCase + split-sentences) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 475 | # Step 5: Preprocess for T5 (BertWordPieceCase) | COMMENT |
| LOW⚡ | …s/common_pile_dataset/create_common_pile_ci_dataset.py | 497 | # Step 6: Clean up and verify | COMMENT |
| LOW⚡ | …/unit_tests/transformer/test_multi_latent_attention.py | 2305 | # Step 1: MXFP8 GEMM. Same as TE Linear's forward under fp8_autocast. | COMMENT |
| LOW⚡ | …/unit_tests/transformer/test_multi_latent_attention.py | 2331 | # Step 3: Quantize. Same quantizer + format as _QuantizeKVForFusedAttn uses for K/V. | COMMENT |
| LOW | …/unit_tests/transformer/test_multi_latent_attention.py | 2318 | # Step 2: RoPE. fused_apply_mla_rope_for_q expects (S, B, NH, Q_HEAD_DIM) when | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2625 | # Step 1: Forward pass for all 3 requests | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2634 | # Step 2: Forward pass where req 10 finishes, req 11 continues. Req 999 is NOT scheduled. | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2723 | # Step 1: All 3 requests are active, process forward pass | COMMENT |
| LOW⚡ | …/unit_tests/inference/contexts/test_dynamic_context.py | 2732 | # Step 2: Both decode requests finish, chunked prefill NOT scheduled this step. | COMMENT |
| LOW | …/unit_tests/inference/contexts/test_dynamic_context.py | 2656 | # Step 3: Add the next chunk. It should sit exactly at the boundary (index 1) and inherit the state. | COMMENT |
| LOW | …nference/contexts/test_dynamic_context_mtp_kv_cache.py | 1377 | # Step 1: the batch is one over the cap, so the spare holder is force-paused. | COMMENT |
| LOW | …nference/contexts/test_dynamic_context_mtp_kv_cache.py | 1388 | # Step 2: retire one filler so the paused row fits back in the active window. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4434 | # Step 1: schedule and launch the first prefill chunk as the async primer. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4438 | # Step 2: resolve the first chunk and launch the second chunk. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4505 | # Step 5: resolve that batch and launch the next decode/prefill batch. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4513 | # Step 6: resolve request 1's final decode and launch request 2's final | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4655 | # Step 1: A's full prompt plus B's first token fill the 256-token budget. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4662 | # Step 2: B's final one-token chunk. | COMMENT |
| LOW⚡ | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4668 | # Step 3: resolve B's prefill output. | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4456 | # Step 3: resolve request 1's second chunk and launch the batch containing | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4479 | # Step 4: resolve that mixed batch and launch request 1's decode with | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4537 | # Step 7: resolve request 2's final prefill and launch request 3's final | COMMENT |
| LOW | …ts/unit_tests/inference/engines/test_dynamic_engine.py | 4549 | # Step 8: resolve request 3's final prefill output. | COMMENT |
| LOW⚡ | …l_tests/python_test_utils/compute_golden_statistics.py | 11 | # Step 1: Run batch tests (from megatron-rl directory): | STRING |
| LOW⚡ | …l_tests/python_test_utils/compute_golden_statistics.py | 15 | # Step 2: Wait for jobs to complete, then compute statistics: | STRING |
| LOW⚡ | examples/multimodal/data_loading/image_processing.py | 1109 | # Step 2: Check if total tokens is within budget | COMMENT |
| LOW⚡ | examples/multimodal/data_loading/image_processing.py | 1116 | # Step 3: We're over budget, need to scale down | COMMENT |
| LOW | examples/multimodal/data_loading/image_processing.py | 1098 | # Step 1: Process each media with current token budget | COMMENT |
| LOW⚡ | .github/scripts/oncall_manager.py | 135 | # Step 1: Add new oncall first (include current members to avoid removing anyone yet) | COMMENT |
| LOW⚡ | .github/scripts/oncall_manager.py | 142 | # Step 2: Now set the usergroup to contain only the new oncall | COMMENT |
| LOW⚡ | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1935 | # Step 3: Split parameter groups to meet communication segmentation requirements. | COMMENT |
| LOW⚡ | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1989 | # Step 4: Generate the groups of collective buckets, where each group aggregates | COMMENT |
| LOW | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1815 | # Step 0: Register new FSDP unit modules. | COMMENT |
| LOW | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1848 | # Step 1: Group the parameters according to their execution order and attributes. | COMMENT |
| LOW | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1886 | # Step 2: Bucket the parameters based on the guide bucket size. | COMMENT |
| LOW | …e/transformer/custom_layers/batch_invariant_kernels.py | 312 | # Step 1: Find maximum value in the row for numerical stability | COMMENT |
| LOW | …e/transformer/custom_layers/batch_invariant_kernels.py | 324 | # Step 2: Compute sum of exp(x - max_val) | COMMENT |
| LOW | …e/transformer/custom_layers/batch_invariant_kernels.py | 340 | # Step 3: Compute final log_softmax values: x - max_val - log_sum_exp | COMMENT |
| LOW⚡ | megatron/core/datasets/data_schedule.py | 558 | # Step 1: Fetch batches and gather global sequence lengths | COMMENT |
| LOW⚡ | megatron/core/datasets/data_schedule.py | 563 | # Step 2: Check required sample keys | COMMENT |
| LOW⚡ | megatron/core/datasets/data_schedule.py | 569 | # Step 3: Schedule samples into groups | COMMENT |
| LOW⚡ | megatron/core/datasets/data_schedule.py | 606 | # Step 5: Build packed microbatches | COMMENT |
| LOW⚡ | megatron/core/datasets/data_schedule.py | 609 | # Step 6: Calculate FLOPs info | COMMENT |
| LOW | megatron/core/datasets/data_schedule.py | 582 | # Step 4: Reroute samples to DCP ranks | COMMENT |
| LOW | megatron/core/datasets/data_schedule.py | 622 | # Step 7: Broadcast to PP group (for middle PP stages) | COMMENT |
| LOW | megatron/core/datasets/data_schedule.py | 638 | # Step 8: Broadcast to TP group (for non-TP-0 ranks) | COMMENT |
| LOW | megatron/core/datasets/data_schedule.py | 652 | # Step 9: create data_iterator and handle VPP if enabled | COMMENT |
| LOW | …n/core/dist_checkpointing/strategies/fully_parallel.py | 267 | # Step 3: load part of the checkpoint. | COMMENT |
| LOW | …n/core/dist_checkpointing/strategies/fully_parallel.py | 321 | # Step 4: exchange data between ranks | COMMENT |
| LOW⚡ | …gatron/core/resharding/nvshmem_copy_service/service.py | 252 | # Step 1: Segment tasks (break large tasks into chunks) | COMMENT |
| LOW⚡ | …gatron/core/resharding/nvshmem_copy_service/service.py | 262 | # Step 2: Pack tasks into workload groups | COMMENT |
| LOW⚡ | …gatron/core/resharding/nvshmem_copy_service/service.py | 269 | # Step 3: Schedule workloads to iterations | COMMENT |
| LOW⚡ | …gatron/core/resharding/nvshmem_copy_service/service.py | 278 | # Step 4: Prepare iteration schedules | COMMENT |
| 11 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tasks/finetune_utils.py | 121 | # shuffling so we can just use a simple infinite loop. | COMMENT |
| MEDIUM | tests/unit_tests/test_inference.py | 108 | # we are replicating what lm-eval-harness::TemplateLM::_encode_pair does | COMMENT |
| MEDIUM | tests/unit_tests/test_model_configs.py | 13 | # does not set an explicit METRICS list. Sourced from the functional-test harness | COMMENT |
| MEDIUM | …s/checkpoint/test_gpt_hybrid_conversion_parallelism.py | 45 | # harness launched pytest. When that default PG is multi-rank (e.g. Megatron's | COMMENT |
| MEDIUM | …/unit_tests/optimizer/test_layer_sharded_e2e_parity.py | 30 | # Plain bf16 run: the harness defaults to FP8, which is orthogonal here. ``fp8=None`` turns | COMMENT |
| MEDIUM | …/unit_tests/optimizer/test_layer_sharded_e2e_parity.py | 31 | # it off; the harness's positional ``recipe`` must then not be "mxfp8", because validate_args | COMMENT |
| MEDIUM | …sts/unit_tests/determinism/kernels/test_ssm_kernels.py | 161 | # layout the Mamba mixer dispatches; the harness must hand them over unchanged. | COMMENT |
| MEDIUM | …ests/determinism/correctness/test_transformer_layer.py | 210 | # stress harness relies on the connection count the launcher | COMMENT |
| LOW | …t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py | 353 | # DP: Only relevant when using HSDP, where we need the flattened DP group for data parallelism. (Otherwise, just pas | COMMENT |
| LOW | …t_tests/distributed/mfsdp_v1/test_mfsdp_fully_shard.py | 355 | # DP-Shard-CP: Only required if using CP. Otherwise, just pass dp_shard to FSDP. | COMMENT |
| MEDIUM | tests/unit_tests/transformer/test_utils.py | 427 | """Test a comprehensive scenario with multiple configurations.""" | STRING |
| MEDIUM⚡ | tests/unit_tests/training/models/test_base.py | 223 | """from_dict() reconstructs configs from serialized dicts, handles nested dataclasses, and is robust to unknown keys | STRING |
| LOW | …s/unit_tests/dist_checkpointing/models/test_mlp_glu.py | 129 | # Load happens in-place, so we can just use the same tensors | COMMENT |
| MEDIUM | …neralized_tensor_parallel/test_gtp_fp8_param_gather.py | 373 | # Pure BF16 GTP reference: fp8=None overrides the harness default (recipe inert). | COMMENT |
| MEDIUM | …tests/generalized_tensor_parallel/test_gtp_muon_dcp.py | 257 | # aliasing. The shared harness builds its own mock args, so wrap init_basic_mock_args to | COMMENT |
| MEDIUM | tests/unit_tests/inference/test_endpoints_common.py | 119 | # --- HTTP harness, shared with the other endpoint test modules -------------------------------- | COMMENT |
| MEDIUM | …/unit_tests/inference/contexts/test_dynamic_context.py | 3121 | # 6. Verify seamless append (no legacy offset math needed) | COMMENT |
| MEDIUM | …s/inference/engines/test_dynamic_engine_async_sched.py | 434 | # Only list a pair when the harness below observes its production call/state | COMMENT |
| MEDIUM | …s/inference/engines/test_dynamic_engine_async_sched.py | 608 | # final candidate; the harness still requires exact async-repeat top-N. | COMMENT |
| MEDIUM | …es/test_dynamic_engine_parallel_and_chunked_prefill.py | 1242 | # it just imposes requirements the harness never set. For EP (not expert TP) they are: | COMMENT |
| MEDIUM | …cipes/h100/gpt-dynamic-inference-with-coordinator.yaml | 103 | # skills/run-performance-tests/SKILL.md for the harness it runs under. | COMMENT |
| MEDIUM | …inference_server_smoke_tp1_pp1_dp8_583m/serve_smoke.py | 43 | # the JET harness expects at ``logs/*/*/attempt_0/*/std*.log``) while still | COMMENT |
| LOW | …sts/functional_tests/shell_test_utils/_run_training.sh | 105 | # If value is "true", just use the key | COMMENT |
| LOW | examples/multimodal/dataset_helpers.py | 275 | ), # TODO: handle class toekn length correctly(not just use 1) | CODE |
| LOW | examples/multimodal/data_loading/task_encoder.py | 481 | # Some videos are too short or low-fps, just use the whole video, like sthv2, smit, llava-hound (2fps) | COMMENT |
| LOW | examples/multimodal/data_loading/image_processing.py | 1133 | # If there was not scaling down, we're stuck just use min_num_patches per media, else | COMMENT |
| LOW | megatron/core/optimizer_param_scheduler.py | 235 | # If the learning rate is constant, just return the initial value. | COMMENT |
| LOW | megatron/core/timers.py | 476 | # polutes the runs list, so we just add each as a scalar | COMMENT |
| MEDIUM | megatron/core/ssm/triton_cache_manager.py | 69 | # use temp dir to be robust against program interruptions | COMMENT |
| LOW | …ore/tokenizers/text/parsers/qwen3_coder_tool_parser.py | 167 | # These map to vLLM types but we just use dictionaries for now | COMMENT |
| LOW⚡ | megatron/core/tensor_parallel/random.py | 202 | # already graphable, just return it. | COMMENT |
| LOW⚡ | megatron/core/tensor_parallel/random.py | 208 | # already non-graphable, just return it. | COMMENT |
| LOW | megatron/core/tensor_parallel/random.py | 121 | # if not using cuda graphs, just use the builtin pytorch function | COMMENT |
| MEDIUM⚡ | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1999 | # Set aggregate buckets by FSDP units, i.e. buckets pertaining to the same | COMMENT |
| MEDIUM | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 2632 | # to leverage NCCL UBR for high-precision gradient reduction with | COMMENT |
| MEDIUM | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 3632 | # for a seamless user experience and coverage for ZeRO-1 and ZeRO-2? | COMMENT |
| MEDIUM | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 5091 | # TODO(@cspades): Clean up this logic in conjunction with | COMMENT |
| LOW | …atron/core/distributed/fsdp/src/megatron_fsdp/utils.py | 237 | # if not using cuda graphs, just use the builtin pytorch function | COMMENT |
| LOW | megatron/core/transformer/transformer_config.py | 3388 | # so just set both if either is specified. | COMMENT |
| LOW | megatron/core/transformer/transformer_layer.py | 1534 | # elements in bias_chunks are the same for all chunks, so we can just use the first one | COMMENT |
| MEDIUM | megatron/core/transformer/mlp.py | 212 | # Weight resharding across TP sizes will have aforementioned problems. | COMMENT |
| MEDIUM | megatron/core/datasets/indexed_dataset.py | 6 | # Essentially re-written in entirety | COMMENT |
| MEDIUM | …e/models/common/embeddings/language_model_embedding.py | 149 | # the original tensor from being garbage collected. Clone to facilitate GC. | COMMENT |
| LOW | megatron/core/models/bert/bert_model.py | 204 | # For local layer spec we just use b1ss | COMMENT |
| LOW | megatron/core/extensions/transformer_engine.py | 1185 | # TODO should we ditch normalization config and just use spec to choose LayerNorm vs RMSNorm? | COMMENT |
| MEDIUM | …nce/data_parallel_inference_coordinator/coordinator.py | 582 | # Todo [Siddharth]: Make this more robust to handle invalid messages. | COMMENT |
| MEDIUM | megatron/training/checkpointing.py | 1832 | # in the serialized payload is a robust, unpickle-free signal that the | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …core/transformer/heterogeneous/heterogeneous_config.py | 177 | "n_heads_in_group": null, | STRING |
| HIGH | …er/experimental_attention_variant/dsa_cudnn_kernels.py | 1666 | """KL(target || predict) over selected top-K entries. | STRING |
| HIGH | …er/experimental_attention_variant/dsa_cudnn_kernels.py | 1718 | """KL(target || predict) over the **full** KV axis, averaged over ``(B, S_q)``. | STRING |
| HIGH⚡ | …mer/experimental_attention_variant/dsa_indexer_loss.py | 25 | """Return elementwise ``KL(target || predict)`` contributions.""" | STRING |
| HIGH⚡ | …mer/experimental_attention_variant/dsa_indexer_loss.py | 35 | """Return ``KL(target || predict)`` reduced over the key dimension.""" | STRING |
| HIGH⚡ | …mer/experimental_attention_variant/dsa_indexer_loss.py | 43 | """Return ``KL(target || predict)`` reduced over every dimension.""" | STRING |
| HIGH | …_attention_variant/csa_utils/fused_sparse_attention.py | 774 | """KL(target || predict) reduced over ``(B, S_q)`` and scaled by loss_coeff. | STRING |
| HIGH | …_attention_variant/csa_utils/fused_sparse_attention.py | 875 | """KL(target || predict) over the **full** KV axis, averaged over ``(B, S_q)``. | STRING |
| HIGH | …mer/experimental_attention_variant/ops/tilelang_dsa.py | 383 | """Compute KL(target || index) sum for one [s_chunk, topk] chunk.""" | STRING |
| HIGH | …e/pipeline_parallel/fine_grained_activation_offload.py | 590 | self.push(chunk) | CODE |
| HIGH | …e/pipeline_parallel/fine_grained_activation_offload.py | 798 | self.push(cur_chunk) | CODE |
| HIGH | megatron/core/allocator/vmm_symm_allocator.py | 88 | if (result == CUDA_SUCCESS && supported) { | CODE |
| HIGH | megatron/core/allocator/vmm_symm_allocator.py | 134 | if (create_result != CUDA_SUCCESS && fabric_added) { | CODE |
| HIGH⚡ | megatron/core/inference/inference_request.py | 489 | Each block hash is computed as SHA-256(parent_digest || block_bytes), where | STRING |
| HIGH | megatron/core/inference/unified_memory.py | 113 | if (device != prev_device && device >= 0) cudaSetDevice(device); | CODE |
| HIGH | megatron/core/inference/unified_memory.py | 149 | if (device != prev_device && prev_device >= 0) cudaSetDevice(prev_device); | CODE |
| HIGH | megatron/core/inference/moe/activations.py | 259 | [M, ffn_hidden] BF16 (rows >= n_rows undefined). | STRING |
| HIGH | megatron/core/inference/moe/vllm_fused_moe.py | 668 | `input` are undefined). | STRING |
| HIGH | megatron/training/arguments.py | 3551 | "n_heads_in_group": null, | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/run_dynamic_text_generation_server.py | 168 | _defaults += ["--language-model-type", "placeholder"] | CODE |
| LOW⚡ | tests/unit_tests/transformer/test_full_cuda_graph.py | 151 | dummy_data = torch.ones(1, 4) | CODE |
| LOW⚡ | tests/unit_tests/transformer/test_full_cuda_graph.py | 156 | return model(dummy_data), loss_func | CODE |
| LOW⚡ | tests/unit_tests/pipeline_parallel/test_schedules.py | 593 | dummy_data = torch.ones(1, 4) | CODE |
| LOW⚡ | tests/unit_tests/pipeline_parallel/test_schedules.py | 598 | return model(dummy_data), loss_func | CODE |
| LOW | …sts/unit_tests/dist_checkpointing/test_safe_globals.py | 31 | dummy_obj = Namespace(dummy_value=0) | CODE |
| LOW | …s/inference/test_dynamic_text_generation_server_cli.py | 24 | ["--language-model-type", "placeholder", "--tokenizer-prompt-format", "mistral"], | CODE |
| LOW⚡ | tests/unit_tests/rl/test_rl_utils.py | 1962 | # Adapter stamps pass through; bare empties are inferred to 'placeholder'. | COMMENT |
| LOW⚡ | tests/unit_tests/rl/test_rl_utils.py | 1964 | ["ok", "ok", "placeholder"], | CODE |
| LOW⚡ | tests/unit_tests/rl/test_rl_utils.py | 1965 | ["masked", "placeholder", "placeholder"], | CODE |
| LOW | tests/unit_tests/rl/test_rl_utils.py | 1726 | group.append("placeholder") | CODE |
| LOW | tests/unit_tests/rl/test_rl_utils.py | 1727 | rollout_statuses.append(["placeholder", "placeholder"]) | CODE |
| LOW | tests/unit_tests/rl/test_rl_utils.py | 1883 | rollout_statuses=[["placeholder", "placeholder"]], | CODE |
| LOW | tests/unit_tests/rl/test_rl_utils.py | 1919 | problem_id="placeholder", | CODE |
| LOW | examples/multimodal/text_generation_mistral_clip.sh | 7 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | examples/multimodal/text_generation_mistral_clip.sh | 8 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | …imodal/nvlm/run_text_generation_yi_34b_internvit_6b.sh | 7 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | …imodal/nvlm/run_text_generation_yi_34b_internvit_6b.sh | 8 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | …ultimodal/nvlm/run_text_generation_qwen25_7b_siglip.sh | 8 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | …ultimodal/nvlm/run_text_generation_qwen25_7b_siglip.sh | 9 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | …al/nvlm/run_text_generation_qwen20_72b_internvit_6b.sh | 8 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | …al/nvlm/run_text_generation_qwen20_72b_internvit_6b.sh | 9 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | …/nvlm/run_text_generation_qwen25_7b_internvit_video.sh | 8 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | …/nvlm/run_text_generation_qwen25_7b_internvit_video.sh | 9 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | …al/llama_3p1_nemotron_nano_vl_8b_v1/text_generation.sh | 7 | INPUT_IMAGE_PATH="placeholder" | CODE |
| LOW | …al/llama_3p1_nemotron_nano_vl_8b_v1/text_generation.sh | 8 | GROUNDTRUTH_PATH="placeholder" | CODE |
| LOW | megatron/rl/rl_utils.py | 1481 | rollout_status = 'placeholder' | CODE |
| LOW | megatron/rl/types/rollout.py | 12 | KNOWN_ROLLOUT_STATUSES = ('ok', 'placeholder', 'masked', 'graded') | CODE |
| LOW | megatron/rl/agent/rollout_pipeline.py | 510 | return rollout.is_placeholder and rollout.rollout_status in ('ok', 'placeholder') | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/functional_tests/shell_test_utils/run_ci_test.sh | 369 | # orchestration's resumable retrigger. | COMMENT |
| MEDIUM⚡ | megatron/core/ssm/ssm_inference.py | 188 | # Shared orchestration. | COMMENT |
| MEDIUM | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 1901 | # We may share the embedding model weight and the final output layer, | COMMENT |
| MEDIUM | megatron/core/inference/apis/async_llm.py | 239 | # SIGINT or out-of-band orchestration). | COMMENT |
| MEDIUM | megatron/core/inference/apis/llm.py | 232 | # SIGINT or out-of-band orchestration). | COMMENT |
| MEDIUM | …t_generation_controllers/text_generation_controller.py | 3453 | # Pick the context window that we need to pass through the network. | COMMENT |
| MEDIUM | megatron/rl/rl_utils.py | 1413 | # A turn must never exceed the model's context window. | COMMENT |
| MEDIUM | megatron/rl/inference/megatron.py | 114 | # detokenizing prompt tokens, especially expensive for long prompts in agentic RL. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tools/checkpoint/loader_llava.py | 52 | CODE | |
| MEDIUM | …nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py | 183 | CODE | |
| MEDIUM | …nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py | 184 | CODE | |
| MEDIUM | …nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py | 185 | CODE | |
| MEDIUM | …nce_tp1_pp1_583m_cuda_graphs_validation/cuda_graphs.py | 186 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/bisect.sh | 5 | # Usage: | COMMENT |
| LOW | tools/common_pile_dataset/setup_common_pile_dataset.sh | 9 | # Usage: | COMMENT |
| LOW | …unctional_tests/shell_test_utils/run_batch_ci_tests.sh | 5 | # Usage: | COMMENT |
| LOW | …nference/advanced/run_prefix_cache_lru_resume_repro.sh | 12 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .github/workflows/claude-copy-to-main.yml | 203 | echo "🤖 **This PR was auto-generated by Claude** via the Claude copy workflow." | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …/inference/test_data_parallel_inference_coordinator.py | 245 | self.ep_consensus_interval = 1 # TODO: add tests for interval > 1 (e.g. verify pausing is delayed by at most N | CODE |
| LOW | …ibuted/fsdp/src/megatron_fsdp/param_and_grad_buffer.py | 5443 | # TODO: Add validation checks for the legality of DTensor. | COMMENT |
| LOW | megatron/core/transformer/multi_token_prediction.py | 2063 | # TODO: implement block-based recompute for MTP | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | megatron/core/process_groups_config.py | 818 | """Check if this rank has a language model. | STRING |
| MEDIUM | megatron/core/process_groups_config.py | 821 | True if this rank has a language model, False otherwise. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …s/unit_tests/training/config/test_instantiate_utils.py | 52 | def test_function(arg1=None, arg2=None, **kwargs): | CODE |
| LOW | tests/unit_tests/training/config/test_yaml_utils.py | 45 | def test_function(): | CODE |
| LOW | megatron/core/utils.py | 2972 | def my_function(): | STRING |