Repository Analysis

pytorch/torchtitan

A PyTorch native platform for training generative AI models

20.9 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of pytorch/torchtitan, a Python project with 5,532 GitHub stars. SynthScan v2.0 examined 133,971 lines of code across 505 source files, recording 2325 pattern matches distributed across 19 syntactic categories. The overall adjusted score of 20.9 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

20.9
Adjusted Score
20.9
Raw Score
100%
Time Factor
2026-07-14
Last Push
5.5K
Stars
Python
Language
134.0K
Lines of Code
505
Files
2.3K
Pattern Hits
2026-07-14
Scan Date
0.06
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 28MEDIUM 173LOW 2124

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 2325 distinct pattern matches across 19 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers1312 hits · 1219 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/test_fused_swiglu.py55 def test_saves_in_stock_layout(self):CODE
LOWtests/unit_tests/test_fused_swiglu.py64 def test_fused_checkpoint_loads_into_stock(self):CODE
LOWtests/unit_tests/test_fused_swiglu.py76 def test_stock_checkpoint_loads_into_fused(self):CODE
LOWtests/unit_tests/test_fused_swiglu.py106 def test_strict_load_reports_missing(self):CODE
LOWtests/unit_tests/test_fused_swiglu.py114 def test_hf_adapter_roundtrip(self):CODE
LOWtests/unit_tests/test_dataloader.py56 def test_dataloader_yields_correct_batches(self):CODE
LOWtests/unit_tests/test_dataloader.py87 def test_load_state_dict_missing_rank_warning_includes_rank_id(self):CODE
LOWtests/unit_tests/test_dataloader.py105 def test_validate_kwargs_rejects_invalid_kwargs(self):CODE
LOWtests/unit_tests/test_dataloader.py120 def test_config_batch_size_overwritten_by_explicit_batch_size(self):CODE
LOWtests/unit_tests/test_dataloader.py144 def test_build_dataloader_with_trainer_config(self):CODE
LOWtests/unit_tests/test_dataloader.py165 def test_positions_matching_sequences(self):CODE
LOWtests/unit_tests/test_dataloader.py218 def test_rejects_empty_sources(self):CODE
LOWtests/unit_tests/test_dataloader.py223 def test_rejects_mixed_infinite(self):CODE
LOWtests/unit_tests/test_dataloader.py234 def test_construction_batch_size_and_num_workers(self):CODE
LOWtests/unit_tests/test_dataloader.py248 def test_yields_input_and_positions_keys(self):CODE
LOWtests/unit_tests/test_dataloader.py265 def test_single_source_equivalent_to_huggingfacetextdataloader(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py36 def test_inductor_backend_left_unchanged(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py41 def test_aot_eager_with_flex_scoops(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py46 def test_aot_eager_without_flex_left_unchanged(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py51 def test_other_backend_with_flex_raises(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py56 def test_region_annotation_follows_flag(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py90 def _triton_flex_kernel_count(backend) -> int:CODE
LOWtests/unit_tests/test_compile_regional_inductor.py118 def test_plain_aot_eager_decomposes_flex(self):CODE
LOWtests/unit_tests/test_compile_regional_inductor.py122 def test_regional_scoop_lowers_flex_to_triton(self):CODE
LOWtests/unit_tests/test_rmsnorm.py26 def test_config_build_without_fields_raises(self):CODE
LOWtests/unit_tests/test_rmsnorm.py49 def test_elementwise_affine_false(self):CODE
LOWtests/unit_tests/test_rmsnorm.py63 def test_normalized_shape_required_at_config_init(self):CODE
LOWtests/unit_tests/test_fused_qkv.py60 def test_fused_checkpoint_loads_into_stock(self):CODE
LOWtests/unit_tests/test_fused_qkv.py79 def test_stock_checkpoint_loads_into_fused(self):CODE
LOWtests/unit_tests/test_fused_qkv.py98 def test_hf_adapter_roundtrip(self):CODE
LOWtests/unit_tests/test_fused_qkv.py139 def test_forward_outputs_are_contiguous_and_correct(self):CODE
LOWtests/unit_tests/test_fused_qkv.py161 def test_raw_pointer_read_needs_contiguous(self):CODE
LOWtests/unit_tests/test_gpt_oss_moe.py20def test_swiglu_matches_reference_formula_and_gradients():CODE
LOWtests/unit_tests/test_tokenizer.py334 def test_download_and_build_tokenizer(self, test_repo_id):CODE
LOWtests/unit_tests/test_tokenizer.py397 def test_apply_chat_template_renders_chatml(self):CODE
LOWtests/unit_tests/test_tokenizer.py407 def test_apply_chat_template_raises_when_no_template(self):CODE
LOWtests/unit_tests/test_tokenizer.py412 def test_raise_exception_available_in_template(self):CODE
LOWtests/unit_tests/test_tokenizer.py67 def test_works_without_tokenizer_config(self):CODE
LOWtests/unit_tests/test_tokenizer.py93 def test_same_bos_eos_token_is_inferred_for_both_ids(self):CODE
LOWtests/unit_tests/test_tokenizer.py462 def test_auto_loads_chat_template_from_config(self):CODE
LOWtests/unit_tests/test_tokenizer.py467 def test_no_chat_template_when_config_lacks_field(self):CODE
LOWtests/unit_tests/test_tokenizer.py489 def test_auto_loads_chat_template_from_jinja_file(self):CODE
LOWtests/unit_tests/test_tokenizer.py512 def test_jinja_file_takes_priority_over_inline(self):CODE
LOWtests/unit_tests/test_dataset_checkpointing.py71 def test_map_style_shuffle_on_reloop(self):CODE
LOWtests/unit_tests/test_dataset_checkpointing.py139 def test_interleaved_resumption(self):CODE
LOWtests/unit_tests/test_dataset_checkpointing.py171 def test_interleaved_resumption_mid_epoch(self):CODE
LOWtests/unit_tests/test_dataset_checkpointing.py211 def _build_interleaved_dataloader(self, batch_size, seq_len, world_size, rank):CODE
LOWtests/unit_tests/test_module.py310 def test_sequential_init_states(self):CODE
LOWtests/unit_tests/test_module.py317 def test_containers_are_module(self):CODE
LOWtests/unit_tests/test_module.py327 def test_param_init_on_instance(self):CODE
LOWtests/unit_tests/test_module.py335 def test_no_param_init_by_default(self):CODE
LOWtests/unit_tests/test_module.py341 def test_init_states_uses_config_param_init(self):CODE
LOWtests/unit_tests/test_module.py30 def test_default_init_states_no_param_init_raises(self):CODE
LOWtests/unit_tests/test_module.py42 def test_init_states_falls_back_to_reset_parameters(self):CODE
LOWtests/unit_tests/test_module.py57 def test_init_states_auto_recurses(self):CODE
LOWtests/unit_tests/test_module.py75 def test__init_self_buffers_called(self):CODE
LOWtests/unit_tests/test_module.py133 def test_module_hierarchy_is_flat(self):CODE
LOWtests/unit_tests/test_module.py149 def test_nn_module_init_called_once(self):CODE
LOWtests/unit_tests/test_module.py189 def test_init_states_calls_reset_parameters(self):CODE
LOWtests/unit_tests/test_module.py198 def test_init_states_uses_custom_param_init(self):CODE
1252 more matches not shown…
Decorative Section Separators127 hits · 436 pts
SeverityFileLineSnippetContext
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py109 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py111 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py119 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py121 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py129 # ------------------------------------------------------------------COMMENT
MEDIUMtests/unit_tests/test_tp_kv_heads_validation.py131 # ------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py86# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py88# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py319# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py321# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py510# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py512# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py649# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py651# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py705# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py707# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py755# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py757# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py1525# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py1527# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py47# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py49# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py298# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py300# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py544# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py546# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py790# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py792# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py990# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py992# ---------------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py1209 # ------------------------------------------------------------------COMMENT
MEDIUM…ts/unit_tests/observability/test_structured_logging.py1215 # ------------------------------------------------------------------COMMENT
MEDIUMscripts/loss_compare.py301# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py303# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py71# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py73# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py84# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py86# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py210# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py212# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py407# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py409# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py492# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py494# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py566# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py568# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py832# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py834# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py876# =============================================================================COMMENT
MEDIUMscripts/loss_compare.py878# =============================================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py47# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py49# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py76# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py78# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py230# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py232# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py263# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py265# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py322# ============================================================COMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py324# ============================================================COMMENT
67 more matches not shown…
Unused Imports236 hits · 226 pts
SeverityFileLineSnippetContext
LOWtorchtitan/overrides/helion_rope.py37CODE
LOWtorchtitan/distributed/spmd_types.py9CODE
LOWtorchtitan/distributed/spmd_types.py25CODE
LOWtorchtitan/distributed/fsdp.py13CODE
LOWtorchtitan/distributed/fsdp.py24CODE
LOWtorchtitan/distributed/__init__.py7CODE
LOWtorchtitan/distributed/utils.py7CODE
LOWtorchtitan/distributed/utils.py501CODE
LOWtorchtitan/distributed/parallel_dims.py7CODE
LOWtorchtitan/distributed/deepep/inference_override.py26CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py7CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py7CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py7CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py7CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py7CODE
LOWtorchtitan/config/__init__.py15CODE
LOWtorchtitan/config/__init__.py15CODE
LOWtorchtitan/config/__init__.py15CODE
LOWtorchtitan/config/__init__.py15CODE
LOWtorchtitan/config/__init__.py15CODE
LOWtorchtitan/config/__init__.py22CODE
LOWtorchtitan/config/__init__.py23CODE
LOWtorchtitan/config/__init__.py24CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/__init__.py25CODE
LOWtorchtitan/config/override.py32CODE
LOWtorchtitan/experiments/graph_trainer/precompile.py7CODE
LOWtorchtitan/experiments/graph_trainer/precompile.py96CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py14CODE
LOWtorchtitan/experiments/graph_trainer/ep_eager_chunk.py14CODE
LOWtorchtitan/experiments/graph_trainer/custom_codegen.py9CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py21CODE
LOWtorchtitan/experiments/graph_trainer/registry.py14CODE
LOWtorchtitan/experiments/graph_trainer/graph_utils.py11CODE
LOWtorchtitan/experiments/graph_trainer/memory_policy.py16CODE
LOWtorchtitan/experiments/graph_trainer/inductor_passes.py14CODE
LOWtorchtitan/experiments/graph_trainer/storage.py7CODE
LOW…tan/experiments/graph_trainer/ep_process_group_pass.py17CODE
LOW…eriments/graph_trainer/log_activation_memory_policy.py9CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py62CODE
LOWtorchtitan/experiments/graph_trainer/passes.py25CODE
LOWtorchtitan/experiments/graph_trainer/passes.py79CODE
LOWtorchtitan/experiments/graph_trainer/ep_overlap_pass.py55CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py110CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py113CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py116CODE
LOW…tan/experiments/graph_trainer/tests/test_precompile.py379CODE
LOW…n/experiments/graph_trainer/deepseek_v3/parallelize.py76CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py7CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py10CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py10CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py14CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py14CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py18CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py18CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py18CODE
176 more matches not shown…
Structural Annotation Overuse102 hits · 151 pts
SeverityFileLineSnippetContext
LOW.ci/docker/common/install_conda.sh52 # WARNING: This is a HACK from PyTorch core to be able to build PyTorch on 22.04.COMMENT
LOWtests/unit_tests/test_tokenizer.py344 # Step 1: Download tokenizer filesCOMMENT
LOWtests/unit_tests/test_tokenizer.py359 # Step 2: Load tokenizer using our functionCOMMENT
LOWtests/unit_tests/test_tokenizer.py365 # Step 3: Load tokenizer using official Tokenizer library (if available)COMMENT
LOWtests/unit_tests/test_tokenizer.py372 # Step 4: Load tokenizer using transformers AutoTokenizer (if available)COMMENT
LOWtests/unit_tests/test_tokenizer.py381 # Step 5: Compare underlying tokenizer attributes (only if official tokenizer is available)COMMENT
LOWtests/unit_tests/test_tokenizer.py387 # Step 6: Compare with transformers tokenizer if availableCOMMENT
LOWtests/unit_tests/test_lr_scheduler.py95 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py96 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py97 1.0, # Step 2: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py98 7.0 / 8.0, # Step 3: 7/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py99 6.0 / 8.0, # Step 4: 3/4 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py100 5.0 / 8.0, # Step 5: 5/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py101 4.0 / 8.0, # Step 6: 1/2 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py102 3.0 / 8.0, # Step 7: 3/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py103 2.0 / 8.0, # Step 8: 1/4 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py104 1.0 / 8.0, # Step 9: 1/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py138 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py139 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py140 1.0, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py141 1.0, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py142 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py143 1.0, # Step 5: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py144 0.8, # Step 6: Linear decay starts (80% of max LR)CODE
LOWtests/unit_tests/test_lr_scheduler.py145 0.6, # Step 7: 60% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py146 0.4, # Step 8: 40% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py147 0.2, # Step 9: 20% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py204 0.2, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py205 0.4, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py206 0.6, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py207 0.8, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py208 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py241 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py242 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py243 1.0, # Step 2: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py244 1.0, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py245 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py246 1.0, # Step 5: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py247 1.0, # Step 6: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py248 1.0, # Step 7: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py249 1.0, # Step 8: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py250 1.0, # Step 9: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py284 0.2, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py285 0.4, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py286 0.6, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py287 0.8, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py288 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py289 1.0, # Step 5: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py290 0.8, # Step 6: Linear decay starts (80% of max LR)CODE
LOWtests/unit_tests/test_lr_scheduler.py291 0.6, # Step 7: 60% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py292 0.4, # Step 8: 40% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py293 0.2, # Step 9: 20% of max LRCODE
LOWscripts/checkpoint_compat_test.py228 # Step 1: Reference run at load_commit (full training from scratch)COMMENT
LOWscripts/checkpoint_compat_test.py237 # Step 2: Save run at save_commit (partial training + checkpoint)COMMENT
LOWscripts/checkpoint_compat_test.py253 # Step 3: Resume run at load_commit (load checkpoint, train to end)COMMENT
LOWscripts/checkpoint_compat_test.py268 # Step 4: CompareCOMMENT
LOW.github/workflows/integration_test_8gpu_torchft.yaml32 # Step 1: Dynamically compute the matrix based on conditionsCOMMENT
LOW.github/workflows/integration_test_8gpu_torchft.yaml40 # Step 2: Use the dynamic matrix in the build-test jobCOMMENT
LOW.github/workflows/integration_test_8gpu_rl.yaml29 # Step 1: Dynamically compute the matrix based on conditionsCOMMENT
LOW.github/workflows/integration_test_8gpu_rl.yaml39 # Step 2: Use the dynamic matrix in the build-test jobCOMMENT
42 more matches not shown…
Verbosity Indicators91 hits · 131 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/test_tokenizer.py344 # Step 1: Download tokenizer filesCOMMENT
LOWtests/unit_tests/test_tokenizer.py359 # Step 2: Load tokenizer using our functionCOMMENT
LOWtests/unit_tests/test_tokenizer.py365 # Step 3: Load tokenizer using official Tokenizer library (if available)COMMENT
LOWtests/unit_tests/test_tokenizer.py372 # Step 4: Load tokenizer using transformers AutoTokenizer (if available)COMMENT
LOWtests/unit_tests/test_tokenizer.py381 # Step 5: Compare underlying tokenizer attributes (only if official tokenizer is available)COMMENT
LOWtests/unit_tests/test_tokenizer.py387 # Step 6: Compare with transformers tokenizer if availableCOMMENT
LOWtests/unit_tests/test_lr_scheduler.py95 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py96 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py97 1.0, # Step 2: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py98 7.0 / 8.0, # Step 3: 7/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py99 6.0 / 8.0, # Step 4: 3/4 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py100 5.0 / 8.0, # Step 5: 5/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py101 4.0 / 8.0, # Step 6: 1/2 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py102 3.0 / 8.0, # Step 7: 3/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py103 2.0 / 8.0, # Step 8: 1/4 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py104 1.0 / 8.0, # Step 9: 1/8 of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py138 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py139 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py140 1.0, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py141 1.0, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py142 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py143 1.0, # Step 5: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py144 0.8, # Step 6: Linear decay starts (80% of max LR)CODE
LOWtests/unit_tests/test_lr_scheduler.py145 0.6, # Step 7: 60% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py146 0.4, # Step 8: 40% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py147 0.2, # Step 9: 20% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py204 0.2, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py205 0.4, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py206 0.6, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py207 0.8, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py208 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py241 0.5, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py242 1.0, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py243 1.0, # Step 2: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py244 1.0, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py245 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py246 1.0, # Step 5: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py247 1.0, # Step 6: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py248 1.0, # Step 7: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py249 1.0, # Step 8: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py250 1.0, # Step 9: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py284 0.2, # Step 0: 50% of max LR (warmup)CODE
LOWtests/unit_tests/test_lr_scheduler.py285 0.4, # Step 1: 100% of max LR (warmup complete)CODE
LOWtests/unit_tests/test_lr_scheduler.py286 0.6, # Step 2: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py287 0.8, # Step 3: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py288 1.0, # Step 4: Stable phaseCODE
LOWtests/unit_tests/test_lr_scheduler.py289 1.0, # Step 5: We maunally added step of stable phase, to prevent LR from dropping to 0 at last stepCODE
LOWtests/unit_tests/test_lr_scheduler.py290 0.8, # Step 6: Linear decay starts (80% of max LR)CODE
LOWtests/unit_tests/test_lr_scheduler.py291 0.6, # Step 7: 60% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py292 0.4, # Step 8: 40% of max LRCODE
LOWtests/unit_tests/test_lr_scheduler.py293 0.2, # Step 9: 20% of max LRCODE
LOWscripts/checkpoint_compat_test.py228 # Step 1: Reference run at load_commit (full training from scratch)COMMENT
LOWscripts/checkpoint_compat_test.py237 # Step 2: Save run at save_commit (partial training + checkpoint)COMMENT
LOWscripts/checkpoint_compat_test.py253 # Step 3: Resume run at load_commit (load checkpoint, train to end)COMMENT
LOWscripts/checkpoint_compat_test.py268 # Step 4: CompareCOMMENT
LOW.github/workflows/integration_test_8gpu_torchft.yaml32 # Step 1: Dynamically compute the matrix based on conditionsCOMMENT
LOW.github/workflows/integration_test_8gpu_torchft.yaml40 # Step 2: Use the dynamic matrix in the build-test jobCOMMENT
LOW.github/workflows/integration_test_8gpu_rl.yaml29 # Step 1: Dynamically compute the matrix based on conditionsCOMMENT
LOW.github/workflows/integration_test_8gpu_rl.yaml39 # Step 2: Use the dynamic matrix in the build-test jobCOMMENT
LOW.github/workflows/integration_test_8gpu_models.yaml32 # Step 1: Dynamically compute the matrix based on conditionsCOMMENT
31 more matches not shown…
Deep Nesting125 hits · 116 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/test_dataloader.py165CODE
LOWtests/unit_tests/test_tokenizer.py117CODE
LOWtests/unit_tests/test_tokenizer.py467CODE
LOWtests/unit_tests/test_tokenizer.py489CODE
LOWtests/unit_tests/test_tokenizer.py512CODE
LOWtests/unit_tests/test_dataset_checkpointing.py35CODE
LOWtests/unit_tests/test_chat_dataset.py351CODE
LOWtests/unit_tests/test_chat_dataset.py502CODE
LOWtests/unit_tests/test_embedding.py110CODE
LOWtests/unit_tests/test_mm_dataset_checkpointing.py53CODE
LOWtests/unit_tests/test_loss.py287CODE
LOWtests/unit_tests/test_fsdp_moe_sharding.py59CODE
LOWtests/integration_tests/run_tests.py208CODE
LOWtests/integration_tests/features.py14CODE
LOWtests/assets/cc12m_test/pack_test_dataset.py13CODE
LOWscripts/download_hf_assets.py13CODE
LOW.github/scripts/todo_debt_tracker.py120CODE
LOW.github/scripts/todo_debt_tracker.py237CODE
LOW.github/scripts/todo_debt_tracker.py338CODE
LOWtorchtitan/trainer.py224CODE
LOWtorchtitan/trainer.py879CODE
LOWtorchtitan/trainer.py151CODE
LOWtorchtitan/trainer.py171CODE
LOWtorchtitan/tools/utils.py93CODE
LOWtorchtitan/distributed/fsdp.py81CODE
LOWtorchtitan/distributed/activation_checkpoint.py31CODE
LOWtorchtitan/distributed/activation_checkpoint.py72CODE
LOWtorchtitan/distributed/parallel_dims.py88CODE
LOWtorchtitan/distributed/context_parallel.py35CODE
LOWtorchtitan/distributed/pipeline_parallel.py315CODE
LOWtorchtitan/distributed/pipeline_parallel.py428CODE
LOWtorchtitan/config/configurable.py162CODE
LOWtorchtitan/config/configurable.py46CODE
LOWtorchtitan/config/configurable.py75CODE
LOWtorchtitan/config/configurable.py49CODE
LOWtorchtitan/config/manager.py48CODE
LOW…tan/experiments/transformers_modeling_backend/model.py477CODE
LOW…tan/experiments/transformers_modeling_backend/model.py520CODE
LOW…/experiments/transformers_modeling_backend/pipeline.py35CODE
LOW…/experiments/transformers_modeling_backend/pipeline.py144CODE
LOW…/experiments/transformers_modeling_backend/pipeline.py186CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py815CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py1008CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py426CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py466CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py1168CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py574CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py596CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py701CODE
LOWtorchtitan/experiments/graph_trainer/cpu_offload.py303CODE
LOWtorchtitan/experiments/graph_trainer/cpu_offload.py412CODE
LOWtorchtitan/experiments/graph_trainer/cudagraph.py328CODE
LOWtorchtitan/experiments/graph_trainer/cudagraph.py466CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_patterns.py74CODE
LOWtorchtitan/experiments/graph_trainer/dynamic_shapes.py94CODE
LOWtorchtitan/experiments/graph_trainer/jit_backend.py18CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py263CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py1439CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py1596CODE
LOWtorchtitan/experiments/graph_trainer/ep_overlap_pass.py572CODE
65 more matches not shown…
Modern Structural Boilerplate92 hits · 93 pts
SeverityFileLineSnippetContext
LOWtests/integration_tests/__init__.py10__all__ = [CODE
LOWtests/integration_tests/h100.py12logger = logging.getLogger(__name__)CODE
LOWtorchtitan/overrides/helion_rope.py69__all__ = ["HelionComplexRoPE", "HelionCosSinRoPE", "HelionRoPE"]CODE
LOWtorchtitan/overrides/fused_swiglu.py68__all__ = [CODE
LOWtorchtitan/distributed/spmd_types.py31__all__ = [CODE
LOWtorchtitan/distributed/__init__.py9__all__ = [CODE
LOWtorchtitan/distributed/utils.py39def set_spmd_backend(spmd_backend: str) -> None:CODE
LOWtorchtitan/distributed/utils.py291def set_batch_invariance(enable: bool) -> None:CODE
LOWtorchtitan/distributed/parallel_dims.py22__all__ = ["MeshAxisName", "ParallelDims", "SpmdLayout", "unfold_dp_axes"]CODE
LOWtorchtitan/distributed/pipeline_parallel.py37__all__ = ["pipeline_llm"]CODE
LOWtorchtitan/distributed/deepep/hybridep.py543__all__ = [CODE
LOWtorchtitan/distributed/minimal_async_ep/__init__.py15__all__ = [CODE
LOWtorchtitan/distributed/minimal_async_ep/api.py999__all__ = [CODE
LOWtorchtitan/config/configurable.py15logger = logging.getLogger(__name__)CODE
LOWtorchtitan/config/__init__.py34__all__ = [CODE
LOW…/experiments/transformers_modeling_backend/__init__.py15__all__ = [CODE
LOWtorchtitan/experiments/graph_trainer/custom_codegen.py418 def _setup_custom_codegen(self) -> None:CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py185def _set_direction_meta(node: fx.Node, *, is_backward: bool) -> None:CODE
LOWtorchtitan/experiments/graph_trainer/ep_overlap_pass.py159def _set_copy_non_blocking(node: fx.Node, non_blocking: bool) -> None:CODE
LOW…ents/graph_trainer/tests/test_bitwise_deterministic.py70def _set_deterministic(seed: int = SEED) -> None:CODE
LOW…xperiments/graph_trainer/tests/test_sac_peak_memory.py31def _set_deterministic() -> None:CODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_passes.py419 def _setup(self) -> None:CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/runner.py46__all__ = [CODE
LOW…chtitan/experiments/graph_trainer/graph_pp/__init__.py25__all__ = [CODE
LOWtorchtitan/experiments/torchft/__init__.py14__all__ = [CODE
LOWtorchtitan/experiments/torchft/optimizer.py19__all__ = ["TorchFTOptimizersContainer"]CODE
LOWtorchtitan/experiments/torchft/config/__init__.py10__all__ = [CODE
LOW…chtitan/experiments/torchft/tests/integration_tests.py16logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/torchft/diloco/__init__.py9__all__ = [CODE
LOWtorchtitan/experiments/forge/__init__.py9__all__ = ["ForgeEngine"]CODE
LOWtorchtitan/experiments/rl/controller.py141logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/renderer.py16logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/__init__.py54__all__ = [CODE
LOWtorchtitan/experiments/rl/train.py45logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/batch_invariance.py22logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/losses/__init__.py10__all__ = ["DAPOLoss", "GRPOLoss"]CODE
LOWtorchtitan/experiments/rl/tests/test_bitwise_parity.py86logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/tests/test_bitwise_parity.py177def _set_generator_determinism(debug) -> None:CODE
LOWtorchtitan/experiments/rl/models/attention.py35logger = logging.getLogger(__name__)CODE
LOW…titan/experiments/rl/observability/metrics/__init__.py14__all__ = [CODE
LOW…itan/experiments/rl/observability/metrics/processor.py30__all__ = [CODE
LOW…rchtitan/experiments/rl/observability/metrics/types.py17__all__ = [CODE
LOWtorchtitan/experiments/rl/rollout/rollouter.py33logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/rollout/__init__.py17__all__ = [CODE
LOWtorchtitan/experiments/rl/components/batcher.py26logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/rubrics/__init__.py9__all__ = ["RewardFn", "Rubric", "RubricOutput"]CODE
LOWtorchtitan/experiments/rl/examples/search_r1/rubric.py127__all__ = ["RewardExactMatch"]CODE
LOWtorchtitan/experiments/rl/examples/search_r1/env.py26logger = logging.getLogger(__name__)CODE
LOW…rchtitan/experiments/rl/examples/search_r1/__init__.py15__all__ = [CODE
LOW…htitan/experiments/rl/examples/alphabet_sort/rubric.py116__all__ = ["RewardAlphabetSort", "score_sorted_list"]STRING
LOW…itan/experiments/rl/examples/alphabet_sort/__init__.py17__all__ = [CODE
LOWtorchtitan/experiments/rl/actors/generator.py58logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/actors/generator.py917 def _set_determinism(debug: DebugConfig) -> None:CODE
LOWtorchtitan/experiments/rl/actors/trainer.py45logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/environment/token.py20logger = logging.getLogger(__name__)CODE
LOWtorchtitan/experiments/rl/environment/__init__.py14__all__ = [CODE
LOWtorchtitan/experiments/rl/routing/__init__.py23__all__ = [CODE
LOW…titan/experiments/rl/routing/inter_generator_router.py123 def _set_state(self, h: _GeneratorHandle, state: _GeneratorState) -> None:CODE
LOWtorchtitan/models/llama3/__init__.py37__all__ = [CODE
LOWtorchtitan/models/gpt_oss/__init__.py47__all__ = [CODE
32 more matches not shown…
Self-Referential Comments26 hits · 84 pts
SeverityFileLineSnippetContext
MEDIUMtests/unit_tests/test_set_determinism.py46 # Create a world_mesh mockCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py19 # Create a simple model with parametersCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py21 # Create an optimizerCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py28 # Create an optimizer containerCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py41 # Create a trainer config with the specified parametersCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py78 # Create a job config with 10 steps, 2 warmup steps, and linear decayCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py121 # Create a job config with 10 steps, 2 warmup steps, 3 stable steps, and 5 decay stepsCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py163 # Create a job config with a minimum learning rateCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py187 # Create a job config where warmup steps > training stepsCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py224 # Create a job config with 10 steps, 2 warmup steps, and no decay phaseCOMMENT
MEDIUMtests/unit_tests/test_lr_scheduler.py266 # Create a job config where warmup + decay steps > training stepsCOMMENT
MEDIUMtests/unit_tests/test_checkpoint.py589 # Create a fake model with freqs_cis and other parametersCOMMENT
MEDIUMtests/assets/cc12m_test/pack_test_dataset.py23 # Create a TarWriter object to write the dataset to a tar archiveCOMMENT
MEDIUMscripts/loss_compare.py319 # Create the output folderCOMMENT
MEDIUMscripts/loss_compare.py749 # Create a test caseCOMMENT
MEDIUM…ripts/checkpoint_conversion/numerical_tests_example.py151 # Define the set of outputs to test loss forCOMMENT
MEDIUM.github/scripts/update_version.sh6 # Create a variable named __version__ at the end of __init__.pyCOMMENT
MEDIUMtorchtitan/distributed/activation_checkpoint.py7# This file provides the util functions to apply activation checkpointing to the model.COMMENT
MEDIUMtorchtitan/distributed/pipeline_parallel.py447 # Create a set of modules to keep for faster lookupCOMMENT
MEDIUM…/experiments/transformers_modeling_backend/pipeline.py191 # Create a set of modules to keep for faster lookupCOMMENT
MEDIUM…chtitan/experiments/graph_trainer/tests/test_passes.py386 # Create a second PG to simulate expert-FSDP.COMMENT
MEDIUM…experiments/graph_trainer/tests/test_custom_codegen.py206 # Create main moduleCOMMENT
MEDIUMtorchtitan/experiments/torchft/manager.py199 # Create the outer optimizer based on the inner optimizer parameters.COMMENT
MEDIUMtorchtitan/experiments/torchft/diloco/utils.py48 # Create a set of modules to keep for faster lookupCOMMENT
MEDIUMtorchtitan/experiments/rl/environment/token.py226 # Create a new list to avoid mutating previous statesCOMMENT
MEDIUMtorchtitan/models/common/moe.py174 """This class implements token-choice routing. In token-choice top-K routing, each token isSTRING
Docstring Block Structure13 hits · 65 pts
SeverityFileLineSnippetContext
HIGHtorchtitan/distributed/utils.py562 Clip the gradient norm of an iterable of parameters. Gradient norm clipping requires computing the gradient noSTRING
HIGHtorchtitan/distributed/parallel_dims.py419Get a device mesh by dimension name(s), returning None if not enabled. Args: dims: Names of the mesSTRING
HIGHtorchtitan/distributed/parallel_dims.py482Get a device mesh by dimension name(s), raising if not available. Args: dims: Names of the mesh dimSTRING
HIGHtorchtitan/distributed/context_parallel.py168 Shard inputs and attention masks across the context parallel mesh. This function distributes input tensors acrSTRING
HIGHtorchtitan/distributed/pipeline_parallel.py538Create pipeline stages based on specified module names for each stage. Also used by Graph PP to split the model intSTRING
HIGH…/experiments/transformers_modeling_backend/pipeline.py151 This API creates pipeline stages based on specified module names for each stage. Some model restrictions incluSTRING
HIGH…itan/experiments/graph_trainer/graph_pp/split_di_dw.py156Split a backward graph into input-gradient and weight-gradient graphs. Contract: Input: backward(saveSTRING
HIGHtorchtitan/experiments/graph_trainer/graph_pp/runner.py714Register GraphPP action handlers on a runtime PP schedule. Args: schedule (_PipelineScheduleRuntime): RuntiSTRING
HIGH…htitan/experiments/graph_trainer/graph_pp/partition.py445Partition a post-pass GraphTrainer joint graph into GraphPP callables. The caller passes a flat ``minimal_fx_tracerSTRING
HIGH…ments/graph_trainer/graph_pp/split_fsdp_collectives.py95Split forward FSDP all-gather chains from a forward graph. Contract: unshard(param_shards_and_replicated_paraSTRING
HIGH…ments/graph_trainer/graph_pp/split_fsdp_collectives.py251Split backward FSDP/DDP/HSDP reduce-grad epilogues. Contract: bw_no_fsdp(original_backward_inputs) ->STRING
HIGH…/experiments/graph_trainer/graph_pp/graph_multiplex.py48Concatenate backward and forward graphs into one boxed GraphPP callable. Contract: ``OVERLAP_F_B`` schedule aSTRING
HIGHtorchtitan/experiments/torchft/diloco/utils.py19 This API creates fragments based on specified module names for each fragment. This method updates the model in STRING
Excessive Try-Catch Wrapping47 hits · 58 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/test_tp_kv_heads_validation.py38except Exception:CODE
LOWtests/unit_tests/test_tokenizer.py369 except Exception as e:CODE
LOWtests/unit_tests/test_tokenizer.py378 except Exception as e:CODE
LOWtests/unit_tests/test_checkpoint.py768 except Exception as e:CODE
LOWtests/integration_tests/run_tests.py261 except Exception as e:CODE
LOWtests/integration_tests/run_tests.py268 except Exception as e:CODE
MEDIUMscripts/loss_compare.py294 log_print(f"Error: Import file does not exist: {import_result}")CODE
MEDIUMscripts/loss_compare.py315 log_print(f"Error: Output folder '{output_folder}' already exists")CODE
MEDIUMscripts/loss_compare.py729 log_print("Error: baseline-only mode requires --import-result")CODE
MEDIUMscripts/loss_compare.py733 log_print("Error: No losses found in baseline")CODE
MEDIUMscripts/loss_compare.py737 log_print("Error: No losses found in test")CODE
MEDIUMscripts/loss_compare.py746 log_print("Error: No losses found in import file")CODE
MEDIUMscripts/loss_compare.py259 log_print("Error: All settings are identical")CODE
MEDIUMscripts/loss_compare.py272 log_print(f"Error: --steps must be a positive integer, got: {steps}")CODE
MEDIUMscripts/loss_compare.py277 log_print("Error: --import-result requires --assert-equal")CODE
LOW.github/scripts/todo_debt_tracker.py45 except Exception:CODE
LOW.github/scripts/todo_debt_tracker.py116 except Exception as e:CODE
LOW.github/scripts/todo_debt_tracker.py183 except Exception:CODE
LOW.github/scripts/todo_debt_tracker.py303 except Exception as e:CODE
MEDIUM.github/scripts/todo_debt_tracker.py304 print(f"Error reading {file_path}: {e}", file=sys.stderr)CODE
LOWtorchtitan/__init__.py11except Exception:CODE
LOWtorchtitan/train.py66 except Exception:CODE
LOW…tan/experiments/transformers_modeling_backend/model.py445 except Exception as e:CODE
LOWtorchtitan/experiments/graph_trainer/custom_codegen.py64 except Exception:CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py218 except Exception:CODE
LOWtorchtitan/experiments/graph_trainer/cpu_offload.py167 except Exception:CODE
LOWtorchtitan/experiments/graph_trainer/cpu_offload.py179 except Exception:CODE
LOW…eriments/graph_trainer/log_activation_memory_policy.py73 except Exception:CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py221 except Exception as exc:CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py241 except Exception as exc:CODE
LOWtorchtitan/experiments/forge/example_train.py440 except Exception:CODE
LOWtorchtitan/experiments/rl/controller.py388 except Exception:CODE
LOWtorchtitan/experiments/rl/controller.py408 except Exception:CODE
LOWtorchtitan/experiments/rl/controller.py414 except Exception:CODE
LOWtorchtitan/experiments/rl/controller.py902 except Exception:CODE
LOW…itan/experiments/rl/observability/metrics/processor.py212 except Exception:CODE
LOW…itan/experiments/rl/observability/metrics/processor.py223 except Exception:CODE
LOWtorchtitan/experiments/rl/rollout/rollouter.py300 except Exception:CODE
LOWtorchtitan/experiments/rl/examples/search_r1/env.py90 except Exception as exc:CODE
LOWtorchtitan/experiments/rl/actors/generator.py1119 except Exception as exc:CODE
LOWtorchtitan/experiments/rl/actors/generator.py1393 except Exception:CODE
MEDIUMtorchtitan/experiments/rl/scripts/loss_compare.py292 log_print("Error: no losses found in TensorBoard events")CODE
LOWtorchtitan/experiments/rl/environment/token.py165 except Exception:CODE
LOWtorchtitan/observability/structured_logger/README.md147 except Exception:CODE
LOWtorchtitan/components/metrics.py434 except Exception as e:CODE
LOWtorchtitan/hf_datasets/multimodal/utils/video.py93 except Exception as e:CODE
LOWtorchtitan/hf_datasets/multimodal/utils/image.py114 except Exception as e:CODE
Over-Commented Block53 hits · 53 pts
SeverityFileLineSnippetContext
LOWrun_train.sh1#!/usr/bin/bashCOMMENT
LOW.ci/docker/common/utils.sh1#!/bin/bashCOMMENT
LOW.ci/docker/common/install_deepep.sh1#!/bin/bashCOMMENT
LOWtests/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtests/unit_tests/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtests/unit_tests/observability/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtests/integration_tests/features.py341 ],COMMENT
LOW.github/workflows/integration_test_8gpu_rl.yaml101 # the E2E suite can't collide with the generator's distributed initCOMMENT
LOW.github/scripts/run_8xgpu_integration_tests.sh1#!/bin/bashCOMMENT
LOWtorchtitan/tools/utils.py121 return 756e12COMMENT
LOWtorchtitan/tools/utils.py161 # - #ops: 512COMMENT
LOWtorchtitan/distributed/utils.py181 torch.backends.cudnn.benchmark = FalseCOMMENT
LOWtorchtitan/distributed/deepep/deepep.py81#COMMENT
LOWtorchtitan/distributed/deepep/inference_override.py41) -> DeepEPTokenDispatcher.Config:COMMENT
LOW…n/experiments/transformers_modeling_backend/configs.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/experiments/graph_trainer/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOW…itan/experiments/graph_trainer/run_train_precompile.sh1#!/usr/bin/bashCOMMENT
LOWtorchtitan/experiments/graph_trainer/inductor_passes.py101 val = node.meta["val"]COMMENT
LOWtorchtitan/experiments/graph_trainer/simple_fsdp.py301 ),COMMENT
LOWtorchtitan/experiments/graph_trainer/tests/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOW…n/experiments/graph_trainer/tests/integration_tests.py181 # Uses the SDPA backend: the default FlexAttention + CP +COMMENT
LOW…n/experiments/graph_trainer/tests/integration_tests.py621 # ("expected size 2==2, stride 2048==1048576 at dim=0"), i.e. a badCOMMENT
LOW…titan/experiments/graph_trainer/tests/test_numerics.py641 importlib.util.find_spec("autoparallel"),COMMENT
LOWtorchtitan/experiments/torchft/tests/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/experiments/rl/controller.py561 max_num_seqs=max_num_seqs,COMMENT
LOWtorchtitan/experiments/rl/tests/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/experiments/rl/models/attention.py141 )COMMENT
LOWtorchtitan/experiments/rl/models/vllm_wrapper.py301 # spmd_types parameter init needs the current mesh to materializeCOMMENT
LOWtorchtitan/experiments/rl/observability/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/experiments/rl/components/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/experiments/rl/examples/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOW…periments/rl/examples/alphabet_sort/config_registry.py621 # FULL_AND_PIECEWISE: decode captured FULL (incl. the expand MoE), prefill breakable.COMMENT
LOWtorchtitan/experiments/rl/actors/generator.py821 engine_kwargs = dict(COMMENT
LOWtorchtitan/experiments/rl/actors/generator.py1241 )COMMENT
LOWtorchtitan/experiments/rl/scripts/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOW…titan/experiments/rl/routing/inter_generator_router.py221 # for now.COMMENT
LOWtorchtitan/models/flux/model/model.py81 # per-token linear params per block (excluding modulation, whichCOMMENT
LOWtorchtitan/models/common/attention.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/models/common/moe.py401 output (Partial) is redistributed to sp_layout. MoE.forward()COMMENT
LOWtorchtitan/models/common/token_dispatcher.py761 layout feeds the grouped-GEMM expert path directly (no permute). Combine isCOMMENT
LOWtorchtitan/observability/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/components/lr_scheduler.py221COMMENT
LOWtorchtitan/components/loss.py861 # Note: this is not safe if downstream accidentally runs tensor ops afterCOMMENT
LOWtorchtitan/components/quantization/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/hf_datasets/multimodal/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWtorchtitan/ops/__init__.py1# Copyright (c) Meta Platforms, Inc. and affiliates.COMMENT
LOWagent_tooling/numerics_debugging/activation_tracer.py81 on ``input_hashes`` / ``output_hash`` for comparison.COMMENT
LOWagent_tooling/numerics_debugging/activation_tracer.py201 # IndexingCOMMENT
LOWagent_tooling/numerics_debugging/activation_tracer.py561 record_ids=True,COMMENT
LOWagent_tooling/numerics_debugging/activation_tracer.py641 # through to the metadata block below.COMMENT
LOWagent_tooling/numerics_debugging/compare_numerics.py61 stats: dict[str, str] = field(default_factory=dict)COMMENT
LOWagent_tooling/numerics_debugging/compare_numerics.py181COMMENT
LOWagent_tooling/numerics_debugging/compare_numerics.py661 # log_tensor_hashes. Comma-separated lists of floats — weCOMMENT
Cross-File Repetition9 hits · 45 pts
SeverityFileLineSnippetContext
HIGHtests/integration_tests/features.py0key is the config file name and value is a list of overridedefinitions that is used to generate variations of integratioSTRING
HIGHtests/integration_tests/h100.py0key is the config file name and value is a list of overridedefinitions that is used to generate variations of integratioSTRING
HIGHtests/integration_tests/flux.py0key is the config file name and value is a list of overridedefinitions that is used to generate variations of integratioSTRING
HIGH…ansformers_modeling_backend/tests/integration_tests.py0key is the config file name and value is a list of overridedefinitions that is used to generate variations of integratioSTRING
HIGH…chtitan/experiments/torchft/tests/integration_tests.py0key is the config file name and value is a list of overridedefinitions that is used to generate variations of integratioSTRING
HIGH…periments/transformers_modeling_backend/parallelize.py0apply tensor parallelism, activation checkpointing, torch.compile, and data parallelism to the qwen3.5 model. note: the STRING
HIGH…htitan/experiments/graph_trainer/llama3/parallelize.py0apply tensor parallelism, activation checkpointing, torch.compile, and data parallelism to the qwen3.5 model. note: the STRING
HIGHtorchtitan/models/llama3/parallelize.py0apply tensor parallelism, activation checkpointing, torch.compile, and data parallelism to the qwen3.5 model. note: the STRING
HIGHtorchtitan/models/qwen3_5/parallelize.py0apply tensor parallelism, activation checkpointing, torch.compile, and data parallelism to the qwen3.5 model. note: the STRING
Fake / Example Data50 hits · 44 pts
SeverityFileLineSnippetContext
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py91 for placeholder in gm.graph.find_nodes(op="placeholder"):CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py130 if n.all_input_nodes[0].op == "placeholder":CODE
LOWtorchtitan/experiments/graph_trainer/fsdp_passes.py1120 if n in visited or n.op in ("placeholder", "get_attr"):CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py312 or node.op != "placeholder"CODE
LOWtorchtitan/experiments/graph_trainer/ep_pass_utils.py329 if node.op != "placeholder" or (val := tensor_meta(node)) is None:CODE
LOWtorchtitan/experiments/graph_trainer/memory_policy.py64 for node in gm.graph.find_nodes(op="placeholder"):CODE
LOWtorchtitan/experiments/graph_trainer/inductor_passes.py100 if node.op == "placeholder" and "val" in node.meta:CODE
LOWtorchtitan/experiments/graph_trainer/inductor_passes.py272 if node.op in ("placeholder", "output"):CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py573 if node.op != "placeholder":CODE
LOWtorchtitan/experiments/graph_trainer/ep_chunk_pass.py2313 for idx, node in enumerate(n for n in gm.graph.nodes if n.op == "placeholder")CODE
LOWtorchtitan/experiments/graph_trainer/ep_overlap_pass.py81_GRAPH_BOUNDARY_OPS = {"placeholder", "get_attr"}CODE
LOWtorchtitan/experiments/graph_trainer/debug_utils.py34 if node.op == "placeholder":CODE
LOW…n/experiments/graph_trainer/tests/test_trace_module.py89 if node.op != "placeholder":CODE
LOW…n/experiments/graph_trainer/tests/test_trace_module.py172 if node.op == "placeholder"CODE
LOW…n/experiments/graph_trainer/tests/test_trace_module.py431 if node.op == "placeholder"CODE
LOW…n/experiments/graph_trainer/tests/test_trace_module.py455 if node.op == "placeholder"CODE
LOW…n/experiments/graph_trainer/tests/test_trace_module.py476 if node.op == "placeholder"CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py2048 self.assertEqual(node.args[0].op, "placeholder")CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5421 if node.op == "placeholder":CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5528 if node.op == "placeholder":CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5547 self.assertEqual(node.args[0].op, "placeholder")CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5625 self.assertEqual(relu_node.args[0].op, "placeholder")CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5724 if node.op == "placeholder":CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5824 if node.op == "placeholder":CODE
LOW…chtitan/experiments/graph_trainer/tests/test_passes.py5865 if node.op == "placeholder":CODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_runner.py479 for node in gm.graph.find_nodes(op="placeholder"):CODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_runner.py636 bw_gm.graph.find_nodes(op="placeholder")[0].meta["val"] = bw_fakeCODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_runner.py637 fw_gm.graph.find_nodes(op="placeholder")[0].meta["val"] = fw_fakeCODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_runner.py652 placeholders = multiplexed.graph.find_nodes(op="placeholder")CODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_runner.py696 fw_gm.graph.find_nodes(op="placeholder")[0].meta["val"] = fw_fakeCODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_passes.py658 sharded_param = gm.graph.find_nodes(op="placeholder")[0]CODE
LOW…xperiments/graph_trainer/tests/test_graph_pp_passes.py533 return tuple(node.name for node in gm.graph.find_nodes(op="placeholder"))CODE
LOW…itan/experiments/graph_trainer/graph_pp/split_di_dw.py200 for placeholder in list(bw_gm.graph.find_nodes(op="placeholder")):CODE
LOW…itan/experiments/graph_trainer/graph_pp/split_di_dw.py215 placeholders = list(bw_gm.graph.find_nodes(op="placeholder"))CODE
LOW…htitan/experiments/graph_trainer/graph_pp/partition.py197 for node in gm.graph.find_nodes(op="placeholder"):CODE
LOW…htitan/experiments/graph_trainer/graph_pp/partition.py320 if mutated_base.op != "placeholder" and mutated_base in backward_nodes:CODE
LOW…htitan/experiments/graph_trainer/graph_pp/partition.py348 and node.op == "placeholder"CODE
LOW…htitan/experiments/graph_trainer/graph_pp/partition.py475 placeholders = list(joint.graph.find_nodes(op="placeholder"))CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py289 return tuple(node.name for node in gm.graph.find_nodes(op="placeholder"))CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py322 return {node for node in node_closure(values) if node.op == "placeholder"}CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py386 if node.op != "placeholder":CODE
LOWtorchtitan/experiments/graph_trainer/graph_pp/utils.py399 for node in gm.graph.find_nodes(op="placeholder"):CODE
LOW…an/experiments/graph_trainer/graph_pp/graph_builder.py384 placeholders = self.modules.fw.graph.find_nodes(op="placeholder")CODE
LOW…an/experiments/graph_trainer/graph_pp/graph_builder.py750 if node.op == "placeholder":CODE
LOW…ments/graph_trainer/graph_pp/split_fsdp_collectives.py132 placeholders = graph.find_nodes(op="placeholder")CODE
LOW…ments/graph_trainer/graph_pp/split_fsdp_collectives.py288 placeholders = graph.find_nodes(op="placeholder")CODE
LOW…/experiments/graph_trainer/graph_pp/graph_multiplex.py104 fw_placeholders = multiplexed_gm.graph.find_nodes(op="placeholder")CODE
LOW…/experiments/graph_trainer/graph_pp/graph_multiplex.py109 for node in bw_gm.graph.find_nodes(op="placeholder"):CODE
LOW…/experiments/graph_trainer/graph_pp/graph_multiplex.py121 (node for node in multiplexed_gm.graph.nodes if node.op != "placeholder"),CODE
LOW…/experiments/graph_trainer/graph_pp/graph_multiplex.py128 bw_nodes = dropwhile(lambda node: node.op == "placeholder", bw_nodes)CODE
Redundant / Tautological Comments18 hits · 25 pts
SeverityFileLineSnippetContext
LOWtests/unit_tests/test_checkpoint.py96 # Check if this is async_with_pinned_mem mode by looking for async_stager parameterCOMMENT
LOWtests/unit_tests/test_checkpoint.py530 cfg.interval = 10 # Set interval to 10 so step 1 wouldn't normally trigger saveCODE
LOWtests/integration_tests/flux.py124 # Check if we have enough GPUsCOMMENT
LOWtests/assets/cc12m_test/pack_test_dataset.py38 # Write the file and its metadata to the TarWriterCOMMENT
LOWscripts/loss_compare.py313 # Check if output folder already existsCOMMENT
LOWscripts/loss_compare.py683 # Check if losses were extracted successfullyCOMMENT
LOWscripts/loss_compare.py1113 # Check if git working directory is clean before switching commitsCOMMENT
LOWtorchtitan/config/manager.py221 # Check if we are using tokenizer.model, if so then we need to alert users to redownload the tokenizerCOMMENT
LOW…chtitan/experiments/torchft/tests/integration_tests.py105 # Check if we have enough GPUsCOMMENT
LOWtorchtitan/experiments/rl/generate.py237 # Print resultsCOMMENT
LOWtorchtitan/experiments/rl/actors/generator.py1007 # Add the request to the queue; the engine loop will admit + process it.COMMENT
LOWtorchtitan/experiments/rl/actors/trainer.py275 # Check if seq_length passed the max_seq_lenCOMMENT
LOWtorchtitan/models/flux/validate.py139 # Set model to eval modeCOMMENT
LOWtorchtitan/models/flux/model/state_dict_adapter.py38 # Check if safetensors index file existsCOMMENT
LOWtorchtitan/components/metrics.py219 # Check if the loss-visible rank is included in LOG_RANK environment variableCOMMENT
LOWtorchtitan/components/metrics.py386 # Check if any logging backend is enabledCOMMENT
LOWtorchtitan/components/loss.py654 # Check if it's training model or validation modeCOMMENT
LOWtorchtitan/components/validate.py221 # Set model to eval modeCOMMENT
AI Slop Vocabulary8 hits · 24 pts
SeverityFileLineSnippetContext
MEDIUMtests/unit_tests/test_tokenizer.py177 # 2. Compare vocabularies with more comprehensive samplingCOMMENT
MEDIUMtests/unit_tests/test_tokenizer.py202 # Test a random sample of tokens (more comprehensive than just common words)COMMENT
MEDIUMtests/unit_tests/test_helion_rope.py27# tests below stay robust if a sibling test (e.g. test_override.py) callsCOMMENT
MEDIUMtorchtitan/distributed/utils.py99 # is no robust way to distinguish the two and `full_tensor()` may result inCOMMENT
MEDIUMtorchtitan/experiments/rl/routing/strategies.py161 # easy to implement, but not robust for all scenarios. Revisit with anCOMMENT
MEDIUMtorchtitan/experiments/rl/routing/strategies.py162 # more robust approach.COMMENT
MEDIUMtorchtitan/models/flux/flux_datasets.py227 # TODO: Add support for robust data loading and error handling.COMMENT
MEDIUMtorchtitan/components/tokenizer.py371 # Process added_tokens_decoder (comprehensive special token definitions)COMMENT
Cross-Language Confusion5 hits · 24 pts
SeverityFileLineSnippetContext
HIGHscripts/loss_compare.py444 log_print(" - To commit: git add -A && git commit -m 'message'")CODE
HIGH…ripts/checkpoint_conversion/numerical_tests_qwen3_5.py53 """KL(softmax(b) || softmax(a)) — F.kl_div(log Q, P) computes KL(P || Q)."""STRING
HIGHagent_tooling/numerics_debugging/compare_numerics.py1016 const hidden = el.dataset.hideByCategory || el.dataset.hideByOpFilter;CODE
HIGHagent_tooling/numerics_debugging/compare_numerics.py1028let _opFilterTimer = null;CODE
HIGHagent_tooling/numerics_debugging/compare_numerics.py1037 let re = null;CODE
AI Structural Patterns9 hits · 8 pts
SeverityFileLineSnippetContext
LOW…tan/experiments/transformers_modeling_backend/model.py788CODE
LOWtorchtitan/experiments/graph_trainer/simple_fsdp.py258CODE
LOW…ents/graph_trainer/tests/test_bitwise_deterministic.py184CODE
LOW…experiments/graph_trainer/tests/_trainer_test_utils.py23CODE
LOWtorchtitan/models/llama3/model.py50CODE
LOWtorchtitan/models/gpt_oss/model.py157CODE
LOWtorchtitan/models/common/decoder.py284CODE
LOWtorchtitan/models/qwen3_5/model.py744CODE
LOWtorchtitan/hf_datasets/multimodal/mm_datasets.py301CODE
Modern AI Meta-Vocabulary1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUMtorchtitan/experiments/rl/examples/search_r1/README.md1# Search-R1: multi-turn retrieval-augmented GRPOCOMMENT
AI Response Leakage1 hit · 2 pts
SeverityFileLineSnippetContext
HIGHtorchtitan/experiments/rl/actors/generator.py634 # The loop now executes the decision. In this example: STEP.STRING