FareedKhan-dev/train-llm-from-scratch

Pattern Findings

The scanner identified 317 distinct pattern matches across 16 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Unused Imports135 hits · 135 pts

Severity	File	Line	Context
LOW	ui/metrics.py	3	CODE
LOW	ui/theme.py	3	CODE
LOW	ui/stages.py	7	CODE
LOW	ui/jobs.py	10	CODE
LOW	ui/app.py	7	CODE
LOW	ui/docs_render.py	11	CODE
LOW	ui/stage_page.py	7	CODE
LOW	ui/config_forms.py	9	CODE
LOW	ui/pages/8_Evaluate.py	3	CODE
LOW	ui/pages/9_Chat.py	3	CODE
LOW	ui/pages/1_Data.py	3	CODE
LOW	config/presets.py	27	CODE
LOW	config/post_training_config.py	20	CODE
LOW	config/loader.py	28	CODE
LOW	tests/test_inference.py	3	CODE
LOW	tests/test_post_training_losses.py	3	CODE
LOW	tests/conftest.py	12	CODE
LOW	tests/conftest.py	29	CODE
LOW	tests/test_classic_model.py	3	CODE
LOW	tests/test_student_track.py	8	CODE
LOW	tests/test_model_report.py	3	CODE
LOW	tests/test_tokenizer.py	3	CODE
LOW	tests/test_ddp_moe.py	3	CODE
LOW	tests/test_factory.py	3	CODE
LOW	tests/test_checkpoint_io.py	10	CODE
LOW	tests/test_optim.py	3	CODE
LOW	tests/test_preference_truncation.py	10	CODE
LOW	tests/test_checkpoint_resume.py	8	CODE
LOW	tests/test_type_checks.py	3	CODE
LOW	tests/test_modern_model.py	9	CODE
LOW	tests/test_config.py	3	CODE
LOW	tests/test_device.py	3	CODE
LOW	tests/test_sampling.py	3	CODE
LOW	tests/test_pipeline_e2e.py	12	CODE
LOW	tests/test_lora.py	3	CODE
LOW	docs/diagrams/make_plots.py	14	CODE
LOW	scripts/eval_post_training.py	16	CODE
LOW	scripts/benchmark.py	14	CODE
LOW	scripts/train_ppo.py	13	CODE
LOW	scripts/prepare_tiny_data.py	29	CODE
LOW	scripts/prepare_preference_data.py	14	CODE
LOW	scripts/train_sft.py	13	CODE
LOW	scripts/train_dpo.py	15	CODE
LOW	scripts/train_reward.py	13	CODE
LOW	scripts/chat.py	16	CODE
LOW	scripts/train_grpo.py	19	CODE
LOW	scripts/pretrain_base.py	21	CODE
LOW	scripts/generate_text.py	19	CODE
LOW	scripts/prepare_rl_prompts.py	12	CODE
LOW	scripts/prepare_pretrain_data.py	18	CODE
LOW	scripts/train_transformer.py	21	CODE
LOW	scripts/prepare_sft_data.py	17	CODE
LOW	scripts/model_report.py	20	CODE
LOW	data_loader/sft_dataset.py	9	CODE
LOW	data_loader/prompt_dataset.py	7	CODE
LOW	data_loader/preference_dataset.py	21	CODE
LOW	src/checkpoint.py	14	CODE
LOW	src/device.py	11	CODE
LOW	src/post_training/evaluation.py	9	CODE
LOW	src/post_training/chat_template.py	31	CODE
75 more matches not shown…

Hyper-Verbose Identifiers108 hits · 121 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_inference.py	30	def test_greedy_speculative_decoding_equals_greedy_target_decoding() -> None:	CODE
LOW	tests/test_inference.py	38	def test_speculative_decoding_respects_the_trained_window() -> None:	CODE
LOW	tests/test_inference.py	60	def test_a_perfect_draft_is_always_accepted() -> None:	CODE
LOW	tests/test_inference.py	68	def test_sampling_keeps_the_target_distribution_whatever_the_draft() -> None:	CODE
LOW	tests/test_inference.py	97	def test_quantized_models_give_nearly_the_same_logits(arch: str) -> None:	CODE
LOW	tests/test_post_training_losses.py	22	def test_dpo_matches_the_formula_and_label_smoothing_hedges() -> None:	CODE
LOW⚡	tests/test_post_training_losses.py	35	def test_ipo_is_zero_at_its_target_gap() -> None:	CODE
LOW⚡	tests/test_post_training_losses.py	45	def test_simpo_is_reference_free_and_length_normalized() -> None:	CODE
LOW⚡	tests/test_post_training_losses.py	55	def test_group_advantages_std_and_dr_grpo() -> None:	CODE
LOW⚡	tests/test_post_training_losses.py	63	def test_loss_aggregation_modes() -> None:	CODE
LOW⚡	tests/test_post_training_losses.py	71	def test_skipped_groups_do_not_shrink_the_update() -> None:	CODE
LOW	tests/test_post_training_losses.py	94	def test_grpo_default_is_unchanged_and_clip_higher_widens_the_range() -> None:	CODE
LOW	tests/test_post_training_losses.py	111	def test_gspo_uses_one_ratio_per_answer() -> None:	CODE
LOW	tests/test_classic_model.py	15	def test_forward_shapes_loss_and_causality() -> None:	CODE
LOW	tests/test_classic_model.py	26	def test_forward_embedding_works_with_several_blocks() -> None:	CODE
LOW	tests/test_student_track.py	64	def test_set_overrides_any_config_value(monkeypatch: pytest.MonkeyPatch, tiny_data: dict[str, str]) -> None:	CODE
LOW	tests/test_student_track.py	78	def test_missing_data_explains_how_to_make_it(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:	CODE
LOW	tests/test_student_track.py	84	def test_presets_are_consistent() -> None:	CODE
LOW	tests/test_model_report.py	20	def test_flops_match_the_model_formula(extra: dict) -> None:	CODE
LOW	tests/test_model_report.py	28	def test_the_meta_device_builds_big_models_without_memory() -> None:	CODE
LOW	tests/verify_data_and_eval.py	39	def verify_sft_mask_alignment():	CODE
LOW	tests/verify_data_and_eval.py	82	def verify_rl_prompts_and_gold():	CODE
LOW	tests/test_tokenizer.py	26	def test_round_trip_is_lossless(tok: BPETokenizer, text: str) -> None:	CODE
LOW	tests/test_tokenizer.py	30	def test_training_learns_frequent_pieces_first(tok: BPETokenizer) -> None:	CODE
LOW	tests/test_tokenizer.py	43	def test_training_is_deterministic() -> None:	CODE
LOW	tests/test_tokenizer.py	69	def test_fast_training_learns_the_same_merges_as_the_textbook_algorithm() -> None:	CODE
LOW	tests/test_tokenizer.py	82	def test_specs_round_trip_and_safe_decode(tok: BPETokenizer, tmp_path) -> None:	CODE
LOW	tests/test_tokenizer.py	94	def test_vocab_must_hold_the_bytes() -> None:	CODE
LOW	tests/test_ddp_moe.py	16	def test_ddp_wrap_lets_moe_experts_skip_a_step(tmp_path: Path) -> None:	CODE
LOW	tests/test_post_training_smoke.py	30	def test_forward_hidden_matches_forward():	CODE
LOW	tests/test_post_training_smoke.py	40	def test_compute_logprobs_matches_manual():	CODE
LOW	tests/test_post_training_smoke.py	53	def test_rollout_logprobs_consistent():	CODE
LOW	tests/test_post_training_smoke.py	68	def test_context_cap_enforced():	CODE
LOW	tests/test_post_training_smoke.py	76	def test_value_and_reward_heads():	CODE
LOW	tests/test_post_training_smoke.py	94	def test_frozen_copy_and_reductions():	CODE
LOW	tests/test_post_training_smoke.py	106	def test_chat_template_masking():	CODE
LOW	tests/test_post_training_smoke.py	130	def test_preference_truncation_preserves_responses():	CODE
LOW⚡	tests/test_factory.py	22	def test_build_model_picks_the_architecture() -> None:	CODE
LOW⚡	tests/test_factory.py	31	def test_cli_style_overrides_build_a_modern_model() -> None:	CODE
LOW⚡	tests/test_factory.py	38	def test_modern_checkpoints_round_trip(tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	42	def test_strip_wrapper_prefixes_handles_any_nesting() -> None:	CODE
LOW	tests/test_checkpoint_io.py	62	def test_unwrap_model_removes_compile_and_data_parallel(cfg: SFTConfig) -> None:	CODE
LOW	tests/test_checkpoint_io.py	69	def test_compiled_model_checkpoint_round_trip(cfg: SFTConfig, tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	82	def test_old_prefixed_checkpoints_still_load(cfg: SFTConfig, tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	93	def test_reward_checkpoint_loads_for_chat_and_as_reward_model(cfg: SFTConfig, tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	107	def test_legacy_checkpoint_dims_are_read_from_config(tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	116	def test_causal_masks_are_not_saved_and_old_checkpoints_with_them_still_load(tmp_path: str) -> None:	CODE
LOW	tests/test_checkpoint_io.py	129	def test_missing_parameters_fail_loudly(cfg: SFTConfig) -> None:	CODE
LOW	tests/test_checkpoint_io.py	136	def test_legacy_trainer_resumes_from_prefixed_checkpoint(tmp_path: str) -> None:	CODE
LOW	tests/test_optim.py	13	def test_newton_schulz_flattens_the_singular_values() -> None:	CODE
LOW	tests/test_optim.py	25	def test_muon_matches_pytorch_reference() -> None:	CODE
LOW	tests/test_optim.py	44	def test_muon_adamw_groups_match_torch_adamw() -> None:	CODE
LOW	tests/test_optim.py	58	def test_muon_groups_keep_embeddings_and_head_on_adamw() -> None:	CODE
LOW	tests/test_optim.py	70	def test_both_optimizers_train_a_tiny_model(name: str) -> None:	CODE
LOW	tests/test_optim.py	85	def test_schedules_hit_their_anchor_points() -> None:	CODE
LOW	tests/test_preference_truncation.py	22	def test_short_pairs_are_encoded_exactly_like_a_full_conversation() -> None:	CODE
LOW	tests/test_preference_truncation.py	31	def test_long_prompt_keeps_both_full_answers() -> None:	CODE
LOW	tests/test_preference_truncation.py	43	def test_long_prompt_and_long_answers_split_the_window() -> None:	CODE
LOW⚡	tests/test_preference_truncation.py	54	def test_late_difference_is_kept_by_shortening_the_prompt() -> None:	CODE
LOW⚡	tests/test_preference_truncation.py	61	def test_indistinguishable_pairs_do_not_crash() -> None:	CODE
48 more matches not shown…

Self-Referential Comments8 hits · 26 pts

Severity	File	Line	Snippet	Context
MEDIUM	scripts/data_preprocess.py	32	# Create an HDF5 file for output	COMMENT
MEDIUM	scripts/train_transformer.py	378	# Create a batch iterator for evaluation.	COMMENT
MEDIUM	scripts/train_transformer.py	654	# Define a window size for averaging recent losses in the training loop.	COMMENT
MEDIUM	scripts/train_transformer.py	661	# Create a batch iterator for the training data.	COMMENT
MEDIUM	scripts/train_transformer.py	675	# Create a progress bar to monitor training progress.	COMMENT
MEDIUM⚡	scripts/data_download.py	40	# Define the path for the validation file	COMMENT
MEDIUM	data_loader/data_loader.py	33	# Create an array of indices for examples and shuffle them for randomness	COMMENT
MEDIUM	data_loader/data_loader.py	68	# Create a dummy HDF5 file	COMMENT

Empty Error Handlers4 hits · 20 pts

Severity	File	Line	Snippet	Context
HIGH	ui/metrics.py	27	except json.JSONDecodeError:	CODE
HIGH	config/loader.py	120	except ValueError:	CODE
HIGH	tests/conftest.py	30	except ImportError: # beartype is a dev dependency; plain runs still work without it	CODE
HIGH	tests/test_checkpoint_resume.py	92	except RuntimeError:	CODE

Cross-Language Confusion5 hits · 20 pts

Severity	File	Line	Snippet	Context
HIGH	ui/config_forms.py	34	# str / str\|None / unknown -> text; empty string means "unset" (-> null) for optionals	COMMENT
HIGH	scripts/train_transformer.py	500	"""Split ``key=value``. Values are read as JSON (true, 2, 1e-3, null), anything else as text."""	STRING
HIGH	scripts/train_transformer.py	541	else "python scripts/data_download.py && python scripts/data_preprocess.py")	CODE
HIGH	data_loader/prompt_dataset.py	3	verifiable numeric answer, or null) and yields lists of sampled rows. Rows are sharded	STRING
HIGH	src/post_training/ppo.py	99	"""Mean approximate KL(old \|\| new) over response tokens (a PPO health metric)."""	STRING

Structural Annotation Overuse13 hits · 20 pts

Severity	File	Line	Snippet	Context
LOW	README.md	200	## Step 1: Preparing the Data	COMMENT
LOW	README.md	319	## Step 2: The Model, Built From Small Pieces	COMMENT
LOW	README.md	483	## Step 3: Pretraining the Base Model	COMMENT
LOW	README.md	580	## Step 4: Generating Text	COMMENT
LOW	README.md	615	## Step 5: Post-Training, Turning a Base Model Into an Assistant	COMMENT
LOW	README.md	759	## Step 6: Evaluation	COMMENT
LOW	README.md	778	## Step 7: Talking to the Model	COMMENT
LOW	docs/student/README.md	36	## Step 1: Install	COMMENT
LOW	docs/student/README.md	48	## Step 2: Prepare the data	COMMENT
LOW	docs/student/README.md	82	## Step 3: Train	COMMENT
LOW	docs/student/README.md	106	## Step 4: Generate	COMMENT
LOW	docs/foundations/bpe.md	45	## Step 1: split the text into chunks	COMMENT
LOW	docs/foundations/bpe.md	64	## Step 2: count pairs, merge, repeat	COMMENT

Deep Nesting12 hits · 12 pts

Severity	File	Line	Context
LOW	ui/metrics.py	17	CODE
LOW	scripts/data_preprocess.py	11	CODE
LOW	scripts/train_ppo.py	65	CODE
LOW	scripts/prepare_tiny_data.py	53	CODE
LOW	scripts/train_sft.py	71	CODE
LOW	scripts/train_dpo.py	94	CODE
LOW	scripts/train_reward.py	65	CODE
LOW	scripts/train_grpo.py	57	CODE
LOW	scripts/pretrain_base.py	70	CODE
LOW	scripts/prepare_pretrain_data.py	76	CODE
LOW	scripts/train_transformer.py	551	CODE
LOW	src/checkpoint.py	43	CODE

Excessive Try-Catch Wrapping9 hits · 12 pts

Severity	File	Line	Snippet	Context
LOW	ui/theme.py	59	except Exception: # noqa: BLE001	CODE
LOW	scripts/data_preprocess.py	73	except Exception as e:	CODE
MEDIUM	scripts/data_preprocess.py	75	print(f"An error occurred while processing line in {filename}: {e}")	CODE
MEDIUM	scripts/data_preprocess.py	99	print(f"Error: Training directory not found: {args.train_dir}")	CODE
MEDIUM	scripts/data_preprocess.py	102	print(f"Error: Validation directory not found: {args.val_dir}")	CODE
LOW	scripts/prepare_preference_data.py	86	except Exception as e: # noqa: BLE001	CODE
LOW	scripts/train_transformer.py	281	except Exception:	CODE
LOW	scripts/prepare_sft_data.py	89	except Exception as e: # noqa: BLE001	CODE
LOW	src/post_training/logging_utils.py	42	except Exception as e: # noqa: BLE001 - logging must never crash training	CODE

Modern Structural Boilerplate10 hits · 10 pts

Severity	File	Line	Snippet	Context
LOW	scripts/train_transformer.py	224	def set_optimizer_lr(optimizer: torch.optim.Optimizer, lr: float) -> None:	CODE
LOW	src/post_training/utils.py	43	def set_seed(seed: int) -> None:	CODE
LOW	src/post_training/rollout.py	34	__all__ = [	CODE
LOW	src/post_training/optim.py	20	__all__ = ["build_optimizer", "configure_optimizer", "cosine_lr", "lr_at", "set_lr"]	CODE
LOW	src/post_training/rewards/__init__.py	16	__all__ = [	CODE
LOW	src/tokenizer/__init__.py	76	__all__ = [	CODE
LOW	src/optim/__init__.py	38	__all__ = [	CODE
LOW	src/optim/schedules.py	78	def set_lr(optimizer: torch.optim.Optimizer, lr: float) -> None:	CODE
LOW	src/models/modern/__init__.py	13	__all__ = [	CODE
LOW	src/inference/__init__.py	5	__all__ = ["filter_logits", "sample_next_token"]	CODE

Redundant / Tautological Comments4 hits · 6 pts

Severity	File	Line	Snippet	Context
LOW	scripts/data_download.py	25	with open(file_name, 'wb') as f: # Open file for writing in binary mode	CODE
LOW⚡	scripts/data_download.py	42	if not os.path.exists(val_file_path): # Check if the validation file already exists	CODE
LOW⚡	scripts/data_download.py	52	if not os.path.exists(file_path): # Check if the file already exists	CODE
LOW	data_loader/data_loader.py	42	# Check if the current batch exceeds the number of available examples	COMMENT

Docstring Block Structure1 hit · 5 pts

Severity	File	Line	Snippet	Context
HIGH	config/loader.py	143	Resolve ``cfg_cls`` from ``base.json`` + the stage JSON + CLI overrides. Args: cfg_cls: the stage data	STRING

Decorative Section Separators1 hit · 3 pts

Severity	File	Line	Snippet	Context
MEDIUM	docs/diagrams/make_plots.py	90	# ---------------------------------------------------------------------------------------------	COMMENT

AI Structural Patterns3 hits · 3 pts

Severity	File	Line	Context
LOW	src/optim/muon.py	76	CODE
LOW	src/models/attention.py	61	CODE
LOW	src/inference/speculative.py	72	CODE

Modern AI Meta-Vocabulary1 hit · 2 pts

Severity	File	Line	Snippet	Context
MEDIUM	README.md	587	idx_cond = idx[:, -self.context_length:] # never look back further than the context window	CODE

Over-Commented Block2 hits · 2 pts

Severity	File	Line	Snippet	Context
LOW	scripts/run_posttraining.sh	1	#!/usr/bin/env bash	COMMENT
LOW	scripts/render_diagrams.sh	1	#!/usr/bin/env bash	COMMENT

Fake / Example Data1 hit · 1 pts

Severity	File	Line	Snippet	Context
LOW	data_loader/data_loader.py	70	dummy_data_path = "dummy_data.h5"	CODE

Maintenance and Governance Review

Risk Review · v3

What These Metrics Mean

Pattern-Signal History

Severity Breakdown

Directory Score Breakdown

Pattern Findings