Repository Analysis

rasbt/LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

8.0 Low AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of rasbt/LLMs-from-scratch, a Jupyter Notebook project with 99,046 GitHub stars. SynthScan v2.0 examined 57,987 lines of code across 273 source files, recording 323 pattern matches distributed across 13 syntactic categories. The overall adjusted score of 8.0 places this repository in the Low AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

8.0
Adjusted Score
8.0
Raw Score
100%
Time Factor
2026-07-11
Last Push
99.0K
Stars
Jupyter Notebook
Language
58.0K
Lines of Code
273
Files
323
Pattern Hits
2026-07-14
Scan Date
0.03
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 8MEDIUM 63LOW 252

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 323 distinct pattern matches across 13 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Self-Referential Comments45 hits · 108 pts
SeverityFileLineSnippetContext
MEDIUMch07/01_main-chapter-code/previous_chapters.py467 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch07/01_main-chapter-code/previous_chapters.py468 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch07/01_main-chapter-code/exercise_experiments.py286 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch07/01_main-chapter-code/exercise_experiments.py287 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch07/01_main-chapter-code/ollama_evaluate.py15 # Create the data payload as a dictionaryCOMMENT
MEDIUMch07/01_main-chapter-code/gpt_instruction_finetuning.py135 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch07/01_main-chapter-code/gpt_instruction_finetuning.py136 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch07/04_preference-tuning-with-dpo/previous_chapters.py468 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch07/04_preference-tuning-with-dpo/previous_chapters.py469 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch06/01_main-chapter-code/gpt_class_finetune.py230 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch06/01_main-chapter-code/gpt_class_finetune.py231 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch06/01_main-chapter-code/gpt_download.py111 # Define the block size for reading the fileSTRING
MEDIUMch06/02_bonus_additional-experiments/gpt_download.py111 # Define the block size for reading the fileSTRING
MEDIUM…_bonus_imdb-classification/download_prepare_dataset.py58 # Create a DataFrame for each file and add it to the listCOMMENT
MEDIUMch06/03_bonus_imdb-classification/gpt_download.py111 # Define the block size for reading the fileSTRING
MEDIUM…6/03_bonus_imdb-classification/train_sklearn_logreg.py65 # Create a dummy classifier with the strategy to predict the most frequent classCOMMENT
MEDIUM…nal-aws-sagemaker-notebook/cloudformation-template.yml59 # Create a startup script that will run in the backgroundCOMMENT
MEDIUM…nal-aws-sagemaker-notebook/cloudformation-template.yml104 # Create a flag file to indicate setup is completeCOMMENT
MEDIUMappendix-D/01_main-chapter-code/previous_chapters.py305 # Create a second x-axis for tokens seenCOMMENT
MEDIUMappendix-D/01_main-chapter-code/previous_chapters.py306 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMappendix-E/01_main-chapter-code/previous_chapters.py540 # Create a second x-axis for tokens seenCOMMENT
MEDIUMappendix-E/01_main-chapter-code/previous_chapters.py541 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMappendix-E/01_main-chapter-code/gpt_download.py111 # Define the block size for reading the fileSTRING
MEDIUMch05/16_qwen3.5/qwen3_5_transformers.py97 """This function is intended to align with the l2norm implementation in the FLA library."""STRING
MEDIUMch05/01_main-chapter-code/gpt_train.py122 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/01_main-chapter-code/gpt_train.py123 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch05/01_main-chapter-code/gpt_download.py111 # Define the block size for reading the fileSTRING
MEDIUMch05/01_main-chapter-code/gpt_generate.py77 # Define the block size for reading the fileCOMMENT
MEDIUMch05/07_gpt_to_llama/tests/tests_rope_and_parts.py113 # Create a module to store the imported functions and classesCOMMENT
MEDIUMch05/10_llm-training-speed/01_opt_single_gpu.py361 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/10_llm-training-speed/01_opt_single_gpu.py362 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch05/10_llm-training-speed/00_orig.py397 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/10_llm-training-speed/00_orig.py398 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py426 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py427 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch05/05_bonus_hparam_tuning/hparam_search.py18# Define a grid of hyperparameters to search overCOMMENT
MEDIUMch05/18_muon/gpt_train_muon.py166 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/18_muon/gpt_train_muon.py167 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMch05/18_muon/gpt_train.py122 # Create a second x-axis for tokens seenCOMMENT
MEDIUMch05/18_muon/gpt_train.py123 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMpkg/llms_from_scratch/ch06.py225 # Create a second x-axis for examples seenCOMMENT
MEDIUMpkg/llms_from_scratch/ch06.py226 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
MEDIUMpkg/llms_from_scratch/ch07.py214 # Create the data payload as a dictionaryCOMMENT
MEDIUMpkg/llms_from_scratch/ch05.py236 # Create a second x-axis for tokens seenCOMMENT
MEDIUMpkg/llms_from_scratch/ch05.py237 ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axisCODE
Hyper-Verbose Identifiers89 hits · 89 pts
SeverityFileLineSnippetContext
LOWch07/01_main-chapter-code/previous_chapters.py338def generate_and_print_sample(model, tokenizer, device, start_context):CODE
LOWch07/01_main-chapter-code/exercise_experiments.py190def custom_collate_with_masking_fn(CODE
LOWch07/02_dataset-utilities/find-near-duplicates.py76def find_print_and_remove_near_duplicates(json_data, remove_duplicates=False, threshold=0.75):CODE
LOWch07/04_preference-tuning-with-dpo/previous_chapters.py339def generate_and_print_sample(model, tokenizer, device, start_context):CODE
LOWch06/01_main-chapter-code/gpt_class_finetune.py26def download_and_unzip_spam_data(url, zip_path, extracted_path, data_file_path):CODE
LOW…_bonus_imdb-classification/download_prepare_dataset.py31def download_and_extract_dataset(dataset_url, target_file, directory):CODE
LOW…_bonus_imdb-classification/download_prepare_dataset.py51def load_dataset_to_dataframe(basepath="aclImdb", labels={"pos": 1, "neg": 0}):CODE
LOWappendix-D/01_main-chapter-code/previous_chapters.py282def generate_and_print_sample(model, tokenizer, device, start_context):CODE
LOWappendix-E/01_main-chapter-code/previous_chapters.py364def download_and_unzip_spam_data(url, zip_path, extracted_path, data_file_path):CODE
LOWch04/06_swa/gpt_with_kv_mha.py253def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/06_swa/tests.py12def test_cached_prefill_matches_uncached_swa():CODE
LOWch04/06_swa/tests.py37def test_swa_matches_base_model_when_window_equals_context():CODE
LOWch04/06_swa/plot_memory_estimates_swa.py57def calc_kv_bytes_total_mha_swa(CODE
LOWch04/06_swa/plot_memory_estimates_swa.py75def calc_kv_bytes_total_gqa_swa(CODE
LOWch04/06_swa/gpt_with_kv_swa.py294def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/07_moe/plot_memory_estimates_moe.py16def calc_moe_active_and_total(CODE
LOWch04/07_moe/plot_memory_estimates_moe.py42def plot_active_params_vs_experts(CODE
LOWch04/07_moe/gpt_with_kv_moe.py339def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/07_moe/memory_estimator_moe.py39def estimate_params_and_hidden(CODE
LOWch04/07_moe/gpt_with_kv_ffn.py279def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/04_gqa/gpt_with_kv_mha.py253def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/04_gqa/gpt_with_kv_gqa.py265def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/04_gqa/plot_memory_estimates_gqa.py23def plot_abs_kv_vs_context_multi_groups():CODE
LOWch04/09_dsa/gpt_with_kv_dsa.py389def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/09_dsa/test_dsa.py16def import_transformers_dsa_model():CODE
LOWch04/09_dsa/test_dsa.py89def test_indexer_matches_transformers_reference():CODE
LOWch04/09_dsa/test_dsa.py163def dense_attention_reference(attn, x):CODE
LOWch04/09_dsa/test_dsa.py181def test_topk_full_equals_dense():CODE
LOWch04/10_kv-sharing/gpt_with_kv_mha.py253def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/10_kv-sharing/gpt_with_kv_sharing.py270def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/10_kv-sharing/tests.py10def test_kv_sharing_matches_mha_when_all_layers_produce_kv():CODE
LOWch04/10_kv-sharing/tests.py55def test_only_producer_layers_store_kv_cache():CODE
LOWch04/10_kv-sharing/tests.py84def test_memory_estimator_counts_cached_layers():CODE
LOW…04/08_deltanet/plot_memory_estimates_gated_deltanet.py27def calc_kv_bytes_total_deltanet_no_conv(batch, emb_dim, n_layers, bytes_per_elem, n_heads):CODE
LOWch04/05_mla/plot_memory_estimates_mla.py33def plot_abs_kv_vs_context_multiple():CODE
LOWch04/05_mla/gpt_with_kv_mla.py261def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/05_mla/gpt_with_kv_mha.py253def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/03_kv-cache/gpt_with_kv_cache.py280def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/03_kv-cache/README.md173def generate_text_simple_cached(model, idx, max_new_tokens,CODE
LOWch04/03_kv-cache/gpt_with_kv_cache_optimized.py306def generate_text_simple_cached(model, idx, max_new_tokens, context_size=None, use_cache=True):CODE
LOWch04/03_kv-cache/tests.py32def test_gpt_model_equivalence_not_cached(ModelClass):CODE
LOWch04/03_kv-cache/tests.py66def test_gpt_model_equivalence_cached(ModelClass):CODE
LOWch04/03_kv-cache/tests.py113def test_context_overflow_bug():CODE
LOWch04/03_kv-cache/tests.py150def test_prefill_chunking_basic():CODE
LOWch02/05_bpe-from-scratch/tests.py11def import_definitions_from_notebook(fullname, names):CODE
LOWch02/05_bpe-from-scratch/tests.py185def test_no_eot_aliasing_and_disallowed_logic(imported_module, gpt2_files):CODE
LOWch02/05_bpe-from-scratch/tests.py214def test_newline_roundtrip_and_equivalence(imported_module, gpt2_files, text):CODE
LOWch02/05_bpe-from-scratch/tests.py234def test_space_newline_space_patterns(imported_module, gpt2_files):CODE
LOWch02/05_bpe-from-scratch/tests.py250def test_multiple_leading_spaces_roundtrip(imported_module, gpt2_files):CODE
LOWch05/16_qwen3.5/qwen3_5_transformers.py66def apply_mask_to_padding_states(hidden_states, attention_mask):CODE
LOWch05/16_qwen3.5/qwen3_5_transformers.py102def torch_chunk_gated_delta_rule(CODE
LOWch05/16_qwen3.5/qwen3_5_transformers.py182def torch_recurrent_gated_delta_rule(CODE
LOWch05/01_main-chapter-code/gpt_train.py61def generate_and_print_sample(model, tokenizer, device, start_context):CODE
LOWch05/15_tiny-aya/tests/test_tiny_aya_nb.py54def test_dummy_tiny_aya_forward(dummy_cfg_base, dummy_input, import_notebook_defs):CODE
LOWch05/15_tiny-aya/tests/test_tiny_aya_nb.py64def test_tiny_aya_base_equivalence_with_transformers(import_notebook_defs):CODE
LOWch05/15_tiny-aya/tests/test_tiny_aya_kvcache_nb.py55def test_dummy_tiny_aya_forward(dummy_cfg_base, dummy_input, import_notebook_defs):CODE
LOWch05/15_tiny-aya/tests/test_tiny_aya_kvcache_nb.py65def test_tiny_aya_base_equivalence_with_transformers(import_notebook_defs):CODE
LOWch05/07_gpt_to_llama/tests/tests_rope_and_parts.py97 def import_definitions_from_notebook(notebooks):CODE
LOWch05/10_llm-training-speed/01_opt_single_gpu.py255def generate_and_print_sample(model, tokenizer, device, start_context):CODE
LOWch05/10_llm-training-speed/01_opt_single_gpu.py269def train_model_simple_with_timing(model, train_loader, val_loader, optimizer, device,CODE
29 more matches not shown…
Deep Nesting47 hits · 47 pts
SeverityFileLineSnippetContext
LOWch07/01_main-chapter-code/gpt_download.py48CODE
LOWch07/01_main-chapter-code/gpt_download.py49CODE
LOWch07/02_dataset-utilities/find-near-duplicates.py41CODE
LOWch06/01_main-chapter-code/gpt_download.py47CODE
LOWch06/01_main-chapter-code/gpt_download.py48CODE
LOWch06/02_bonus_additional-experiments/gpt_download.py47CODE
LOWch06/02_bonus_additional-experiments/gpt_download.py48CODE
LOW…bonus_additional-experiments/additional_experiments.py257CODE
LOW…_bonus_imdb-classification/download_prepare_dataset.py31CODE
LOW…_bonus_imdb-classification/download_prepare_dataset.py51CODE
LOWch06/03_bonus_imdb-classification/gpt_download.py47CODE
LOWch06/03_bonus_imdb-classification/gpt_download.py48CODE
LOW…nstalling-python-libraries/python_environment_check.py20CODE
LOWappendix-E/01_main-chapter-code/gpt_download.py47CODE
LOWappendix-E/01_main-chapter-code/gpt_download.py48CODE
LOWch04/07_moe/gpt_with_kv_moe.py339CODE
LOWch04/07_moe/gpt_with_kv_ffn.py279CODE
LOWch02/02_bonus_bytepair-encoder/bpe_openai_gpt2.py148CODE
LOWch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py248CODE
LOW…3_bonus_pretraining_on_gutenberg/pretraining_simple.py80CODE
LOWch05/12_gemma3/tests/gemma3_layer_debugger.py206CODE
LOWch05/01_main-chapter-code/gpt_download.py48CODE
LOWch05/01_main-chapter-code/gpt_download.py49CODE
LOWch05/01_main-chapter-code/gpt_generate.py62CODE
LOWch05/15_tiny-aya/tests/tiny_aya_layer_debugger.py199CODE
LOWch05/07_gpt_to_llama/tests/tests_rope_and_parts.py96CODE
LOWch05/07_gpt_to_llama/tests/tests_rope_and_parts.py97CODE
LOWch05/13_olmo3/tests/olmo3_layer_debugger.py251CODE
LOW…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py258CODE
LOW…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py270CODE
LOWch05/10_llm-training-speed/01_opt_single_gpu.py269CODE
LOWch05/10_llm-training-speed/00_orig.py305CODE
LOWch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py314CODE
LOW…qwen3-chat-interface/qwen3-chat-interface-multiturn.py33CODE
LOW…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py32CODE
LOW.github/scripts/check_double_quotes.py76CODE
LOWpkg/llms_from_scratch/utils.py17CODE
LOWpkg/llms_from_scratch/utils.py39CODE
LOWpkg/llms_from_scratch/utils.py110CODE
LOWpkg/llms_from_scratch/utils.py153CODE
LOWpkg/llms_from_scratch/qwen3.py653CODE
LOWpkg/llms_from_scratch/ch05.py280CODE
LOWpkg/llms_from_scratch/ch05.py281CODE
LOWpkg/llms_from_scratch/appendix_d.py23CODE
LOWpkg/llms_from_scratch/tests/test_qwen3.py63CODE
LOWpkg/llms_from_scratch/tests/test_qwen3.py630CODE
LOWpkg/llms_from_scratch/tests/test_qwen3.py704CODE
Cross-File Repetition8 hits · 40 pts
SeverityFileLineSnippetContext
HIGHch06/01_main-chapter-code/gpt_download.py0def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.getSTRING
HIGHch06/02_bonus_additional-experiments/gpt_download.py0def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.getSTRING
HIGHch06/03_bonus_imdb-classification/gpt_download.py0def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.getSTRING
HIGHappendix-E/01_main-chapter-code/gpt_download.py0def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.getSTRING
HIGHch05/01_main-chapter-code/gpt_download.py0def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.getSTRING
HIGHch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py0arguments: rank: a unique process id world_size: total number of processes in the groupSTRING
HIGHappendix-A/01_main-chapter-code/DDP-script.py0arguments: rank: a unique process id world_size: total number of processes in the groupSTRING
HIGHappendix-A/01_main-chapter-code/DDP-script-torchrun.py0arguments: rank: a unique process id world_size: total number of processes in the groupSTRING
Decorative Section Separators10 hits · 39 pts
SeverityFileLineSnippetContext
MEDIUM…qwen3-chat-interface/qwen3-chat-interface-multiturn.py22# ============================================================COMMENT
MEDIUM…qwen3-chat-interface/qwen3-chat-interface-multiturn.py24# ============================================================COMMENT
MEDIUM…qwen3-chat-interface/qwen3-chat-interface-multiturn.py30# ============================================================COMMENT
MEDIUM…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py21# ============================================================COMMENT
MEDIUM…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py23# ============================================================COMMENT
MEDIUM…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py29# ============================================================COMMENT
MEDIUMpkg/llms_from_scratch/llama3.py208# ==============================================================================COMMENT
MEDIUMpkg/llms_from_scratch/llama3.py257# ==============================================================================COMMENT
MEDIUMpkg/llms_from_scratch/qwen3.py332# ==============================================================================COMMENT
MEDIUMpkg/llms_from_scratch/qwen3.py381# ==============================================================================COMMENT
Excessive Try-Catch Wrapping29 hits · 35 pts
SeverityFileLineSnippetContext
LOWch07/01_main-chapter-code/gpt_download.py91 except Exception as e:CODE
LOWch06/01_main-chapter-code/gpt_download.py91 except Exception as e:CODE
LOWch06/02_bonus_additional-experiments/gpt_download.py91 except Exception as e:CODE
LOWch06/03_bonus_imdb-classification/gpt_download.py91 except Exception as e:CODE
LOW…nstalling-python-libraries/python_environment_check.py90 except Exception as e:CODE
LOWappendix-E/01_main-chapter-code/gpt_download.py91 except Exception as e:CODE
LOWch04/06_swa/plot_memory_estimates_swa.py38 except Exception:CODE
LOWch04/06_swa/memory_estimator_swa.py37 except Exception:CODE
LOWch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py20 except Exception:CODE
LOWch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py39except Exception:CODE
MEDIUMch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py14def _import_qwen3_5_classes():CODE
LOWch05/16_qwen3.5/tests/test_qwen3_5_nb.py22 except Exception:CODE
LOWch05/16_qwen3.5/tests/test_qwen3_5_nb.py43 except Exception:CODE
MEDIUMch05/16_qwen3.5/tests/test_qwen3_5_nb.py16def _import_qwen3_5_classes():CODE
LOWch05/01_main-chapter-code/gpt_download.py91 except Exception as e:CODE
LOWch05/17_gemma4/tests/test_gemma4_nb.py77 except Exception:CODE
MEDIUMch05/17_gemma4/tests/test_gemma4_nb.py71def gemma4_transformers_module():CODE
LOW…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py20 except Exception:CODE
LOW…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py42except Exception:CODE
MEDIUM…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py15def _import_gemma4_classes():CODE
LOW…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py20 except Exception:CODE
LOW…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py42except Exception:CODE
MEDIUM…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py15def _import_gemma4_classes():CODE
LOW.github/scripts/check_double_quotes.py111 except Exception as e:CODE
MEDIUM.github/scripts/check_double_quotes.py104def check_file(path):CODE
LOWpkg/llms_from_scratch/utils.py171 except Exception as e:CODE
LOWpkg/llms_from_scratch/ch05.py323 except Exception as e:CODE
LOWpkg/llms_from_scratch/tests/test_qwen3.py77 except Exception:CODE
LOWpkg/llms_from_scratch/tests/test_qwen3.py654 except Exception:CODE
Redundant / Tautological Comments28 hits · 30 pts
SeverityFileLineSnippetContext
LOWch07/01_main-chapter-code/previous_chapters.py301 model.train() # Set model to training modeCODE
LOWch07/01_main-chapter-code/gpt_download.py55 # Check if file exists and has same sizeCOMMENT
LOWch07/04_preference-tuning-with-dpo/previous_chapters.py302 model.train() # Set model to training modeCODE
LOWch06/01_main-chapter-code/gpt_class_finetune.py190 model.train() # Set model to training modeCODE
LOWch06/01_main-chapter-code/gpt_download.py54 # Check if file exists and has same sizeCOMMENT
LOWch06/01_main-chapter-code/gpt_download.py104 # Check if file exists and has the same sizeSTRING
LOWch06/02_bonus_additional-experiments/gpt_download.py54 # Check if file exists and has same sizeCOMMENT
LOWch06/02_bonus_additional-experiments/gpt_download.py104 # Check if file exists and has the same sizeSTRING
LOW…bonus_additional-experiments/additional_experiments.py338 model.train() # Set model to training modeCODE
LOWch06/03_bonus_imdb-classification/train_bert_hf_spam.py237 model.train() # Set model to training modeCODE
LOWch06/03_bonus_imdb-classification/train_gpt.py184 model.train() # Set model to training modeCODE
LOWch06/03_bonus_imdb-classification/gpt_download.py54 # Check if file exists and has same sizeCOMMENT
LOWch06/03_bonus_imdb-classification/gpt_download.py104 # Check if file exists and has the same sizeSTRING
LOWch06/03_bonus_imdb-classification/train_gpt_muon.py219 model.train() # Set model to training modeCODE
LOWch06/03_bonus_imdb-classification/train_bert_hf.py140 model.train() # Set model to training modeCODE
LOW…nal-aws-sagemaker-notebook/cloudformation-template.yml119 # Check if setup is still running or not startedCOMMENT
LOWappendix-E/01_main-chapter-code/previous_chapters.py500 model.train() # Set model to training modeCODE
LOWappendix-E/01_main-chapter-code/gpt_download.py54 # Check if file exists and has same sizeCOMMENT
LOWappendix-E/01_main-chapter-code/gpt_download.py104 # Check if file exists and has the same sizeSTRING
LOWch05/01_main-chapter-code/gpt_train.py84 model.train() # Set model to training modeCODE
LOWch05/01_main-chapter-code/gpt_download.py55 # Check if file exists and has same sizeCOMMENT
LOWch05/01_main-chapter-code/gpt_download.py104 # Check if file exists and has the same sizeSTRING
LOWch05/01_main-chapter-code/gpt_generate.py70 # Check if file exists and has the same sizeCOMMENT
LOWch05/18_muon/gpt_train_muon.py120 model.train() # Set model to training modeCODE
LOWch05/18_muon/gpt_train.py84 model.train() # Set model to training modeCODE
LOWpkg/llms_from_scratch/ch06.py185 model.train() # Set model to training modeCODE
LOWpkg/llms_from_scratch/ch05.py70 model.train() # Set model to training modeCODE
LOWpkg/llms_from_scratch/ch05.py287 # Check if file exists and has same sizeCOMMENT
Unused Imports28 hits · 28 pts
SeverityFileLineSnippetContext
LOW.github/scripts/test_check_links.py3CODE
LOW.github/scripts/check_links.py14CODE
LOWpkg/llms_from_scratch/kv_cache/gpt2.py6CODE
LOWpkg/llms_from_scratch/kv_cache/generate.py6CODE
LOWpkg/llms_from_scratch/kv_cache/llama3.py6CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py6CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/generate.py6CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py6CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
LOWpkg/llms_from_scratch/kv_cache_batched/qwen3.py7CODE
Over-Commented Block24 hits · 24 pts
SeverityFileLineSnippetContext
LOWch06/01_main-chapter-code/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOW…6/02_bonus_additional-experiments/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch06/03_bonus_imdb-classification/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch04/07_moe/gpt_with_kv_ffn.py141 torch.sqrt(torch.tensor(2.0 / torch.pi)) *COMMENT
LOWch04/09_dsa/gpt_with_kv_dsa.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch04/09_dsa/gpt_with_kv_dsa.py21COMMENT
LOWch04/05_mla/gpt_with_kv_mla.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch02/02_bonus_bytepair-encoder/bpe_openai_gpt2.py1# Source: https://github.com/openai/gpt-2/blob/master/src/encoder.pyCOMMENT
LOWch05/01_main-chapter-code/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch05/07_gpt_to_llama/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWch05/10_llm-training-speed/01_opt_single_gpu.py501 epochs_tensor = torch.linspace(0, OTHER_SETTINGS["num_epochs"], len(train_losses))COMMENT
LOWch05/10_llm-training-speed/00_orig.py521 ###########################COMMENT
LOWch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py601 # torch.save(model._orig_mod.state_dict(), "model.pth")COMMENT
LOWch05/18_muon/previous_chapters.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOW.github/scripts/check_links.py1#!/usr/bin/env -S uv run --scriptCOMMENT
LOWpkg/llms_from_scratch/ch07.py41# if not os.path.exists(file_path):COMMENT
LOWpkg/llms_from_scratch/__init__.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWpkg/llms_from_scratch/llama3.py201 # Combine heads, where self.d_out = self.num_heads * self.head_dimCOMMENT
LOWpkg/llms_from_scratch/llama3.py221# │ │ │ │ │ │ │ │COMMENT
LOWpkg/llms_from_scratch/llama3.py241# [ x0 x1 x2 x3 x4 x5 x6 x7 ]COMMENT
LOWpkg/llms_from_scratch/qwen3.py341#COMMENT
LOWpkg/llms_from_scratch/qwen3.py361# 2) Interleaved (even/odd) style (original paper, Llama repo):COMMENT
LOWpkg/llms_from_scratch/kv_cache/__init__.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
LOWpkg/llms_from_scratch/kv_cache_batched/__init__.py1# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).COMMENT
Structural Annotation Overuse11 hits · 16 pts
SeverityFileLineSnippetContext
LOWch07/06_user_interface/README.md16## Step 1: Install dependenciesCOMMENT
LOWch07/06_user_interface/README.md27## Step 2: Run `app` codeCOMMENT
LOWch06/03_bonus_imdb-classification/README.md29## Step 1: Install DependenciesCOMMENT
LOWch06/03_bonus_imdb-classification/README.md38## Step 2: Download DatasetCOMMENT
LOWch06/03_bonus_imdb-classification/README.md50## Step 3: Run ModelsCOMMENT
LOWch06/04_user_interface/README.md16## Step 1: Install dependenciesCOMMENT
LOWch06/04_user_interface/README.md27## Step 2: Run `app` codeCOMMENT
LOWch05/06_user_interface/README.md16## Step 1: Install dependenciesCOMMENT
LOWch05/06_user_interface/README.md27## Step 2: Run `app` codeCOMMENT
LOWch05/11_qwen3/qwen3-chat-interface/README.md16## Step 1: Install dependenciesCOMMENT
LOWch05/11_qwen3/qwen3-chat-interface/README.md34## Step 2: Run `app` codeCOMMENT
AI Slop Vocabulary1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUMpkg/llms_from_scratch/ch07.py33# The `requests` version above is more robustCOMMENT
Slop Phrases1 hit · 2 pts
SeverityFileLineSnippetContext
MEDIUM…-tuning-with-dpo/instruction-data-with-preference.json3520 "chosen": "It's worth noting that the most popular vegetable in the world is actually the potato.",CODE
AI Structural Patterns2 hits · 2 pts
SeverityFileLineSnippetContext
LOWch05/16_qwen3.5/qwen3_5_transformers.py93CODE
LOWch05/16_qwen3.5/qwen3_5_transformers.py411CODE