rasbt/LLMs-from-scratch

Pattern Findings

302 matches across 11 categories. Click a row to expand file-level details.

Self-Referential Comments45 hits · 108 pts

Severity	File	Line	Snippet
MEDIUM	ch07/01_main-chapter-code/previous_chapters.py	467	# Create a second x-axis for tokens seen
MEDIUM	ch07/01_main-chapter-code/previous_chapters.py	468	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch07/01_main-chapter-code/exercise_experiments.py	286	# Create a second x-axis for tokens seen
MEDIUM	ch07/01_main-chapter-code/exercise_experiments.py	287	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch07/01_main-chapter-code/ollama_evaluate.py	15	# Create the data payload as a dictionary
MEDIUM	ch07/01_main-chapter-code/gpt_instruction_finetuning.py	135	# Create a second x-axis for tokens seen
MEDIUM	ch07/01_main-chapter-code/gpt_instruction_finetuning.py	136	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch07/04_preference-tuning-with-dpo/previous_chapters.py	468	# Create a second x-axis for tokens seen
MEDIUM	ch07/04_preference-tuning-with-dpo/previous_chapters.py	469	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch06/01_main-chapter-code/gpt_class_finetune.py	230	# Create a second x-axis for tokens seen
MEDIUM	ch06/01_main-chapter-code/gpt_class_finetune.py	231	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch06/01_main-chapter-code/gpt_download.py	111	# Define the block size for reading the file
MEDIUM	ch06/02_bonus_additional-experiments/gpt_download.py	111	# Define the block size for reading the file
MEDIUM	…_bonus_imdb-classification/download_prepare_dataset.py	58	# Create a DataFrame for each file and add it to the list
MEDIUM	ch06/03_bonus_imdb-classification/gpt_download.py	111	# Define the block size for reading the file
MEDIUM	…6/03_bonus_imdb-classification/train_sklearn_logreg.py	65	# Create a dummy classifier with the strategy to predict the most frequent class
MEDIUM	…nal-aws-sagemaker-notebook/cloudformation-template.yml	59	# Create a startup script that will run in the background
MEDIUM	…nal-aws-sagemaker-notebook/cloudformation-template.yml	104	# Create a flag file to indicate setup is complete
MEDIUM	appendix-D/01_main-chapter-code/previous_chapters.py	305	# Create a second x-axis for tokens seen
MEDIUM	appendix-D/01_main-chapter-code/previous_chapters.py	306	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	appendix-E/01_main-chapter-code/previous_chapters.py	540	# Create a second x-axis for tokens seen
MEDIUM	appendix-E/01_main-chapter-code/previous_chapters.py	541	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	appendix-E/01_main-chapter-code/gpt_download.py	111	# Define the block size for reading the file
MEDIUM	ch05/16_qwen3.5/qwen3_5_transformers.py	97	"""This function is intended to align with the l2norm implementation in the FLA library."""
MEDIUM	ch05/01_main-chapter-code/gpt_train.py	122	# Create a second x-axis for tokens seen
MEDIUM	ch05/01_main-chapter-code/gpt_train.py	123	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch05/01_main-chapter-code/gpt_download.py	111	# Define the block size for reading the file
MEDIUM	ch05/01_main-chapter-code/gpt_generate.py	77	# Define the block size for reading the file
MEDIUM	ch05/07_gpt_to_llama/tests/tests_rope_and_parts.py	113	# Create a module to store the imported functions and classes
MEDIUM	ch05/10_llm-training-speed/01_opt_single_gpu.py	361	# Create a second x-axis for tokens seen
MEDIUM	ch05/10_llm-training-speed/01_opt_single_gpu.py	362	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch05/10_llm-training-speed/00_orig.py	397	# Create a second x-axis for tokens seen
MEDIUM	ch05/10_llm-training-speed/00_orig.py	398	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py	426	# Create a second x-axis for tokens seen
MEDIUM	ch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py	427	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch05/05_bonus_hparam_tuning/hparam_search.py	18	# Define a grid of hyperparameters to search over
MEDIUM	ch05/18_muon/gpt_train_muon.py	166	# Create a second x-axis for tokens seen
MEDIUM	ch05/18_muon/gpt_train_muon.py	167	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	ch05/18_muon/gpt_train.py	122	# Create a second x-axis for tokens seen
MEDIUM	ch05/18_muon/gpt_train.py	123	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	pkg/llms_from_scratch/ch06.py	225	# Create a second x-axis for examples seen
MEDIUM	pkg/llms_from_scratch/ch06.py	226	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis
MEDIUM	pkg/llms_from_scratch/ch07.py	214	# Create the data payload as a dictionary
MEDIUM	pkg/llms_from_scratch/ch05.py	236	# Create a second x-axis for tokens seen
MEDIUM	pkg/llms_from_scratch/ch05.py	237	ax2 = ax1.twiny() # Create a second x-axis that shares the same y-axis

Hyper-Verbose Identifiers84 hits · 84 pts

Severity	File	Line	Snippet
LOW	ch07/01_main-chapter-code/previous_chapters.py	338	def generate_and_print_sample(model, tokenizer, device, start_context):
LOW	ch07/01_main-chapter-code/exercise_experiments.py	190	def custom_collate_with_masking_fn(
LOW	ch07/02_dataset-utilities/find-near-duplicates.py	76	def find_print_and_remove_near_duplicates(json_data, remove_duplicates=False, threshold=0.75):
LOW	ch07/04_preference-tuning-with-dpo/previous_chapters.py	339	def generate_and_print_sample(model, tokenizer, device, start_context):
LOW	ch06/01_main-chapter-code/gpt_class_finetune.py	26	def download_and_unzip_spam_data(url, zip_path, extracted_path, data_file_path):
LOW	…_bonus_imdb-classification/download_prepare_dataset.py	31	def download_and_extract_dataset(dataset_url, target_file, directory):
LOW	…_bonus_imdb-classification/download_prepare_dataset.py	51	def load_dataset_to_dataframe(basepath="aclImdb", labels={"pos": 1, "neg": 0}):
LOW	appendix-D/01_main-chapter-code/previous_chapters.py	282	def generate_and_print_sample(model, tokenizer, device, start_context):
LOW	appendix-E/01_main-chapter-code/previous_chapters.py	364	def download_and_unzip_spam_data(url, zip_path, extracted_path, data_file_path):
LOW	ch04/06_swa/gpt_with_kv_mha.py	253	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/06_swa/tests.py	12	def test_cached_prefill_matches_uncached_swa():
LOW	ch04/06_swa/tests.py	37	def test_swa_matches_base_model_when_window_equals_context():
LOW	ch04/06_swa/plot_memory_estimates_swa.py	57	def calc_kv_bytes_total_mha_swa(
LOW	ch04/06_swa/plot_memory_estimates_swa.py	75	def calc_kv_bytes_total_gqa_swa(
LOW	ch04/06_swa/gpt_with_kv_swa.py	294	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/07_moe/plot_memory_estimates_moe.py	16	def calc_moe_active_and_total(
LOW	ch04/07_moe/plot_memory_estimates_moe.py	42	def plot_active_params_vs_experts(
LOW	ch04/07_moe/gpt_with_kv_moe.py	339	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/07_moe/memory_estimator_moe.py	39	def estimate_params_and_hidden(
LOW	ch04/07_moe/gpt_with_kv_ffn.py	279	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/04_gqa/gpt_with_kv_mha.py	253	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/04_gqa/gpt_with_kv_gqa.py	265	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/04_gqa/plot_memory_estimates_gqa.py	23	def plot_abs_kv_vs_context_multi_groups():
LOW	ch04/09_dsa/gpt_with_kv_dsa.py	389	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/09_dsa/test_dsa.py	16	def import_transformers_dsa_model():
LOW	ch04/09_dsa/test_dsa.py	89	def test_indexer_matches_transformers_reference():
LOW	ch04/09_dsa/test_dsa.py	163	def dense_attention_reference(attn, x):
LOW	ch04/09_dsa/test_dsa.py	181	def test_topk_full_equals_dense():
LOW	ch04/10_kv-sharing/gpt_with_kv_mha.py	253	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/10_kv-sharing/gpt_with_kv_sharing.py	270	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/10_kv-sharing/tests.py	10	def test_kv_sharing_matches_mha_when_all_layers_produce_kv():
LOW	ch04/10_kv-sharing/tests.py	55	def test_only_producer_layers_store_kv_cache():
LOW	ch04/10_kv-sharing/tests.py	84	def test_memory_estimator_counts_cached_layers():
LOW	…04/08_deltanet/plot_memory_estimates_gated_deltanet.py	27	def calc_kv_bytes_total_deltanet_no_conv(batch, emb_dim, n_layers, bytes_per_elem, n_heads):
LOW	ch04/05_mla/plot_memory_estimates_mla.py	33	def plot_abs_kv_vs_context_multiple():
LOW	ch04/05_mla/gpt_with_kv_mla.py	261	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/05_mla/gpt_with_kv_mha.py	253	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/03_kv-cache/gpt_with_kv_cache.py	280	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/03_kv-cache/README.md	173	def generate_text_simple_cached(model, idx, max_new_tokens,
LOW	ch04/03_kv-cache/gpt_with_kv_cache_optimized.py	306	def generate_text_simple_cached(model, idx, max_new_tokens, context_size=None, use_cache=True):
LOW	ch04/03_kv-cache/tests.py	32	def test_gpt_model_equivalence_not_cached(ModelClass):
LOW	ch04/03_kv-cache/tests.py	66	def test_gpt_model_equivalence_cached(ModelClass):
LOW	ch04/03_kv-cache/tests.py	113	def test_context_overflow_bug():
LOW	ch04/03_kv-cache/tests.py	150	def test_prefill_chunking_basic():
LOW	ch02/05_bpe-from-scratch/tests.py	11	def import_definitions_from_notebook(fullname, names):
LOW	ch02/05_bpe-from-scratch/tests.py	185	def test_no_eot_aliasing_and_disallowed_logic(imported_module, gpt2_files):
LOW	ch02/05_bpe-from-scratch/tests.py	214	def test_newline_roundtrip_and_equivalence(imported_module, gpt2_files, text):
LOW	ch02/05_bpe-from-scratch/tests.py	234	def test_space_newline_space_patterns(imported_module, gpt2_files):
LOW	ch02/05_bpe-from-scratch/tests.py	250	def test_multiple_leading_spaces_roundtrip(imported_module, gpt2_files):
LOW	ch05/16_qwen3.5/qwen3_5_transformers.py	66	def apply_mask_to_padding_states(hidden_states, attention_mask):
LOW	ch05/16_qwen3.5/qwen3_5_transformers.py	102	def torch_chunk_gated_delta_rule(
LOW	ch05/16_qwen3.5/qwen3_5_transformers.py	182	def torch_recurrent_gated_delta_rule(
LOW	ch05/01_main-chapter-code/gpt_train.py	61	def generate_and_print_sample(model, tokenizer, device, start_context):
LOW	ch05/15_tiny-aya/tests/test_tiny_aya_nb.py	54	def test_dummy_tiny_aya_forward(dummy_cfg_base, dummy_input, import_notebook_defs):
LOW	ch05/15_tiny-aya/tests/test_tiny_aya_nb.py	64	def test_tiny_aya_base_equivalence_with_transformers(import_notebook_defs):
LOW	ch05/15_tiny-aya/tests/test_tiny_aya_kvcache_nb.py	55	def test_dummy_tiny_aya_forward(dummy_cfg_base, dummy_input, import_notebook_defs):
LOW	ch05/15_tiny-aya/tests/test_tiny_aya_kvcache_nb.py	65	def test_tiny_aya_base_equivalence_with_transformers(import_notebook_defs):
LOW	ch05/07_gpt_to_llama/tests/tests_rope_and_parts.py	97	def import_definitions_from_notebook(notebooks):
LOW	ch05/10_llm-training-speed/01_opt_single_gpu.py	255	def generate_and_print_sample(model, tokenizer, device, start_context):
LOW	ch05/10_llm-training-speed/01_opt_single_gpu.py	269	def train_model_simple_with_timing(model, train_loader, val_loader, optimizer, device,
24 more matches not shown…

Deep Nesting47 hits · 47 pts

Severity	File	Line
LOW	ch07/01_main-chapter-code/gpt_download.py	48
LOW	ch07/01_main-chapter-code/gpt_download.py	49
LOW	ch07/02_dataset-utilities/find-near-duplicates.py	41
LOW	ch06/01_main-chapter-code/gpt_download.py	47
LOW	ch06/01_main-chapter-code/gpt_download.py	48
LOW	ch06/02_bonus_additional-experiments/gpt_download.py	47
LOW	ch06/02_bonus_additional-experiments/gpt_download.py	48
LOW	…bonus_additional-experiments/additional_experiments.py	257
LOW	…_bonus_imdb-classification/download_prepare_dataset.py	31
LOW	…_bonus_imdb-classification/download_prepare_dataset.py	51
LOW	ch06/03_bonus_imdb-classification/gpt_download.py	47
LOW	ch06/03_bonus_imdb-classification/gpt_download.py	48
LOW	…nstalling-python-libraries/python_environment_check.py	20
LOW	appendix-E/01_main-chapter-code/gpt_download.py	47
LOW	appendix-E/01_main-chapter-code/gpt_download.py	48
LOW	ch04/07_moe/gpt_with_kv_moe.py	339
LOW	ch04/07_moe/gpt_with_kv_ffn.py	279
LOW	ch02/02_bonus_bytepair-encoder/bpe_openai_gpt2.py	148
LOW	ch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py	248
LOW	…3_bonus_pretraining_on_gutenberg/pretraining_simple.py	80
LOW	ch05/12_gemma3/tests/gemma3_layer_debugger.py	206
LOW	ch05/01_main-chapter-code/gpt_download.py	48
LOW	ch05/01_main-chapter-code/gpt_download.py	49
LOW	ch05/01_main-chapter-code/gpt_generate.py	62
LOW	ch05/15_tiny-aya/tests/tiny_aya_layer_debugger.py	199
LOW	ch05/07_gpt_to_llama/tests/tests_rope_and_parts.py	96
LOW	ch05/07_gpt_to_llama/tests/tests_rope_and_parts.py	97
LOW	ch05/13_olmo3/tests/olmo3_layer_debugger.py	251
LOW	…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py	258
LOW	…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py	270
LOW	ch05/10_llm-training-speed/01_opt_single_gpu.py	269
LOW	ch05/10_llm-training-speed/00_orig.py	305
LOW	ch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py	314
LOW	…qwen3-chat-interface/qwen3-chat-interface-multiturn.py	33
LOW	…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py	32
LOW	.github/scripts/check_double_quotes.py	76
LOW	pkg/llms_from_scratch/utils.py	17
LOW	pkg/llms_from_scratch/utils.py	39
LOW	pkg/llms_from_scratch/utils.py	110
LOW	pkg/llms_from_scratch/utils.py	153
LOW	pkg/llms_from_scratch/qwen3.py	653
LOW	pkg/llms_from_scratch/ch05.py	280
LOW	pkg/llms_from_scratch/ch05.py	281
LOW	pkg/llms_from_scratch/appendix_d.py	23
LOW	pkg/llms_from_scratch/tests/test_qwen3.py	63
LOW	pkg/llms_from_scratch/tests/test_qwen3.py	614
LOW	pkg/llms_from_scratch/tests/test_qwen3.py	688

Cross-File Repetition8 hits · 40 pts

Severity	File	Snippet
HIGH	ch06/01_main-chapter-code/gpt_download.py	def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.get
HIGH	ch06/02_bonus_additional-experiments/gpt_download.py	def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.get
HIGH	ch06/03_bonus_imdb-classification/gpt_download.py	def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.get
HIGH	appendix-E/01_main-chapter-code/gpt_download.py	def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.get
HIGH	ch05/01_main-chapter-code/gpt_download.py	def download_file(url, destination): # send a get request to download the file in streaming mode response = requests.get
HIGH	ch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py	arguments: rank: a unique process id world_size: total number of processes in the group
HIGH	appendix-A/01_main-chapter-code/DDP-script.py	arguments: rank: a unique process id world_size: total number of processes in the group
HIGH	appendix-A/01_main-chapter-code/DDP-script-torchrun.py	arguments: rank: a unique process id world_size: total number of processes in the group

Decorative Section Separators10 hits · 39 pts

Severity	File	Line	Snippet
MEDIUM	…qwen3-chat-interface/qwen3-chat-interface-multiturn.py	22	# ============================================================
MEDIUM	…qwen3-chat-interface/qwen3-chat-interface-multiturn.py	24	# ============================================================
MEDIUM	…qwen3-chat-interface/qwen3-chat-interface-multiturn.py	30	# ============================================================
MEDIUM	…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py	21	# ============================================================
MEDIUM	…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py	23	# ============================================================
MEDIUM	…/11_qwen3/qwen3-chat-interface/qwen3-chat-interface.py	29	# ============================================================
MEDIUM	pkg/llms_from_scratch/llama3.py	208	# ==============================================================================
MEDIUM	pkg/llms_from_scratch/llama3.py	257	# ==============================================================================
MEDIUM	pkg/llms_from_scratch/qwen3.py	332	# ==============================================================================
MEDIUM	pkg/llms_from_scratch/qwen3.py	381	# ==============================================================================

Excessive Try-Catch Wrapping29 hits · 35 pts

Severity	File	Line	Snippet
LOW	ch07/01_main-chapter-code/gpt_download.py	91	except Exception as e:
LOW	ch06/01_main-chapter-code/gpt_download.py	91	except Exception as e:
LOW	ch06/02_bonus_additional-experiments/gpt_download.py	91	except Exception as e:
LOW	ch06/03_bonus_imdb-classification/gpt_download.py	91	except Exception as e:
LOW	…nstalling-python-libraries/python_environment_check.py	90	except Exception as e:
LOW	appendix-E/01_main-chapter-code/gpt_download.py	91	except Exception as e:
LOW	ch04/06_swa/plot_memory_estimates_swa.py	38	except Exception:
LOW	ch04/06_swa/memory_estimator_swa.py	37	except Exception:
LOW	ch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py	20	except Exception:
LOW	ch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py	39	except Exception:
MEDIUM	ch05/16_qwen3.5/tests/qwen3_5_layer_debugger.py	14	def _import_qwen3_5_classes():
LOW	ch05/16_qwen3.5/tests/test_qwen3_5_nb.py	22	except Exception:
LOW	ch05/16_qwen3.5/tests/test_qwen3_5_nb.py	43	except Exception:
MEDIUM	ch05/16_qwen3.5/tests/test_qwen3_5_nb.py	16	def _import_qwen3_5_classes():
LOW	ch05/01_main-chapter-code/gpt_download.py	91	except Exception as e:
LOW	ch05/17_gemma4/tests/test_gemma4_nb.py	77	except Exception:
MEDIUM	ch05/17_gemma4/tests/test_gemma4_nb.py	71	def gemma4_transformers_module():
LOW	…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py	20	except Exception:
LOW	…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py	42	except Exception:
MEDIUM	…/17_gemma4/tests/test_e4b/gemma4_e4b_layer_debugger.py	15	def _import_gemma4_classes():
LOW	…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py	20	except Exception:
LOW	…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py	42	except Exception:
MEDIUM	…/17_gemma4/tests/test_e2b/gemma4_e2b_layer_debugger.py	15	def _import_gemma4_classes():
LOW	.github/scripts/check_double_quotes.py	111	except Exception as e:
MEDIUM	.github/scripts/check_double_quotes.py	104	def check_file(path):
LOW	pkg/llms_from_scratch/utils.py	171	except Exception as e:
LOW	pkg/llms_from_scratch/ch05.py	323	except Exception as e:
LOW	pkg/llms_from_scratch/tests/test_qwen3.py	77	except Exception:
LOW	pkg/llms_from_scratch/tests/test_qwen3.py	638	except Exception:

Redundant / Tautological Comments28 hits · 30 pts

Severity	File	Line	Snippet
LOW	ch07/01_main-chapter-code/previous_chapters.py	301	model.train() # Set model to training mode
LOW	ch07/01_main-chapter-code/gpt_download.py	55	# Check if file exists and has same size
LOW	ch07/04_preference-tuning-with-dpo/previous_chapters.py	302	model.train() # Set model to training mode
LOW	ch06/01_main-chapter-code/gpt_class_finetune.py	190	model.train() # Set model to training mode
LOW	ch06/01_main-chapter-code/gpt_download.py	54	# Check if file exists and has same size
LOW	ch06/01_main-chapter-code/gpt_download.py	104	# Check if file exists and has the same size
LOW	ch06/02_bonus_additional-experiments/gpt_download.py	54	# Check if file exists and has same size
LOW	ch06/02_bonus_additional-experiments/gpt_download.py	104	# Check if file exists and has the same size
LOW	…bonus_additional-experiments/additional_experiments.py	338	model.train() # Set model to training mode
LOW	ch06/03_bonus_imdb-classification/train_bert_hf_spam.py	237	model.train() # Set model to training mode
LOW	ch06/03_bonus_imdb-classification/train_gpt.py	184	model.train() # Set model to training mode
LOW	ch06/03_bonus_imdb-classification/gpt_download.py	54	# Check if file exists and has same size
LOW	ch06/03_bonus_imdb-classification/gpt_download.py	104	# Check if file exists and has the same size
LOW	ch06/03_bonus_imdb-classification/train_gpt_muon.py	219	model.train() # Set model to training mode
LOW	ch06/03_bonus_imdb-classification/train_bert_hf.py	140	model.train() # Set model to training mode
LOW	…nal-aws-sagemaker-notebook/cloudformation-template.yml	119	# Check if setup is still running or not started
LOW	appendix-E/01_main-chapter-code/previous_chapters.py	500	model.train() # Set model to training mode
LOW	appendix-E/01_main-chapter-code/gpt_download.py	54	# Check if file exists and has same size
LOW	appendix-E/01_main-chapter-code/gpt_download.py	104	# Check if file exists and has the same size
LOW	ch05/01_main-chapter-code/gpt_train.py	84	model.train() # Set model to training mode
LOW	ch05/01_main-chapter-code/gpt_download.py	55	# Check if file exists and has same size
LOW	ch05/01_main-chapter-code/gpt_download.py	104	# Check if file exists and has the same size
LOW	ch05/01_main-chapter-code/gpt_generate.py	70	# Check if file exists and has the same size
LOW	ch05/18_muon/gpt_train_muon.py	120	model.train() # Set model to training mode
LOW	ch05/18_muon/gpt_train.py	84	model.train() # Set model to training mode
LOW	pkg/llms_from_scratch/ch06.py	185	model.train() # Set model to training mode
LOW	pkg/llms_from_scratch/ch05.py	70	model.train() # Set model to training mode
LOW	pkg/llms_from_scratch/ch05.py	287	# Check if file exists and has same size

Unused Imports26 hits · 26 pts

Severity	File	Line
LOW	pkg/llms_from_scratch/kv_cache/gpt2.py	6
LOW	pkg/llms_from_scratch/kv_cache/generate.py	6
LOW	pkg/llms_from_scratch/kv_cache/llama3.py	6
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	6
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/generate.py	6
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	6
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7
LOW	pkg/llms_from_scratch/kv_cache_batched/qwen3.py	7

Over-Commented Block23 hits · 23 pts

Severity	File	Line	Snippet
LOW	ch06/01_main-chapter-code/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	…6/02_bonus_additional-experiments/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch06/03_bonus_imdb-classification/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch04/07_moe/gpt_with_kv_ffn.py	141	torch.sqrt(torch.tensor(2.0 / torch.pi)) *
LOW	ch04/09_dsa/gpt_with_kv_dsa.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch04/09_dsa/gpt_with_kv_dsa.py	21
LOW	ch04/05_mla/gpt_with_kv_mla.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch02/02_bonus_bytepair-encoder/bpe_openai_gpt2.py	1	# Source: https://github.com/openai/gpt-2/blob/master/src/encoder.py
LOW	ch05/01_main-chapter-code/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch05/07_gpt_to_llama/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	ch05/10_llm-training-speed/01_opt_single_gpu.py	501	epochs_tensor = torch.linspace(0, OTHER_SETTINGS["num_epochs"], len(train_losses))
LOW	ch05/10_llm-training-speed/00_orig.py	521	###########################
LOW	ch05/10_llm-training-speed/02_opt_multi_gpu_ddp.py	601	# torch.save(model._orig_mod.state_dict(), "model.pth")
LOW	ch05/18_muon/previous_chapters.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	pkg/llms_from_scratch/ch07.py	41	# if not os.path.exists(file_path):
LOW	pkg/llms_from_scratch/__init__.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	pkg/llms_from_scratch/llama3.py	201	# Combine heads, where self.d_out = self.num_heads * self.head_dim
LOW	pkg/llms_from_scratch/llama3.py	221	# │ │ │ │ │ │ │ │
LOW	pkg/llms_from_scratch/llama3.py	241	# [ x0 x1 x2 x3 x4 x5 x6 x7 ]
LOW	pkg/llms_from_scratch/qwen3.py	341	#
LOW	pkg/llms_from_scratch/qwen3.py	361	# 2) Interleaved (even/odd) style (original paper, Llama repo):
LOW	pkg/llms_from_scratch/kv_cache/__init__.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).
LOW	pkg/llms_from_scratch/kv_cache_batched/__init__.py	1	# Copyright (c) Sebastian Raschka under Apache License 2.0 (see LICENSE.txt).

AI Slop Vocabulary1 hit · 3 pts

Severity	File	Line	Snippet
MEDIUM	pkg/llms_from_scratch/ch07.py	33	# The `requests` version above is more robust

Slop Phrases1 hit · 2 pts

Severity	File	Line	Snippet
MEDIUM	…-tuning-with-dpo/instruction-data-with-preference.json	3520	"chosen": "It's worth noting that the most popular vegetable in the world is actually the potato.",

Score History

Severity Breakdown

Pattern Findings