Repository Analysis

stas00/ml-engineering

Machine Learning Engineering Open Book

4.7 Likely human-written View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of stas00/ml-engineering, a Python project with 18,403 GitHub stars. SynthScan v2.0 examined 22,358 lines of code across 92 source files, recording 91 pattern matches distributed across 10 syntactic categories. The overall adjusted score of 4.7 places this repository in the Likely human-written band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

4.7
Adjusted Score
4.7
Raw Score
100%
Time Factor
2026-07-09
Last Push
18.4K
Stars
Python
Language
22.4K
Lines of Code
92
Files
91
Pattern Hits
2026-07-14
Scan Date
0.04
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 4MEDIUM 4LOW 83

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 91 distinct pattern matches across 10 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Unused Imports40 hits · 40 pts
SeverityFileLineSnippetContext
LOWtraining/fault-tolerance/slurm-status.py13CODE
LOWtraining/fault-tolerance/slurm-status.py14CODE
LOWtraining/fault-tolerance/slurm-status.py15CODE
LOWtraining/fault-tolerance/slurm-status.py16CODE
LOWtraining/fault-tolerance/slurm-status.py21CODE
LOWtraining/fault-tolerance/fs-watchdog.py16CODE
LOWnetwork/benchmarks/all_reduce_bench.py106CODE
LOWnetwork/benchmarks/all_reduce_bench.py121CODE
LOWstorage/fio-json-extract.py13CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py21CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py22CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py24CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py25CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py26CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py27CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py28CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py29CODE
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py30CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py24CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py26CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py27CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py28CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py29CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py30CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py31CODE
LOWdebug/tiny-scripts/m4-ds-unpack.py32CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py52CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py53CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py56CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py58CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py59CODE
LOWdebug/tiny-scripts/cm4-synthetic-testing.py52CODE
LOWdebug/tiny-scripts/cm4-synthetic-testing.py56CODE
LOWdebug/tiny-scripts/cm4-synthetic-testing.py58CODE
LOWdebug/tiny-scripts/cm4-synthetic-testing.py59CODE
LOWdebug/tiny-scripts/mt5-make-tiny-model.py28CODE
LOWdebug/tiny-scripts/mt5-make-tiny-model.py29CODE
LOWdebug/tiny-scripts/mt5-make-tiny-model.py30CODE
LOWdebug/tiny-scripts/mt5-make-tiny-model.py32CODE
LOWdebug/tiny-scripts/mt5-make-tiny-model.py33CODE
AI Response Leakage3 hits · 15 pts
SeverityFileLineSnippetContext
HIGHtraining/model-parallelism/README.md172[ZeRO++](https://arxiv.org/abs/2306.10209) solves the 2nd limitation by introducing Hierarchical Weight Partition for ZeCODE
HIGHinference/README.md274One other much simpler solution for [input-grounded tasks](#input-grounded-tasks), is to use [ngram prompt lookup decodiCODE
HIGHinference/README.md274One other much simpler solution for [input-grounded tasks](#input-grounded-tasks), is to use [ngram prompt lookup decodiCODE
Over-Commented Block14 hits · 14 pts
SeverityFileLineSnippetContext
LOWtraining/tools/printflock.py1# If you have ever done multi-gpu work and tried to `print` for debugging you quickly discoveredCOMMENT
LOWtraining/tools/multi-gpu-non-interleaved-print.py1#!/usr/bin/env pythonCOMMENT
LOWtraining/checkpoints/torch-checkpoint-shrink.py1#!/usr/bin/env pythonCOMMENT
LOWtraining/performance/benchmarks/numa/numa-set.sh1#!/usr/bin/bashCOMMENT
LOWnetwork/benchmarks/all_gather_object_vs_all_gather.py1#!/usr/bin/env pythonCOMMENT
LOWtesting/testing_utils.py1# I developed the bulk of this library while I worked at HFCOMMENT
LOWdebug/underflow_overflow.py1# Copyright 2020 The HuggingFace Team. All rights reserved.COMMENT
LOWdebug/tiny-scripts/openwebtext-10k.py1# coding=utf-8COMMENT
LOWdebug/tiny-scripts/general-pmd-ds-unpack.py1# unpack the desired datasets records into a filesystem-based subdir structure which can then beCOMMENT
LOWdebug/tiny-scripts/oscar-en-10k.py1# coding=utf-8COMMENT
LOWdebug/tiny-scripts/m4-ds-unpack.py1# unpack the desired datasets records into a filesystem-based subdir structure which can then beCOMMENT
LOWdebug/tiny-scripts/c4-en-10k.py1# coding=utf-8COMMENT
LOWdebug/tiny-scripts/fsmt-make-super-tiny-model.py1#!/usr/bin/env pythonCOMMENT
LOWdebug/tiny-scripts/mt5-make-tiny-model.py1#!/usr/bin/env pythonCOMMENT
Hyper-Verbose Identifiers16 hits · 14 pts
SeverityFileLineSnippetContext
LOWtraining/tools/main_process_first.py76def main_process_by_path_first(path):CODE
LOWtraining/tools/main_process_first.py103def global_main_process_first():CODE
LOWtraining/fault-tolerance/slurm-status.py44def send_email_alert_job_not_scheduled(job_name):STRING
LOWtraining/fault-tolerance/slurm-status.py63def check_running_on_jean_zay():STRING
LOWtraining/fault-tolerance/fs-watchdog.py54def check_running_on_jean_zay():STRING
LOWtraining/fault-tolerance/fs-watchdog.py114 def analyse_partition_idrquota(partition_name, partition_flag, alert_bytes_threshold, alert_inodes_threshold):STRING
LOWtesting/testing_utils.py128def require_torch_non_multi_gpu(test_case):CODE
LOWtesting/testing_utils.py203def require_bnb_non_decorator():CODE
LOWtesting/testing_utils.py308def parameterized_custom_name_func_join_params(func, param_num, param):CODE
LOWtesting/testing_utils.py323 def test_determinism_wrt_rank(self, num_workers, pad_dataset):STRING
LOWtesting/testing_utils.py777 def get_auto_remove_tmp_dir_str(self, *args, **kwargs):CODE
LOWtesting/testing_utils.py954def pytest_terminal_summary_main(tr, id):CODE
LOWtesting/README.md345def test_example_with_multi_gpu():CODE
LOWtesting/README.md586 def test_something_involving_local_locations(self):CODE
LOWtesting/README.md598 def test_something_involving_stringified_locations(self):CODE
LOWdebug/NicerTrace.py122 def localtrace_trace_and_count(self, frame, why, arg):CODE
Modern AI Meta-Vocabulary3 hits · 7 pts
SeverityFileLineSnippetContext
MEDIUMtraining/fault-tolerance/README.md185The section on [Crontab Emulation](../../orchestration/slurm/users.md#crontab-emulation) discusses how to implement cronCODE
MEDIUMorchestration/README.md25## Other orchestration frameworksCOMMENT
MEDIUMinference/README.md57For example [input-grounded tasks](#input-grounded-tasks) give the model a lot of additional information in the prompt. CODE
Deep Nesting6 hits · 6 pts
SeverityFileLineSnippetContext
LOWtraining/fault-tolerance/slurm-status.py118CODE
LOWtesting/testing_utils.py954CODE
LOWdebug/NicerTrace.py73CODE
LOWdebug/underflow_overflow.py214CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py124CODE
LOWdebug/tiny-scripts/general-pmd-synthetic-testing.py134CODE
Excessive Try-Catch Wrapping5 hits · 5 pts
SeverityFileLineSnippetContext
LOWtraining/emulate-multi-node.md148except Exception:CODE
LOWtesting/testing_utils.py46except Exception:CODE
LOWtesting/README.md804 except Exception as e:CODE
LOWdebug/torch-distributed-gpu-test.py110except Exception:CODE
LOWdebug/code/see-mem-usage.py20except Exception:CODE
Cross-Language Confusion1 hit · 2 pts
SeverityFileLineSnippetContext
HIGHdebug/tiny-scripts/cm4-synthetic-testing.py19# perl -le 'qx[touch texts_$_.null] for map {sprintf "%02d", $_} 0..38'STRING
Fake / Example Data2 hits · 2 pts
SeverityFileLineSnippetContext
LOWtraining/fault-tolerance/slurm-status.py28TO_ADDRS = ["bigscience@googlegroups.com", "foo@bar.com"] # wants a listCODE
LOWtraining/fault-tolerance/fs-watchdog.py22TO_ADDRS = ["bigscience@googlegroups.com", "foo@bar.com"] # wants a listCODE
AI Slop Vocabulary1 hit · 1 pts
SeverityFileLineSnippetContext
MEDIUMtesting/testing_utils.py397 # environment, so to make the test robust a combined entry of both streams is availableSTRING