A coding agent that evolves its own source, in public — 200 lines of Rust on day one, every commit since agent-written and tests-gated.
This report presents the forensic synthetic code analysis of yologdev/yoyo-evolve, a Rust project with 1,868 GitHub stars. SynthScan v2.0 examined 202,620 lines of code across 212 source files, recording 1080 pattern matches distributed across 15 syntactic categories. The overall adjusted score of 9.4 places this repository in the Low AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 1080 distinct pattern matches across 15 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/harness_logic.sh | 23 | # ── session budget helper ──────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 40 | # ── gate thresholds vs the timeouts they must cover ────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 56 | # ── coupled numbers: prompts must agree with the code ──────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 68 | # ── ordering invariant: NO function called above its definition ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 97 | # ── failed-test name extraction: exercise evolve.sh's OWN regex ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 106 | # ── scope-review checklist: prompt items == parser items (#712) ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 224 | # ── cross-file coupling: evolve.sh gates vs evolve.yml budgets ─────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 242 | # ── GASP: exactly one writer to the state store (#683) ─────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 297 | # ── eval-fix loop: the no-progress detector ────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 314 | # ── the no-progress COUNTER, not just its constants ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 36 | // ── --help ────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 78 | // ── --version ─────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 117 | // ── Empty stdin (piped mode) ──────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 212 | // ── Unknown flags ─────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 234 | // ── --no-color suppresses ANSI codes ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 271 | // ── Missing API key ──────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 337 | // ── Flags requiring values show clear errors ──────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 380 | // ── /help output lists all documented commands ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 453 | // ── --no-color output contains no ANSI escape sequences ───────────── | COMMENT |
| MEDIUM | tests/integration.rs | 498 | // ── Multiple unknown flags each produce warnings ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 538 | // ── --system-file with nonexistent file shows useful error ────────── | COMMENT |
| MEDIUM | tests/integration.rs | 586 | // ── Piped input with bad API key (needs network) ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 588 | // ── --thinking without a value ─────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 611 | // ── --verbose flag accepted ───────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 649 | // ── --allow and --deny flags accepted ─────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 719 | // ── --model without value (specific exit code + error format) ─────── | COMMENT |
| MEDIUM | tests/integration.rs | 746 | // ── Unknown slash-command-like arguments don't crash ──────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 770 | // ── Piped input with bad API key (needs network) ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 817 | // ── Error message quality ─────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 921 | // ── Flag combinations ─────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1002 | // ── Exit codes ────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1059 | // ── Output format ─────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1159 | // ── Edge cases ────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1336 | // ── UX timing tests ───────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1488 | // ── Dogfood UX verification tests (Issue #69) ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1488 | // ── Dogfood UX verification tests (Issue #69) ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1696 | // ── --allow-dir and --deny-dir flags ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1809 | // ── /plan command ──────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1825 | // ── --image flag ───────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1915 | // ── Benchmark-relevant properties ─────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1989 | // ── Setup wizard wiring (Issue #157) ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1989 | // ── Setup wizard wiring (Issue #157) ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2125 | // ── --no-bell ─────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2146 | // ── /map command ───────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2164 | // ── MCP collision guard (Day 39) ───────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2193 | // ── Skill loader contract regression guard ──────────────────────────── | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 774 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 776 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 167 | # ── Markers ─────────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1088 | # ── I/O (kept at the call sites; everything above is pure) ──────────────────── | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1298 | # ── Self-tests ──────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1886 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1889 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 59 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 61 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 91 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 93 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 208 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 210 | # -------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/check_assertion_weakening.py | 331 | # -------------------------------------------------------------------------------------- | COMMENT |
| 281 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/orphan_modules.rs | 1 | //! Orphan-module gate: every `*.rs` file under `src/` must be named by some | COMMENT |
| LOW | tests/orphan_modules.rs | 21 | //! as "checked; clean". | COMMENT |
| LOW | tests/orphan_modules.rs | 41 | COMMENT | |
| LOW | tests/doc_version_claims.rs | 1 | //! Freshness gate for prose claims about what an **external dependency** | COMMENT |
| LOW | tests/doc_version_claims.rs | 21 | //! This lives in `tests/` rather than inside the modules it scans on purpose: | COMMENT |
| LOW | tests/doc_version_claims.rs | 81 | COMMENT | |
| LOW | tests/doc_version_claims.rs | 221 | // Line numbers are 1-based and several markers in one file are all seen — | COMMENT |
| LOW | tests/feature_gated_tests.rs | 1 | //! Feature-gated test-visibility gate: every `tests/*.rs` file whose first | COMMENT |
| LOW | tests/feature_gated_tests.rs | 21 | //! `tests/blind_round_grades.rs`, `tests/orphan_modules.rs`, | COMMENT |
| LOW | tests/feature_gated_tests.rs | 41 | //! that I knew a file was invisible to the harness — it is not evidence that | COMMENT |
| LOW | tests/system_prompt_chokepoint.rs | 1 | //! Choke-point gate: every production `with_system_prompt` call must route | COMMENT |
| LOW | tests/system_prompt_chokepoint.rs | 21 | //! is the safe one: it fails loudly rather than passing quietly. | COMMENT |
| LOW | tests/blind_round_grades.rs | 1 | //! Structural gate: a blind round that predicted must also grade. | COMMENT |
| LOW | tests/blind_round_grades.rs | 21 | //! | COMMENT |
| LOW | tests/blind_round_grades.rs | 41 | //! partially graded round did teach something. Same message shape (round, | COMMENT |
| LOW | tests/blind_round_grades.rs | 61 | //! | COMMENT |
| LOW | tests/blind_round_grades.rs | 81 | /// without its grade and that I knew. The ratchet in branch 3 is what keeps it | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 1 | //! Cargo-spawning-test gate: a `#[test]` under `src/` that reaches a `cargo` | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 21 | //! gives an N/N counter that feels like closure while the property stays | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 41 | //! `Command::new("cargo")`) finds **zero** violations today and would be | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 61 | //! 1. **One hop only.** A test reaching a spawner through an intermediate | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 81 | //! and the scanner is pinned by its own table test. | COMMENT |
| LOW | tests/harness_logic.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | tests/harness_logic.sh | 241 | COMMENT | |
| LOW | tests/module_size.rs | 1 | //! Structural smoke gate: a deterministic module line-count cap over `src/`. | COMMENT |
| LOW | tests/module_size.rs | 21 | //! blowing 50+ lines past the cap in one task *is* the design event. That | COMMENT |
| LOW | tests/module_size.rs | 41 | //! damage is to branch 3: a stale-high entry does not break the ratchet, it | COMMENT |
| LOW | tests/module_size.rs | 61 | COMMENT | |
| LOW | tests/module_size.rs | 81 | /// reverting the task teaches nothing the warning cannot. | COMMENT |
| LOW | tests/module_size.rs | 101 | /// with **no reader**. It goes to the stderr of a *passing* test, and the only | COMMENT |
| LOW | tests/module_size.rs | 121 | /// | COMMENT |
| LOW | tests/module_size.rs | 141 | // Day 164 (#728): +98 lines — `/skill install`'s destination becomes a third | COMMENT |
| LOW | tests/module_size.rs | 161 | // conflicted files honestly instead of "all strategies failed". | COMMENT |
| LOW | tests/module_size.rs | 181 | // (map, annotation, helper, and the test pinning them); see #724. | COMMENT |
| LOW | tests/module_size.rs | 201 | // guaranteed never-forecast status and leading the list that steers the | COMMENT |
| LOW | tests/module_size.rs | 221 | // pinning that a text-less newest turn yields None, not an older turn's text. | COMMENT |
| LOW | tests/module_size.rs | 241 | // `print_context_usage` / `contextual_hint` with their doc comments, one | COMMENT |
| LOW | tests/module_size.rs | 261 | // mechanical, plus the test that pins the emitted bytes. | COMMENT |
| LOW | tests/module_size.rs | 281 | // matcher this deliberately avoids. | COMMENT |
| LOW | tests/module_size.rs | 301 | // guard (two constructors) + three Arc-path enforcement tests. The tests | COMMENT |
| LOW | tests/module_size.rs | 321 | // build_watch_fix_prompt, with its own test; see #724. | COMMENT |
| LOW | tests/module_size.rs | 361 | ceiling: usize, | COMMENT |
| LOW | tests/module_size.rs | 501 | /// I/O stays at the single call site. | COMMENT |
| LOW | tests/global_state_races.rs | 1 | //! Shared-global test-race gate: a `#[test]` in `src/` that writes a | COMMENT |
| LOW | tests/global_state_races.rs | 21 | //! buys the enumeration. | COMMENT |
| LOW | tests/global_state_races.rs | 41 | //! off after eight days of nobody reading a passing test's warning. | COMMENT |
| LOW | tests/global_state_races.rs | 61 | //! none in `src/` today, and the scanner is pinned by its own table test. | COMMENT |
| LOW | tests/global_state_races.rs | 181 | // test IS the global setter (`enable_quiet` is `QUIET.set(true)`, | COMMENT |
| LOW | tests/gasp_session_end_guard.rs | 1 | //! Source-level ordering guard for `gasp::session_end`'s session-node close (#849). | COMMENT |
| LOW | tests/gasp_session_end_guard.rs | 21 | //! **Why this file is not feature-gated, deliberately.** `src/gasp.rs` sits | COMMENT |
| LOW | tests/gasp_cli_run_ordering.rs | 1 | //! #831 — a four-call `yoyo gasp` session must not interrupt its own run. | COMMENT |
| LOW | tests/gasp_cli_run_ordering.rs | 21 | //! | COMMENT |
| LOW | tests/gasp_cli_run_ordering.rs | 41 | //! The whole file is `#![cfg(feature = "gasp")]`, so a **plain** `cargo test` | COMMENT |
| LOW | tests/gasp_cli_run_ordering.rs | 221 | COMMENT | |
| LOW | tests/integration.rs | 2161 | // is in unit tests (map_in_help_text). | COMMENT |
| LOW | tests/integration.rs | 2241 | "skill-evolve meta-skill must be present" | COMMENT |
| LOW | .cargo/mutants.toml | 1 | # cargo-mutants configuration for yoyo | COMMENT |
| LOW | .cargo/mutants.toml | 21 | # failed to start. The live key is `exclude_re`: a list of regexps matched | COMMENT |
| LOW | .cargo/mutants.toml | 41 | # CLAUDE.md records mutation readings for these five files: | COMMENT |
| LOW | scripts/measure_abstentions.py | 181 | # a silently-absorbed unknown inside my own meter is the defect this tool exists for) | COMMENT |
| 500 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/harness_logic.sh | 2 | # Structural tests for the harness and its cross-file contracts — the pure | COMMENT |
| MEDIUM | tests/harness_logic.sh | 6 | # The harness grew to ~3k lines of interlocking gates whose only automated | COMMENT |
| MEDIUM | tests/harness_logic.sh | 354 | # Non-empty diff — green work exists. Must NOT convert the harness's | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 342 | # text*, not harness stdout, so a log containing nothing but an echoed prompt would | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 348 | # harness output. | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 357 | # SESSION_FOOTER are deliberately absent: they are bookends the harness prints whether | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 780 | # the `audit-log` branch does not contain the harness's stdout — it contains agent | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 781 | # transcripts and files the harness writes. So measuring a session *directory* for | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 788 | # Why artifacts rather than more greps: these files are written by the harness into the | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 209 | # no-harness-output one rather than replacing it. Buckets are referred to BY NAME and | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 222 | # Why that inverts the metric: "the harness printed something" and "the harness got far | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 248 | # Anchored shapes. Each is the harness's WHOLE emitted line from its first word to | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 312 | # the PROVIDER's string, not the harness's, and the phrase list mirrors | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 320 | # Tells that a line is my own writing (or a tool quoting a file) rather than harness | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1011 | # no unquoted harness marker was not measured at all, so it must not stand in for a | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1212 | # carries harness markers" and "age comes from a day-N-<ts> filename" — used to | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1308 | # 1. The four real harness lines classify correctly (verbatim from evolve.sh). | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1403 | # A real log carries harness stdout; without a phase marker this | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1814 | # A real log carries harness stdout; without a phase marker this | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2005 | # (g) Byte-identity for the healthy path: with harness output everywhere, `grade` | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2026 | # three `=== Day N (...) ===` headers each — harness output present, no phase. | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2143 | # carries harness markers" and "age comes from a day-N-<ts> filename" — composed | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2186 | # assertion must not overwrite a stamp the harness itself wrote. | COMMENT |
| MEDIUM⚡ | scripts/test_scan_commitments.py | 167 | # an empty result — "couldn't check" must not reach the harness as | COMMENT |
| MEDIUM | scripts/test_scan_commitments.py | 384 | """The exact failure mode the human named on #589: when the harness | STRING |
| MEDIUM | scripts/evolve.sh | 422 | # Inline cleanup is robust to that risk; PID-suffixed worktree paths bound | COMMENT |
| MEDIUM | scripts/evolve.sh | 472 | # Keep core skills in ./skills, but allow the harness to fetch reusable external | COMMENT |
| MEDIUM | scripts/evolve.sh | 814 | # progress and the harness fails open on a green build+test. Each body ends by | COMMENT |
| MEDIUM | scripts/evolve.sh | 1003 | # tagged source=discussion (#589, the harness half of #582) — promises made | COMMENT |
| MEDIUM | scripts/evolve.sh | 1734 | # the harness-side backstop if it writes more files anyway. | COMMENT |
| MEDIUM | scripts/evolve.sh | 2593 | # The harness is deliberately fail-open everywhere else in | COMMENT |
| MEDIUM | scripts/skill_evolve.sh | 36 | # the eval fact must name THIS harness's actual oracle (no clippy, no | COMMENT |
| MEDIUM | scripts/skill_evolve.sh | 316 | # harness-side belt that turns them into actual constraints. | COMMENT |
| MEDIUM | scripts/format_issues.py | 203 | # to a bare "..."). The recovery command itself lives in the harness's | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 226 | # Verbatim title the harness writes when Phase A2 produces no task files. | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 1339 | # lookup: the harness checkout is shallow (already documented for | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 1948 | # so the consumer exists and needs no harness change. ONE line, so it cannot | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 3424 | # The harness's real numbers, 2026-08-27T07:19Z: a full page of 20 whose | STRING |
| MEDIUM | scripts/extract_trajectory.py | 3695 | # State 3: the harness path. Near-miss guard — a discriminator tested only | STRING |
| MEDIUM | scripts/dream.sh | 222 | # Write ONLY these two files (the harness reverts anything else — this is a hard limit): | COMMENT |
| MEDIUM | scripts/bench/humaneval_run.sh | 6 | # It COMPOSES the single-case harness `humaneval_one.sh` (do NOT reinvent the | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 19 | # over the small inline problem set the single-case harness encodes. | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 28 | # from the single-case harness. Do NOT hardcode the list independently. | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 30 | # NOTE: this is a Kind:evolve script — a bench harness for yoyo's own capability | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 37 | # anywhere. The single-case harness resolves its own REPO_ROOT/cwd, so we only | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 100 | # other = the harness itself errored (crash, missing key, build failure). | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 101 | # We count both 1 and other as "not passed", but track harness errors distinctly | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 102 | # so a harness bug isn't silently scored as a wrong answer. | COMMENT |
| MEDIUM | scripts/bench/humaneval_run.sh | 137 | # arithmetic if python3 is absent (matches the single-case harness's posture). | COMMENT |
| MEDIUM | scripts/bench/humaneval_run.sh | 149 | # headline so a harness bug is never silently scored as a wrong answer. | COMMENT |
| MEDIUM | src/main_tests.rs | 32 | // The caller already trims, but we should be robust to \n/doctor\n etc. | COMMENT |
| MEDIUM | src/prompt.rs | 2029 | /// #686: the harness greps phase logs for `refused this request`. This | COMMENT |
| MEDIUM | src/help.rs | 1794 | "#827: harness-facing door onto the ported GASP arms, behind the \ | CODE |
| MEDIUM | src/commands_info.rs | 1298 | /// Uses serde_json for robust parsing. | COMMENT |
| MEDIUM | src/tool_wrappers.rs | 912 | /// Session-wide cap on calls to high-leverage tools (`web_search`, `sub_agent`). | COMMENT |
| MEDIUM | src/format/markdown.rs | 278 | // Definitely not a fence, header, or block element — flush as | COMMENT |
| MEDIUM | src/format/markdown.rs | 321 | // Definitely not a closing fence — flush as code content immediately | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 42 | def test_colon_dash_form_is_caught_too(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 46 | def test_apostrophe_outside_an_expansion_is_fine(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 51 | def test_label_appears_in_the_report(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 55 | def test_nested_braces_do_not_end_the_block_early(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 63 | def test_reported_line_number_is_the_block_start(self): | CODE |
| LOW | scripts/test_lint_evolve_heredocs.py | 89 | def test_no_argument_still_checks_the_real_evolve_sh(self): | CODE |
| LOW | scripts/measure_abstentions.py | 422 | def classify_provider_error_line(line): | CODE |
| LOW | scripts/measure_abstentions.py | 449 | def classify_harness_output_line(line): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 43 | def test_skips_issues_with_no_bot_comment(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 48 | def test_skips_issues_with_no_comments(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 53 | def test_finds_last_bot_comment(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 87 | def test_truncates_long_bodies(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 95 | def test_truncates_long_git_log(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 102 | def test_extracts_first_text_block(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 107 | def test_returns_none_for_malformed_json(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 111 | def test_returns_none_for_no_text_block(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 115 | def test_returns_none_for_empty_content(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 127 | def test_empty_issues_skips_api_call(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 133 | def test_issues_without_bot_comment_skip_api(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 140 | def test_renders_outstanding_block(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 158 | def test_no_outstanding_means_no_blocks(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 177 | def test_unknown_issue_number_in_response_is_skipped(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 69 | def test_includes_up_to_two_prior_comments(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 226 | def test_discussion_source_renders_discussion_header(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 244 | def test_missing_source_defaults_to_issue(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 261 | def test_issue_and_discussion_same_number_dont_collide(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 297 | def test_unknown_source_treated_as_issue(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 360 | def test_bot_comment_present_is_triaged(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 383 | def test_bot_comment_dropped_by_unpaginated_fetch_is_false_negative(self): | CODE |
| LOW | scripts/check_assertion_weakening.py | 225 | def classify_assertion_change( | CODE |
| LOW | scripts/format_discussions.py | 193 | def _paginate_replies_from_cursor(comment_id, start_cursor): | STRING |
| LOW | scripts/lint_evolve_heredocs.py | 45 | def find_param_expansion_blocks(src): | CODE |
| LOW | scripts/extract_trajectory.py | 500 | def collect_subsystem_commits() -> str | None: | CODE |
| LOW | scripts/extract_trajectory.py | 528 | def render_subsystem_concentration( | CODE |
| LOW | scripts/extract_trajectory.py | 1031 | def collect_failed_ci_fingerprints(repo: str, now: datetime | None = None) -> CiScan: | CODE |
| LOW | scripts/extract_trajectory.py | 1780 | def collect_epistemic_blind_spots() -> tuple[list[str], list[str]]: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | CLAUDE.md | 193 | - `tool_wrappers.rs` — Tool decorator types (GuardedTool, TruncatingTool, ConfirmTool, ArcGuardedTool, AutoCheckTool, Re | CODE |
| MEDIUM⚡ | CLAUDE.md | 209 | - `format/mod.rs` — Color, constants, utility functions, re-exports, contextual command hints (`HintContext`, `contextua | CODE |
| MEDIUM⚡ | CLAUDE.md | 227 | - `commands_project.rs` — `/context`, `/init`, `/docs`, project-type detection, `auto_context_for_prompt` (automatic fil | CODE |
| MEDIUM | CLAUDE.md | 265 | - `commands_spawn.rs` — `/spawn` subagent orchestration: SpawnTracker, worktree isolation (with a symlink pre-flight — ` | CODE |
| MEDIUM | journals/llm-wiki.md | 17 | Added MCP documentation to the README so external agents can actually discover the server, created `mcp.json` as the sta | CODE |
| MEDIUM | journals/llm-wiki.md | 23 | Extended the MCP server with three new tools: `create_page`, `update_page`, and `agent_context` — so external agents can | CODE |
| MEDIUM⚡ | scripts/test_scan_commitments.py | 178 | # Defensive: LLM hallucinates an issue number we didn't pass in. | COMMENT |
| MEDIUM | scripts/scan_commitments.py | 361 | # hallucinations are a signal the prompt or model is misbehaving. | COMMENT |
| MEDIUM | skills/skill-creator/SKILL.md | 104 | ### 6. Generate the SKILL.md scaffold | COMMENT |
| MEDIUM | src/commands_session.rs | 156 | /// This runs BEFORE a prompt attempt (not after) to prevent overflow during agentic execution. | COMMENT |
| MEDIUM | src/commands_spawn.rs | 1093 | // failure is a dim note, never a blocker (#341, orchestration scale). | COMMENT |
| MEDIUM | src/commands_spawn.rs | 3499 | // --- spawn manifest (#341: codified/replayable orchestration) --- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/gasp_shim.sh | 417 | except Exception as e: # one broken stream must not stop the others | CODE |
| LOW | scripts/gasp_shim.sh | 453 | except Exception as e: | CODE |
| LOW | scripts/yoyo_context.sh | 73 | except Exception as e: | CODE |
| MEDIUM | scripts/format_discussions.py | 243 | print(f"Error: REPO must be in 'owner/name' format, got: '{repo}'", file=sys.stderr) | CODE |
| MEDIUM | scripts/format_discussions.py | 249 | print(f"Error: invalid repo format: '{repo}'", file=sys.stderr) | CODE |
| MEDIUM | scripts/format_discussions.py | 312 | print("Error: repository not found in GraphQL response", file=sys.stderr) | STRING |
| LOW | scripts/evolve.sh | 314 | except Exception as exc: | CODE |
| LOW | scripts/evolve.sh | 938 | except Exception: print(0)' 2>/dev/null || echo 0) | CODE |
| LOW | scripts/evolve.sh | 1045 | except Exception as e: | CODE |
| LOW | scripts/evolve.sh | 1057 | except Exception: | CODE |
| LOW | scripts/evolve.sh | 3031 | except Exception: | CODE |
| LOW | scripts/create_address_book.sh | 125 | except Exception as e: | CODE |
| MEDIUM | scripts/create_address_book.sh | 126 | print(f'Error checking for existing Address Book: {e}', file=sys.stderr) | CODE |
| LOW | scripts/bench/humaneval_one.sh | 241 | except Exception as exc: # syntax error, import error, etc. | CODE |
| LOW | scripts/bench/humaneval_one.sh | 308 | except Exception as exc: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/commands_update.rs | 99 | // Step 1: Check for latest version | COMMENT |
| LOW | src/commands_update.rs | 137 | // Step 2: Detect platform and find the right asset | COMMENT |
| LOW | src/commands_update.rs | 167 | // Step 3: Confirm with user | COMMENT |
| LOW | src/commands_update.rs | 182 | // Step 4: Download | COMMENT |
| LOW | src/commands_update.rs | 205 | // Step 5: Extract and replace. The extract dir is per-run, and created | COMMENT |
| LOW | src/setup.rs | 364 | // Step 1: Choose provider | COMMENT |
| LOW | src/setup.rs | 402 | // Step 2: API key (skip for ollama, special flow for bedrock) | COMMENT |
| LOW | src/setup.rs | 592 | // Step 3: Model preference | COMMENT |
| LOW | src/setup.rs | 630 | // Step 4: Offer to save config (three choices) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/commands_update.rs | 99 | // Step 1: Check for latest version | COMMENT |
| LOW | src/commands_update.rs | 137 | // Step 2: Detect platform and find the right asset | COMMENT |
| LOW | src/commands_update.rs | 167 | // Step 3: Confirm with user | COMMENT |
| LOW | src/commands_update.rs | 182 | // Step 4: Download | COMMENT |
| LOW | src/commands_update.rs | 205 | // Step 5: Extract and replace. The extract dir is per-run, and created | COMMENT |
| LOW | src/setup.rs | 364 | // Step 1: Choose provider | COMMENT |
| LOW | src/setup.rs | 402 | // Step 2: API key (skip for ollama, special flow for bedrock) | COMMENT |
| LOW | src/setup.rs | 592 | // Step 3: Model preference | COMMENT |
| LOW | src/setup.rs | 630 | // Step 4: Offer to save config (three choices) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/measure_abstentions.py | 735 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 225 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 349 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 576 | CODE | |
| LOW | scripts/format_discussions.py | 491 | CODE | |
| LOW | scripts/scan_commitments.py | 219 | CODE | |
| LOW | scripts/scan_commitments.py | 288 | CODE | |
| LOW | scripts/lint_evolve_heredocs.py | 45 | CODE | |
| LOW | scripts/build_site.py | 52 | CODE | |
| LOW | scripts/refresh_sponsors.py | 331 | CODE | |
| LOW | scripts/format_issues.py | 133 | CODE | |
| LOW | scripts/extract_trajectory.py | 176 | CODE | |
| LOW | scripts/extract_trajectory.py | 1031 | CODE | |
| LOW | scripts/extract_trajectory.py | 1240 | CODE | |
| LOW | scripts/extract_trajectory.py | 1458 | CODE | |
| LOW | scripts/extract_trajectory.py | 1481 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/yoyo_context.sh | 5 | # Usage: | COMMENT |
| LOW | scripts/reset_day.sh | 4 | # Usage: | COMMENT |
| LOW | scripts/evolve.sh | 7 | # Usage: | COMMENT |
| LOW | scripts/social.sh | 7 | # Usage: | COMMENT |
| LOW | scripts/evolve-local.sh | 4 | # Usage: | COMMENT |
| LOW | scripts/create_address_book.sh | 11 | # Usage: | COMMENT |
| LOW | scripts/run_mutants.sh | 4 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/integration.rs | 1836 | .env("ANTHROPIC_API_KEY", "sk-test-fake-key") | CODE |
| LOW | tests/integration.rs | 1896 | .env("ANTHROPIC_API_KEY", "sk-test-fake-key") | CODE |
| LOW | src/commands_git_review.rs | 1087 | assert!(colored.contains("John Doe")); | CODE |
| LOW | src/commands_skill.rs | 1620 | "test@test.com", | CODE |
| LOW | src/commands_info.rs | 2567 | run(&["config", "user.email", "test@test.com"]); | CODE |
| LOW | src/commands_info.rs | 2918 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/prompt_retry.rs | 1754 | std::env::set_var(key, "fake-key"); | CODE |
| LOW | src/commands_git.rs | 2687 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/commands_revisit.rs | 1055 | out.contains("(issue #999999)") && out.contains("placeholder"), | CODE |
| LOW | src/commands_spawn.rs | 3160 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/commands_file.rs | 2032 | .args(["config", "user.email", "test@test.com"]) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | CLAUDE.md | 218 | - `git_commit_msg.rs` — the deterministic, token-free commit-message generator behind `/commit` (`generate_commit_messag | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/skill_evolve_report.py | 22 | CODE | |
| LOW | scripts/skill_evolve_report.py | 23 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 50 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/evolve.sh | 1282 | # Check if assessment was produced | COMMENT |
| LOW | scripts/evolve.sh | 1568 | # Check if planning agent produced tasks | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | src/setup.rs | 99 | config.push_str("# This file contains a secret — keep it out of version control.\n"); | CODE |