A coding agent that evolves its own source, in public — 200 lines of Rust on day one, every commit since agent-written and tests-gated.
This report summarizes scanner evidence for yologdev/yoyo-evolve, a Rust project with 1,892 GitHub stars. SynthScan examined 267,356 lines of code across 268 source files, recording 1710 pattern matches distributed across 16 syntactic categories. The legacy adjusted pattern signal is 9.6 (Moderate pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 1262 finding(s).
Are imports declared and package/API names verified? 21 finding(s).
Are wrappers, guards, and repeated structures necessary? 514 finding(s).
Which changes need focused human review? 1710 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 1710 distinct pattern matches across 16 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/print_stdout_contract.rs | 997 | // ── The whole contract in one scenario ─────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 31 | # ── session budget helper ──────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 59 | # ── gate thresholds vs the timeouts they must cover ────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 75 | # ── coupled numbers: prompts must agree with the code ──────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 87 | # ── ordering invariant: NO function called above its definition ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 116 | # ── failed-test name extraction: exercise evolve.sh's OWN regex ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 125 | # ── scope-review checklist: prompt items == parser items (#712) ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 224 | # ── cron cadence: evolve.yml owns it, and nothing restates it ──────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 252 | # ── cross-file coupling: evolve.sh gates vs evolve.yml budgets ─────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 270 | # ── GASP: exactly one writer to the state store (#683) ─────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 325 | # ── eval-fix loop: the no-progress detector ────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 342 | # ── the no-progress COUNTER, not just its constants ────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 442 | # ── revert-receipt index + expiry ──────────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 591 | # ── accept verdict word (#915) ───────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 607 | # ── API-error detector: real failures abort, quoted JSON does not ────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 637 | # ── model: .yoyo.toml is the single source, nothing hardcodes an id ───────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 755 | # ── a reverted task's work is saved before the reset (#997) ───────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 791 | # ── main push: retry, and never echo a failure into success ──────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 849 | # ── dream.sh: post-agent outcome ───────────────────────────────────────── | COMMENT |
| MEDIUM | tests/harness_logic.sh | 883 | # ── dream.sh: the cooldown checkpoint is bounded to its own file ───────── | COMMENT |
| MEDIUM | tests/integration.rs | 36 | // ── --help ────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 78 | // ── --version ─────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 117 | // ── Empty stdin (piped mode) ──────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 212 | // ── Unknown flags ─────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 234 | // ── --no-color suppresses ANSI codes ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 271 | // ── Missing API key ──────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 337 | // ── Flags requiring values show clear errors ──────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 380 | // ── /help output lists all documented commands ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 453 | // ── --no-color output contains no ANSI escape sequences ───────────── | COMMENT |
| MEDIUM | tests/integration.rs | 498 | // ── Multiple unknown flags each produce warnings ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 538 | // ── --system-file with nonexistent file shows useful error ────────── | COMMENT |
| MEDIUM | tests/integration.rs | 586 | // ── Piped input with bad API key (needs network) ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 588 | // ── --thinking without a value ─────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 611 | // ── --verbose flag accepted ───────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 649 | // ── --allow and --deny flags accepted ─────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 719 | // ── --model without value (specific exit code + error format) ─────── | COMMENT |
| MEDIUM | tests/integration.rs | 746 | // ── Unknown slash-command-like arguments don't crash ──────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 770 | // ── Piped input with bad API key (needs network) ──────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 817 | // ── Error message quality ─────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 921 | // ── Flag combinations ─────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1002 | // ── Exit codes ────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1059 | // ── Output format ─────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1159 | // ── Edge cases ────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1336 | // ── UX timing tests ───────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1488 | // ── Dogfood UX verification tests (Issue #69) ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1488 | // ── Dogfood UX verification tests (Issue #69) ────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1696 | // ── --allow-dir and --deny-dir flags ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1809 | // ── /plan command ──────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1825 | // ── --image flag ───────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1915 | // ── Benchmark-relevant properties ─────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1989 | // ── Setup wizard wiring (Issue #157) ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 1989 | // ── Setup wizard wiring (Issue #157) ──────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2125 | // ── --no-bell ─────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2146 | // ── /map command ───────────────────────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2164 | // ── MCP collision guard (Day 39) ───────────────────────────────────── | COMMENT |
| MEDIUM | tests/integration.rs | 2193 | // ── Skill loader contract regression guard ──────────────────────────── | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 774 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 776 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 167 | # ── Markers ─────────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1088 | # ── I/O (kept at the call sites; everything above is pure) ──────────────────── | COMMENT |
| 345 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .yoyo.toml | 1 | # yoyo configuration for the evolution loop. | COMMENT |
| LOW | .yoyo.toml | 21 | COMMENT | |
| LOW | build.rs | 21 | } | COMMENT |
| LOW | tests/orphan_modules.rs | 1 | //! Orphan-module gate: every `*.rs` file under `src/` must be named by some | COMMENT |
| LOW | tests/orphan_modules.rs | 21 | //! as "checked; clean". | COMMENT |
| LOW | tests/orphan_modules.rs | 41 | COMMENT | |
| LOW | tests/doc_version_claims.rs | 1 | //! Freshness gate for prose claims about what an **external dependency** | COMMENT |
| LOW | tests/doc_version_claims.rs | 21 | //! This lives in `tests/` rather than inside the modules it scans on purpose: | COMMENT |
| LOW | tests/doc_version_claims.rs | 81 | COMMENT | |
| LOW | tests/git_chokepoint.rs | 1 | //! Git-chokepoint gate: every `std::process::Command::new("git")` in **non-test** | COMMENT |
| LOW | tests/git_chokepoint.rs | 21 | //! **What it buys, and it outlives quotepath.** *Any* future global applied at the | COMMENT |
| LOW | tests/git_chokepoint.rs | 41 | //! It is **anti-vacuous**, and that branch is asserted **first**: if the scan finds | COMMENT |
| LOW | tests/git_chokepoint.rs | 61 | //! covers **every** site inside that function — three of the seven entries | COMMENT |
| LOW | tests/git_chokepoint.rs | 81 | /// one entry vouch for a sibling it knows nothing about. | COMMENT |
| LOW | tests/git_chokepoint.rs | 421 | COMMENT | |
| LOW | tests/feature_gated_tests.rs | 1 | //! Feature-gated test-visibility gate: every `tests/*.rs` file whose first | COMMENT |
| LOW | tests/feature_gated_tests.rs | 21 | //! `tests/blind_round_grades.rs`, `tests/orphan_modules.rs`, | COMMENT |
| LOW | tests/feature_gated_tests.rs | 41 | //! that I knew a file was invisible to the harness — it is not evidence that | COMMENT |
| LOW | tests/print_stdout_contract.rs | 1 | //! #966: a process-level witness for the `-p` stdout contract. | COMMENT |
| LOW | tests/print_stdout_contract.rs | 721 | assert_eq!( | COMMENT |
| LOW | tests/print_stdout_contract.rs | 1001 | // each added a test for ITS OWN emitter. The tests above therefore each | COMMENT |
| LOW | tests/build_vcs_info.rs | 1 | //! #994: a crates.io install compiles from the unpacked registry source, which | COMMENT |
| LOW | tests/doc_symbols.rs | 1 | //! Doc-symbol freshness gate: every backticked Rust-shaped symbol named in | COMMENT |
| LOW | tests/doc_symbols.rs | 21 | //! The asymmetry is the design, not an oversight: | COMMENT |
| LOW | tests/doc_symbols.rs | 41 | //! |---|---| | COMMENT |
| LOW | tests/doc_symbols.rs | 61 | //! direction** — a registered name that now *exists* in `src/` is fatal too, | COMMENT |
| LOW | tests/doc_symbols.rs | 101 | CAT_MARKER, | COMMENT |
| LOW | tests/system_prompt_chokepoint.rs | 1 | //! Choke-point gate: every production `with_system_prompt` call must route | COMMENT |
| LOW | tests/system_prompt_chokepoint.rs | 21 | //! is the safe one: it fails loudly rather than passing quietly. | COMMENT |
| LOW | tests/git_config_injection.rs | 1 | //! A **repository** must not be able to make git execute a program on our host. | COMMENT |
| LOW | tests/git_config_injection.rs | 21 | //! underneath all of it, before any prompt, and it is not a model-generated tool | COMMENT |
| LOW | tests/git_config_injection.rs | 41 | //! ## Stated limit | COMMENT |
| LOW | tests/blind_round_grades.rs | 1 | //! Structural gate: a blind round that predicted must also grade. | COMMENT |
| LOW | tests/blind_round_grades.rs | 21 | //! | COMMENT |
| LOW | tests/blind_round_grades.rs | 41 | //! partially graded round did teach something. Same message shape (round, | COMMENT |
| LOW | tests/blind_round_grades.rs | 61 | //! | COMMENT |
| LOW | tests/blind_round_grades.rs | 81 | /// without its grade and that I knew. The ratchet in branch 3 is what keeps it | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 1 | //! Cargo-spawning-test gate: a `#[test]` under `src/` that reaches a `cargo` | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 21 | //! gives an N/N counter that feels like closure while the property stays | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 41 | //! `Command::new("cargo")`) finds **zero** violations today and would be | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 61 | //! 1. **One hop only.** A test reaching a spawner through an intermediate | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 81 | //! and the scanner is pinned by its own table test. | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 101 | /// — a registered test that stops reaching a spawn is fatal (the ratchet). | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 121 | // | COMMENT |
| LOW | tests/cargo_spawning_tests.rs | 821 | /// now been paid, so that clause would forbid the ratchet's own terminal | COMMENT |
| LOW | tests/harness_logic.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | tests/harness_logic.sh | 261 | [ "$b" -ge "$NEED" ] && ok "attempt budget ${b}s clears A-phases+one task (${NEED}s)" \ | COMMENT |
| LOW | tests/harness_logic.sh | 341 | COMMENT | |
| LOW | tests/module_size.rs | 1 | //! Structural smoke gate: a deterministic module line-count cap over `src/`. | COMMENT |
| LOW | tests/module_size.rs | 21 | //! blowing 50+ lines past the cap in one task *is* the design event. That | COMMENT |
| LOW | tests/module_size.rs | 41 | //! damage is to branch 3: a stale-high entry does not break the ratchet, it | COMMENT |
| LOW | tests/module_size.rs | 61 | COMMENT | |
| LOW | tests/module_size.rs | 81 | /// reverting the task teaches nothing the warning cannot. | COMMENT |
| LOW | tests/module_size.rs | 101 | /// with **no reader**. It goes to the stderr of a *passing* test, and the only | COMMENT |
| LOW | tests/module_size.rs | 121 | /// | COMMENT |
| LOW | tests/module_size.rs | 141 | // and their table tests were extracted to `src/commands_risk_unhittable.rs` | COMMENT |
| LOW | tests/module_size.rs | 161 | // says why the wrap is not tidiness, and the weak source-level guard | COMMENT |
| LOW | tests/module_size.rs | 181 | // builds `ModelConfig::deepseek` instead of `ModelConfig::openai`, so the | COMMENT |
| LOW | tests/module_size.rs | 201 | // +98 and not more); the rest is the new source, its doc comments, and three | COMMENT |
| LOW | tests/module_size.rs | 221 | // test `hook()` helper) now read `HookPhase::as_str`, the single source. | COMMENT |
| 982 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/harness_logic.sh | 2 | # Structural tests for the harness and its cross-file contracts — the pure | COMMENT |
| MEDIUM | tests/harness_logic.sh | 6 | # The harness grew to ~3k lines of interlocking gates whose only automated | COMMENT |
| MEDIUM | tests/harness_logic.sh | 22 | # dream.sh is the sibling harness script and, until 2026-09-22, had NO automated | COMMENT |
| MEDIUM | tests/harness_logic.sh | 388 | # Non-empty diff — green work exists. Must NOT convert the harness's | COMMENT |
| MEDIUM | tests/harness_logic.sh | 445 | # the harness suite was green on a change it could not see. Extracts the REAL | COMMENT |
| MEDIUM | tests/harness_logic.sh | 527 | # The harness runs under set -e. A BARE call (not inside an `if`, which | COMMENT |
| MEDIUM | tests/harness_logic.sh | 661 | # Every harness script resolves the model the same way, from .yoyo.toml. | COMMENT |
| MEDIUM | tests/gasp_cli_run_ordering.rs | 174 | /// Split out for #915: the unverified path must carry the harness's own | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 342 | # text*, not harness stdout, so a log containing nothing but an echoed prompt would | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 348 | # harness output. | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 357 | # SESSION_FOOTER are deliberately absent: they are bookends the harness prints whether | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 780 | # the `audit-log` branch does not contain the harness's stdout — it contains agent | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 781 | # transcripts and files the harness writes. So measuring a session *directory* for | COMMENT |
| MEDIUM⚡ | scripts/measure_abstentions.py | 788 | # Why artifacts rather than more greps: these files are written by the harness into the | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 209 | # no-harness-output one rather than replacing it. Buckets are referred to BY NAME and | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 222 | # Why that inverts the metric: "the harness printed something" and "the harness got far | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 248 | # Anchored shapes. Each is the harness's WHOLE emitted line from its first word to | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 312 | # the PROVIDER's string, not the harness's, and the phrase list mirrors | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 320 | # Tells that a line is my own writing (or a tool quoting a file) rather than harness | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1011 | # no unquoted harness marker was not measured at all, so it must not stand in for a | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1212 | # carries harness markers" and "age comes from a day-N-<ts> filename" — used to | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1308 | # 1. The four real harness lines classify correctly (verbatim from evolve.sh). | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1403 | # A real log carries harness stdout; without a phase marker this | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 1814 | # A real log carries harness stdout; without a phase marker this | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2005 | # (g) Byte-identity for the healthy path: with harness output everywhere, `grade` | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2026 | # three `=== Day N (...) ===` headers each — harness output present, no phase. | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2143 | # carries harness markers" and "age comes from a day-N-<ts> filename" — composed | COMMENT |
| MEDIUM | scripts/measure_abstentions.py | 2186 | # assertion must not overwrite a stamp the harness itself wrote. | COMMENT |
| MEDIUM⚡ | scripts/counterfactual_green.py | 1884 | # on this harness that is a per-run fact: `scripts/evolve.sh` checks out shallow and is | COMMENT |
| MEDIUM⚡ | scripts/test_scan_commitments.py | 178 | # an empty result — "couldn't check" must not reach the harness as | COMMENT |
| MEDIUM | scripts/test_scan_commitments.py | 390 | """The exact failure mode the human named on #589: when the harness | STRING |
| MEDIUM⚡ | scripts/check_assertion_weakening.py | 395 | # #[test] and #[tokio::test]. The attribute form, which is how Rust's own harness works. | COMMENT |
| MEDIUM | scripts/evolve.sh | 463 | # Inline cleanup is robust to that risk; PID-suffixed worktree paths bound | COMMENT |
| MEDIUM | scripts/evolve.sh | 513 | # Keep core skills in ./skills, but allow the harness to fetch reusable external | COMMENT |
| MEDIUM | scripts/evolve.sh | 1039 | # progress and the harness fails open on a green build+test. Each body ends by | COMMENT |
| MEDIUM | scripts/evolve.sh | 1229 | # tagged source=discussion (#589, the harness half of #582) — promises made | COMMENT |
| MEDIUM | scripts/evolve.sh | 1527 | # being spent on it. So the harness runs it here instead, before slot allocation. | COMMENT |
| MEDIUM | scripts/evolve.sh | 2036 | # the harness log will ever carry. | COMMENT |
| MEDIUM | scripts/evolve.sh | 2131 | # the harness-side backstop if it writes more files anyway. | COMMENT |
| MEDIUM | scripts/evolve.sh | 3028 | # The harness is deliberately fail-open everywhere else in | COMMENT |
| MEDIUM | scripts/evolve.sh | 3341 | # the harness's own reason carried so the decision node names it. | COMMENT |
| MEDIUM | scripts/evolve.sh | 4093 | # Anything dirty here was written by the harness itself AFTER the wrap-up | COMMENT |
| MEDIUM | scripts/evolve.sh | 4096 | # is harness state, not agent work: commit it under its own name so the | COMMENT |
| MEDIUM | scripts/skill_evolve.sh | 36 | # the eval fact must name THIS harness's actual oracle (no clippy, no | COMMENT |
| MEDIUM | scripts/skill_evolve.sh | 350 | # harness-side belt that turns them into actual constraints. | COMMENT |
| MEDIUM | scripts/format_issues.py | 203 | # to a bare "..."). The recovery command itself lives in the harness's | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 272 | # Verbatim title the harness writes when Phase A2 produces no task files. | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 1589 | # the harness never emits" are different facts with different remedies, and a | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 1689 | # carrying the emission's text in a shape the harness never emits is a | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 2142 | # lookup: the harness checkout is shallow (already documented for | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 2885 | # files -> the harness wrote its fallback task -> the fallback committed | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 2905 | # four above. Measured cause: the harness checkout is SHALLOW (50 commits, | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 4187 | # so the consumer exists and needs no harness change. ONE line, so it cannot | COMMENT |
| MEDIUM | scripts/extract_trajectory.py | 5821 | # The harness's real numbers, 2026-08-27T07:19Z: a full page of 20 whose | STRING |
| MEDIUM | scripts/extract_trajectory.py | 5998 | # The harness receipt, day 220 20:52, verbatim: branch=still-live | STRING |
| MEDIUM | scripts/extract_trajectory.py | 6253 | # State 3: the harness path. Near-miss guard — a discriminator tested only | STRING |
| MEDIUM | scripts/dream.sh | 254 | # Write ONLY these two files (the harness reverts anything else — this is a hard limit): | COMMENT |
| MEDIUM | scripts/bench/humaneval_run.sh | 6 | # It COMPOSES the single-case harness `humaneval_one.sh` (do NOT reinvent the | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 19 | # over the small inline problem set the single-case harness encodes. | COMMENT |
| MEDIUM⚡ | scripts/bench/humaneval_run.sh | 28 | # from the single-case harness. Do NOT hardcode the list independently. | COMMENT |
| 14 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/test_gasp_delivery.py | 78 | def test_publish_and_repeat_do_not_duplicate(self): | CODE |
| LOW | scripts/test_gasp_delivery.py | 86 | def test_two_concurrent_advances_are_retried_and_preserved(self): | CODE |
| LOW | scripts/test_gasp_delivery.py | 104 | def test_acknowledgement_lost_on_last_attempt_is_verified(self): | CODE |
| LOW | scripts/test_gasp_delivery.py | 119 | def test_mirror_conflict_preserves_original_and_remote_main(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 134 | def test_invalid_state_is_backed_up_but_never_published(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 144 | def test_malformed_envelope_is_backed_up_without_publishing(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 153 | def test_denied_main_can_recover_without_reexecuting_run(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 166 | def test_all_remote_writes_denied_is_local_only(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 174 | def test_reducer_version_is_pinned_to_runtime(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 181 | def test_published_legacy_skip_does_not_block_new_valid_run(self): | CODE |
| LOW⚡ | scripts/test_gasp_delivery.py | 188 | def test_unreachable_remote_has_bounded_retries_and_local_anchor(self): | CODE |
| LOW | scripts/test_gasp_delivery.py | 204 | def test_shim_exit_trap_does_not_repeat_session_end_after_failed_delivery(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 42 | def test_colon_dash_form_is_caught_too(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 46 | def test_apostrophe_outside_an_expansion_is_fine(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 51 | def test_label_appears_in_the_report(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 55 | def test_nested_braces_do_not_end_the_block_early(self): | CODE |
| LOW⚡ | scripts/test_lint_evolve_heredocs.py | 63 | def test_reported_line_number_is_the_block_start(self): | CODE |
| LOW | scripts/test_lint_evolve_heredocs.py | 89 | def test_no_argument_still_checks_the_real_evolve_sh(self): | CODE |
| LOW | scripts/measure_abstentions.py | 422 | def classify_provider_error_line(line): | CODE |
| LOW | scripts/measure_abstentions.py | 449 | def classify_harness_output_line(line): | CODE |
| LOW⚡ | scripts/counterfactual_green.py | 363 | def strip_rust_strings_and_comments(line: str, in_string: bool) -> tuple[str, bool]: | CODE |
| LOW | scripts/counterfactual_green.py | 433 | def test_diff_is_register_only(diff_text: str) -> bool: | CODE |
| LOW | scripts/counterfactual_green.py | 1111 | def parse_grandfathered_register(text): | CODE |
| LOW | scripts/counterfactual_green.py | 1143 | def partition_register_listed(candidates, register): | CODE |
| LOW | scripts/counterfactual_green.py | 1202 | def classify_src_test_readability(rows, has_parent_test_module) -> str: | CODE |
| LOW | scripts/counterfactual_green.py | 1303 | def classify_splice_eligibility(rows, register, has_parent_test_module) -> str: | CODE |
| LOW | scripts/counterfactual_green.py | 1543 | def census_shape_split_by_population(rows: list[CensusRow], shape_of) -> dict: | CODE |
| LOW | scripts/counterfactual_green.py | 1595 | def diff_mentions_assertion_shape(diff_text: str) -> bool: | CODE |
| LOW | scripts/counterfactual_green.py | 1643 | def commit_fix_loop_wall_facts(root: str, sha: str): | CODE |
| LOW | scripts/counterfactual_green.py | 2098 | def commit_src_test_readability(root: str, sha: str) -> str: | CODE |
| LOW | scripts/counterfactual_green.py | 3317 | def read_module_size_register(root: str): | STRING |
| LOW | scripts/counterfactual_green.py | 3461 | def order_src_test_only_by_eligibility(rows, eligibility_of): | STRING |
| LOW⚡ | scripts/test_scan_commitments.py | 45 | def test_skips_issues_with_no_bot_comment(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 50 | def test_skips_issues_with_no_comments(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 55 | def test_finds_last_bot_comment(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 89 | def test_truncates_long_bodies(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 97 | def test_truncates_long_git_log(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 104 | def test_extracts_json_object(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 108 | def test_tolerates_code_fence_and_prose(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 114 | def test_first_object_wins_over_trailing_data(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 121 | def test_returns_none_for_malformed_json(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 124 | def test_returns_none_for_no_json_object(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 127 | def test_returns_none_for_empty_or_non_object(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 138 | def test_empty_issues_skips_api_call(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 144 | def test_issues_without_bot_comment_skip_api(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 151 | def test_renders_outstanding_block(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 169 | def test_no_outstanding_means_no_blocks(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 187 | def test_unknown_issue_number_in_response_is_skipped(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 487 | def test_success_reads_the_output_file_not_stdout(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 494 | def test_receives_payload_and_system_file(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 509 | def test_rate_limit_is_transient(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 514 | def test_empty_response_is_unknown_not_zero(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 520 | def test_missing_binary_is_config(self): | CODE |
| LOW⚡ | scripts/test_scan_commitments.py | 528 | def test_missing_bot_login_is_config(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 71 | def test_includes_up_to_two_prior_comments(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 233 | def test_discussion_source_renders_discussion_header(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 251 | def test_missing_source_defaults_to_issue(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 268 | def test_issue_and_discussion_same_number_dont_collide(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 304 | def test_unknown_source_treated_as_issue(self): | CODE |
| LOW | scripts/test_scan_commitments.py | 366 | def test_bot_comment_present_is_triaged(self): | CODE |
| 27 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | scripts/test_scan_commitments.py | 489 | 'cat >/dev/null; echo "stdout decoy"\n' | CODE |
| HIGH⚡ | scripts/test_scan_commitments.py | 506 | self._run('cat >/dev/null; echo " error: Auth error: HTTP 401 Unauthorized: {}" >&2; exit 0\n') | CODE |
| HIGH⚡ | scripts/test_scan_commitments.py | 511 | self._run('cat >/dev/null; echo " error: API error: HTTP 429 Too Many Requests: {}" >&2; exit 1\n') | CODE |
| HIGH⚡ | scripts/test_scan_commitments.py | 517 | self._run('cat >/dev/null; echo \'{"outstanding_commitments": []}\'; exit 0\n') | CODE |
| HIGH | scripts/extract_trajectory.py | 4288 | # refusal too — 0 UNEARNED rows means coverage is undefined, not complete. | COMMENT |
| HIGH | scripts/extract_trajectory.py | 3812 | f"recorded, so coverage is undefined, not complete.{junk}" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | journals/llm-wiki.md | 17 | Added MCP documentation to the README so external agents can actually discover the server, created `mcp.json` as the sta | CODE |
| MEDIUM | journals/llm-wiki.md | 23 | Extended the MCP server with three new tools: `create_page`, `update_page`, and `agent_context` — so external agents can | CODE |
| MEDIUM | scripts/counterfactual_green.py | 219 | # literals and comments. These are the residue characters that cannot: tuple scaffolding, | COMMENT |
| MEDIUM⚡ | scripts/counterfactual_green.py | 3050 | # ever been run; this is the scaffolding that makes the reading the deliverable instead. | STRING |
| MEDIUM⚡ | scripts/test_scan_commitments.py | 188 | # Defensive: LLM hallucinates an issue number we didn't pass in. | COMMENT |
| MEDIUM | scripts/scan_commitments.py | 335 | # hallucinations are a signal the prompt or model is misbehaving. | COMMENT |
| MEDIUM | skills/skill-creator/SKILL.md | 104 | ### 6. Generate the SKILL.md scaffold | COMMENT |
| MEDIUM | src/commands_session.rs | 175 | /// This runs BEFORE a prompt attempt (not after) to prevent overflow during agentic execution. | COMMENT |
| MEDIUM | src/commands_spawn.rs | 1145 | // failure is a dim note, never a blocker (#341, orchestration scale). | COMMENT |
| MEDIUM | src/commands_spawn.rs | 4048 | // --- spawn manifest (#341: codified/replayable orchestration) --- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | scripts/build_site.py | 644 | except (ValueError, AttributeError): | CODE |
| HIGH | scripts/format_issues.py | 254 | except (json.JSONDecodeError, FileNotFoundError): | CODE |
| HIGH | scripts/format_issues.py | 261 | except ValueError: | CODE |
| HIGH | scripts/extract_trajectory.py | 4088 | except FileNotFoundError: | CODE |
| HIGH | scripts/extract_trajectory.py | 104 | except (ProcessLookupError, PermissionError, OSError): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/gasp_shim.sh | 443 | except Exception as e: # one broken stream must not stop the others | CODE |
| LOW | scripts/gasp_shim.sh | 479 | except Exception as e: | CODE |
| LOW | scripts/counterfactual_green.py | 3397 | except Exception: | STRING |
| LOW | scripts/yoyo_context.sh | 73 | except Exception as e: | CODE |
| MEDIUM | scripts/format_discussions.py | 243 | print(f"Error: REPO must be in 'owner/name' format, got: '{repo}'", file=sys.stderr) | CODE |
| MEDIUM | scripts/format_discussions.py | 249 | print(f"Error: invalid repo format: '{repo}'", file=sys.stderr) | CODE |
| MEDIUM | scripts/format_discussions.py | 312 | print("Error: repository not found in GraphQL response", file=sys.stderr) | STRING |
| LOW | scripts/evolve.sh | 330 | except Exception as exc: | CODE |
| LOW | scripts/evolve.sh | 355 | except Exception: | CODE |
| LOW | scripts/evolve.sh | 1163 | except Exception: print(0)' 2>/dev/null || echo 0) | CODE |
| LOW | scripts/evolve.sh | 1271 | except Exception as e: | CODE |
| LOW | scripts/evolve.sh | 1283 | except Exception: | CODE |
| LOW | scripts/evolve.sh | 3501 | except Exception: | CODE |
| LOW | scripts/create_address_book.sh | 125 | except Exception as e: | CODE |
| MEDIUM | scripts/create_address_book.sh | 126 | print(f'Error checking for existing Address Book: {e}', file=sys.stderr) | CODE |
| LOW | scripts/bench/humaneval_one.sh | 241 | except Exception as exc: # syntax error, import error, etc. | CODE |
| LOW | scripts/bench/humaneval_one.sh | 308 | except Exception as exc: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/gasp_delivery.py | 74 | CODE | |
| LOW | scripts/measure_abstentions.py | 735 | CODE | |
| LOW | scripts/counterfactual_green.py | 504 | CODE | |
| LOW | scripts/counterfactual_green.py | 2434 | CODE | |
| LOW | scripts/counterfactual_green.py | 3638 | CODE | |
| LOW | scripts/counterfactual_green.py | 4167 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 715 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 879 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 1004 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 2351 | CODE | |
| LOW | scripts/format_discussions.py | 491 | CODE | |
| LOW | scripts/scan_commitments.py | 286 | CODE | |
| LOW | scripts/lint_evolve_heredocs.py | 45 | CODE | |
| LOW | scripts/build_site.py | 52 | CODE | |
| LOW | scripts/refresh_sponsors.py | 331 | CODE | |
| LOW | scripts/format_issues.py | 133 | CODE | |
| LOW | scripts/extract_trajectory.py | 1295 | CODE | |
| LOW | scripts/extract_trajectory.py | 1775 | CODE | |
| LOW | scripts/extract_trajectory.py | 2281 | CODE | |
| LOW | scripts/extract_trajectory.py | 2317 | CODE | |
| LOW | scripts/extract_trajectory.py | 2590 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/commands_update.rs | 99 | // Step 1: Check for latest version | COMMENT |
| LOW | src/commands_update.rs | 137 | // Step 2: Detect platform and find the right asset | COMMENT |
| LOW | src/commands_update.rs | 167 | // Step 3: Confirm with user | COMMENT |
| LOW | src/commands_update.rs | 182 | // Step 4: Download | COMMENT |
| LOW | src/commands_update.rs | 205 | // Step 5: Extract and replace. The extract dir is per-run, and created | COMMENT |
| LOW | src/setup.rs | 364 | // Step 1: Choose provider | COMMENT |
| LOW | src/setup.rs | 402 | // Step 2: API key (skip for ollama, special flow for bedrock) | COMMENT |
| LOW | src/setup.rs | 592 | // Step 3: Model preference | COMMENT |
| LOW | src/setup.rs | 630 | // Step 4: Offer to save config (three choices) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/commands_update.rs | 99 | // Step 1: Check for latest version | COMMENT |
| LOW | src/commands_update.rs | 137 | // Step 2: Detect platform and find the right asset | COMMENT |
| LOW | src/commands_update.rs | 167 | // Step 3: Confirm with user | COMMENT |
| LOW | src/commands_update.rs | 182 | // Step 4: Download | COMMENT |
| LOW | src/commands_update.rs | 205 | // Step 5: Extract and replace. The extract dir is per-run, and created | COMMENT |
| LOW | src/setup.rs | 364 | // Step 1: Choose provider | COMMENT |
| LOW | src/setup.rs | 402 | // Step 2: API key (skip for ollama, special flow for bedrock) | COMMENT |
| LOW | src/setup.rs | 592 | // Step 3: Model preference | COMMENT |
| LOW | src/setup.rs | 630 | // Step 4: Offer to save config (three choices) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/yoyo_context.sh | 5 | # Usage: | COMMENT |
| LOW | scripts/reset_day.sh | 4 | # Usage: | COMMENT |
| LOW | scripts/evolve.sh | 7 | # Usage: | COMMENT |
| LOW | scripts/social.sh | 7 | # Usage: | COMMENT |
| LOW | scripts/evolve-local.sh | 4 | # Usage: | COMMENT |
| LOW | scripts/create_address_book.sh | 11 | # Usage: | COMMENT |
| LOW | scripts/run_mutants.sh | 4 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/integration.rs | 1836 | .env("ANTHROPIC_API_KEY", "sk-test-fake-key") | CODE |
| LOW | tests/integration.rs | 1896 | .env("ANTHROPIC_API_KEY", "sk-test-fake-key") | CODE |
| LOW | src/commands_git_review.rs | 1097 | assert!(colored.contains("John Doe")); | CODE |
| LOW | src/commands_skill.rs | 1653 | "test@test.com", | CODE |
| LOW | src/commands_info.rs | 2947 | run(&["config", "user.email", "test@test.com"]); | CODE |
| LOW | src/commands_info.rs | 3298 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/prompt_retry.rs | 1857 | std::env::set_var(key, "fake-key"); | CODE |
| LOW | src/commands_git.rs | 2934 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/commands_revisit.rs | 1055 | out.contains("(issue #999999)") && out.contains("placeholder"), | CODE |
| LOW | src/commands_spawn.rs | 3510 | .args(["config", "user.email", "test@test.com"]) | CODE |
| LOW | src/commands_file.rs | 2076 | .args(["config", "user.email", "test@test.com"]) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/skill_evolve_report.py | 22 | CODE | |
| LOW | scripts/skill_evolve_report.py | 23 | CODE | |
| LOW | scripts/counterfactual_green.py | 121 | CODE | |
| LOW | scripts/check_assertion_weakening.py | 50 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/evolve.sh | 1509 | # Check if assessment was produced | COMMENT |
| LOW | scripts/evolve.sh | 1880 | # Check if planning agent produced tasks | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | src/setup.rs | 99 | config.push_str("# This file contains a secret — keep it out of version control.\n"); | CODE |