AI Slop Code RIsk Monitor

microsoft/agent-lightning

The absolute trainer to light up AI agents.

19.7 Elevated pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for microsoft/agent-lightning, a Python project with 18,646 GitHub stars. SynthScan examined 17,701 lines of code across 139 source files, recording 251 pattern matches distributed across 13 syntactic categories. The legacy adjusted pattern signal is 19.7 (Elevated pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

19.7
Adjusted Signal
19.7
Raw Pattern Score
100%
Time Factor
2026-10-09
Last Push
18.6K
Stars
Python
Language
17.7K
Lines of Code
139
Files
251
Pattern Hits
2026-10-12
Scan Date
0.12
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 37.2/100 · MEDIUM

Can maintainers explain intent, assumptions, and trade-offs? 146 finding(s).

Hallucinated Dependency & Supply Chain Risk 18.3/100 · LOW

Are imports declared and package/API names verified? 51 finding(s).

Agent Over-Engineering Ratio 49.8/100 · MEDIUM

Are wrappers, guards, and repeated structures necessary? 176 finding(s).

PR Review Velocity Impact 63.1/100 · HIGH

Which changes need focused human review? 251 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 1HIGH 15MEDIUM 9LOW 226

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 251 distinct pattern matches across 13 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers122 hits · 132 pts
SeverityFileLineSnippetContext
LOW⚡tests/test_client.py29def test_sync_key_replaces_authorization_case_insensitively(headers: httpx.Headers | dict[str, str]) -> None:CODE
LOW⚡tests/test_client.py39async def test_async_key_replaces_authorization_case_insensitively(headers: httpx.Headers | dict[str, str]) -> None:CODE
LOW⚡tests/test_client.py49def test_sync_without_key_preserves_authorization(key: str | None) -> None:CODE
LOW⚡tests/test_client.py56async def test_async_without_key_preserves_authorization(key: str | None) -> None:CODE
LOWtests/server/test_endpoints.py83def test_patch_rollout_rejects_unknown_top_level_fields(CODE
LOWtests/server/test_endpoints.py99def test_list_rollouts_filters_by_state_in(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_endpoints.py169def test_enqueue_with_client_rollout_id_is_idempotent(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_endpoints.py202def test_re_enqueue_does_not_clobber_a_running_rollout(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_endpoints.py225def test_triplet_events_keep_last_model_request_for_duplicate_prompt(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_endpoints.py274def test_triplet_events_do_not_dedupe_without_valid_prompt_tokens(CODE
LOWtests/server/test_endpoints.py329def test_proxy_completion_endpoint(client: TestClient, auth_headers: dict[str, str], monkeypatch):CODE
LOWtests/server/test_endpoints.py370def test_proxy_rejects_non_object_json_body(CODE
LOWtests/server/test_endpoints.py387def test_proxy_error_triplet_preserves_status(client: TestClient, auth_headers: dict[str, str], monkeypatch):CODE
LOWtests/server/test_endpoints.py422def test_proxy_admin_endpoints(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_endpoints.py460def test_terminal_rollouts_cursor_pagination(client: TestClient, auth_headers: dict[str, str]):CODE
LOWtests/server/test_proxy_disconnect.py55async def test_disconnect_drains_only_abandoned_request(CODE
LOWtests/server/test_proxy_disconnect.py142async def test_cancelled_handler_joins_forwarding_and_disconnect_watcher(CODE
LOWtests/controller/test_local_reconciler_platform.py16async def test_run_fails_before_reconciling_or_spawning_on_native_windows(CODE
LOWtests/controller/test_local_reconciler_platform.py43async def test_run_reconciles_and_shuts_down_on_supported_platform(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/controller/test_k8s_reconciler.py11def test_build_job_spec_uses_agentlightning_labels() -> None:CODE
LOWtests/controller/test_local_reconciler_env.py24async def test_spawn_uses_agent_url_for_worker_endpoints(CODE
LOWtests/controller/test_local_reconciler.py72async def test_shutdown_does_not_spawn_queued_rollouts(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/controller/test_local_reconciler.py84async def test_normal_reconcile_still_spawns_queued_rollouts(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/controller/test_local_reconciler.py95async def test_reconcile_does_not_spawn_after_stop_requested_during_poll(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/controller/test_local_reconciler.py121async def test_shutdown_still_fails_running_rollout_without_local_process() -> None:CODE
LOWtests/controller/test_local_reconciler.py135async def test_timeout_reconciliation_retries_and_retains_process_record(CODE
LOWtests/controller/test_local_reconciler.py187async def test_unmarked_terminal_process_retries_and_retains_record(CODE
LOWtests/controller/test_local_reconciler.py215async def test_run_stops_then_shutdown_kills_after_final_reconcile(CODE
LOWtests/controller/test_local_reconciler.py252async def test_timeout_during_run_retries_original_reason_during_shutdown_without_rekill(CODE
LOWtests/verl/test_rollout_level_advantage.py58def _compute_rollout_level_advantage(CODE
LOWtests/verl/test_rollout_level_advantage.py74def test_rollout_level_advantage_collapses_rollouts_and_broadcasts_triplets() -> None:CODE
LOW⚡tests/verl/test_rollout_level_advantage.py117def test_rollout_level_advantage_requires_rollout_ids() -> None:CODE
LOW⚡tests/verl/test_rollout_level_advantage.py125def test_rollout_level_advantage_rejects_inconsistent_rollout_rewards() -> None:CODE
LOW⚡tests/verl/test_rollout_level_advantage.py132def test_rollout_level_advantage_rejects_non_scalar_token_advantages() -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py55def test_build_completed_rollout_skips_error_and_empty_model_requests() -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py148def _triplet_model_request_event(prompt_token_ids: object, response_token_ids: list[int]) -> Event:CODE
LOWtests/verl/test_agl_rollout_manager.py163def test_extract_image_urls_from_messages() -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py187def test_build_completed_rollout_aligns_image_urls_with_triplets() -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py225def test_build_completed_rollout_aligns_images_without_valid_prompt_token_ids(prompt_token_ids: object) -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py252def test_aligned_image_urls_count_mismatch_returns_none(capsys: pytest.CaptureFixture[str]) -> None:CODE
LOWtests/verl/test_agl_rollout_manager.py259def test_aligned_image_urls_text_only_rollout_returns_none_without_warning(CODE
LOWtests/verl/test_per_rollout_loss.py36def test_normalize_advantages_by_rollout() -> None:CODE
LOWtests/verl/test_per_rollout_loss.py60def test_policy_loss_matches_masked_sum() -> None:CODE
LOWtests/verl/test_per_rollout_loss.py77def test_normalize_advantages_validates_inputs() -> None:CODE
LOWtests/verl/test_rollout_adapter.py42def test_routed_experts_batch_preserves_padding_and_truncation(dtype: type[np.generic]) -> None:CODE
LOWtests/verl/test_rollout_adapter.py62def test_routed_experts_batch_releases_decoded_rows(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/verl/test_rollout_adapter.py82def test_routed_experts_batch_rejects_short_route_sequence() -> None:CODE
LOWtests/verl/test_rollout_adapter.py202def test_trajectory_aggregation_preserves_input_log_probs(prefix_mismatch: bool, missing_log_probs: bool) -> None:CODE
LOWtests/verl/test_rollout_adapter.py266def test_trajectory_prefix_mismatch_uploads_trace_merge_table_to_wandb(CODE
LOWtests/verl/test_rollout_adapter.py393def test_validation_uploads_all_compact_rollout_trajectories_to_wandb_zip(CODE
LOWtests/verl/test_rollout_adapter.py542def test_transition_image_rows_attach_multi_modal_inputs_and_mrope_position_ids(CODE
LOWtests/verl/test_rollout_adapter.py570def test_text_only_rows_keep_original_behavior_with_processor() -> None:CODE
LOWtests/verl/test_rollout_adapter.py587def test_trajectory_level_with_image_triplets_raises() -> None:CODE
LOWtests/verl/test_rollout_adapter.py603def test_truncated_image_prompt_falls_back_to_text_only(CODE
LOWtests/verl/test_rollout_adapter.py638def test_processor_without_vision_output_marks_image_row_for_drop(CODE
LOWtests/verl/test_rollout_adapter.py656def test_failed_image_rows_are_removed_by_trainer_filter(failure: str, monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOWtests/verl/test_rollout_adapter.py695def test_is_mrope_processor_detection() -> None:CODE
LOWtests/verl/test_rollout_adapter.py705def test_load_pil_image_decodes_data_url() -> None:CODE
LOWtests/verl/test_rollout_adapter.py719def test_load_pil_image_fetches_remote_url_by_default(monkeypatch: pytest.MonkeyPatch) -> None:CODE
LOW⚡tests/examples/test_swe_smith_images.py65def test_openai_image_name_matches_job_template_convention() -> None:CODE
62 more matches not shown…
Unused Imports47 hits · 46 pts
SeverityFileLineSnippetContext
LOWtests/server/conftest.py5CODE
LOWtests/server/test_endpoints.py5CODE
LOWtests/verl/test_rollout_level_advantage.py3CODE
LOWtests/verl/test_agl_rollout_manager.py5CODE
LOWtests/verl/test_per_rollout_loss.py3CODE
LOWtests/verl/test_rollout_adapter.py3CODE
LOWtests/examples/test_swe_smith_images.py3CODE
LOWtests/examples/test_swe_smith_agent.py12CODE
LOWdocs/macros/source_links.py3CODE
LOWexamples/calc_x/calc_agent.py8CODE
LOWexamples/swe_smith/pull_images.py12CODE
LOWexamples/swe_smith/train_smith_agent_moe.py4CODE
LOWexamples/swe_smith/train_smith_agent.py3CODE
LOWexamples/swe_smith/train_smith_agent_megatron.py5CODE
LOWexamples/swe_smith/agents/smith_agent.py5CODE
LOWexamples/search_r1/retrieval_server.py3CODE
LOWexamples/search_r1/train_search_r1_agent.py3CODE
LOWexamples/search_r1/agents/qa_em.py17CODE
LOWexamples/search_r1/agents/search_r1_agent.py3CODE
LOWexamples/science_world/train_sw_agent.py22CODE
LOWexamples/science_world/agents/sw_agent.py28CODE
LOWexamples/gsm8k/gsm8k_agent.py8CODE
LOWexamples/multimodal_qa/multimodal_qa_agent.py8CODE
LOWexamples/llm-in-sandbox/train_llm_in_sandbox.py6CODE
LOWexamples/llm-in-sandbox/agents/runner.py6CODE
LOWscripts/check_headers.py5CODE
LOWagentlightning/hooks.py5CODE
LOWagentlightning/client.py5CODE
LOWagentlightning/schemas.py5CODE
LOWagentlightning/server/store.py9CODE
LOWagentlightning/server/proxy.py5CODE
LOWagentlightning/server/app.py5CODE
LOWagentlightning/server/__main__.py5CODE
LOWagentlightning/server/routes/models.py5CODE
LOWagentlightning/server/routes/events.py5CODE
LOWagentlightning/server/routes/proxy.py5CODE
LOWagentlightning/server/routes/rollouts.py5CODE
LOWagentlightning/controller/k8s_reconciler.py12CODE
LOWagentlightning/controller/local_reconciler.py5CODE
LOWagentlightning/controller/__main__.py5CODE
LOWagentlightning/verl/agl_rollout_manager.py5CODE
LOWagentlightning/verl/rollout_adapter.py5CODE
LOWagentlightning/verl/per_rollout_loss.py5CODE
LOWagentlightning/verl/entrypoint.py13CODE
LOWagentlightning/verl/rollout_level_advantage.py5CODE
LOWagentlightning/verl/vllm_compat.py5CODE
LOWagentlightning/verl/trainer.py5CODE
Cross-File Repetition9 hits · 45 pts
SeverityFileLineSnippetContext
HIGHexamples/calc_x/train_calc_agent.py0load gsm8k datasets, build config, and launch verl training via agent lightning.STRING
HIGHexamples/search_r1/train_search_r1_agent.py0load gsm8k datasets, build config, and launch verl training via agent lightning.STRING
HIGHexamples/llm-in-sandbox/train_llm_in_sandbox.py0load gsm8k datasets, build config, and launch verl training via agent lightning.STRING
HIGHexamples/gsm8k/train_gsm8k_agent.py0load gsm8k datasets, build config, and launch verl training via agent lightning.STRING
HIGHexamples/search_r1/train_search_r1_agent.py0build the full omegaconf config by merging base + overrides.STRING
HIGHexamples/science_world/train_sw_agent.py0build the full omegaconf config by merging base + overrides.STRING
HIGHexamples/gsm8k/train_gsm8k_agent.py0build the full omegaconf config by merging base + overrides.STRING
HIGHexamples/multimodal_qa/train_multimodal_qa.py0build the full omegaconf config by merging base + overrides.STRING
HIGHexamples/llm-in-sandbox/train_llm_in_sandbox.py0build the full omegaconf config by merging base + overrides.STRING
Excessive Try-Catch Wrapping31 hits · 32 pts
SeverityFileLineSnippetContext
LOWexamples/calc_x/eval_utils.py66 except Exception:CODE
LOWexamples/swe_smith/pull_images.py106 except Exception:CODE
LOWexamples/swe_smith/pull_images.py184 except Exception as exc:CODE
LOWexamples/swe_smith/agents/smith_agent.py354 except Exception as exc:CODE
LOWexamples/swe_smith/agents/smith_agent.py571 except Exception as exc:CODE
LOWexamples/swe_smith/agents/smith_agent.py838 except Exception as exc:CODE
MEDIUMexamples/swe_smith/agents/smith_agent.py561def capture_patch() -> str:CODE
LOWexamples/science_world/agents/sw_agent.py224 except Exception as e:CODE
LOWexamples/llm-in-sandbox/agents/runner.py127 except Exception as exc:CODE
LOWagentlightning/client.py58 except Exception as exc:CODE
LOWagentlightning/client.py76 except Exception as exc:CODE
LOWagentlightning/controller/k8s_reconciler.py152 except Exception:CODE
LOWagentlightning/controller/k8s_reconciler.py266 except Exception as exc:CODE
LOWagentlightning/controller/k8s_reconciler.py296 except Exception:CODE
LOWagentlightning/controller/k8s_reconciler.py360 except Exception as exc:CODE
LOWagentlightning/controller/local_reconciler.py48 except Exception:CODE
LOWagentlightning/controller/local_reconciler.py129 except Exception:CODE
LOWagentlightning/controller/local_reconciler.py250 except Exception as e:CODE
LOWagentlightning/controller/local_reconciler.py268 except Exception:CODE
LOWagentlightning/controller/local_reconciler.py296 except Exception as e:CODE
LOWagentlightning/verl/agl_rollout_manager.py311 except Exception as exc:CODE
LOWagentlightning/verl/agl_rollout_manager.py325 except Exception as exc:CODE
LOWagentlightning/verl/agl_rollout_manager.py454 except Exception:CODE
LOWagentlightning/verl/agl_rollout_manager.py465 except Exception:CODE
LOWagentlightning/verl/rollout_adapter.py56 except Exception:CODE
LOW⚡agentlightning/verl/rollout_adapter.py127 except Exception as exc:CODE
LOWagentlightning/verl/rollout_adapter.py162 except Exception as exc:CODE
LOWagentlightning/verl/rollout_adapter.py715 except Exception as exc:CODE
LOWagentlightning/verl/rollout_adapter.py740 except Exception as exc:CODE
LOWagentlightning/verl/rollout_adapter.py756 except Exception:CODE
LOWagentlightning/verl/entrypoint.py134 except Exception as exc:CODE
Cross-Language Confusion3 hits · 15 pts
SeverityFileLineSnippetContext
HIGHtests/examples/test_swe_smith_agent.py207 "cd /testbed && git checkout 614b134 -- pkg/x.py",CODE
HIGHexamples/swe_smith/agents/smith_agent.py97commands connected with && or ||).CODE
HIGHexamples/swe_smith/agents/smith_agent.py155- That block MUST contain EXACTLY ONE command (or several joined with && or ||).CODE
Empty Error Handlers3 hits · 15 pts
SeverityFileLineSnippetContext
HIGHexamples/calc_x/eval_utils.py66except Exception:CODE
HIGHagentlightning/controller/k8s_reconciler.py158except TimeoutError:CODE
HIGHagentlightning/controller/local_reconciler.py134except TimeoutError:CODE
Deep Nesting15 hits · 15 pts
SeverityFileLineSnippetContext
LOWexamples/swe_smith/pull_images.py149CODE
LOWexamples/swe_smith/agents/smith_agent.py719CODE
LOWexamples/search_r1/retrieval_server.py124CODE
LOWagentlightning/server/routes/events.py125CODE
LOWagentlightning/controller/k8s_reconciler.py161CODE
LOWagentlightning/controller/k8s_reconciler.py300CODE
LOWagentlightning/controller/local_reconciler.py137CODE
LOWagentlightning/verl/agl_rollout_manager.py146CODE
LOWagentlightning/verl/agl_rollout_manager.py174CODE
LOWagentlightning/verl/agl_rollout_manager.py540CODE
LOWagentlightning/verl/agl_rollout_manager.py596CODE
LOWagentlightning/verl/rollout_adapter.py403CODE
LOWagentlightning/verl/trainer.py290CODE
LOWagentlightning/verl/trainer.py661CODE
LOWagentlightning/verl/trainer.py302CODE
Decorative Section Separators4 hits · 12 pts
SeverityFileLineSnippetContext
MEDIUMtests/verl/test_rollout_adapter.py449# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/verl/test_rollout_adapter.py451# ---------------------------------------------------------------------------COMMENT
MEDIUMagentlightning/verl/rollout_adapter.py219# ---------------------------------------------------------------------------COMMENT
MEDIUMagentlightning/verl/rollout_adapter.py226# ---------------------------------------------------------------------------COMMENT
AI Slop Vocabulary4 hits · 12 pts
SeverityFileLineSnippetContext
MEDIUMtests/examples/test_swe_smith_agent.py245 # Network/package-install/test-harness-tamper routes are reward-hackingCOMMENT
MEDIUMtests/examples/test_swe_smith_agent.py294 # .git moved out of the worktree and harness now targets it explicitly.COMMENT
MEDIUMexamples/swe_smith/agents/smith_agent.py28# harness still reaches it via --git-dir/--work-tree. See also _forbidden_action.COMMENT
MEDIUMexamples/swe_smith/agents/smith_agent.py75# Writing test-harness/config files (conftest, .pth, sitecustomize) can forceCOMMENT
Hallucination Indicators1 hit · 10 pts
SeverityFileLineSnippetContext
CRITICALagentlightning/verl/trainer.py632 loss_mode = self.config.actor_rollout_ref.actor.policy_loss.get("loss_mode", "vanilla")CODE
Modern Structural Boilerplate7 hits · 8 pts
SeverityFileLineSnippetContext
LOWexamples/science_world/agents/sw_agent.py75def _setup_logging() -> None:CODE
LOWagentlightning/verl/agl_rollout_manager.py682__all__ = [CODE
LOWagentlightning/verl/rollout_adapter.py902__all__ = ["RolloutAdapter"]CODE
LOWagentlightning/verl/entrypoint.py28__all__ = [CODE
LOWagentlightning/verl/dataset.py12__all__ = [CODE
LOW⚡agentlightning/verl/vllm_compat.py10__all__ = ["apply_vllm_compat_patches"]CODE
LOW⚡agentlightning/verl/vllm_compat.py12logger = logging.getLogger(__name__)CODE
Structural Annotation Overuse3 hits · 4 pts
SeverityFileLineSnippetContext
LOWdocs/00-installation.md7#### Step 1: UV SyncCOMMENT
LOWdocs/00-installation.md18#### Step 2: Install `verl` and FlashAttentionCOMMENT
LOWdocs/00-installation.md35#### Step 3: W&B LoginCOMMENT
Over-Commented Block2 hits · 2 pts
SeverityFileLineSnippetContext
LOWexamples/swe_smith/requirements.txt1# Extra dependencies for the swe_smith example, beyond the Agent Lightning rootCOMMENT
LOWexamples/search_r1/agents/qa_em.py1# Copyright (c) Microsoft. All rights reserved.COMMENT