AI Slop Code RIsk Monitor

UKGovernmentBEIS/inspect_ai

Inspect: A framework for large language model evaluations

31.9 High pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for UKGovernmentBEIS/inspect_ai, a Python project with 2,983 GitHub stars. SynthScan examined 710,576 lines of code across 2094 source files, recording 18058 pattern matches distributed across 23 syntactic categories. The legacy adjusted pattern signal is 31.9 (High pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

31.9
Adjusted Signal
31.9
Raw Pattern Score
100%
Time Factor
2026-10-10
Last Push
3.0K
Stars
Python
Language
710.6K
Lines of Code
2.1K
Files
18.1K
Pattern Hits
2026-10-12
Scan Date
0.29
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 60.5/100 · HIGH

Can maintainers explain intent, assumptions, and trade-offs? 13966 finding(s).

Hallucinated Dependency & Supply Chain Risk 13.5/100 · LOW

Are imports declared and package/API names verified? 1730 finding(s).

Agent Over-Engineering Ratio 72.6/100 · HIGH

Are wrappers, guards, and repeated structures necessary? 14938 finding(s).

PR Review Velocity Impact 81.9/100 · CRITICAL

Which changes need focused human review? 18058 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 6HIGH 598MEDIUM 1609LOW 15845

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 18058 distinct pattern matches across 23 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers12500 hits · 10906 pts
SeverityFileLineSnippetContext
LOWdesign/ctl/config-log-persistence.md161def effective_generate_config(log: EvalLog) -> GenerateConfig: ...CODE
LOWtests/test_task_cancel.py34def test_abort_cancel_produces_error_status() -> None:CODE
LOWtests/test_task_cancel.py93def test_abort_cancel_not_retried_without_task_retries() -> None:CODE
LOWtests/test_task_cancel.py166def test_score_resolution_cancel_completes_eval() -> None:CODE
LOWtests/test_task_cancel.py223def test_error_resolution_cancel_completes_eval() -> None:CODE
LOWtests/test_task_cancel.py265def test_error_resolution_rejected_when_samples_fail_on_error() -> None:CODE
LOWtests/test_task_cancel.py274 def error_resolution_rejected_solver():CODE
LOWtests/test_task_cancel.py303def test_error_resolution_downgraded_for_materializing_fail_on_error_sample() -> None:CODE
LOWtests/test_task_cancel.py329 def downgrade_resolution_solver():CODE
LOWtests/test_task_cancel.py366def test_drain_resolution_cancels_materializing_sample() -> None:CODE
LOWtests/test_task_cancel.py446def test_sample_cancelled_interrupt_action() -> None:CODE
LOWtests/test_task_cancel.py455 def cancelled_interrupt_solver():CODE
LOWtests/test_task_cancel.py493def test_score_resolution_sweep_preserves_cancelled_sample() -> None:CODE
LOWtests/test_task_cancel.py506 def cancelled_then_score_solver():CODE
LOWtests/test_task_cancel.py543def test_interrupt_in_retry_drain_window_resolves_cancelled() -> None:CODE
LOWtests/test_task_cancel.py571 def drain_window_error_solver():CODE
LOWtests/test_task_cancel.py642def test_external_interrupt_with_pending_resolution_logs_cancelled(CODE
LOWtests/test_task_cancel.py828def test_sample_cancel_while_initializing_resolves_at_start(CODE
LOWtests/test_task_cancel.py909def test_sample_cancel_while_initializing_wins_over_later_task_score(CODE
LOWtests/test_task_cancel.py958def test_sample_cancel_while_initializing_then_init_failure_abandons(CODE
LOWtests/test_task_cancel.py1036def test_errored_attempt_marked_retry_pending() -> None:CODE
LOWtests/test_eval_set_capture.py68def test_eval_set_capture_manifest(CODE
LOWtests/test_eval_set_capture.py108def test_eval_set_capture_limit_and_epochs(CODE
LOWtests/test_eval_set_capture.py126def test_eval_set_capture_max_samples(CODE
LOWtests/test_eval_set_capture.py137def test_eval_set_capture_model_roles(CODE
LOWtests/test_eval_set_capture.py148def test_eval_set_capture_model_roles_list(CODE
LOWtests/test_eval_set_capture.py162def test_eval_set_capture_adhoc_task(CODE
LOWtests/test_eval_set_capture.py191def test_eval_set_capture_task_source_error(CODE
LOWtests/test_eval_set_capture.py204def test_eval_set_capture_subprocess_protocol(tmp_path: Path) -> None:CODE
LOWtests/test_eval_set_capture.py347def test_eval_set_capture_schema_stability() -> None:CODE
LOW⚡tests/test_sample_limits.py141def test_message_limit_generate():CODE
LOW⚡tests/test_sample_limits.py145def test_message_limit_append():CODE
LOW⚡tests/test_sample_limits.py149def test_message_limit_overwrite():CODE
LOW⚡tests/test_sample_limits.py153def test_message_limit_reached_before_assistant_message():CODE
LOWtests/test_sample_limits.py70def looping_concurrecy_solver():CODE
LOWtests/test_sample_limits.py169def test_message_limit_does_not_apply_to_scorer():CODE
LOWtests/test_sample_limits.py218def test_token_limit_reached_exactly_refuses_next_generate():CODE
LOWtests/test_sample_limits.py239def test_token_limit_does_not_apply_to_scorer():CODE
LOWtests/test_sample_limits.py285def test_output_token_limit_string_form():CODE
LOWtests/test_sample_limits.py320def test_token_limit_type_absent_for_int_limit():CODE
LOWtests/test_sample_limits.py367def test_turn_limit_does_not_apply_to_scorer():CODE
LOWtests/test_sample_limits.py415def test_sample_limits_available_to_scorer():CODE
LOWtests/test_sample_limits.py446def test_solver_scorer_combined_timeout():CODE
LOWtests/test_sample_limits.py455def test_solver_scorer_combined_timeout_exceeded():CODE
LOWtests/test_sample_limits.py466def test_solver_timeout_scored():CODE
LOWtests/test_sample_limits.py475def test_solver_timeout_not_scored():CODE
LOWtests/test_sample_limits.py494def test_working_limit_reporting():CODE
LOWtests/test_sample_limits.py511def test_working_limit_does_not_raise_during_sandbox_teardown() -> None:CODE
LOWtests/test_sample_limits.py526def check_working_limit_event(log: EvalLog, working_limit: int):CODE
LOWtests/test_sample_limits.py580def test_cost_limit_without_cost_data_errors() -> None:CODE
LOWtests/test_sample_limits.py592def test_model_without_cost_data_errors() -> None:CODE
LOWtests/test_sample_limits.py609def test_cost_data_without_cost_limit_tracks_cost() -> None:CODE
LOWtests/test_sample_limits.py640def test_cost_data_keyed_by_full_model_string_tracks_cost() -> None:CODE
LOWtests/test_sample_limits.py677def test_two_models_both_with_cost_data_tracks_cost() -> None:CODE
LOWtests/test_sample_limits.py728def test_task_level_cost_limit_without_cost_data_errors() -> None:CODE
LOWtests/test_sample_limits.py740def test_task_level_cost_limit() -> None:CODE
LOWtests/test_sample_limits.py771def test_model_cost_config_file() -> None:CODE
LOWtests/test_sample_limits.py803def test_model_cost_config_dict() -> None:CODE
LOWtests/test_sample_limits.py831def test_operator_limit_records_reason() -> None:CODE
LOWtests/test_retry.py67def test_eval_retry_with_task_name():CODE
12440 more matches not shown…
Decorative Section Separators1315 hits · 4850 pts
SeverityFileLineSnippetContext
MEDIUMdesign/s3-incremental-log-flush.md762#480 (independent) ──────────────────────────────────────┐COMMENT
MEDIUMtests/test_task_cancel.py721# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/test_task_cancel.py724# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/conftest.py35# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/conftest.py49# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/conftest.py370# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/conftest.py388# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py138# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py140# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py312# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py314# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py525# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py527# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py601# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py603# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py696# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py698# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py826# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/test_release_workflow.py828# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_ctl.py8989# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_ctl.py8991# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py102# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py104# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py213# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py215# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py276# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py278# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py437# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py439# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py579# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py581# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py950# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py952# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1028# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1030# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1356# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1358# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1395# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_pause.py1397# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py31# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py33# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py132# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py134# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py217# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_buffer.py219# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py381# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py383# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py462# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py464# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py1189# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_score.py1191# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py193# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py195# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py1342# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py1344# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py1712# ---------------------------------------------------------------------------COMMENT
MEDIUMtests/_control/test_score.py1714# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_server.py1756# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_server.py1758# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡tests/_control/test_cancel.py134# ---------------------------------------------------------------------------COMMENT
1255 more matches not shown…
Empty Error Handlers312 hits · 1560 pts
SeverityFileLineSnippetContext
HIGHtests/test_task_cancel.py708except KeyboardInterrupt:CODE
HIGHtests/conftest.py1113except BaseException:CODE
HIGHtests/test_eval_set.py205except Exception:CODE
HIGHtests/test_eval_set.py3164except KeyboardInterrupt:CODE
HIGHtests/test_eval_set.py874except KeyboardInterrupt:CODE
HIGHtests/test_cancellation_logging.py289except KeyboardInterrupt:CODE
HIGHtests/test_model_env_mismatch.py101except Exception:CODE
HIGHtests/test_model_env_mismatch.py279except (ValueError, PrerequisiteError):CODE
HIGHtests/test_model_env_mismatch.py50except Exception:CODE
HIGHtests/test_model_env_mismatch.py136except Exception:CODE
HIGHtests/test_model_env_mismatch.py168except Exception:CODE
HIGHtests/test_model_env_mismatch.py194except Exception:CODE
HIGHtests/test_model_env_mismatch.py221except Exception:CODE
HIGHtests/test_model_env_mismatch.py247except Exception:CODE
HIGHtests/tools/test_tools_bridge.py879except Exception:CODE
HIGHtests/tools/test_tools_bridge.py1198except Exception:CODE
HIGHtests/tools/test_mcp_tools.py180except anyio.ClosedResourceError:CODE
HIGHtests/tools/test_tool_with.py34except Exception:CODE
HIGHtests/util/test_media_resolver.py57except ValueError:CODE
HIGHtests/util/test_asyncfiles.py391except EndOfStream:CODE
HIGHtests/util/test_asyncfiles.py417except EndOfStream:CODE
HIGHtests/input/test_input_acp.py104except ValueError:CODE
HIGHtests/agent/deepagent/test_agent_tool.py266except Exception:CODE
HIGHtests/agent/test_channel/test_channel.py202except BaseException:CODE
HIGHtests/agent/test_acp/test_span_boundary.py184except Exception:CODE
HIGHtests/agent/test_acp/test_server_dispatch.py71except OSError:CODE
HIGHtests/agent/test_acp/test_server_dispatch.py221except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_server_dispatch.py226except Exception:CODE
HIGHtests/agent/test_acp/test_server_dispatch.py64except OSError:CODE
HIGHtests/agent/test_acp/test_server_forwarding.py775except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_server_forwarding.py779except Exception:CODE
HIGHtests/agent/test_acp/test_server_forwarding.py89except OSError:CODE
HIGHtests/agent/test_acp/test_server_forwarding.py203except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_server_forwarding.py208except Exception:CODE
HIGHtests/agent/test_acp/test_server_forwarding.py82except OSError:CODE
HIGHtests/agent/test_acp/test_server_forwarding.py843except Exception:CODE
HIGHtests/agent/test_acp/test_standard_acp_client.py81except OSError:CODE
HIGHtests/agent/test_acp/test_standard_acp_client.py192except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_standard_acp_client.py197except Exception:CODE
HIGHtests/agent/test_acp/test_standard_acp_client.py74except OSError:CODE
HIGHtests/agent/test_acp/test_raw_events.py87except OSError:CODE
HIGHtests/agent/test_acp/test_raw_events.py174except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_raw_events.py179except Exception:CODE
HIGHtests/agent/test_acp/test_raw_events.py80except OSError:CODE
HIGHtests/agent/test_acp/test_raw_events.py351except asyncio.TimeoutError:CODE
HIGHtests/agent/test_acp/test_action_methods.py555except OSError:CODE
HIGHtests/agent/test_acp/test_action_methods.py610except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_action_methods.py615except Exception:CODE
HIGHtests/agent/test_acp/test_action_methods.py548except OSError:CODE
HIGHtests/agent/test_acp/test_discovery.py72except OSError:CODE
HIGHtests/agent/test_acp/test_discovery.py68except OSError:CODE
HIGHtests/agent/test_acp/test_elicitation_e2e.py87except OSError:CODE
HIGHtests/agent/test_acp/test_elicitation_e2e.py163except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_elicitation_e2e.py168except Exception:CODE
HIGHtests/agent/test_acp/test_elicitation_e2e.py83except OSError:CODE
HIGHtests/agent/test_acp/test_stdio_bridge.py69except OSError:CODE
HIGHtests/agent/test_acp/test_stdio_bridge.py65except OSError:CODE
HIGHtests/agent/test_acp/test_server_session_info.py59except OSError:CODE
HIGHtests/agent/test_acp/test_server_session_info.py153except (asyncio.CancelledError, Exception):CODE
HIGHtests/agent/test_acp/test_server_session_info.py158except Exception:CODE
252 more matches not shown…
Unused Imports1569 hits · 1289 pts
SeverityFileLineSnippetContext
LOWtests/conftest.py30CODE
LOWtests/test_release_workflow.py10CODE
LOWtests/test_eval_set_scanner.py3166CODE
LOWtests/test_disk_sample_store.py23CODE
LOWtests/test_disk_sample_store.py24CODE
LOWtests/_control/test_state.py20CODE
LOWtests/_control/test_cancel.py22CODE
LOWtests/tools/test_mcp_session_isolation.py14CODE
LOWtests/tools/test_sandbox_docker_and_local.py20CODE
LOW…ools/sandbox_tools_utils/test_sandbox_tools_digests.py9CODE
LOWtests/checkpoint/test_eval_checkpoints.py3CODE
LOWtests/checkpoint/test_sample_checkpoints.py3CODE
LOWtests/checkpoint/test_resume_copy.py11CODE
LOWtests/checkpoint/test_resolve.py3CODE
LOWtests/checkpoint/docker_projects.py3CODE
LOWtests/checkpoint/conftest.py11CODE
LOWtests/checkpoint/hydrate_interrupt_harness.py27CODE
LOWtests/checkpoint/test_checkpoint_scoring_resume_e2e.py23CODE
LOWtests/checkpoint/resume_kill_thinking_harness.py35CODE
LOWtests/checkpoint/test_parse.py3CODE
LOWtests/checkpoint/test_staging_dir.py3CODE
LOWtests/checkpoint/test_snapshot_strategy.py15CODE
LOWtests/checkpoint/test_sandbox_dir.py12CODE
LOWtests/checkpoint/test_current_checkpointer_post_exit.py31CODE
LOWtests/checkpoint/test_sandbox_restic_repo.py20CODE
LOWtests/checkpoint/test_sample_runtime.py3CODE
LOWtests/checkpoint/test_layout_paths.py3CODE
LOWtests/checkpoint/test_host_egress.py9CODE
LOWtests/checkpoint/test_checkpoint_e2e.py40CODE
LOWtests/checkpoint/test_checkpointer.py10CODE
LOWtests/checkpoint/test_fs_copy_s3.py12CODE
LOWtests/checkpoint/resume_scoring_kill_harness.py24CODE
LOWtests/checkpoint/test_host_egress_s3.py9CODE
LOWtests/checkpoint/resume_kill_harness.py21CODE
LOWtests/checkpoint/test_sandbox_egress_restic.py24CODE
LOWtests/checkpoint/test_restore_repo.py11CODE
LOWtests/checkpoint/test_schemas.py9CODE
LOWtests/checkpoint/test_sandbox_egress_extract.py14CODE
LOWtests/checkpoint/test_sandbox_paths.py3CODE
LOWtests/checkpoint/test_normalize.py10CODE
LOWtests/checkpoint/test_restore_scope.py9CODE
LOWtests/util/test_restic_ops.py15CODE
LOWtests/util/test_zipfile_multiframe.py8CODE
LOWtests/util/test_download.py3CODE
LOWtests/util/test_restic_summary.py9CODE
LOWtests/util/test_store_from_events.py7CODE
LOWtests/util/test_async_zip.py7CODE
LOWtests/util/test_async_zip.py22CODE
LOWtests/util/test_limit_working.py1CODE
LOWtests/util/test_compression_transcoding.py3CODE
LOWtests/util/test_restic_binary.py19CODE
LOWtests/input/test_input_acp.py11CODE
LOWtests/input/test_input_panel.py11CODE
LOWtests/agent/test_bridge_additional_tools.py16CODE
LOWtests/agent/test_bridge_agent_message.py25CODE
LOWtests/agent/test_bridge_google_messages.py18CODE
LOWtests/agent/test_bridge_function_call_id.py11CODE
LOWtests/agent/test_bridge_tool_search.py9CODE
LOWtests/agent/test_bridge_provider_errors.py10CODE
LOWtests/agent/test_react_checkpoint_resume.py1CODE
1509 more matches not shown…
Cross-Language Confusion115 hits · 669 pts
SeverityFileLineSnippetContext
HIGH⚡tests/test_eval_set_overrides.py659 """`model_dump_json()` writes every unset field as null, and that has to be readable.STRING
HIGHtests/test_eval_set_overrides.py683 Read as an instruction, `{"max_connections": null}` replaced a definition'sSTRING
HIGHtests/test_docker_workflow.py44 'inspect-tool-support && touch "$CANARY"',CODE
HIGH⚡tests/_control/test_ctl.py662 # a running sample with nothing pending (activity null) keeps theCOMMENT
HIGHtests/_control/test_log_dir.py301 # live-only state is null, not a false zeroCOMMENT
HIGHtests/tools/sandbox_tools_utils/test_sandbox_tools.py800 "input": f"echo {flag} > /flag && chmod 400 /flag; ls -al /flag && cat /flag",CODE
HIGHtests/tools/sandbox_tools_utils/test_sandbox_tools.py904 f"touch {SANDBOX_TOOLS_DIR}/planted && echo CREATED; "CODE
HIGHtests/tools/sandbox_tools_utils/test_sandbox_tools.py905 f"mv {SANDBOX_TOOLS_DIR} {SANDBOX_TOOLS_DIR}.moved && echo MOVED; "CODE
HIGHtests/tools/sandbox_tools_utils/test_sandbox_tools.py997 "input": "command -v gzip || echo no-gzip; echo probe-done",CODE
HIGH…ls/sandbox_tools_utils/test_sandbox_tools_injection.py152TAR_XZF_STDIN = "tar xzf - || { cat >/dev/null; exit 1; }"CODE
HIGH…ls/sandbox_tools_utils/test_sandbox_tools_injection.py153TAR_XF_STDIN = "tar xf - || { cat >/dev/null; exit 1; }"CODE
HIGH⚡tests/util/test_store_from_events.py516 """Add operation should include value even when it's None (explicit null)."""STRING
HIGHtests/util/sandbox/test_privileged.py282 'mkdir -p -- "$1" && tar --version >/dev/null && sh -c \'printf %s "$PATH"\'',CODE
HIGHtests/util/sandbox/test_privileged.py593 'tar --version >/dev/null && rm -rf /root/pinned-check && printf %s "$PATH"',CODE
HIGHtests/util/sandbox/test_privileged.py607 ["sh", "-c", "tar --version >/dev/null && cat > ./ok"],CODE
HIGH⚡tests/util/sandbox/test_framework_directory.py732 local, str(target), ["sh", "-c", "pwd -P && touch marker"], user=NoneCODE
HIGHtests/util/sandbox/test_framework_directory.py762 'mv -- "$1" "$2" && ln -s -- "$3" "$1" && touch ./marker && pwd -P',CODE
HIGHtests/util/sandbox/test_sandbox_service.py1139_DECOY = "mkdir -p /tmp/decoy && chmod 777 /tmp/decoy && touch /tmp/decoy/sentinel"CODE
HIGHtests/util/sandbox/test_sandbox_service.py1233 f"mkdir -m 700 {svc}/responses && echo stale > {svc}/responses/stale.json",CODE
HIGHtests/agent/test_bridge_function_call_id.py21 """Each function_call item gets a non-null, unique id; call_id round-trips."""STRING
HIGH⚡tests/agent/test_agent_human.py985 "touch /root/target && chmod 644 /root/target && "CODE
HIGH⚡tests/agent/test_agent_human.py986 "rm /home/nonroot/.bashrc && ln -s /root/target /home/nonroot/.bashrc",CODE
HIGHtests/agent/test_agent_human.py913 docker_sandbox, "cat /root/.bashrc 2>/dev/null || true"CODE
HIGHtests/agent/test_agent_human.py952 f"mkdir -p /home/nonroot/decoy && ln -s /home/nonroot/decoy {HUMAN_AGENT_DIR}",CODE
HIGHtests/agent/test_agent_human.py968 f"ls -A {HUMAN_AGENT_DIR} /home/nonroot/decoy 2>/dev/null || true",CODE
HIGHtests/scorer/test_metric.py210 Serialized as JSON constants (NaN/Infinity) rather than null, via bothSTRING
HIGH⚡tests/scorer/test_reducers.py755 # With fewer than k scored epochs the estimator is undefined, so theCOMMENT
HIGH⚡tests/scorer/test_math.py630 assert plain.equals(latex)CODE
HIGHtests/_cli/test_util.py71 "critic={model: mockllm/model, temperature: 0.2, model_args: {custom_outputs: null}}",CODE
HIGHtests/_cli/test_util.py223 "{model: mockllm/model, model_args: {custom_outputs: null}, "CODE
HIGHtests/model/providers/test_anthropic.py3200 'head -c 9 /dev/urandom | base64 > /tmp/seed && echo "seed-$(cat /tmp/seed)"'CODE
HIGHtests/model/providers/test_anthropic.py3202 pending_cmd = 'sleep 5 && echo "marker-$(cat /tmp/seed)"'CODE
HIGHtests/model/providers/test_bedrock_structured_output.py38 nickname: Optional[str] = None # renders as anyOf:[string, null]CODE
HIGH…ts/model/providers/test_anthropic_cache_diagnostics.py364 """`{cache_miss_reason: null}` means comparison still pending — no warning."""STRING
HIGHtests/model/providers/test_litellm_proxy.py1897 '"invalid_request_error", "param": null, "code": "invalid_prompt"}}\n'CODE
HIGHtests/model/providers/test_openai_responses.py360 Meta's Model API streams a refusal-only response whose `model` is null;STRING
HIGH⚡tests/log/test_message_pool.py1052 Regression: if persistence stores call_refs as [] instead of null,STRING
HIGHtests/log/test_eval_log.py1640 null_value = re.compile(r'"value":\s*(null|\[\s*null|\{"a":\s*null)')CODE
HIGHtests/_eval/task/test_hf.py382 # hard-fails under the C locale (ascii) and cp1252 (0x81 undefined);COMMENT
HIGH⚡…/_remote_tools/_web_browser/playwright_page_crawler.py231 "(document.scrollingElement || document.body).scrollTop ="CODE
HIGH⚡…/_remote_tools/_web_browser/playwright_page_crawler.py232 " (document.scrollingElement || document.body).scrollTop -"CODE
HIGH⚡…/_remote_tools/_web_browser/playwright_page_crawler.py237 "(document.scrollingElement || document.body).scrollTop ="CODE
HIGH⚡…/_remote_tools/_web_browser/playwright_page_crawler.py238 " (document.scrollingElement || document.body).scrollTop +"CODE
HIGH⚡…sandbox_tools/tests/test_text_editor_validated_path.py329 b'{"/x": [null]}',CODE
HIGHsrc/inspect_sandbox_tools/tests/test_exec_remote.py292 "command": "echo hello_from_kill && echo err_from_kill >&2 && sleep 100"CODE
HIGHsrc/inspect_sandbox_tools/tests/test_exec_remote.py412 "params": {"command": "echo stdout_msg && echo stderr_msg >&2"},CODE
HIGH…_sandbox_tools/tests/test_mcp_server_session_reader.py31 b'{"jsonrpc":"2.0","id":null,'CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py10 sd.push("A")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py19 sd.push("A")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py25 sd.push("B")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py34 sd.push("A")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py39 sd.push("B")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py48 sd.push("A")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py51 sd.push("B")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py54 sd.push("C")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py63 sd.push("data")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py66 sd.push("") # retransmit, no new dataCODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py76 sd.push("A")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py80 sd.push("B")CODE
HIGH⚡…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py90 sd.push(("out1", "err1"))CODE
55 more matches not shown…
Deep Nesting628 hits · 536 pts
SeverityFileLineSnippetContext
LOWtests/conftest.py529CODE
LOWtests/test_eval_set_scanner.py1104CODE
LOWtests/_control/test_pause.py1400CODE
LOWtests/_control/test_pause.py1430CODE
LOWtests/_control/test_pause.py1431CODE
LOWtests/_control/test_limits.py1257CODE
LOWtests/_control/test_requeue.py1621CODE
LOWtests/tools/test_mcp_tools.py87CODE
LOWtests/tools/test_mcp_tools.py646CODE
LOWtests/tools/test_mcp_tools.py128CODE
LOWtests/tools/test_mcp_tools.py137CODE
LOWtests/tools/test_skill.py559CODE
LOWtests/checkpoint/test_checkpoint_e2e.py179CODE
LOWtests/util/sandbox/test_docker_cleanup.py272CODE
LOWtests/util/sandbox/test_local_sandbox.py29CODE
LOWtests/agent/test_agent_compaction.py208CODE
LOWtests/agent/test_agent_react.py153CODE
LOWtests/agent/test_agent_react.py154CODE
LOWtests/agent/test_channel/test_channel.py134CODE
LOWtests/agent/test_acp/test_server_dispatch.py45CODE
LOWtests/agent/test_acp/test_server_dispatch.py173CODE
LOWtests/agent/test_acp/test_server_forwarding.py63CODE
LOWtests/agent/test_acp/test_server_forwarding.py157CODE
LOWtests/agent/test_acp/test_standard_acp_client.py55CODE
LOWtests/agent/test_acp/test_standard_acp_client.py225CODE
LOWtests/agent/test_acp/test_standard_acp_client.py149CODE
LOWtests/agent/test_acp/test_raw_events.py62CODE
LOWtests/agent/test_acp/test_raw_events.py313CODE
LOWtests/agent/test_acp/test_raw_events.py133CODE
LOWtests/agent/test_acp/test_action_methods.py529CODE
LOWtests/agent/test_acp/test_action_methods.py572CODE
LOWtests/agent/test_acp/test_discovery.py36CODE
LOWtests/agent/test_acp/test_elicitation_e2e.py61CODE
LOWtests/agent/test_acp/test_elicitation_e2e.py116CODE
LOWtests/agent/test_acp/test_stdio_bridge.py43CODE
LOWtests/agent/test_acp/test_server_session_info.py115CODE
LOWtests/agent/test_acp/test_server.py29CODE
LOWtests/agent/test_acp/test_approval.py1445CODE
LOWtests/agent/test_acp/test_approval.py1499CODE
LOWtests/agent/test_acp/test_tui/test_client.py89CODE
LOWtests/_cli/test_acp_cli.py71CODE
LOWtests/model/test_compaction_edit_server_tools.py60CODE
LOWtests/model/test_compaction_edit_thinking.py48CODE
LOWtests/model/test_inline_media_serialization.py154CODE
LOWtests/model/test_compaction.py783CODE
LOWtests/model/test_model_info.py357CODE
LOWtests/model/test_model_info.py432CODE
LOWtests/model/providers/test_openai_prompt_cache.py129CODE
LOWtests/model/providers/test_openai_prompt_cache.py516CODE
LOWtests/model/providers/test_anthropic.py74CODE
LOWtests/model/providers/test_anthropic_cache_control.py87CODE
LOWtests/model/providers/test_perplexity.py31CODE
LOWtests/model/providers/test_litellm_proxy.py2051CODE
LOWtests/view/test_sample_list_states.py368CODE
LOWtests/view/test_sample_list_states.py371CODE
LOWtests/log/test_log_attachments.py57CODE
LOWtests/log/test_log_attachments.py67CODE
LOWtests/log/test_task_log.py1717CODE
LOWtests/log/test_eval_log.py450CODE
LOWtests/log/test_eval_log.py1594CODE
568 more matches not shown…
Docstring Block Structure102 hits · 510 pts
SeverityFileLineSnippetContext
HIGH…ort/src/inspect_tool_support/_util/json_rpc_helpers.py43 Validates RPC method parameters and handles the method execution. This function validates the provided parametSTRING
HIGHsrc/inspect_sandbox_tools/tests/conftest.py80Execute an RPC request via the CLI and return the parsed response. Args: request: The JSON-RPC request dictSTRING
HIGHsrc/inspect_sandbox_tools/tests/test_exec_remote.py34Poll an exec_remote job until it completes or times out. Args: rpc_client: The RPC client function to use fSTRING
HIGH…ls/src/inspect_sandbox_tools/_util/json_rpc_helpers.py46 Validates RPC method parameters and handles the method execution. This function validates the provided parametSTRING
HIGHsrc/inspect_ai/_util/registry.py426Create a registry object. Creates objects registered via decorator (e.g. `@task`, `@solver`). Note that this caSTRING
HIGHsrc/inspect_ai/_util/registry.py502Lookup RegistryInfo for an object. Args: o (object): Object to lookup info for Returns: RegistSTRING
HIGHsrc/inspect_ai/_util/flag_values.py29An option that is a bare flag or an integer. Args: value: The text given, or `None` for the bare flag. STRING
HIGHsrc/inspect_ai/_util/download.py44Download a file and verify its SHA256 checksum. If `dest` already exists and its checksum matches, the download is STRING
HIGHsrc/inspect_ai/_util/download.py112Download a Google Drive file via `gdown` and verify SHA256. Useful for fetching public-link Google Drive assets (daSTRING
HIGHsrc/inspect_ai/_util/format.py49Format a template string, optionally preserving unknown placeholders. Args: template: A string containing {STRING
HIGHsrc/inspect_ai/_util/_json_rpc.py159 Execute a JSON-RPC command expecting a scalar result. Args: method (str): The JSON-RPC method to call. STRING
HIGHsrc/inspect_ai/_util/_json_rpc.py199 Execute a JSON-RPC command to a sandbox environment expecting a model result. Args: method (str): The JSSTRING
HIGHsrc/inspect_ai/_util/_json_rpc.py233 Execute a JSON-RPC notification to a sandbox environment. A notification is a JSON-RPC request that doesn't exSTRING
HIGHsrc/inspect_ai/_util/_json_rpc.py342 Generate a string description of an RPC call. Args: method (str): The name of the RPC method. STRING
HIGHsrc/inspect_ai/_util/local_server.py294 Start a server with the given command and handle potential errors. Args: base_cmd: List of base commanSTRING
HIGHsrc/inspect_ai/_util/file.py587 Convert a string into a safe filename by removing or replacing unsafe characters. Args: s (str): The iSTRING
HIGHsrc/inspect_ai/_util/dateutil.py148Parse and normalize ISO datetime string or datetime instance to UTC. For legacy string temporal fields that cannot STRING
HIGHsrc/inspect_ai/_util/_async.py41Runs all of the passed async functions and collects their results. The results will be returned in the same order aSTRING
HIGHsrc/inspect_ai/_util/text.py141Convert a str to float, handling exponent characters and Unicode fractions. The Python isnumeric() function returnsSTRING
HIGHsrc/inspect_ai/_util/images.py183Validate an inline image returned by a model provider. Provider output is untrusted, so only inline data URIs are aSTRING
HIGHsrc/inspect_ai/_util/async_zip.py438Open a ZIP member and stream its decompressed contents. Must be used as an async context manager to ensure propSTRING
HIGHsrc/inspect_ai/_util/atomic_write.py103Context manager for atomic local-file writes with durability guarantees. Implements the write-to-temporary-then-atoSTRING
HIGHsrc/inspect_ai/core/_imports.py41Check every import in a module or package against an allowlist. Reads source files with `ast` rather than importingSTRING
HIGHsrc/inspect_ai/util/_json.py325Convert JSON schema to Pydantic BaseModel. Handles nested objects, arrays, and basic validations. Args: STRING
HIGHsrc/inspect_ai/util/_subprocess.py112Execute and wait for a subprocess. Convenience method for solvers, scorers, and tools to launch subprocesses. ASTRING
HIGHsrc/inspect_ai/util/_store.py227Convert a JsonChange to a jsonpatch operation dict with validation. Args: change: The JsonChange to convertSTRING
HIGHsrc/inspect_ai/util/_notify.py62Build an `Apprise` instance for the eval's notification config. Notification URLs frequently carry secrets (Slack tSTRING
HIGHsrc/inspect_ai/util/_panel.py82Create an input panel in the task display. There can only be a single instance of an InputPanel with a given 'tSTRING
HIGHsrc/inspect_ai/util/_restic/resolver.py129Return a path to a usable restic binary for the given platform. Downloads the archive on cache miss, retrying transSTRING
HIGHsrc/inspect_ai/util/_sandbox/exec_remote.py674Run a command and return the result without streaming. Submits the command, polls until completion, and returns ExeSTRING
HIGHsrc/inspect_ai/util/_sandbox/compose.py459Parse a Docker Compose file into a ComposeConfig. Args: file: Path to the compose file. multiple_seSTRING
HIGHsrc/inspect_ai/util/_sandbox/context.py187Get a SandboxEnvironment that satisfies all the given injection requirements. Args: injectables: Single SanSTRING
HIGHsrc/inspect_ai/util/_sandbox/context.py449Find the best sandbox for injection and return it with needed injections. Args: injectables: List of detectSTRING
HIGHsrc/inspect_ai/util/_sandbox/environment.py215Execute a command within a sandbox environment. The current working directory for execution will be the per-samSTRING
HIGHsrc/inspect_ai/util/_sandbox/environment.py311Read a file from the sandbox environment. By default, file size is limited to 100 MiB. You may change this by sSTRING
HIGHsrc/inspect_ai/util/_sandbox/environment.py342Information required to connect to sandbox environment. Args: user: User to login as. ReturnSTRING
HIGHsrc/inspect_ai/util/_sandbox/environment.py381Start a command and return a process handle or result. In streaming mode (stream=True), the function returns onSTRING
HIGHsrc/inspect_ai/util/_sandbox/environment.py454Verify and return a reference to a subclass of SandboxEnvironment. Args: sandbox_cls: Class of sandbSTRING
HIGHsrc/inspect_ai/util/_sandbox/_privileged.py139Run ``cmd`` in ``sandbox`` as ``user`` with utilities resolved via :data:`SYSTEM_PATH`. A drop-in for ``sandbox.exeSTRING
HIGHsrc/inspect_ai/util/_sandbox/_privileged.py222Look ``name`` up on the *image's* ``PATH`` with a ``which`` resolved via :data:`SYSTEM_PATH`. For the questions thaSTRING
HIGHsrc/inspect_ai/util/_sandbox/_framework_directory.py573Create or adopt ``path`` as a root-owned framework directory, if root works. Runs :func:`ensure_framework_directorySTRING
HIGHsrc/inspect_ai/util/_sandbox/_framework_directory.py715Verify ``path`` and then run ``cmd`` with the verified directory as cwd. The verification and the command run in thSTRING
HIGHsrc/inspect_ai/util/_sandbox/_json_rpc_transport.py72Execute an RPC request using the sandbox transport. Args: method: The JSON-RPC method to call. STRING
HIGHsrc/inspect_ai/_lfs/_client.py39Get download URLs for LFS objects via the batch API. Chunks requests to avoid exceeding GitHub's payload size limitSTRING
HIGHsrc/inspect_ai/_lfs/resolver.py20Resolve a directory that may contain LFS pointer files. Recursively checks source_dir for LFS pointers. If none areSTRING
HIGHsrc/inspect_ai/solver/_solver.py85Contribute to solving an evaluation task. Transform a `TaskState`, returning the new state. Solvers may STRING
HIGHsrc/inspect_ai/solver/_solver.py163Decorator for registering solvers. Args: name: Optional name for solver. If the decorator has nSTRING
HIGHsrc/inspect_ai/scorer/_scorer.py139Decorator for registering scorers. Args: metrics: One or more metrics to calculate over the scoSTRING
HIGHsrc/inspect_ai/scorer/_metric.py335Compute a metric on a list of scores. Args: scores: List of scores. Returns: MetriSTRING
HIGHsrc/inspect_ai/scorer/_score.py15Score a model conversation. Score a model conversation (you may pass `TaskState` or `AgentState` as the value fSTRING
HIGHsrc/inspect_ai/scorer/_metrics/aggregate.py25Apply `agg` to a single key extracted from each dict-valued `Score.value`. Many scorers emit dict-valued scores (muSTRING
HIGHsrc/inspect_ai/_cli/util.py189Parse model roles from CLI args. Supports key-value, YAML, and JSON formats. Args: model_roles: Tuple of stSTRING
HIGHsrc/inspect_ai/_cli/util.py251Parse model specs from CLI args. Each spec is one YAML or JSON mapping. A spec holds `model` (required), `base_url`STRING
HIGHsrc/inspect_ai/_cli/eval.py1874Resolve `--model-spec` into one model per spec. A spec builds a `Model`, and `get_model()` returns a `Model` unchanSTRING
HIGHsrc/inspect_ai/model/_internal.py15 Extracts and removes a smuggled <internal>...</internal> tag from the content string, if present. Note: STRING
HIGHsrc/inspect_ai/model/_openai_responses.py2456Convert CompactedResponse to a list of ChatMessages. The compact endpoint returns the complete new context window, STRING
HIGHsrc/inspect_ai/model/_google_convert.py76Convert Google GenerateContentResponse into Inspect ModelOutput. Args: response: Google GenerateContentRespSTRING
HIGHsrc/inspect_ai/model/_model_info.py289Get model information including context window, output tokens, etc. Looks up model information from a local databasSTRING
HIGHsrc/inspect_ai/model/_model.py514Tokenize text into token IDs using the model's tokenizer. Override in providers that support server-side tokeniSTRING
HIGHsrc/inspect_ai/model/_model.py647Compact messages using provider-native compaction. Some model providers (e.g., OpenAI Codex models) support natSTRING
42 more matches not shown…
Excessive Try-Catch Wrapping435 hits · 445 pts
SeverityFileLineSnippetContext
LOWdesign/bridge-host-tool-events.md663 except Exception as ex:CODE
LOWdesign/deepagent-background.md137 except Exception as ex:CODE
LOWdesign/recover.md320 except Exception:CODE
LOWdesign/recover.md401 except Exception:CODE
LOWtests/test_retry.py394 except Exception:CODE
LOWtests/conftest.py490 except Exception as ex:CODE
LOWtests/conftest.py574 except Exception as ex:CODE
LOW⚡tests/test_eval_set.py198 except Exception:CODE
LOW⚡tests/test_eval_set.py205 except Exception:CODE
LOWtests/test_model_env_mismatch.py50 except Exception:CODE
LOWtests/test_model_env_mismatch.py101 except Exception:CODE
LOWtests/test_model_env_mismatch.py136 except Exception:CODE
LOWtests/test_model_env_mismatch.py168 except Exception:CODE
LOWtests/test_model_env_mismatch.py194 except Exception:CODE
LOWtests/test_model_env_mismatch.py221 except Exception:CODE
LOWtests/test_model_env_mismatch.py247 except Exception:CODE
LOWtests/tools/test_tools_bridge.py879 except Exception:CODE
LOWtests/tools/test_tools_bridge.py1162 except Exception as ex:CODE
LOWtests/tools/test_tools_bridge.py1198 except Exception:CODE
LOWtests/tools/test_sandbox_docker_and_local.py135 except Exception:CODE
LOWtests/tools/test_tool_with.py34 except Exception:CODE
MEDIUMtests/tools/test_tool_with.py27def test_tool_with_validation():CODE
LOWtests/tools/sandbox_tools_utils/test_sandbox_tools.py646 except Exception as ex:CODE
LOWtests/util/test_async.py44 except Exception as group:CODE
LOWtests/util/test_async.py57 except Exception as group:CODE
MEDIUMtests/util/test_async.py30def fail():CODE
LOW⚡tests/util/test_anyio.py66 except Exception as inner_tool_error:CODE
LOW⚡tests/util/test_anyio.py72 except Exception as e:CODE
LOW⚡tests/util/test_anyio.py74 except Exception as wow:CODE
LOWtests/util/sandbox/test_sandbox_service.py1103 except Exception as ex:CODE
LOWtests/util/sandbox/test_sandbox_service.py115 except Exception as e:STRING
LOWtests/agent/test_bridge_refusals.py243 except Exception as ex:CODE
MEDIUMtests/agent/deepagent/test_agent_tool.py258def import ToolDefCODE
LOWtests/agent/deepagent/test_agent_tool.py266 except Exception:CODE
LOWtests/agent/test_acp/test_span_boundary.py184 except Exception:CODE
MEDIUMtests/agent/test_acp/test_server_dispatch.py173def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_server_dispatch.py226 except Exception:CODE
MEDIUMtests/agent/test_acp/test_server_forwarding.py157def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_server_forwarding.py208 except Exception:CODE
LOWtests/agent/test_acp/test_server_forwarding.py779 except Exception:CODE
LOWtests/agent/test_acp/test_server_forwarding.py843 except Exception:CODE
MEDIUMtests/agent/test_acp/test_standard_acp_client.py149def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_standard_acp_client.py197 except Exception:CODE
MEDIUMtests/agent/test_acp/test_raw_events.py133def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_raw_events.py179 except Exception:CODE
MEDIUMtests/agent/test_acp/test_action_methods.py572def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_action_methods.py615 except Exception:CODE
LOWtests/agent/test_acp/test_elicitation_e2e.py168 except Exception:CODE
MEDIUMtests/agent/test_acp/test_elicitation_e2e.py116def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_server_session_info.py158 except Exception:CODE
MEDIUMtests/agent/test_acp/test_server_session_info.py115def _read_loop(self) -> None:CODE
LOW⚡tests/agent/test_acp/test_server.py266 except Exception:CODE
LOW⚡tests/agent/test_acp/test_server.py318 except Exception:CODE
LOWtests/agent/test_acp/test_server.py238 except Exception:CODE
LOWtests/agent/test_acp/test_server.py387 except Exception:CODE
LOWtests/agent/test_acp/test_server.py432 except Exception:CODE
MEDIUMtests/agent/test_acp/test_approval.py1499def _read_loop(self) -> None:CODE
LOWtests/agent/test_acp/test_approval.py1552 except Exception:CODE
LOW⚡tests/agent/test_acp/test_tui/test_defer_body.py147 except Exception:CODE
LOW⚡tests/agent/test_acp/test_tui/test_defer_body.py155 except Exception:CODE
375 more matches not shown…
Cross-File Repetition68 hits · 340 pts
SeverityFileLineSnippetContext
HIGHtests/tools/test_tools_bridge.py0add two numbers. args: x: first number to add. y: second number to add.STRING
HIGHtests/sentinel/test_sentinel_config.py0add two numbers. args: x: first number to add. y: second number to add.STRING
HIGHtests/sentinel/test_sentinel_human.py0add two numbers. args: x: first number to add. y: second number to add.STRING
HIGHtests/sentinel/test_sentinel_dispatch.py0add two numbers. args: x: first number to add. y: second number to add.STRING
HIGHtests/model/providers/test_deepseek.py0add two numbers. args: x: first number to add. y: second number to add.STRING
HIGHtests/tools/test_tools.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/agent/test_react_review.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/agent/test_agent_react.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/solver/test_basic_agent.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/review/test_review.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/model/test_reasoning_claude.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/test_helpers/tools.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/approval/test_approval.py0add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.STRING
HIGHtests/agent/test_acp/test_server_forwarding.py0short /tmp data dir so af_unix paths fit in 104 chars on macos.STRING
HIGHtests/agent/test_acp/test_action_methods.py0short /tmp data dir so af_unix paths fit in 104 chars on macos.STRING
HIGHtests/agent/test_acp/test_elicitation_e2e.py0short /tmp data dir so af_unix paths fit in 104 chars on macos.STRING
HIGHtests/agent/test_acp/test_stdio_bridge.py0short /tmp data dir so af_unix paths fit in 104 chars on macos.STRING
HIGHtests/agent/test_acp/test_approval.py0short /tmp data dir so af_unix paths fit in 104 chars on macos.STRING
HIGHtests/_cli/test_token_limit_flag.py0run the test command and eval its echoed repr back to a python value.STRING
HIGHtests/_cli/test_ctl_server_flag.py0run the test command and eval its echoed repr back to a python value.STRING
HIGHtests/_cli/test_acp_server_flag.py0run the test command and eval its echoed repr back to a python value.STRING
HIGHtests/model/providers/test_anthropic.py0completes each batch at once, answering each request with its id.STRING
HIGHtests/model/providers/test_openai_compatible.py0completes each batch at once, answering each request with its id.STRING
HIGHtests/model/providers/util/test_batch.py0completes each batch at once, answering each request with its id.STRING
HIGHtests/model/providers/test_mistral.py0unset stream is "auto": stream iff the caller passed on_stream.STRING
HIGHtests/model/providers/test_azureai.py0unset stream is "auto": stream iff the caller passed on_stream.STRING
HIGHtests/model/providers/test_bedrock_streaming.py0unset stream is "auto": stream iff the caller passed on_stream.STRING
HIGHtests/model/providers/test_groq.py0unset stream is "auto": stream iff the caller passed on_stream.STRING
HIGHtests/model/providers/test_openai_compatible.py0unset stream is "auto": stream iff the caller passed on_stream.STRING
HIGHtests/model/providers/test_mistral.py0the stream accumulator reconstructs the completion and reports deltas.STRING
HIGHtests/model/providers/test_azureai.py0the stream accumulator reconstructs the completion and reports deltas.STRING
HIGHtests/model/providers/test_groq.py0the stream accumulator reconstructs the completion and reports deltas.STRING
HIGHtests/model/providers/test_mistral.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_azureai.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_grok.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_bedrock_streaming.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_groq.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_sagemaker.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_openai_responses.py0without an on_stream consumer only heartbeat/usage progress runs. explicit streaming=true callers stream without asking STRING
HIGHtests/model/providers/test_google.py0add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.STRING
HIGHdocs/_tools-basics.md0add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.STRING
HIGHexamples/tool_use.py0add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.STRING
HIGH…ort/src/inspect_tool_support/_util/json_rpc_helpers.py0check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.goSTRING
HIGH…ls/src/inspect_sandbox_tools/_util/json_rpc_helpers.py0check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.goSTRING
HIGHsrc/inspect_ai/_util/httpx.py0check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.goSTRING
HIGH…andbox_tools/_remote_tools/_bash_session/tool_types.py0user to run as: a username, or the sandbox default user's identity as captured by the host. switching requires the serveSTRING
HIGH…sandbox_tools/_remote_tools/_exec_remote/tool_types.py0user to run as: a username, or the sandbox default user's identity as captured by the host. switching requires the serveSTRING
HIGH…inspect_sandbox_tools/_remote_tools/_mcp/tool_types.py0user to run as: a username, or the sandbox default user's identity as captured by the host. switching requires the serveSTRING
HIGHsrc/inspect_ai/util/_store_model.py0pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: optiSTRING
HIGHsrc/inspect_ai/solver/_task_state.py0pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: optiSTRING
HIGHsrc/inspect_ai/log/_log.py0pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: optiSTRING
HIGHsrc/inspect_ai/model/_model.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/mistral.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/grok.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/google.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/openai.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/anthropic.py0canonical model name for model info database lookup.STRING
HIGHsrc/inspect_ai/model/_providers/mistral.py0scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adaptiSTRING
HIGHsrc/inspect_ai/model/_providers/grok.py0scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adaptiSTRING
HIGHsrc/inspect_ai/model/_providers/openai_compatible.py0scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adaptiSTRING
8 more matches not shown…
Modern AI Meta-Vocabulary114 hits · 323 pts
SeverityFileLineSnippetContext
MEDIUMCHANGELOG.md2813- ReAct Agent: [truncation](https://inspect.aisi.org.uk/agents.html#truncation) option to trim conversation messages wheCODE
MEDIUMdesign/bridge-host-tool-events.md151 teardown #5464 triggers may pre-empt delivery of the reply; the scaffold'sCODE
MEDIUMdesign/ctl/pause-resume.md140- **Hard pause (hold at next model call) — implemented.** `pause --now`: in addition to quiesce, hold in-flight samples CODE
MEDIUMdesign/ctl/control-channel.md992- **Self-targeting guard hardening.** Open question #8 in this doc — an LLM agent running *inside* an eval shouldn't be CODE
MEDIUM⚡tests/tools/test_tools_bridge.py922# the scaffold reads as tool output, so an unexpected one has to be signalled toCOMMENT
MEDIUMtests/tools/test_tools_bridge.py1152 # the scaffold carries on regardless; the monitor tears the group downCOMMENT
MEDIUMtests/checkpoint/test_checkpointer.py651 # leading scaffolding before the first wrap — dropped by the sliceCOMMENT
MEDIUMtests/checkpoint/test_checkpointer.py671 # seeded wrap and still open at fire time — trailing scaffoldingCOMMENT
MEDIUM⚡tests/agent/test_agent_compaction_native.py49 # the context window. Reasoning + compaction is tested separately inCOMMENT
MEDIUMtests/agent/test_bridge_generation_config.py405 # would leak the scaffold's value to the served model whenever the eval configCOMMENT
MEDIUM⚡tests/agent/test_bridge_track_state.py1387 # the scaffold restarts: title call fires first againCOMMENT
MEDIUMtests/agent/test_bridge_track_state.py259 # main loop, then scaffold compaction: the compacted loop is promoted, soCOMMENT
MEDIUMtests/agent/test_bridge_track_state.py390 # the scaffold re-creates every message: different ids and metadata, andCOMMENT
MEDIUMtests/agent/test_bridge_track_state.py427 # scaffold runs the task under its own (different) system promptCOMMENT
MEDIUMtests/agent/test_bridge_track_state.py604 # title call fires first; the scaffold's store holds the condensed turnCOMMENT
MEDIUMtests/agent/test_bridge_track_state.py1461 # main loop: turn 2 (scaffold round-trips through its own store)COMMENT
MEDIUMtests/agent/test_bridge_refusals.py236 # the scaffold carries on regardless; the monitor tears the group downCOMMENT
MEDIUMtests/agent/test_agent_bridge.py1108 # backend; the scaffold's second call replays the searched assistant turnSTRING
MEDIUMtests/agent/test_agent_bridge.py1119 # both scaffold calls completed (the second replays the first turn)STRING
MEDIUM⚡tests/agent/test_bridge_approval.py817 # the scaffold re-issues the approved call with the keys in a different orderCOMMENT
MEDIUM⚡tests/agent/test_bridge_approval.py2081 # the rejected call never reached the scaffoldCOMMENT
MEDIUM⚡tests/agent/test_bridge_approval.py2215# should behave identically whichever API the scaffold speaks. These drive eachCOMMENT
MEDIUM⚡tests/agent/test_bridge_approval.py2216# dialect impl the way a scaffold's request arrives.COMMENT
MEDIUMtests/agent/test_bridge_approval.py384 # the scaffold gets the approved arguments...COMMENT
MEDIUMtests/agent/test_bridge_approval.py452 # the model is told and generates again; the scaffold sees only the replacementCOMMENT
MEDIUMtests/agent/test_bridge_approval.py1537 # a second call for the same proposal (e.g. a scaffold retrying after aCOMMENT
MEDIUMtests/agent/test_bridge_approval.py1806 # and so does the response handed to the scaffoldCOMMENT
MEDIUMtests/agent/test_bridge_approval.py2040 # what the scaffold would go on to executeCOMMENT
MEDIUM⚡tests/agent/test_acp/test_operator_provenance.py55# note_operator_message — the scaffold's signalCOMMENT
MEDIUMtests/agent/test_acp/test_router_bridge_tools.py150 # the scaffold-provided view renders richly (not args-only)COMMENT
MEDIUM⚡tests/agent/test_acp/test_tui/test_widgets.py915 # Used-only — context window denominator dropped.COMMENT
MEDIUMtests/scorer/test_math.py289 # An integer answer on the final line of a reasoning trace is extracted.COMMENT
MEDIUMtests/model/test_compaction.py1197 # small context window so a modest tool output overflows itCOMMENT
MEDIUMtests/model/test_model_length.py24# model context window sizesCOMMENT
MEDIUMtests/model/test_model_length.py49 # cloudflare serves this model with a reduced context windowCOMMENT
MEDIUMtests/model/test_model_length.py61 # context window for modelCOMMENT
MEDIUMtests/model/test_model_length.py65 # create message that exceeds context windowCOMMENT
MEDIUMtests/model/providers/test_anthropic.py526 # scaffold for should_retryCOMMENT
MEDIUM⚡tests/model/providers/test_vllm.py34 # this allows us to run base models with the chat message scaffolding:COMMENT
MEDIUM⚡tests/model/providers/test_openai_model_names.py89 # input_tokens_name() aliases to the current frontier so the context windowCOMMENT
MEDIUM⚡tests/model/providers/test_grok_model_names.py84 # input_tokens_name() aliases to the current frontier so the context windowCOMMENT
MEDIUM⚡tests/model/providers/test_grok_model_names.py98 # frontier would overstate its context windowCOMMENT
MEDIUMtests/model/providers/test_anthropic_fallback.py542 # scaffold echoes the assistant turn as dicts (incl. the fallback block)COMMENT
MEDIUMtests/model/providers/test_hf.py36 # this allows us to run base models with the chat message scaffolding:COMMENT
MEDIUMtests/model/providers/test_hf.py55 # this allows us to run base models with the chat message scaffolding:COMMENT
MEDIUM⚡tests/model/providers/test_google_model_names.py169 # input_tokens_name() aliases to the current frontier so the context windowCOMMENT
MEDIUMtests/model/providers/test_openai_responses.py2300 # Bridge -> scaffold (assistant text) -> scaffold echoes it back as input.COMMENT
MEDIUMdocs/CHANGELOG.md2813- ReAct Agent: [truncation](https://inspect.aisi.org.uk/agents.html#truncation) option to trim conversation messages wheCODE
MEDIUMdocs/evals/evals.json4400 "paper": "https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/",CODE
MEDIUMdocs/evals/evals.json6246 "paper": "https://www.anthropic.com/research/agentic-misalignment",CODE
MEDIUMsrc/inspect_ai/_control/pause.py256# at the hard gate — the interim-scoring pass's event-driven park ack (noCOMMENT
MEDIUMsrc/inspect_ai/_control/pause.py425 # event-driven park ack (see sample_park_waiter)COMMENT
MEDIUMsrc/inspect_ai/util/_checkpoint/hydrate.py748 # Slice away the live session's scaffolding ahead of the firstCOMMENT
MEDIUMsrc/inspect_ai/util/_checkpoint/hydrate.py783 # resumed session's own structural scaffolding (`init` / `solvers` /COMMENT
MEDIUMsrc/inspect_ai/agent/_react.py275 # check for context window overflowCOMMENT
MEDIUMsrc/inspect_ai/agent/_react.py515 # check for context window overflowCOMMENT
MEDIUMsrc/inspect_ai/agent/_react.py796 # exceed the context window; _handle_overflow's forcedCOMMENT
MEDIUM⚡src/inspect_ai/agent/_acp/inspect_ext.py132# (claude_code, codex, …): their tools are run by the bridged scaffold, not bySTRING
MEDIUM⚡src/inspect_ai/agent/_acp/connection.py1170 # Bind / unbind orchestrationCOMMENT
MEDIUMsrc/inspect_ai/agent/_acp/event_mapping.py723 # and a known context window. ACP semantics: "Tokens currently inCOMMENT
54 more matches not shown…
Self-Referential Comments102 hits · 322 pts
SeverityFileLineSnippetContext
MEDIUMtests/test_eval_set.py3268 # Create a log with the exact task identity eval_set expects, thenCOMMENT
MEDIUMtests/test_model_env_mismatch.py85 # Create a minimal task with proper solver and scorerCOMMENT
MEDIUM⚡tests/tools/test_web_search_tavily.py55 # Create a client with our mock transportCOMMENT
MEDIUMtests/analysis/test_large_string.py11 # Create a DataFrame with a small PyArrow string columnCOMMENT
MEDIUMtests/analysis/test_large_string.py32 # Create a DataFrame with a large PyArrow string columnCOMMENT
MEDIUMtests/analysis/test_large_string.py55 # Create a DataFrame with both small and large columns (same length)COMMENT
MEDIUM⚡tests/util/test_store_nested_types.py89 # Create a store with dict dataCOMMENT
MEDIUM⚡tests/util/test_anyio.py84 # Create a circular reference: exc1.__context__ -> exc2 -> exc1COMMENT
MEDIUMtests/util/test_anyio.py107 # Create a circular reference in exception groupsCOMMENT
MEDIUMtests/util/sandbox/test_docker_compatibility.py530 # Create a task with compose.yaml config (use None for dataset to get a dummy sample)COMMENT
MEDIUMtests/util/sandbox/sandbox_setup_symlink.sh12# Create a nested directory and symlink via “..”COMMENT
MEDIUMtests/util/sandbox/test_docker_compose_config.py160 # Create a ComposeService with x-default extensionCOMMENT
MEDIUM⚡tests/agent/test_agent_react.py661 # Create a tool call for submitCOMMENT
MEDIUM⚡tests/agent/test_agent_react.py668 # Create a tool call for a regular toolCOMMENT
MEDIUM⚡tests/agent/test_agent_react.py675 # Create an assistant message with both reasoning content and tool callsCOMMENT
MEDIUM⚡tests/agent/test_agent_react.py684 # Create an assistant message without reasoning content but with submit tool callCOMMENT
MEDIUM⚡tests/agent/test_agent_react.py690 # Create an assistant message with only regular tool calls (should be unchanged)COMMENT
MEDIUMtests/agent/test_acp/test_action_methods.py973 # Create the pending nested ToolEvent — same construction theCOMMENT
MEDIUM⚡tests/model/test_web_search_exclude_none.py84 # Create a web search with ActionSearch that has sources=None (the bug condition)COMMENT
MEDIUMtests/model/test_compaction_summary.py100 # Create a previous summary messageCOMMENT
MEDIUMtests/model/test_trim_messages.py72 # Create a longer conversationCOMMENT
MEDIUMtests/model/test_trim_messages.py140 # Create a scenario with a tool response without a corresponding assistant messageCOMMENT
MEDIUMtests/model/test_trim_messages.py241 # Create a longer conversationCOMMENT
MEDIUMtests/model/test_trim_messages.py305 # Create a longer conversation with tool callsCOMMENT
MEDIUMtests/model/test_trim_messages.py349 # Create a conversation with alternating user-assistant pairsCOMMENT
MEDIUMtests/model/test_trim_messages.py432 # Create an assistant message with 3 tool calls, but only 1 tool resultCOMMENT
MEDIUMtests/model/test_trim_messages.py465 # Create an assistant message with tool calls, but no tool results at allCOMMENT
MEDIUMtests/model/test_trim_messages.py492 # Create a conversation where trimming will create orphan tool_callsCOMMENT
MEDIUMtests/model/test_token_counting.py120 # Create a conversation with an assistant message containing multiple tool callsCOMMENT
MEDIUMtests/model/test_prompt_logprobs.py464 # Create a minimal instance without calling __init__COMMENT
MEDIUMtests/model/providers/test_anthropic.py699 # Create an assistant message with a single tool call (no tool result)COMMENT
MEDIUMtests/model/providers/test_anthropic.py721 # Create an assistant message with multiple tool calls (no tool results)COMMENT
MEDIUMtests/model/providers/test_anthropic.py753 # Create a tool result message without a preceding assistant messageCOMMENT
MEDIUMtests/model/providers/test_mistral.py111 # Create a simple promptCOMMENT
MEDIUMtests/model/providers/test_transformer_lens.py18 # Create a small HookedTransformer model for testingCOMMENT
MEDIUMtests/model/providers/test_transformer_lens.py34 # Create the model using InspectCOMMENT
MEDIUMtests/model/providers/test_transformer_lens.py143 # Create a small HookedTransformer model for testingCOMMENT
MEDIUMtests/model/providers/test_transformer_lens.py159 # Create a simple task for testingCOMMENT
MEDIUM⚡tests/model/providers/test_google.py456 # Create a mock clientCOMMENT
MEDIUM⚡tests/model/providers/test_google.py459 # Create a message with reasoning + text + tool callCOMMENT
MEDIUMtests/model/providers/test_google.py1028 # Create an assistant message with a single tool call (no tool result)COMMENT
MEDIUMtests/model/providers/test_google.py1053 # Create an assistant message with multiple tool calls (no tool results)COMMENT
MEDIUMtests/model/providers/test_google.py1088 # Create a tool result message without a preceding assistant messageCOMMENT
MEDIUMtests/model/providers/test_openai_responses.py733 # Create a mock CompactedResponse with a ResponseCompactionItemCOMMENT
MEDIUMtests/model/providers/test_openai_responses.py786 # Create a CompactedResponse without a ResponseCompactionItemCOMMENT
MEDIUMtests/model/providers/test_openai_responses.py953 # Create a CompactedResponse with compaction item followed by an output messageCOMMENT
MEDIUMtests/model/providers/test_openai_responses.py1025 # Create a compact response matching real API output:COMMENT
MEDIUMtests/model/providers/test_openai_responses.py1211 # Create a message with phase stored in ContentText.internalCOMMENT
MEDIUMtests/model/providers/util/test_batch_intake.py208 # Create a request that's too large for the available sizeCOMMENT
MEDIUMtests/model/providers/util/test_batch.py630 # Create a batcher with a low max_consecutive_check_failures valueCOMMENT
MEDIUMtests/model/providers/util/test_batch.py701 # Create a batcher that will fail initially then succeedCOMMENT
MEDIUMtests/model/providers/util/test_batch.py760 # Create a request that's close to but under the limitCOMMENT
MEDIUM⚡tests/log/test_log_file_info_fallback.py204 # Create a native-named eval (timestamp prefix -> fast path)COMMENT
MEDIUM⚡tests/log/test_log_file_info_fallback.py211 # Create a custom-named eval (no timestamp -> header fallback)COMMENT
MEDIUM⚡tests/log/test_log_attachments.py463# # Create a long text that should be condensed (> 100 chars)COMMENT
MEDIUM⚡tests/log/test_log_attachments.py467# # Create a model event with long contentCOMMENT
MEDIUMtests/log/test_log_attachments.py539# # Create the same long text that will appear in multiple eventsCOMMENT
MEDIUMtests/log/test_log_attachments.py620 # Create an event with long contentCOMMENT
MEDIUMtests/log/test_recover_e2e.py283 # Create an empty buffer DB so recovery can proceedCOMMENT
MEDIUMtests/log/test_recover_e2e.py402 # Create a crashed logCOMMENT
42 more matches not shown…
Over-Commented Block288 hits · 232 pts
SeverityFileLineSnippetContext
LOWtests/conftest.py361COMMENT
LOWtests/conftest.py381# OS-thread dump (async-signal-safe; catches hard C-level wedges), then — forCOMMENT
LOWtests/test_eval_set_selection.py1301 # Additive though they are, the models forbid extra fields, so a v1 readerCOMMENT
LOWtests/checkpoint/resume_kill_harness.py101COMMENT
LOWtests/checkpoint/resume_kill_harness.py221 return "crashed"COMMENT
LOWtests/util/test_text.py21 # Stripping it to face value would make match(numeric=True)COMMENT
LOWtests/agent/test_acp/test_session.py221COMMENT
LOWtests/agent/test_acp/test_router.py1881 tg.start_soon(lambda: agent(_state()))COMMENT
LOWtests/agent/test_acp/test_standard_acp_client.py241 k.startswith("inspect.") or k.startswith("inspect/")COMMENT
LOWtests/agent/test_acp/test_approval.py561 shim_task.cancel()COMMENT
LOWtests/agent/test_acp/test_tui/test_client.py61 active.agent_name = agent_nameCOMMENT
LOWtests/agent/test_acp/test_tui/test_session_screen.py1461 # by one refresh — pump another pause so focus has landedCOMMENT
LOWtests/model/test_reasoning_claude.py21@skip_if_no_anthropicCOMMENT
LOWtests/model/test_adaptive_connections.py961 # Controller's adaptive bound max should be 50 — the int shorthand.COMMENT
LOWtests/view/test_openapi_schema_generation.py241def test_default_pydantic_makes_defaulted_fields_not_required() -> None:COMMENT
LOWtests/log/test_log_attachments.py461# transcript = Transcript()COMMENT
LOWtests/log/test_log_attachments.py481# stored_event = transcript.events[0]COMMENT
LOWtests/log/test_log_attachments.py501# input=[initial_message],COMMENT
LOWtests/log/test_log_attachments.py521# # Verify the output was condensedCOMMENT
LOWtests/log/test_log_attachments.py541# message1 = ChatMessageUser(content=repeated_text)COMMENT
LOWtests/log/test_log_attachments.py561# )COMMENT
LOWdocs/evals/model_cards.yml1# Benchmarks referenced in recent model cards/announcements.COMMENT
LOWexamples/computer/compose.yaml1services:COMMENT
LOWscripts/release-sandbox-tools.sh181COMMENT
LOW.github/workflows/release.yml1name: ReleaseCOMMENT
LOW.github/workflows/suppressions-comment.yml1# Posts the sticky suppression-ledger delta comment on PRs, fork PRsCOMMENT
LOW.github/workflows/release-pr-checks.yml1name: Release PR checksCOMMENT
LOW.github/workflows/build.yml181 fetch-tags: trueCOMMENT
LOW.github/workflows/build.yml261 # (~2.5 min median, ~8 min p90 of PR wall clock under runnerCOMMENT
LOW.github/workflows/build.yml321 # published binaries were downloaded into.COMMENT
LOW.github/workflows/build.yml601 # Digest file must be in lockstep with the version bump: four non-blankCOMMENT
LOW.github/scripts/check_suppressions.py41COMMENT
LOW…ool_support/_remote_tools/_web_browser/scale_factor.py21#COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py41# AXNode,COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py61# def test_getitem(self):COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py81# }COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py101# }COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py121COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py141# node_data = {COMMENT
LOW…ote_tools/_web_browser/test_accessibility_tree_node.py161# }COMMENT
LOW…t_sandbox_tools/_remote_tools/_exec_remote/__init__.py1# TODO: Consider renaming _remote_tools directory to _remote_services or _json_rpc_servicesCOMMENT
LOW…sandbox_tools/src/inspect_sandbox_tools/_cli/server.py41#COMMENT
LOWsrc/inspect_ai/_control/server.py201# Keep-alive intentCOMMENT
LOWsrc/inspect_ai/_control/server.py801 active_since: float | None = None,COMMENT
LOWsrc/inspect_ai/_control/server.py1041 # naturally (scored on their own terms, no interrupts), thenCOMMENT
LOWsrc/inspect_ai/_control/server.py1201 if result.get("ok") is False:COMMENT
LOWsrc/inspect_ai/_control/server.py1321 # the ambiguity rejection carries the pending callsCOMMENT
LOWsrc/inspect_ai/_control/server.py1381 # after a `/`); `key`/`key_limit` retune a named concurrency() registryCOMMENT
LOWsrc/inspect_ai/_control/server.py1621 # Pause / resume the whole run (the eval-set spelling — oneCOMMENT
LOWsrc/inspect_ai/_control/server.py1641 # retries/min, backoff ratio, and active retry waits per model, plusCOMMENT
LOWsrc/inspect_ai/_control/server.py1801 exc_info=True,COMMENT
LOWsrc/inspect_ai/_control/limits.py361COMMENT
LOWsrc/inspect_ai/_control/cancel.py341 pending = state.task_cancel.cancel_typeCOMMENT
LOWsrc/inspect_ai/_control/cancel.py401 if stamp == "abort":COMMENT
LOWsrc/inspect_ai/_control/scoring.py81# gate (and then quiesce) before reporting it un-scored. A constant to startCOMMENT
LOWsrc/inspect_ai/_control/__init__.py1"""Control channel — HTTP server embedded in each running eval process.COMMENT
LOWsrc/inspect_ai/_control/__init__.py21# non-GET route with a 400 (`_control/strict.py`), atomically — so aCOMMENT
LOWsrc/inspect_ai/_control/__init__.py41# must not 400 their retune on an older strict server, so the CLI includesCOMMENT
LOWsrc/inspect_ai/_control/__init__.py61# server advertises >= 5 (an explicit --author/--reason against anCOMMENT
LOWsrc/inspect_ai/_control/__init__.py81# `ctl config`), joining the version-4 retry knobs. No CLI gate — theCOMMENT
228 more matches not shown…
Modern Structural Boilerplate223 hits · 214 pts
SeverityFileLineSnippetContext
LOW⚡tests/input/test_input_panel.py129def _set_input_value(input_widget: Input, value: str) -> None:CODE
LOWtests/input/test_input_panel.py53 def set_title(self, title: str) -> None:CODE
LOW⚡tests/agent/test_acp/test_tui/test_elicitation_card.py68def _set_input(input_widget: Input, value: str) -> None:CODE
LOWtests/test_package/inspect_package/__init__.py3__all__ = ["simple_score"]CODE
LOW…ridianlabs-ai/inspect-docs/filters/reference/filter.py230 def set_default_title(elem: pf.Element, doc: pf.Doc) -> None:CODE
LOWexamples/hooks/mlflow_tracking.py44logger = logging.getLogger(__name__)CODE
LOWexamples/hooks/trackio_tracking.py41logger = logging.getLogger(__name__)CODE
LOWexamples/hooks/mlflow_tracing.py51logger = logging.getLogger(__name__)CODE
LOW…pect_tool_support/src/inspect_tool_support/__init__.py12__all__ = ["__version__"]CODE
LOW…ool_support/_remote_tools/_web_browser/scale_factor.py5logger = logging.getLogger(__name__)CODE
LOW…sandbox_tools/tests/test_text_editor_validated_path.py193def _set_history_path(monkeypatch: pytest.MonkeyPatch, history_path: Path) -> None:CODE
LOWsrc/inspect_sandbox_tools/tests/conftest.py19logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_sandbox_tools/tests/test_exec_remote.py90def _setup_and_teardown(sandbox_server_cleanup: None) -> None:CODE
LOW…ct_sandbox_tools/src/inspect_sandbox_tools/__init__.py12__all__ = ["__version__"]CODE
LOW…ox_tools/_in_process_tools/_text_editor/text_editor.py27logger = logging.getLogger(__name__)CODE
LOW…box_tools/src/inspect_sandbox_tools/_util/constants.py11__all__ = [CODE
LOW…x_tools/src/inspect_sandbox_tools/_util/user_switch.py54def set_oom_score_adj() -> None:CODE
LOWsrc/inspect_ai/__init__.py32__all__ = [CODE
LOWsrc/inspect_ai/_control/eval_state.py742def set_sample_requeue(eval_id: str, handle: "SampleRequeue | None") -> None:CODE
LOWsrc/inspect_ai/_control/eval_state.py755def set_task_scoring(eval_id: str, handle: "TaskScoring | None") -> None:CODE
LOWsrc/inspect_ai/_control/max_tasks.py58def set_max_tasks_override(value: int | None) -> None:CODE
LOWsrc/inspect_ai/_util/zipfile.py29logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/zipfile.py199__all__ = ["zipfile_compress_kwargs"]CODE
LOWsrc/inspect_ai/_util/log_context.py8def set_run_shape(task_names: Iterable[str], max_epochs: int) -> None:CODE
LOWsrc/inspect_ai/_util/error.py116def set_exception_hook() -> None:CODE
LOWsrc/inspect_ai/_util/transcript.py66def set_transcript_markdown_options(markdown: Markdown) -> None:CODE
LOWsrc/inspect_ai/_util/registry.py92def set_annotations(wrapper: Callable[..., Any], annotations: dict[str, Any]) -> None:CODE
LOWsrc/inspect_ai/_util/registry.py123def set_return_annotation(wrapper: Callable[..., Any], return_type: type[Any]) -> None:CODE
LOWsrc/inspect_ai/_util/httpx.py16logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/background.py50def set_background_task_group(tg: TaskGroup | None) -> None:CODE
LOWsrc/inspect_ai/_util/local_server.py14logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/file.py668logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/event_loop_monitor.py38logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/asyncfiles.py55logger = logging.getLogger(__name__)CODE
LOWsrc/inspect_ai/_util/future.py14 def set_result(self, result: T) -> None:CODE
LOWsrc/inspect_ai/_util/future.py18 def set_exception(self, ex: Exception) -> None:CODE
LOWsrc/inspect_ai/_util/future.py32 def set_future_result(future: "Future[T]", result: T) -> None:CODE
LOWsrc/inspect_ai/_util/future.py36 def set_future_exception(future: "Future[T]", error: Exception) -> None:CODE
LOWsrc/inspect_ai/_util/strenum.py49__all__ = ["StrEnum"]CODE
LOWsrc/inspect_ai/_util/textual/__init__.py3__all__ = ["ElicitationForm"]CODE
LOWsrc/inspect_ai/_display/__init__.py14__all__ = [CODE
LOWsrc/inspect_ai/_display/core/results.py38def set_retry_args_suffix(suffix: str) -> None:CODE
LOWsrc/inspect_ai/_display/core/display.py170 def update_metrics(self, scores: list[TaskDisplayMetric]) -> None: ...CODE
LOWsrc/inspect_ai/_display/core/display.py198 def update_task_count(self, n: int) -> None:CODE
LOWsrc/inspect_ai/_display/core/progress.py70 def update_count(self, complete: int, total: int) -> None:CODE
LOWsrc/inspect_ai/_display/core/progress.py77 def update_score(self, score: str) -> None:CODE
LOWsrc/inspect_ai/_display/core/panel.py23def set_eval_set_id_display(value: str | None) -> None:CODE
LOWsrc/inspect_ai/_display/plain/display.py75 def update_task_count(self, n: int) -> None:CODE
LOWsrc/inspect_ai/_display/plain/display.py222 def update_metrics(self, metrics: list[TaskDisplayMetric]) -> None:CODE
LOWsrc/inspect_ai/_display/textual/display.py85 def update_task_count(self, n: int) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py59def set_flow_content(content: RenderableType) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py244 def update_task_count(self, n: int) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py300 def update_display(self) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py315 def update_scan(self) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py340 def update_title(self) -> None:CODE
LOW⚡src/inspect_ai/_display/textual/app.py365 def update_tasks(self) -> None:CODE
LOW⚡src/inspect_ai/_display/textual/app.py374 def update_samples(self) -> None:CODE
LOW⚡src/inspect_ai/_display/textual/app.py381 def update_footer(self) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py394 def set_unread(unread: int | None) -> None:CODE
LOWsrc/inspect_ai/_display/textual/app.py408 async def set_active_tab(active: str) -> None:CODE
163 more matches not shown…
AI Slop Vocabulary42 hits · 101 pts
SeverityFileLineSnippetContext
MEDIUMtests/test_sample_source.py322 # once epoch 4 completed, every time under its own id (an RL harnessCOMMENT
MEDIUM⚡tests/checkpoint/test_current_checkpointer_post_exit.py224# harness faithfully reproduces the normal span/checkpoint pairing — and thatCOMMENT
MEDIUMtests/checkpoint/resume_kill_harness.py253 # this harness drives — so stamp a fixed amount, giving the budget-carryCOMMENT
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json234 "content": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tackCODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json257 "content": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tackCODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json1545 "result": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tacklCODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json1577 "output": "{\"jsonrpc\": \"2.0\", \"result\": {\"web_url\": \"https://www.aisi.gov.uk/work\", \"main_content\"CODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json2102 "result": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tacklCODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json2134 "output": "{\"jsonrpc\": \"2.0\", \"result\": {\"web_url\": \"https://www.aisi.gov.uk/work\", \"main_content\"CODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json2732 "replaced": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/newsCODE
MEDIUM…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json3577 "cb6d9b3382c23a2729d389c44be2fb26": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.CODE
MEDIUMtests/agent/deepagent/test_deepagent_background.py979 # deterministically without a scheduling barrier the harness lacks.)COMMENT
MEDIUMtests/agent/test_acp/test_deepagent_integration.py479 # ACP harness. The cancel pathway itself is tested inCOMMENT
MEDIUMtests/agent/test_acp/test_server_dispatch.py102 # mirrors what the eval harness produces from the defaultCOMMENT
MEDIUMtests/agent/test_acp/test_action_methods.py957 # Set up the cancel-propagation harness in a real task group.COMMENT
MEDIUMtests/agent/test_acp/test_approval.py1204 # Scan all blocks for the view markdown — robust to layoutCOMMENT
MEDIUMtests/agent/test_acp/test_tui/test_cancel_card.py242 # Don't call original_notify — the harness can't render toasts.COMMENT
MEDIUM⚡tests/agent/test_acp/test_tui/test_widgets.py63# Single-widget harnessCOMMENT
MEDIUMtests/agent/test_acp/test_tui/test_widgets.py1099 # comprehensive fingerprint should detect the change and theCOMMENT
MEDIUMexamples/skills/skills/system-info/scripts/sysinfo.sh3# Outputs comprehensive system details in a structured formatCOMMENT
MEDIUM.github/workflows/test.yml1# Test harness shell. GitHub only exposes workflow_dispatch for workflows thatCOMMENT
MEDIUM…x_tools/src/inspect_sandbox_tools/_util/user_switch.py39 """The identity a child process must switch to in order to run as ``user``.STRING
LOWsrc/inspect_ai/_util/notebook.py53 # notebook cells, if not just return an empty moduleCOMMENT
LOWsrc/inspect_ai/analysis/_dataframe/record.py391 # For now, I'll just return an empty dictCOMMENT
LOWsrc/inspect_ai/util/_limit.py228 # gone out of scope for the sample so we just return that snapshotCOMMENT
MEDIUMsrc/inspect_ai/util/_checkpoint/checkpointer_impl.py581 # checkpoint span. Skip on the harness-driven final fire —COMMENT
MEDIUMsrc/inspect_ai/util/_sandbox/service.py568 # under an eval harness -- a sandbox-awareness leak that wasCOMMENT
MEDIUMsrc/inspect_ai/agent/_react.py228 # to clean exit and the harness will run scoring next. StateCOMMENT
MEDIUMsrc/inspect_ai/agent/_react.py472 # to clean exit and the harness will run scoring next. StateCOMMENT
LOWsrc/inspect_ai/agent/_acp/tui/session_screen.py184 # …); duplicating them in the footer would just add noise.COMMENT
MEDIUMsrc/inspect_ai/agent/_acp/tui/widgets/cancel_card.py232 # ``fails_on_error`` is True, but stay robust if a futureCOMMENT
MEDIUMsrc/inspect_ai/agent/_acp/tui/widgets/cancel_card.py267 # but stay robust if a future refactor changes theCOMMENT
MEDIUMsrc/inspect_ai/agent/_acp/tui/widgets/transcript.py158 # Gate on fingerprint change — the comprehensiveCOMMENT
LOWsrc/inspect_ai/_view/common.py647 # then just return that fileCOMMENT
MEDIUMsrc/inspect_ai/model/_call_tools.py1100 # harness injects the real AgentState below)COMMENT
LOWsrc/inspect_ai/model/_call_tools.py1324 # if the function takes **kwargs: Any then just pass the tool arguments throughCOMMENT
LOWsrc/inspect_ai/model/_providers/openai.py354 # side step that complexity and just use two different batchers.COMMENT
LOWsrc/inspect_ai/log/_recorders/buffer/database.py745 # this will just return no events and no attachments until theCOMMENT
LOWsrc/inspect_ai/log/_recorders/buffer/filestore.py469 # we'll just return NoneCOMMENT
LOWsrc/inspect_ai/_eval/loader.py664 # if there is no solver file then just create from the registry by nameCOMMENT
MEDIUMsrc/inspect_ai/tool/_sandbox_tools_utils/sandbox.py637 # TODO: once we get the github CI/CD actions robust, this should be fatalCOMMENT
LOWsrc/inspect_ai/tool/_mcp/_local.py360 # to the session, we can just return nitCOMMENT
Redundant / Tautological Comments55 hits · 78 pts
SeverityFileLineSnippetContext
LOWtests/model/test_message_ids.py177 # Output message should have an IDCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py117 # Check if the model used server-side toolsCOMMENT
LOWtests/model/test_stable_message_ids.py157 # Output message should have IDCOMMENT
LOWtests/model/providers/test_perplexity.py83 # Check if citations are presentCOMMENT
LOWtests/model/providers/util/test_batch.py102 # Check if batch should failCOMMENT
LOWtests/model/providers/util/test_batch.py108 # Check if batch is "complete" based on elapsed timeCOMMENT
LOWtests/log/test_eval_log.py194 # Set timestamp to a timezone-aware datetime object because when serializing toCOMMENT
LOWexamples/structured.py170 # Check if the output matches the expected RGB formatSTRING
LOWscripts/pypi-release.py745 # Check if there are any other versions presentCOMMENT
LOWscripts/pypi-release.py805 # Check if .pypirc exists or environment variables are setCOMMENT
LOWscripts/pypi-release.py929 # Check if we're ahead of remoteCOMMENT
LOWscripts/pypi-release.py972 # Check if we're in a git repositoryCOMMENT
LOWscripts/pypi-release.py1022 # Check if branch is up to dateCOMMENT
LOWscripts/pypi-release.py1033 # Check if tag already existsCOMMENT
LOWscripts/pypi-release.py1165 # Check if correct version already existsCOMMENT
LOW…t_tool_support/src/inspect_tool_support/_cli/server.py27 # Set umask to handle dynamic user switching scenarios:COMMENT
LOW…ox_tools/_in_process_tools/_text_editor/text_editor.py107 # Read the file contentCOMMENT
LOW…ox_tools/_in_process_tools/_text_editor/text_editor.py112 # Check if old_str is unique in the fileCOMMENT
LOW…ox_tools/_in_process_tools/_text_editor/text_editor.py215 # Check if path existsCOMMENT
LOW…ox_tools/_in_process_tools/_text_editor/text_editor.py224 # Check if the path points to a directoryCOMMENT
LOW…ols/src/inspect_sandbox_tools/_util/pseudo_terminal.py68 # Close the file descriptorsCOMMENT
LOW…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py154 # Check if process has finishedCOMMENT
LOWsrc/inspect_ai/_util/generate_config_args.py129 # Check if it's a file pathCOMMENT
LOWsrc/inspect_ai/_util/generate_config_args.py149 # Check if it looks like a file path that doesn't existCOMMENT
LOWsrc/inspect_ai/_util/platform.py53 # Check if running in VS Code Jupyter notebook or interactive windowCOMMENT
LOWsrc/inspect_ai/_util/platform.py61 # Check if running in a VS Code terminalCOMMENT
LOWsrc/inspect_ai/_util/local_server.py162 os.kill(pid, 0) # Check if process existsCODE
LOWsrc/inspect_ai/_util/local_server.py254 # Check if the process is still aliveCOMMENT
LOWsrc/inspect_ai/_util/file.py456 # Check if it's already a URICOMMENT
LOWsrc/inspect_ai/_util/file.py572 # Open the file using fsspec and retrieve the file's informationCOMMENT
LOWsrc/inspect_ai/analysis/_dataframe/samples/table.py220 # Check if it's specifically the pyarrow offset overflow errorCOMMENT
LOWsrc/inspect_ai/analysis/_dataframe/samples/table.py612 # Check if this is a PyArrow-backed columnCOMMENT
LOWsrc/inspect_ai/util/_json.py385 # Check if it's already Optional/Union with NoneCOMMENT
LOWsrc/inspect_ai/util/_sandbox/self_check.py381 # Write the file byte-by-byte from ASCII-only octal escapes, then cat it, soCOMMENT
LOWsrc/inspect_ai/_cli/score.py311 # Print the results panelCOMMENT
LOWsrc/inspect_ai/model/_openai_responses.py363 # Check if this is a compaction marker messageCOMMENT
LOWsrc/inspect_ai/model/_openai_responses.py1444 # Check if this is a valid OpenAI action (correct type + required fields)COMMENT
LOWsrc/inspect_ai/model/_openai_responses.py1618 # Check if result was cleared during compactionCOMMENT
LOWsrc/inspect_ai/model/_model_info.py162 # Check if the second-to-last part is a known service prefixCOMMENT
LOWsrc/inspect_ai/model/_model_info.py168 # Check if this is a hosting provider that needs org detectionCOMMENT
LOWsrc/inspect_ai/model/_providers/mockllm.py51 # Check if it's a callable functionCOMMENT
LOWsrc/inspect_ai/model/_providers/google.py2046 # Check if this block has an associated thought_signature andCOMMENT
LOWsrc/inspect_ai/model/_providers/anthropic.py1127 # Check if model doesn't support compactionCOMMENT
LOWsrc/inspect_ai/model/_providers/anthropic.py5589 # Check if result was cleared during compactionCOMMENT
LOWsrc/inspect_ai/model/_providers/anthropic.py5806 # Check if previous message has corresponding tool_use blocksCOMMENT
LOWsrc/inspect_ai/model/_providers/anthropic.py5848 # Check if next message has corresponding tool_result blocksCOMMENT
LOWsrc/inspect_ai/model/_providers/sglang.py181 # Check if process is still aliveCOMMENT
LOWsrc/inspect_ai/log/_bundle.py204 # Open the file for writing to save the updated contentSTRING
LOWsrc/inspect_ai/event/_timeline_repr.py202 # Check if span overlaps with any span in the current clusterCOMMENT
LOW…spect_ai/tool/_sandbox_tools_utils/validate_distros.py122 # Check if it's for a supported architectureCOMMENT
LOWsrc/inspect_ai/tool/_sandbox_tools_utils/sandbox.py858 # Check if we're in a git repoCOMMENT
LOWsrc/inspect_ai/tool/_tools/_memory.py281 # Read fileCOMMENT
LOWsrc/inspect_ai/tool/_tools/_memory.py349 # Read file contentCOMMENT
LOWsrc/inspect_ai/tool/_tools/_memory.py354 # Check if old_str is uniqueCOMMENT
LOW…_tools/_computer/_resources/entrypoint/xvfb_startup.sh30# Check if Xvfb is already runningCOMMENT
AI Structural Patterns76 hits · 74 pts
SeverityFileLineSnippetContext
LOWtests/log/test_eval_log.py2648CODE
LOW…/_remote_tools/_web_browser/accessibility_tree_node.py224CODE
LOW…sandbox_tools/_remote_tools/_mcp/mcp_server_session.py143CODE
LOWsrc/inspect_ai/_control/server.py882CODE
LOWsrc/inspect_ai/_control/server.py1395CODE
LOWsrc/inspect_ai/_control/server.py1487CODE
LOWsrc/inspect_ai/_control/limits.py155CODE
LOWsrc/inspect_ai/_control/limits.py242CODE
LOWsrc/inspect_ai/_control/events.py141CODE
LOWsrc/inspect_ai/_control/events.py204CODE
LOWsrc/inspect_ai/_control/eval_state.py446CODE
LOWsrc/inspect_ai/_control/eval_state.py503CODE
LOWsrc/inspect_ai/_control/log_dir/samples.py114CODE
LOWsrc/inspect_ai/util/_subprocess.py74CODE
LOWsrc/inspect_ai/util/_subprocess.py88CODE
LOWsrc/inspect_ai/util/_subprocess.py101CODE
LOWsrc/inspect_ai/util/_subprocess.py159CODE
LOWsrc/inspect_ai/util/_subprocess.py200CODE
LOWsrc/inspect_ai/util/_subtask.py155CODE
LOWsrc/inspect_ai/util/_checkpoint/hydrate.py512CODE
LOWsrc/inspect_ai/util/_sandbox/local.py112CODE
LOWsrc/inspect_ai/util/_sandbox/docker/compose.py75CODE
LOWsrc/inspect_ai/util/_sandbox/docker/compose.py342CODE
LOWsrc/inspect_ai/dataset/_dataset.py32CODE
LOWsrc/inspect_ai/dataset/_sources/file.py14CODE
LOWsrc/inspect_ai/dataset/_sources/csv.py53CODE
LOWsrc/inspect_ai/dataset/_sources/json.py27CODE
LOWsrc/inspect_ai/dataset/_sources/hf.py123CODE
LOWsrc/inspect_ai/agent/_react.py57CODE
LOWsrc/inspect_ai/agent/_bridge/bridge.py101CODE
LOWsrc/inspect_ai/agent/_bridge/types.py57CODE
LOWsrc/inspect_ai/agent/_bridge/sandbox/bridge.py51CODE
LOWsrc/inspect_ai/agent/_bridge/sandbox/types.py42CODE
LOWsrc/inspect_ai/agent/_deepagent/agent_tool.py277CODE
LOWsrc/inspect_ai/agent/_deepagent/plan.py35CODE
LOWsrc/inspect_ai/agent/_deepagent/subagent.py56CODE
LOWsrc/inspect_ai/agent/_deepagent/research.py35CODE
LOWsrc/inspect_ai/agent/_deepagent/prompt.py234CODE
LOWsrc/inspect_ai/agent/_deepagent/deepagent.py50CODE
LOWsrc/inspect_ai/agent/_deepagent/general.py32CODE
LOWsrc/inspect_ai/solver/_basic_agent.py52CODE
LOWsrc/inspect_ai/solver/_task_state.py144CODE
LOWsrc/inspect_ai/scorer/_model.py35CODE
LOWsrc/inspect_ai/scorer/_model.py115CODE
LOWsrc/inspect_ai/_view/fastapi_server.py148CODE
LOWsrc/inspect_ai/_view/fastapi_server.py808CODE
LOWsrc/inspect_ai/_view/view.py24CODE
LOWsrc/inspect_ai/_cli/_scanner.py23CODE
LOWsrc/inspect_ai/_cli/_scanner.py91CODE
LOWsrc/inspect_ai/_cli/eval.py1936CODE
LOWsrc/inspect_ai/_cli/ctl/_fetch.py485CODE
LOWsrc/inspect_ai/_cli/ctl/_sample_read.py448CODE
LOWsrc/inspect_ai/_cli/ctl/_config.py421CODE
LOWsrc/inspect_ai/_cli/ctl/_config.py933CODE
LOWsrc/inspect_ai/model/_model.py2236CODE
LOWsrc/inspect_ai/model/_providers/vllm.py161CODE
LOWsrc/inspect_ai/model/_providers/nnterp.py54CODE
LOWsrc/inspect_ai/model/_providers/openai_compatible.py79CODE
LOWsrc/inspect_ai/model/_providers/openai.py119CODE
LOWsrc/inspect_ai/log/_condense.py260CODE
16 more matches not shown…
Hallucination Indicators6 hits · 60 pts
SeverityFileLineSnippetContext
CRITICALtests/_view/test_view_server.py1413 buffer = inspect_ai.log._recorders.buffer.filestore.SampleBufferFilestore(CODE
CRITICALtests/_view/test_view_server.py1768 inspect_ai.log._recorders.buffer.filestore.SampleManifest(CODE
CRITICALtests/_view/test_view_server.py1779 inspect_ai.log._recorders.buffer.filestore.Segment(CODE
CRITICALtests/_view/test_view_server.py1786 manifest = inspect_ai.log._recorders.buffer.filestore.Manifest(CODE
CRITICALtests/model/test_should_retry_classification.py797 from azure.core.exceptions import HttpResponseErrorCODE
CRITICALtests/model/test_should_retry_classification.py812 from azure.core.exceptions import HttpResponseErrorCODE
Structural Annotation Overuse29 hits · 52 pts
SeverityFileLineSnippetContext
LOW⚡design/recover.md434### Step 1: Read crashed .eval files (done: `162628a3f`)COMMENT
LOW⚡design/recover.md444### Step 2: Read recovery data from sample buffer database (done: `664a9a43b`)COMMENT
LOW⚡design/recover.md454### Step 3: Reconstruct EvalSample from buffer DB data (done: `a05ce7fda`)COMMENT
LOWdesign/recover.md468### Step 4: Write recovered .eval file (done: `493de0a1d`)COMMENT
LOWtests/model/test_compaction_edit_server_tools.py101 # Step 1: Initial request with server-side toolsCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py129 # Step 2: Follow-up to build more historyCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py143 # Step 3: Another follow-upCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py158 # Step 4: Compact history - clear older tool resultsCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py181 # Step 5: Final request with compacted history - THIS SHOULD NOT CAUSE 400 ERRORCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py90 # Step 2: Execute tool calls if presentCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py95 # Step 3: Ask follow-up questionCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py112 # Step 4: Add thank you messageCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py115 # Step 5: Compact history - remove ALL thinking blocksCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py145 # Step 6: Call model with compacted historyCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py152 # Step 7: This should NOT raise an errorCOMMENT
LOWtests/model/test_compaction_edit_thinking.py61 # Step 1: System prompt that STRONGLY requires tool use + user promptCOMMENT
LOWtests/model/test_web_search_exclude_none.py97 # Step 1: Convert OpenAI Response to ChatMessageAssistantCOMMENT
LOWtests/model/test_web_search_exclude_none.py136 # Step 2: Convert ChatMessageAssistant back to OpenAI Input ItemsCOMMENT
LOW⚡tests/model/providers/test_google.py1473 # Step 1: Simulate model returning an image via inline_dataCOMMENT
LOWtests/model/providers/test_google.py1496 # Step 2: Build assistant message and convert back to Google ContentCOMMENT
LOW⚡tests/model/providers/test_openai_responses.py1490 # Step 2: Build an assistant message with the image contentCOMMENT
LOW⚡tests/model/providers/test_openai_responses.py1497 # Step 3: Convert back to input items (simulating the next turn)COMMENT
LOWtests/model/providers/test_openai_responses.py1473 # Step 1: Model returns an image generation callCOMMENT
LOWsrc/inspect_ai/model/_providers/_anthropic_citations.py109 # NOTE: This function cannot assume that this module was responsible for theCOMMENT
LOWsrc/inspect_ai/log/_log.py1306 # WARNING: The order of these fields is important for the log file format.COMMENT
LOWsrc/inspect_ai/log/_recover/_api.py191 # Step 1: Read the crashed .eval file metadataCOMMENT
LOWsrc/inspect_ai/log/_recover/_api.py228 # Step 2: Read buffer DB metadata (lightweight — just summaries)COMMENT
LOWsrc/inspect_ai/log/_recover/_api.py260 # Step 3: Determine recovery path and writeCOMMENT
LOWsrc/inspect_ai/log/_recover/_api.py303 # Step 4: Stream all samples into the recovered file.COMMENT
Fake / Example Data40 hits · 46 pts
SeverityFileLineSnippetContext
LOWdesign/litellm-proxy.md503- The `get_model_info()` dummy-key path already catches exceptions and returnsCODE
LOW⚡tests/tools/test_web_search.py280 "fake-key" if key == "TAVILY_API_KEY" else defaultCODE
LOW⚡tests/tools/test_web_search.py289 "fake-key" if key == "TAVILY_API_KEY" else defaultCODE
LOW⚡tests/tools/test_web_search.py298 "fake-key" if key == "TAVILY_API_KEY" else defaultCODE
LOW⚡tests/tools/test_web_search.py307 return_value=("fake-key", "fake-cse-id"),CODE
LOW⚡tests/tools/test_web_search.py314 return_value=("fake-key", "fake-cse-id"),CODE
LOW⚡tests/tools/test_web_search.py322 "fake-key" if key == "EXA_API_KEY" else defaultCODE
LOW⚡tests/tools/test_web_search.py331 "fake-key" if key == "EXA_API_KEY" else defaultCODE
LOW⚡tests/tools/test_web_search.py340 "fake-key" if key == "EXA_API_KEY" else defaultCODE
LOWtests/tools/test_web_search.py365 return_value=("fake-key", "fake-cse-id"),CODE
LOWtests/tools/test_web_search.py396 return_value=("fake-key", "fake-cse-id"),CODE
LOWtests/tools/test_web_search_exa.py40 with patch.dict("os.environ", {"EXA_API_KEY": "dummy-key"}):CODE
LOW⚡tests/tools/test_web_search_tavily.py59 with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):CODE
LOWtests/tools/test_web_search_tavily.py113 with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):CODE
LOW⚡tests/tools/test_web_search_tavily.py127 with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):CODE
LOW⚡tests/tools/test_web_search_tavily.py144 with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):CODE
LOWtests/tools/test_web_search_google.py94 mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")CODE
LOWtests/tools/test_web_search_google.py141 mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")CODE
LOWtests/tools/test_web_search_google.py174 mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")CODE
LOW⚡tests/util/test_store_nested_types.py97 "street": "123 Main St",CODE
LOWtests/util/test_format_template.py171 address=Address(street="123 Main St", city="Anytown"),CODE
LOW⚡tests/util/test_json_schema.py169 instance = Model(name="Alice", address={"street": "123 Main St", "city": "NYC"})CODE
LOW⚡tests/util/test_json_schema.py171 assert instance.address.street == "123 Main St"CODE
LOWtests/util/test_json_schema.py200 "contact": {"email": "bob@example.com", "phone": "555-1234"},CODE
LOWtests/agent/test_sandbox_agent_bridge_media.py27 api_key="local-fake-key",CODE
LOWtests/model/test_token_counting_integration.py72 assert result[0]["name"] == "placeholder"CODE
LOW⚡tests/model/providers/test_vllm.py37 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOWtests/model/providers/test_vllm.py318 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOWtests/model/providers/test_litellm_proxy_reasoning.py167 id="placeholder",CODE
LOW⚡tests/model/providers/test_hf.py291 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOW⚡tests/model/providers/test_hf.py293 assert chat == "user: Lorem ipsum dolor\n"CODE
LOWtests/model/providers/test_hf.py67 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOWtests/model/providers/test_hf.py93 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOWtests/model/providers/test_hf.py373 message = ChatMessageUser(content="Lorem ipsum dolor")CODE
LOWtests/model/providers/test_hf.py375 assert chat == "[user] Lorem ipsum dolor"CODE
LOWexamples/http_proxy/claude.py56 "FUTUREMODEL_API_KEY": "fm-fake-key-for-demo",CODE
LOW…ct_tool_support/_remote_tools/_web_browser/cdp/a11y.py94 "attribute", "implicit", "stylet", "contents", "placeholder", "relatedElement"CODE
LOW⚡src/inspect_ai/model/_openai_responses.py1224 param["id"] = "dummy-id"CODE
LOWsrc/inspect_ai/model/_openai_responses.py2625 "name": "placeholder",CODE
LOWsrc/inspect_ai/model/_providers/anthropic.py5827 name="placeholder",CODE
Verbosity Indicators23 hits · 41 pts
SeverityFileLineSnippetContext
LOWtests/model/test_compaction_edit_server_tools.py101 # Step 1: Initial request with server-side toolsCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py129 # Step 2: Follow-up to build more historyCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py143 # Step 3: Another follow-upCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py158 # Step 4: Compact history - clear older tool resultsCOMMENT
LOWtests/model/test_compaction_edit_server_tools.py181 # Step 5: Final request with compacted history - THIS SHOULD NOT CAUSE 400 ERRORCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py90 # Step 2: Execute tool calls if presentCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py95 # Step 3: Ask follow-up questionCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py112 # Step 4: Add thank you messageCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py115 # Step 5: Compact history - remove ALL thinking blocksCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py145 # Step 6: Call model with compacted historyCOMMENT
LOW⚡tests/model/test_compaction_edit_thinking.py152 # Step 7: This should NOT raise an errorCOMMENT
LOWtests/model/test_compaction_edit_thinking.py61 # Step 1: System prompt that STRONGLY requires tool use + user promptCOMMENT
LOWtests/model/test_web_search_exclude_none.py97 # Step 1: Convert OpenAI Response to ChatMessageAssistantCOMMENT
LOWtests/model/test_web_search_exclude_none.py136 # Step 2: Convert ChatMessageAssistant back to OpenAI Input ItemsCOMMENT
LOW⚡tests/model/providers/test_google.py1473 # Step 1: Simulate model returning an image via inline_dataCOMMENT
LOWtests/model/providers/test_google.py1496 # Step 2: Build assistant message and convert back to Google ContentCOMMENT
LOW⚡tests/model/providers/test_openai_responses.py1490 # Step 2: Build an assistant message with the image contentCOMMENT
LOW⚡tests/model/providers/test_openai_responses.py1497 # Step 3: Convert back to input items (simulating the next turn)COMMENT
LOWtests/model/providers/test_openai_responses.py1473 # Step 1: Model returns an image generation callCOMMENT
LOWsrc/inspect_ai/log/_recover/_api.py191 # Step 1: Read the crashed .eval file metadataCOMMENT
LOWsrc/inspect_ai/log/_recover/_api.py228 # Step 2: Read buffer DB metadata (lightweight — just summaries)COMMENT
LOWsrc/inspect_ai/log/_recover/_api.py260 # Step 3: Determine recovery path and writeCOMMENT
LOWsrc/inspect_ai/log/_recover/_api.py303 # Step 4: Stream all samples into the recovered file.COMMENT
Dead Code9 hits · 18 pts
SeverityFileLineSnippetContext
MEDIUMtests/tools/test_tool_error.py75CODE
MEDIUM…ts/agent/test_acp/test_tui/test_inline_request_card.py129CODE
MEDIUMtests/model/providers/test_mistral.py956CODE
MEDIUMtests/model/providers/test_sagemaker.py1470CODE
MEDIUMtests/model/providers/test_openai.py714CODE
MEDIUMsrc/inspect_ai/agent/_acp/tui/widgets/cancel_card.py162CODE
MEDIUMsrc/inspect_ai/agent/_acp/tui/widgets/approval_card.py112CODE
MEDIUM…spect_ai/agent/_acp/tui/widgets/inline_request_card.py115CODE
MEDIUM…spect_ai/agent/_acp/tui/widgets/inline_request_card.py126CODE
Synthetic Comment Markers1 hit · 8 pts
SeverityFileLineSnippetContext
HIGHtests/model/providers/test_bedrock_adaptive_thinking.py360 # the base Sonnet 5 keeps forced tool choice as requestedCOMMENT
Overly Generic Function Names6 hits · 6 pts
SeverityFileLineSnippetContext
LOWtests/test_flaky_retry.py168 def test_function():CODE
LOWtests/review/test_review.py540 def helper() -> Agent:CODE
LOW…t_tool_support/src/inspect_tool_support/_cli/server.py16 async def handle_request(request: Request) -> Response:CODE
LOW…sandbox_tools/src/inspect_sandbox_tools/_cli/server.py138 async def handle_request(request: Request) -> Response:CODE
LOWsrc/inspect_ai/_util/_async.py64 async def run_task(func: Callable[[], Awaitable[T]], index: int) -> None:CODE
LOWsrc/inspect_ai/util/_collect.py38 async def run_task(index: int, task: Awaitable[T]) -> None:CODE