UKGovernmentBEIS/inspect_ai

30.1

Adjusted Score

30.1

Raw Score

100%

Time Factor

2026-08-09

Last Push

2.5K

Stars

Python

Language

538.1K

Lines of Code

1.8K

Files

12.6K

Pattern Hits

2026-08-09

Scan Date

0.11

HC Hit Rate

What These Metrics Mean

Adjusted Score: Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score: The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor: The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits: Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate: High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files: Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 6HIGH 201MEDIUM 1417LOW 10978

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 12602 distinct pattern matches across 21 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers8199 hits · 7530 pts

Severity	File	Line	Snippet	Context
LOW	design/ctl/config-log-persistence.md	161	def effective_generate_config(log: EvalLog) -> GenerateConfig: ...	CODE
LOW	design/ctl/generate-progress.md	90	def report_active_model_progress(output_tokens: int \| None = None) -> None:	CODE
LOW	tests/test_task_cancel.py	24	def test_abort_cancel_produces_error_status() -> None:	CODE
LOW	tests/test_task_cancel.py	83	def test_abort_cancel_not_retried_without_task_retries() -> None:	CODE
LOW	tests/test_task_cancel.py	156	def test_score_resolution_cancel_completes_eval() -> None:	CODE
LOW	tests/test_task_cancel.py	212	def test_error_resolution_cancel_completes_eval() -> None:	CODE
LOW	tests/test_task_cancel.py	254	def test_error_resolution_rejected_when_samples_fail_on_error() -> None:	CODE
LOW	tests/test_task_cancel.py	263	def error_resolution_rejected_solver():	CODE
LOW	tests/test_task_cancel.py	292	def test_error_resolution_downgraded_for_materializing_fail_on_error_sample() -> None:	CODE
LOW	tests/test_task_cancel.py	318	def downgrade_resolution_solver():	CODE
LOW	tests/test_task_cancel.py	355	def test_sample_cancelled_interrupt_action() -> None:	CODE
LOW	tests/test_task_cancel.py	364	def cancelled_interrupt_solver():	CODE
LOW	tests/test_task_cancel.py	402	def test_score_resolution_sweep_preserves_cancelled_sample() -> None:	CODE
LOW	tests/test_task_cancel.py	415	def cancelled_then_score_solver():	CODE
LOW	tests/test_task_cancel.py	452	def test_interrupt_in_retry_drain_window_resolves_cancelled() -> None:	CODE
LOW	tests/test_task_cancel.py	479	def drain_window_error_solver():	CODE
LOW	tests/test_task_cancel.py	549	def test_external_interrupt_with_pending_resolution_logs_cancelled(	CODE
LOW	tests/test_task_cancel.py	605	def test_errored_attempt_marked_retry_pending() -> None:	CODE
LOW⚡	tests/test_sample_limits.py	139	def test_message_limit_generate():	CODE
LOW⚡	tests/test_sample_limits.py	143	def test_message_limit_append():	CODE
LOW⚡	tests/test_sample_limits.py	147	def test_message_limit_overwrite():	CODE
LOW⚡	tests/test_sample_limits.py	151	def test_message_limit_reached_before_assistant_message():	CODE
LOW	tests/test_sample_limits.py	68	def looping_concurrecy_solver():	CODE
LOW	tests/test_sample_limits.py	167	def test_message_limit_does_not_apply_to_scorer():	CODE
LOW	tests/test_sample_limits.py	216	def test_token_limit_does_not_apply_to_scorer():	CODE
LOW	tests/test_sample_limits.py	262	def test_output_token_limit_string_form():	CODE
LOW	tests/test_sample_limits.py	297	def test_token_limit_type_absent_for_int_limit():	CODE
LOW	tests/test_sample_limits.py	343	def test_turn_limit_does_not_apply_to_scorer():	CODE
LOW	tests/test_sample_limits.py	391	def test_sample_limits_available_to_scorer():	CODE
LOW	tests/test_sample_limits.py	422	def test_solver_scorer_combined_timeout():	CODE
LOW	tests/test_sample_limits.py	431	def test_solver_scorer_combined_timeout_exceeded():	CODE
LOW	tests/test_sample_limits.py	442	def test_solver_timeout_scored():	CODE
LOW	tests/test_sample_limits.py	451	def test_solver_timeout_not_scored():	CODE
LOW	tests/test_sample_limits.py	470	def test_working_limit_reporting():	CODE
LOW	tests/test_sample_limits.py	487	def test_working_limit_does_not_raise_during_sandbox_teardown() -> None:	CODE
LOW	tests/test_sample_limits.py	502	def check_working_limit_event(log: EvalLog, working_limit: int):	CODE
LOW	tests/test_sample_limits.py	556	def test_cost_limit_without_cost_data_errors() -> None:	CODE
LOW	tests/test_sample_limits.py	568	def test_model_without_cost_data_errors() -> None:	CODE
LOW	tests/test_sample_limits.py	585	def test_cost_data_without_cost_limit_tracks_cost() -> None:	CODE
LOW	tests/test_sample_limits.py	616	def test_cost_data_keyed_by_full_model_string_tracks_cost() -> None:	CODE
LOW	tests/test_sample_limits.py	653	def test_two_models_both_with_cost_data_tracks_cost() -> None:	CODE
LOW	tests/test_sample_limits.py	704	def test_task_level_cost_limit_without_cost_data_errors() -> None:	CODE
LOW	tests/test_sample_limits.py	716	def test_task_level_cost_limit() -> None:	CODE
LOW	tests/test_sample_limits.py	747	def test_model_cost_config_file() -> None:	CODE
LOW	tests/test_sample_limits.py	779	def test_model_cost_config_dict() -> None:	CODE
LOW	tests/test_retry.py	65	def test_eval_retry_with_task_name():	CODE
LOW	tests/test_retry.py	84	def test_eval_retry_with_model_generate_config():	CODE
LOW	tests/test_retry.py	109	def test_eval_retry_preserves_token_limit_type():	CODE
LOW	tests/test_retry.py	126	def test_eval_retry_honors_zero_max_retries(	CODE
LOW	tests/test_retry.py	175	def test_eval_retry_resolves_relative_sandbox_paths():	CODE
LOW	tests/test_retry.py	284	def test_eval_retry_preserves_scorer_attribution(monkeypatch) -> None:	CODE
LOW	tests/test_retry.py	325	def test_eval_retry_preserves_token_usage():	CODE
LOW	tests/test_retry.py	347	def test_eval_retry_token_usage_multi_retry():	CODE
LOW	tests/test_retry.py	400	def _reuse_flush_probe_solver(log_dir: str, probe_dir: str):	CODE
LOW	tests/test_retry.py	448	def test_eval_retry_flushes_reused_samples_during_live_run(tmp_path: Path):	CODE
LOW	tests/test_list_task.py	15	def test_task_list_multiple_file():	CODE
LOW	tests/test_list_task.py	23	def test_task_list_multiple_dir():	CODE
LOW	tests/test_task_retry_error_history.py	61	def test_task_retry_seeds_error_retries_on_sample() -> None:	CODE
LOW	tests/test_task_retry_error_history.py	77	def test_task_retry_retries_in_sample_summaries() -> None:	CODE
LOW	tests/test_task_retry_error_history.py	93	def test_is_cancellation_error_distinguishes_cancellations() -> None:	CODE
8139 more matches not shown…

Decorative Section Separators1141 hits · 4266 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/conftest.py	20	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/conftest.py	34	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	86	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	88	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	197	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	199	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	260	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	262	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	421	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	423	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	563	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	565	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	641	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	643	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	888	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	890	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	927	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_pause.py	929	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	31	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	33	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	132	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	134	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	217	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_buffer.py	219	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_server.py	1057	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_server.py	1059	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	85	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	87	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	379	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	381	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	513	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_cancel.py	515	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	311	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	313	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	424	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	426	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	617	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	619	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	759	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_limits.py	761	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	53	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	55	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	1151	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	1153	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	1595	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_limits.py	1597	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	178	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	180	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	504	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	506	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	919	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	921	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	1061	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/_control/test_requeue.py	1063	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_requeue.py	1142	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_requeue.py	1144	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_requeue.py	1485	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/_control/test_requeue.py	1487	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/tools/test_tools_bridge.py	24	# =============================================================================	COMMENT
MEDIUM	tests/tools/test_tools_bridge.py	26	# =============================================================================	COMMENT
1081 more matches not shown…

Unused Imports1246 hits · 1032 pts

Severity	File	Line	Context
LOW	tests/conftest.py	15	CODE
LOW	tests/test_eval_set_scanner.py	3078	CODE
LOW	tests/test_disk_sample_store.py	22	CODE
LOW	tests/test_disk_sample_store.py	23	CODE
LOW	tests/_control/test_state.py	20	CODE
LOW	tests/checkpoint/test_eval_checkpoints.py	3	CODE
LOW	tests/checkpoint/test_sample_checkpoints.py	3	CODE
LOW	tests/checkpoint/test_resolve.py	3	CODE
LOW	tests/checkpoint/conftest.py	11	CODE
LOW	tests/checkpoint/test_checkpoint_scoring_resume_e2e.py	23	CODE
LOW	tests/checkpoint/resume_kill_thinking_harness.py	35	CODE
LOW	tests/checkpoint/test_parse.py	3	CODE
LOW	tests/checkpoint/test_staging_dir.py	3	CODE
LOW	tests/checkpoint/test_current_checkpointer_post_exit.py	31	CODE
LOW	tests/checkpoint/test_sandbox_restic_repo.py	13	CODE
LOW	tests/checkpoint/test_host_egress.py	9	CODE
LOW	tests/checkpoint/test_checkpoint_e2e.py	30	CODE
LOW	tests/checkpoint/test_checkpointer.py	10	CODE
LOW	tests/checkpoint/test_fs_copy_s3.py	12	CODE
LOW	tests/checkpoint/resume_scoring_kill_harness.py	24	CODE
LOW	tests/checkpoint/test_host_egress_s3.py	9	CODE
LOW	tests/checkpoint/resume_kill_harness.py	20	CODE
LOW	tests/checkpoint/test_schemas.py	9	CODE
LOW	tests/checkpoint/test_sandbox_egress_extract.py	11	CODE
LOW	tests/checkpoint/test_sandbox_paths.py	3	CODE
LOW	tests/checkpoint/test_normalize.py	10	CODE
LOW	tests/util/test_restic_ops.py	10	CODE
LOW	tests/util/test_zipfile_multiframe.py	8	CODE
LOW	tests/util/test_zipfile_multiframe.py	23	CODE
LOW	tests/util/test_download.py	3	CODE
LOW	tests/util/test_restic_summary.py	9	CODE
LOW	tests/util/test_store_from_events.py	7	CODE
LOW	tests/util/test_async_zip.py	7	CODE
LOW	tests/util/test_async_zip.py	22	CODE
LOW	tests/util/test_limit_working.py	1	CODE
LOW	tests/util/test_compression_transcoding.py	3	CODE
LOW	tests/util/test_restic_binary.py	19	CODE
LOW	tests/input/test_input_acp.py	11	CODE
LOW	tests/input/test_input_panel.py	11	CODE
LOW	tests/agent/test_bridge_additional_tools.py	16	CODE
LOW	tests/agent/test_bridge_agent_message.py	25	CODE
LOW	tests/agent/test_bridge_google_messages.py	18	CODE
LOW	tests/agent/test_bridge_tool_search.py	9	CODE
LOW	tests/agent/test_bridge_provider_errors.py	10	CODE
LOW	tests/agent/test_react_checkpoint_resume.py	1	CODE
LOW	tests/agent/test_bridge_namespace_tool.py	3	CODE
LOW	tests/agent/deepagent/test_deepagent_examples.py	18	CODE
LOW	tests/agent/deepagent/test_deepagent_background.py	11	CODE
LOW	tests/agent/test_channel/test_coalesce.py	11	CODE
LOW	tests/agent/test_channel/test_channel.py	3	CODE
LOW	tests/agent/test_acp/test_server_dispatch.py	15	CODE
LOW	tests/agent/test_acp/test_disconnect_log_filter.py	12	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	19	CODE
LOW	tests/agent/test_acp/test_standard_acp_client.py	22	CODE
LOW	tests/agent/test_acp/test_plan_policy.py	11	CODE
LOW	tests/agent/test_acp/test_raw_events.py	19	CODE
LOW	tests/agent/test_acp/test_action_methods.py	14	CODE
LOW	tests/agent/test_acp/test_discovery.py	16	CODE
LOW	tests/agent/test_acp/test_elicitation_e2e.py	21	CODE
LOW	tests/agent/test_acp/test_stdio_bridge.py	11	CODE
1186 more matches not shown…

Deep Nesting522 hits · 456 pts

Severity	File	Line	Context
LOW	tests/test_eval_set_scanner.py	1093	CODE
LOW	tests/_control/test_pause.py	932	CODE
LOW	tests/_control/test_pause.py	962	CODE
LOW	tests/_control/test_pause.py	963	CODE
LOW	tests/_control/test_requeue.py	1393	CODE
LOW	tests/tools/test_mcp_tools.py	82	CODE
LOW	tests/tools/test_mcp_tools.py	500	CODE
LOW	tests/tools/test_mcp_tools.py	123	CODE
LOW	tests/tools/test_mcp_tools.py	132	CODE
LOW	tests/tools/test_skill.py	490	CODE
LOW	tests/agent/test_agent_compaction.py	208	CODE
LOW	tests/agent/test_agent_react.py	148	CODE
LOW	tests/agent/test_agent_react.py	149	CODE
LOW	tests/agent/test_channel/test_channel.py	134	CODE
LOW	tests/agent/test_acp/test_server_dispatch.py	45	CODE
LOW	tests/agent/test_acp/test_server_dispatch.py	173	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	63	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	879	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	157	CODE
LOW	tests/agent/test_acp/test_standard_acp_client.py	54	CODE
LOW	tests/agent/test_acp/test_standard_acp_client.py	224	CODE
LOW	tests/agent/test_acp/test_standard_acp_client.py	148	CODE
LOW	tests/agent/test_acp/test_raw_events.py	62	CODE
LOW	tests/agent/test_acp/test_raw_events.py	313	CODE
LOW	tests/agent/test_acp/test_raw_events.py	133	CODE
LOW	tests/agent/test_acp/test_action_methods.py	471	CODE
LOW	tests/agent/test_acp/test_action_methods.py	514	CODE
LOW	tests/agent/test_acp/test_discovery.py	36	CODE
LOW	tests/agent/test_acp/test_elicitation_e2e.py	61	CODE
LOW	tests/agent/test_acp/test_elicitation_e2e.py	116	CODE
LOW	tests/agent/test_acp/test_stdio_bridge.py	43	CODE
LOW	tests/agent/test_acp/test_server_session_info.py	115	CODE
LOW	tests/agent/test_acp/test_server.py	29	CODE
LOW	tests/agent/test_acp/test_approval.py	1439	CODE
LOW	tests/agent/test_acp/test_approval.py	1493	CODE
LOW	tests/agent/test_acp/test_tui/test_client.py	89	CODE
LOW	tests/_cli/test_acp_cli.py	71	CODE
LOW	tests/model/test_compaction_edit_server_tools.py	60	CODE
LOW	tests/model/test_compaction_edit_thinking.py	48	CODE
LOW	tests/model/test_compaction.py	736	CODE
LOW	tests/model/test_model_info.py	300	CODE
LOW	tests/model/providers/test_anthropic.py	59	CODE
LOW	tests/model/providers/test_anthropic_cache_control.py	55	CODE
LOW	tests/model/providers/test_perplexity.py	22	CODE
LOW	tests/view/test_sample_list_states.py	368	CODE
LOW	tests/view/test_sample_list_states.py	371	CODE
LOW	tests/log/test_eval_log.py	378	CODE
LOW	tests/log/test_eval_log.py	1237	CODE
LOW	tests/log/test_log_formats.py	156	CODE
LOW	tests/log/test_log_formats.py	555	CODE
LOW	tests/log/test_convert.py	625	CODE
LOW	tests/test_helpers/utils.py	36	CODE
LOW	tests/test_helpers/utils.py	72	CODE
LOW	tests/test_helpers/utils.py	96	CODE
LOW	tests/test_helpers/outline/oracle.py	320	CODE
LOW	tests/test_helpers/outline/oracle.py	608	CODE
LOW	tests/test_helpers/outline/oracle.py	632	CODE
LOW	tests/test_helpers/outline/oracle.py	798	CODE
LOW	tests/test_helpers/outline/compare.py	172	CODE
LOW	docs/_scripts/generate_reasoning_defaults_table.py	34	CODE
462 more matches not shown…

Docstring Block Structure90 hits · 450 pts

Severity	File	Line	Snippet	Context
HIGH	…ort/src/inspect_tool_support/_util/json_rpc_helpers.py	43	Validates RPC method parameters and handles the method execution. This function validates the provided paramet	STRING
HIGH	src/inspect_sandbox_tools/tests/conftest.py	79	Execute an RPC request via the CLI and return the parsed response. Args: request: The JSON-RPC request dict	STRING
HIGH	src/inspect_sandbox_tools/tests/test_exec_remote.py	34	Poll an exec_remote job until it completes or times out. Args: rpc_client: The RPC client function to use f	STRING
HIGH	…ls/src/inspect_sandbox_tools/_util/json_rpc_helpers.py	46	Validates RPC method parameters and handles the method execution. This function validates the provided paramet	STRING
HIGH	src/inspect_ai/_util/registry.py	399	Create a registry object. Creates objects registered via decorator (e.g. `@task`, `@solver`). Note that this ca	STRING
HIGH	src/inspect_ai/_util/registry.py	475	Lookup RegistryInfo for an object. Args: o (object): Object to lookup info for Returns: Regist	STRING
HIGH	src/inspect_ai/_util/download.py	44	Download a file and verify its SHA256 checksum. If `dest` already exists and its checksum matches, the download is	STRING
HIGH	src/inspect_ai/_util/download.py	112	Download a Google Drive file via `gdown` and verify SHA256. Useful for fetching public-link Google Drive assets (da	STRING
HIGH	src/inspect_ai/_util/format.py	49	Format a template string, optionally preserving unknown placeholders. Args: template: A string containing {	STRING
HIGH	src/inspect_ai/_util/_json_rpc.py	159	Execute a JSON-RPC command expecting a scalar result. Args: method (str): The JSON-RPC method to call.	STRING
HIGH	src/inspect_ai/_util/_json_rpc.py	199	Execute a JSON-RPC command to a sandbox environment expecting a model result. Args: method (str): The JS	STRING
HIGH	src/inspect_ai/_util/_json_rpc.py	233	Execute a JSON-RPC notification to a sandbox environment. A notification is a JSON-RPC request that doesn't ex	STRING
HIGH	src/inspect_ai/_util/_json_rpc.py	342	Generate a string description of an RPC call. Args: method (str): The name of the RPC method.	STRING
HIGH	src/inspect_ai/_util/local_server.py	294	Start a server with the given command and handle potential errors. Args: base_cmd: List of base comman	STRING
HIGH	src/inspect_ai/_util/file.py	554	Convert a string into a safe filename by removing or replacing unsafe characters. Args: s (str): The i	STRING
HIGH	src/inspect_ai/_util/dateutil.py	160	Parse and normalize ISO datetime string to UTC. For legacy string temporal fields that cannot be converted to UtcDa	STRING
HIGH	src/inspect_ai/_util/_async.py	41	Runs all of the passed async functions and collects their results. The results will be returned in the same order a	STRING
HIGH	src/inspect_ai/_util/text.py	141	Convert a str to float, handling exponent characters and Unicode fractions. The Python isnumeric() function returns	STRING
HIGH	src/inspect_ai/_util/async_zip.py	396	Open a ZIP member and stream its decompressed contents. Must be used as an async context manager to ensure prop	STRING
HIGH	src/inspect_ai/_util/atomic_write.py	103	Context manager for atomic local-file writes with durability guarantees. Implements the write-to-temporary-then-ato	STRING
HIGH	src/inspect_ai/util/_json.py	373	Convert JSON schema to Pydantic BaseModel. Handles nested objects, arrays, and basic validations. Args:	STRING
HIGH	src/inspect_ai/util/_subprocess.py	86	Execute and wait for a subprocess. Convenience method for solvers, scorers, and tools to launch subprocesses. A	STRING
HIGH	src/inspect_ai/util/_store.py	227	Convert a JsonChange to a jsonpatch operation dict with validation. Args: change: The JsonChange to convert	STRING
HIGH	src/inspect_ai/util/_notify.py	62	Build an `Apprise` instance for the eval's notification config. Notification URLs frequently carry secrets (Slack t	STRING
HIGH	src/inspect_ai/util/_panel.py	82	Create an input panel in the task display. There can only be a single instance of an InputPanel with a given 't	STRING
HIGH	src/inspect_ai/util/_restic/resolver.py	129	Return a path to a usable restic binary for the given platform. Downloads the archive on cache miss, retrying trans	STRING
HIGH	src/inspect_ai/util/_sandbox/exec_remote.py	575	Run a command and return the result without streaming. Submits the command, polls until completion, and returns Exe	STRING
HIGH	src/inspect_ai/util/_sandbox/compose.py	367	Parse a Docker Compose file into a ComposeConfig. Args: file: Path to the compose file. multiple_se	STRING
HIGH	src/inspect_ai/util/_sandbox/context.py	173	Get a SandboxEnvironment that satisfies all the given injection requirements. Args: injectables: Single San	STRING
HIGH	src/inspect_ai/util/_sandbox/context.py	409	Find the best sandbox for injection and return it with needed injections. Args: injectables: List of detect	STRING
HIGH	src/inspect_ai/util/_sandbox/environment.py	116	Execute a command within a sandbox environment. The current working directory for execution will be the per-sam	STRING
HIGH	src/inspect_ai/util/_sandbox/environment.py	190	Read a file from the sandbox environment. By default, file size is limited to 100 MiB. You may change this by s	STRING
HIGH	src/inspect_ai/util/_sandbox/environment.py	221	Information required to connect to sandbox environment. Args: user: User to login as. Return	STRING
HIGH	src/inspect_ai/util/_sandbox/environment.py	260	Start a command and return a process handle or result. In streaming mode (stream=True), the function returns on	STRING
HIGH	src/inspect_ai/util/_sandbox/environment.py	333	Verify and return a reference to a subclass of SandboxEnvironment. Args: sandbox_cls: Class of sandb	STRING
HIGH	src/inspect_ai/util/_sandbox/_json_rpc_transport.py	72	Execute an RPC request using the sandbox transport. Args: method: The JSON-RPC method to call.	STRING
HIGH	src/inspect_ai/_lfs/_client.py	39	Get download URLs for LFS objects via the batch API. Chunks requests to avoid exceeding GitHub's payload size limit	STRING
HIGH	src/inspect_ai/_lfs/resolver.py	20	Resolve a directory that may contain LFS pointer files. Recursively checks source_dir for LFS pointers. If none are	STRING
HIGH	src/inspect_ai/solver/_solver.py	85	Contribute to solving an evaluation task. Transform a `TaskState`, returning the new state. Solvers may	STRING
HIGH	src/inspect_ai/solver/_solver.py	163	Decorator for registering solvers. Args: name: Optional name for solver. If the decorator has n	STRING
HIGH	src/inspect_ai/scorer/_scorer.py	139	Decorator for registering scorers. Args: metrics: One or more metrics to calculate over the sco	STRING
HIGH	src/inspect_ai/scorer/_math.py	91	Remove nested boxed expressions, keeping only outermost content. Args: match: Text containing potentially n	STRING
HIGH	src/inspect_ai/scorer/_math.py	114	Extract content from the last \boxed{} or \fbox{} in text. Args: text: Text potentially containing boxed ex	STRING
HIGH	src/inspect_ai/scorer/_math.py	140	Fallback: extract the last integer found in text. Args: text: Text to search for integers. Returns:	STRING
HIGH	src/inspect_ai/scorer/_math.py	220	Strip whitespace and LaTeX newlines from string edges. Args: s: String to strip. Returns: Stri	STRING
HIGH	src/inspect_ai/scorer/_math.py	252	Normalize a LaTeX string for parsing. Removes sizing commands, alignment environments, converts brackets, and p	STRING
HIGH	src/inspect_ai/scorer/_math.py	355	Remove matching outer parentheses if they wrap the entire expression. Args: s: String potentially wrapped i	STRING
HIGH	src/inspect_ai/scorer/_math.py	398	Remove LaTeX spacing commands that interfere with parsing. Args: text: Text containing LaTeX spacing comman	STRING
HIGH	src/inspect_ai/scorer/_metric.py	274	Compute a metric on a list of scores. Args: scores: List of scores. Returns: Metri	STRING
HIGH	src/inspect_ai/scorer/_score.py	15	Score a model conversation. Score a model conversation (you may pass `TaskState` or `AgentState` as the value f	STRING
HIGH	src/inspect_ai/_cli/util.py	233	Parse model roles from CLI args. Supports key-value, YAML, and JSON formats. Args: model_roles: Tuple of st	STRING
HIGH	src/inspect_ai/model/_internal.py	15	Extracts and removes a smuggled <internal>...</internal> tag from the content string, if present. Note:	STRING
HIGH	src/inspect_ai/model/_openai_responses.py	2288	Convert CompactedResponse to a list of ChatMessages. The compact endpoint returns the complete new context window,	STRING
HIGH	src/inspect_ai/model/_google_convert.py	76	Convert Google GenerateContentResponse into Inspect ModelOutput. Args: response: Google GenerateContentResp	STRING
HIGH	src/inspect_ai/model/_model_info.py	265	Get model information including context window, output tokens, etc. Looks up model information from a local databas	STRING
HIGH	src/inspect_ai/model/_model.py	385	Tokenize text into token IDs using the model's tokenizer. Override in providers that support server-side tokeni	STRING
HIGH	src/inspect_ai/model/_model.py	532	Compact messages using provider-native compaction. Some model providers (e.g., OpenAI Codex models) support nat	STRING
HIGH	src/inspect_ai/model/_model.py	1032	Compact messages using provider-native compaction. Delegates to the model provider's native compaction API when	STRING
HIGH	src/inspect_ai/model/_compaction/native.py	75	Compact messages using the provider's native compaction API. Args: model: Target model for compacti	STRING
HIGH	src/inspect_ai/model/_compaction/_compaction.py	417	Perform compaction, iterating if necessary to get under threshold. Args: strategy: Compaction strategy to u	STRING
30 more matches not shown…

Cross-Language Confusion71 hits · 418 pts

Severity	File	Line	Snippet	Context
HIGH⚡	tests/_control/test_ctl.py	239	# a running sample with nothing pending (activity null) keeps the	COMMENT
HIGH	tests/tools/sandbox_tools_utils/test_sandbox_tools.py	252	"input": f"echo {flag} > /flag && chmod 400 /flag; ls -al /flag && cat /flag",	CODE
HIGH⚡	tests/util/test_store_from_events.py	516	"""Add operation should include value even when it's None (explicit null)."""	STRING
HIGH	tests/scorer/test_metric.py	194	Serialized as JSON constants (NaN/Infinity) rather than null, via both	STRING
HIGH	tests/scorer/test_reducers.py	569	# With fewer than k scored epochs the estimator is undefined, so the	COMMENT
HIGH	tests/_cli/test_util.py	69	"critic={model: mockllm/model, temperature: 0.2, model_args: {custom_outputs: null}}",	CODE
HIGH	tests/model/providers/test_anthropic.py	1755	'head -c 9 /dev/urandom \| base64 > /tmp/seed && echo "seed-$(cat /tmp/seed)"'	CODE
HIGH	tests/model/providers/test_anthropic.py	1757	pending_cmd = 'sleep 5 && echo "marker-$(cat /tmp/seed)"'	CODE
HIGH	tests/model/providers/test_bedrock_structured_output.py	38	nickname: Optional[str] = None # renders as anyOf:[string, null]	CODE
HIGH	…ts/model/providers/test_anthropic_cache_diagnostics.py	366	"""`{cache_miss_reason: null}` means comparison still pending — no warning."""	STRING
HIGH⚡	tests/log/test_message_pool.py	1026	Regression: if persistence stores call_refs as [] instead of null,	STRING
HIGH	tests/log/test_eval_log.py	1283	null_value = re.compile(r'"value":\s(null\|\[\snull\|\{"a":\s*null)')	CODE
HIGH⚡	…/_remote_tools/_web_browser/playwright_page_crawler.py	231	"(document.scrollingElement \|\| document.body).scrollTop ="	CODE
HIGH⚡	…/_remote_tools/_web_browser/playwright_page_crawler.py	232	" (document.scrollingElement \|\| document.body).scrollTop -"	CODE
HIGH⚡	…/_remote_tools/_web_browser/playwright_page_crawler.py	237	"(document.scrollingElement \|\| document.body).scrollTop ="	CODE
HIGH⚡	…/_remote_tools/_web_browser/playwright_page_crawler.py	238	" (document.scrollingElement \|\| document.body).scrollTop +"	CODE
HIGH	src/inspect_sandbox_tools/tests/test_exec_remote.py	292	"command": "echo hello_from_kill && echo err_from_kill >&2 && sleep 100"	CODE
HIGH	src/inspect_sandbox_tools/tests/test_exec_remote.py	412	"params": {"command": "echo stdout_msg && echo stderr_msg >&2"},	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	10	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	19	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	25	sd.push("B")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	34	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	39	sd.push("B")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	48	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	51	sd.push("B")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	54	sd.push("C")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	63	sd.push("data")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	66	sd.push("") # retransmit, no new data	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	76	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	80	sd.push("B")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	90	sd.push(("out1", "err1"))	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	96	sd.push(("out2", "err2"))	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	105	sd.push({"a": 1})	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	116	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	119	sd.push("B")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	123	sd.push("C")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	127	sd.push("D")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	136	sd.push("A")	CODE
HIGH⚡	…inspect_sandbox_tools/tests/test_acked_chunk_buffer.py	141	sd.push("B")	CODE
HIGH	…t_sandbox_tools/tests/agent_bridge/test_model_proxy.py	473	yield b'data: {"id":"chatcmpl-stream","object":"chat.completion.chunk","created":1234567890,"model":"gpt-3.5-tur	CODE
HIGH	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	165	self._acked_buffer.push((stdout, stderr))	CODE
HIGH	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	185	self._acked_buffer.push(("", ""))	CODE
HIGH	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	208	self._acked_buffer.push((stdout, stderr))	CODE
HIGH	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	249	self._acked_buffer.push((stdout, stderr))	CODE
HIGH	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	272	self._acked_buffer.push((stdout, stderr))	CODE
HIGH⚡	…ools/_remote_tools/_exec_remote/_acked_chunk_buffer.py	24	sd.push("A")	STRING
HIGH⚡	…ools/_remote_tools/_exec_remote/_acked_chunk_buffer.py	26	sd.push("B")	STRING
HIGH⚡	…ools/_remote_tools/_exec_remote/_acked_chunk_buffer.py	32	sd.push("A")	STRING
HIGH⚡	…ools/_remote_tools/_exec_remote/_acked_chunk_buffer.py	34	sd.push("B")	STRING
HIGH	src/inspect_ai/util/_limit.py	1024	token_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_limit.py	1115	turn_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_limit.py	1199	cost_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_limit.py	1280	message_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_limit.py	1356	time_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_limit.py	1421	working_limit_tree.push(self)	CODE
HIGH	src/inspect_ai/util/_store.py	241	# These operations require a value (None is valid for explicit null)	COMMENT
HIGH⚡	…/inspect_ai/util/_checkpoint/_sandbox_restic/egress.py	77	f" {{ find config -type f 2>/dev/null; "	CODE
HIGH⚡	…/inspect_ai/util/_checkpoint/_sandbox_restic/egress.py	78	f" find keys -type f 2>/dev/null; "	CODE
HIGH⚡	…/inspect_ai/util/_checkpoint/_sandbox_restic/egress.py	79	f" find data -type f 2>/dev/null; "	CODE
HIGH⚡	…/inspect_ai/util/_checkpoint/_sandbox_restic/egress.py	80	f" find index -type f 2>/dev/null; "	CODE
11 more matches not shown…

Excessive Try-Catch Wrapping385 hits · 399 pts

Severity	File	Line	Snippet	Context
LOW	design/deepagent-background.md	137	except Exception as ex:	CODE
LOW	design/recover.md	320	except Exception:	CODE
LOW	design/recover.md	401	except Exception:	CODE
LOW	tests/test_retry.py	392	except Exception:	CODE
LOW⚡	tests/test_eval_set.py	179	except Exception:	CODE
LOW⚡	tests/test_eval_set.py	186	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	41	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	100	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	142	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	189	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	226	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	255	except Exception:	CODE
LOW	tests/test_model_env_mismatch.py	285	except Exception:	CODE
LOW	tests/tools/test_tools_bridge.py	511	except Exception:	CODE
LOW	tests/tools/test_tool_with.py	34	except Exception:	CODE
MEDIUM	tests/tools/test_tool_with.py	27	def test_tool_with_validation():	CODE
LOW⚡	tests/util/test_anyio.py	66	except Exception as inner_tool_error:	CODE
LOW⚡	tests/util/test_anyio.py	72	except Exception as e:	CODE
LOW⚡	tests/util/test_anyio.py	74	except Exception as wow:	CODE
LOW	tests/util/sandbox/test_sandbox_service.py	88	except Exception as e:	STRING
LOW	tests/util/sandbox/test_sandbox_service.py	873	except Exception as e:	STRING
MEDIUM	tests/agent/deepagent/test_agent_tool.py	208	def import ToolDef	CODE
LOW	tests/agent/deepagent/test_agent_tool.py	216	except Exception:	CODE
LOW	tests/agent/test_acp/test_span_boundary.py	184	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_server_dispatch.py	173	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_server_dispatch.py	226	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_server_forwarding.py	157	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	208	except Exception:	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	739	except Exception:	CODE
LOW	tests/agent/test_acp/test_server_forwarding.py	803	except Exception:	CODE
LOW	tests/agent/test_acp/test_standard_acp_client.py	196	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_standard_acp_client.py	148	def _read_loop(self) -> None:	CODE
MEDIUM	tests/agent/test_acp/test_raw_events.py	133	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_raw_events.py	179	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_action_methods.py	514	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_action_methods.py	557	except Exception:	CODE
LOW	tests/agent/test_acp/test_elicitation_e2e.py	168	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_elicitation_e2e.py	116	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_server_session_info.py	158	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_server_session_info.py	115	def _read_loop(self) -> None:	CODE
LOW⚡	tests/agent/test_acp/test_server.py	244	except Exception:	CODE
LOW⚡	tests/agent/test_acp/test_server.py	296	except Exception:	CODE
LOW	tests/agent/test_acp/test_server.py	216	except Exception:	CODE
LOW	tests/agent/test_acp/test_server.py	365	except Exception:	CODE
LOW	tests/agent/test_acp/test_server.py	410	except Exception:	CODE
MEDIUM	tests/agent/test_acp/test_approval.py	1493	def _read_loop(self) -> None:	CODE
LOW	tests/agent/test_acp/test_approval.py	1546	except Exception:	CODE
LOW⚡	tests/agent/test_acp/test_tui/test_defer_body.py	147	except Exception:	CODE
LOW⚡	tests/agent/test_acp/test_tui/test_defer_body.py	155	except Exception:	CODE
LOW	tests/model/test_collapse_user_message.py	132	except Exception as ex:	CODE
LOW	tests/model/providers/test_mistral.py	114	except Exception as e:	CODE
LOW	tests/model/providers/util/test_batch.py	467	except Exception as e:	CODE
LOW	tests/model/providers/util/test_batch.py	613	except Exception as e:	CODE
LOW	tests/model/providers/util/test_batch.py	1043	except Exception as e:	CODE
LOW	…_extensions/meridianlabs-ai/inspect-docs/pre-render.py	477	except Exception:	CODE
LOW	…_extensions/meridianlabs-ai/inspect-docs/pre-render.py	819	except Exception as e:	CODE
LOW	…eridianlabs-ai/inspect-docs/filters/reference/parse.py	157	except Exception:	CODE
LOW	examples/structured.py	193	except Exception as ex:	STRING
LOW	examples/hooks/mlflow_tracking.py	55	except Exception:	CODE
LOW	examples/hooks/mlflow_tracking.py	203	except Exception:	CODE
325 more matches not shown…

Self-Referential Comments106 hits · 332 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/test_eval_set.py	2186	# Create a log with the exact task identity eval_set expects, then	COMMENT
MEDIUM	tests/test_model_env_mismatch.py	90	# Create a minimal task with proper solver and scorer	COMMENT
MEDIUM⚡	tests/tools/test_web_search_tavily.py	55	# Create a client with our mock transport	COMMENT
MEDIUM	tests/analysis/test_large_string.py	11	# Create a DataFrame with a small PyArrow string column	COMMENT
MEDIUM	tests/analysis/test_large_string.py	32	# Create a DataFrame with a large PyArrow string column	COMMENT
MEDIUM	tests/analysis/test_large_string.py	55	# Create a DataFrame with both small and large columns (same length)	COMMENT
MEDIUM⚡	tests/util/test_store_nested_types.py	89	# Create a store with dict data	COMMENT
MEDIUM⚡	tests/util/test_anyio.py	84	# Create a circular reference: exc1.__context__ -> exc2 -> exc1	COMMENT
MEDIUM	tests/util/test_anyio.py	107	# Create a circular reference in exception groups	COMMENT
MEDIUM	tests/util/sandbox/test_docker_compatibility.py	530	# Create a task with compose.yaml config (use None for dataset to get a dummy sample)	COMMENT
MEDIUM	tests/util/sandbox/sandbox_setup_symlink.sh	12	# Create a nested directory and symlink via “..”	COMMENT
MEDIUM	tests/util/sandbox/test_docker_compose_config.py	160	# Create a ComposeService with x-default extension	COMMENT
MEDIUM⚡	tests/agent/test_agent_react.py	582	# Create a tool call for submit	COMMENT
MEDIUM⚡	tests/agent/test_agent_react.py	589	# Create a tool call for a regular tool	COMMENT
MEDIUM⚡	tests/agent/test_agent_react.py	596	# Create an assistant message with both reasoning content and tool calls	COMMENT
MEDIUM⚡	tests/agent/test_agent_react.py	605	# Create an assistant message without reasoning content but with submit tool call	COMMENT
MEDIUM⚡	tests/agent/test_agent_react.py	611	# Create an assistant message with only regular tool calls (should be unchanged)	COMMENT
MEDIUM	tests/agent/test_acp/test_action_methods.py	915	# Create the pending nested ToolEvent — same construction the	COMMENT
MEDIUM⚡	tests/model/test_web_search_exclude_none.py	84	# Create a web search with ActionSearch that has sources=None (the bug condition)	COMMENT
MEDIUM	tests/model/test_compaction_summary.py	69	# Create a previous summary message	COMMENT
MEDIUM	tests/model/test_trim_messages.py	72	# Create a longer conversation	COMMENT
MEDIUM	tests/model/test_trim_messages.py	140	# Create a scenario with a tool response without a corresponding assistant message	COMMENT
MEDIUM	tests/model/test_trim_messages.py	241	# Create a longer conversation	COMMENT
MEDIUM	tests/model/test_trim_messages.py	305	# Create a longer conversation with tool calls	COMMENT
MEDIUM	tests/model/test_trim_messages.py	349	# Create a conversation with alternating user-assistant pairs	COMMENT
MEDIUM	tests/model/test_trim_messages.py	432	# Create an assistant message with 3 tool calls, but only 1 tool result	COMMENT
MEDIUM	tests/model/test_trim_messages.py	465	# Create an assistant message with tool calls, but no tool results at all	COMMENT
MEDIUM	tests/model/test_trim_messages.py	492	# Create a conversation where trimming will create orphan tool_calls	COMMENT
MEDIUM	tests/model/test_token_counting.py	120	# Create a conversation with an assistant message containing multiple tool calls	COMMENT
MEDIUM	tests/model/test_prompt_logprobs.py	463	# Create a minimal instance without calling __init__	COMMENT
MEDIUM	tests/model/providers/test_anthropic.py	500	# Create an assistant message with a single tool call (no tool result)	COMMENT
MEDIUM	tests/model/providers/test_anthropic.py	522	# Create an assistant message with multiple tool calls (no tool results)	COMMENT
MEDIUM	tests/model/providers/test_anthropic.py	554	# Create a tool result message without a preceding assistant message	COMMENT
MEDIUM	tests/model/providers/test_mistral.py	101	# Create a simple prompt	COMMENT
MEDIUM	tests/model/providers/test_transformer_lens.py	18	# Create a small HookedTransformer model for testing	COMMENT
MEDIUM	tests/model/providers/test_transformer_lens.py	34	# Create the model using Inspect	COMMENT
MEDIUM	tests/model/providers/test_transformer_lens.py	143	# Create a small HookedTransformer model for testing	COMMENT
MEDIUM	tests/model/providers/test_transformer_lens.py	159	# Create a simple task for testing	COMMENT
MEDIUM⚡	tests/model/providers/test_google.py	432	# Create a mock client	COMMENT
MEDIUM⚡	tests/model/providers/test_google.py	435	# Create a message with reasoning + text + tool call	COMMENT
MEDIUM	tests/model/providers/test_google.py	896	# Create an assistant message with a single tool call (no tool result)	COMMENT
MEDIUM	tests/model/providers/test_google.py	921	# Create an assistant message with multiple tool calls (no tool results)	COMMENT
MEDIUM	tests/model/providers/test_google.py	956	# Create a tool result message without a preceding assistant message	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	288	# Create a mock Response with an invalid_prompt error	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	517	# Create a mock CompactedResponse with a ResponseCompactionItem	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	570	# Create a CompactedResponse without a ResponseCompactionItem	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	694	# Create a CompactedResponse with compaction item followed by an output message	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	766	# Create a compact response matching real API output:	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	952	# Create a message with phase stored in ContentText.internal	COMMENT
MEDIUM	tests/model/providers/util/test_batch_intake.py	208	# Create a request that's too large for the available size	COMMENT
MEDIUM	tests/model/providers/util/test_batch.py	593	# Create a batcher with a low max_consecutive_check_failures value	COMMENT
MEDIUM	tests/model/providers/util/test_batch.py	664	# Create a batcher that will fail initially then succeed	COMMENT
MEDIUM	tests/model/providers/util/test_batch.py	723	# Create a request that's close to but under the limit	COMMENT
MEDIUM⚡	tests/log/test_log_file_info_fallback.py	204	# Create a native-named eval (timestamp prefix -> fast path)	COMMENT
MEDIUM⚡	tests/log/test_log_file_info_fallback.py	211	# Create a custom-named eval (no timestamp -> header fallback)	COMMENT
MEDIUM⚡	tests/log/test_log_attachments.py	52	# # Create a long text that should be condensed (> 100 chars)	COMMENT
MEDIUM⚡	tests/log/test_log_attachments.py	56	# # Create a model event with long content	COMMENT
MEDIUM	tests/log/test_log_attachments.py	128	# # Create the same long text that will appear in multiple events	COMMENT
MEDIUM	tests/log/test_log_attachments.py	209	# Create an event with long content	COMMENT
MEDIUM	tests/log/test_recover_e2e.py	283	# Create an empty buffer DB so recovery can proceed	COMMENT
46 more matches not shown…

Modern AI Meta-Vocabulary97 hits · 294 pts

Severity	File	Line	Snippet	Context
MEDIUM	CHANGELOG.md	2279	- ReAct Agent: [truncation](https://inspect.aisi.org.uk/agents.html#truncation) option to trim conversation messages whe	CODE
MEDIUM	design/plans/sandbox-tools-onedir.md	83	### 2. Build orchestration — `Dockerfile.pyinstaller`, `build_executable.py`	COMMENT
MEDIUM	design/ctl/pause-resume.md	137	- Hard pause (interrupt-and-requeue) — planned. `pause --now`: interrupt in-flight samples as `cancelled` and requeu	CODE
MEDIUM	design/ctl/control-channel.md	1026	- Self-targeting guard hardening. Open question #8 in this doc — an LLM agent running inside an eval shouldn't be	CODE
MEDIUM	tests/checkpoint/test_checkpointer.py	611	# leading scaffolding before the first wrap — dropped by the slice	COMMENT
MEDIUM	tests/checkpoint/test_checkpointer.py	631	# seeded wrap and still open at fire time — trailing scaffolding	COMMENT
MEDIUM⚡	tests/agent/test_agent_compaction_native.py	49	# the context window. Reasoning + compaction is tested separately in	COMMENT
MEDIUM⚡	tests/agent/test_bridge_track_state.py	819	# the scaffold restarts: title call fires first again	COMMENT
MEDIUM	tests/agent/test_bridge_track_state.py	259	# main loop, then scaffold compaction: the compacted loop is promoted, so	COMMENT
MEDIUM	tests/agent/test_bridge_track_state.py	390	# the scaffold re-creates every message: different ids and metadata, and	COMMENT
MEDIUM	tests/agent/test_bridge_track_state.py	427	# scaffold runs the task under its own (different) system prompt	COMMENT
MEDIUM	tests/agent/test_bridge_track_state.py	604	# title call fires first; the scaffold's store holds the condensed turn	COMMENT
MEDIUM	tests/agent/test_bridge_track_state.py	893	# main loop: turn 2 (scaffold round-trips through its own store)	COMMENT
MEDIUM⚡	tests/agent/test_agent_bridge.py	944	# both scaffold calls completed (the second replays the first turn)	STRING
MEDIUM	tests/agent/test_agent_bridge.py	933	# backend; the scaffold's second call replays the searched assistant turn	STRING
MEDIUM⚡	tests/agent/test_bridge_approval.py	302	# the scaffold gets the approved arguments...	COMMENT
MEDIUM⚡	tests/agent/test_bridge_approval.py	646	# the rejected call never reached the scaffold	COMMENT
MEDIUM⚡	tests/agent/test_bridge_approval.py	780	# should behave identically whichever API the scaffold speaks. These drive each	COMMENT
MEDIUM⚡	tests/agent/test_bridge_approval.py	781	# dialect impl the way a scaffold's request arrives.	COMMENT
MEDIUM	tests/agent/test_bridge_approval.py	605	# what the scaffold would go on to execute	COMMENT
MEDIUM⚡	tests/agent/test_acp/test_operator_provenance.py	55	# note_operator_message — the scaffold's signal	COMMENT
MEDIUM	tests/agent/test_acp/test_router_bridge_tools.py	150	# the scaffold-provided view renders richly (not args-only)	COMMENT
MEDIUM⚡	tests/agent/test_acp/test_tui/test_widgets.py	915	# Used-only — context window denominator dropped.	COMMENT
MEDIUM	tests/model/test_compaction.py	1495	# small context window so a modest tool output overflows it	COMMENT
MEDIUM	tests/model/test_model_length.py	24	# model context window sizes	COMMENT
MEDIUM	tests/model/test_model_length.py	49	# cloudflare serves this model with a reduced context window	COMMENT
MEDIUM	tests/model/test_model_length.py	61	# context window for model	COMMENT
MEDIUM	tests/model/test_model_length.py	65	# create message that exceeds context window	COMMENT
MEDIUM	tests/model/providers/test_anthropic.py	337	# scaffold for should_retry	COMMENT
MEDIUM⚡	tests/model/providers/test_vllm.py	27	# this allows us to run base models with the chat message scaffolding:	COMMENT
MEDIUM⚡	tests/model/providers/test_openai_model_names.py	82	# input_tokens_name() aliases to the current frontier so the context window	COMMENT
MEDIUM⚡	tests/model/providers/test_grok_model_names.py	82	# input_tokens_name() aliases to the current frontier so the context window	COMMENT
MEDIUM⚡	tests/model/providers/test_grok_model_names.py	94	# frontier would overstate its context window	COMMENT
MEDIUM	tests/model/providers/test_anthropic_fallback.py	517	# scaffold echoes the assistant turn as dicts (incl. the fallback block)	COMMENT
MEDIUM	tests/model/providers/test_hf.py	30	# this allows us to run base models with the chat message scaffolding:	COMMENT
MEDIUM	tests/model/providers/test_hf.py	49	# this allows us to run base models with the chat message scaffolding:	COMMENT
MEDIUM⚡	tests/model/providers/test_google_model_names.py	86	# input_tokens_name() aliases to the current frontier so the context window	COMMENT
MEDIUM	tests/model/providers/test_openai_responses.py	1977	# Bridge -> scaffold (assistant text) -> scaffold echoes it back as input.	COMMENT
MEDIUM	docs/CHANGELOG.md	2279	- ReAct Agent: [truncation](https://inspect.aisi.org.uk/agents.html#truncation) option to trim conversation messages whe	CODE
MEDIUM	docs/evals/evals.json	3797	"paper": "https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/",	CODE
MEDIUM	docs/evals/evals.json	5546	"paper": "https://www.anthropic.com/research/agentic-misalignment",	CODE
MEDIUM	src/inspect_ai/util/_checkpoint/hydrate.py	671	# Slice away the live session's scaffolding ahead of the first	COMMENT
MEDIUM	src/inspect_ai/util/_checkpoint/hydrate.py	706	# resumed session's own structural scaffolding (`init` / `solvers` /	COMMENT
MEDIUM	src/inspect_ai/agent/_react.py	247	# check for context window overflow	COMMENT
MEDIUM	src/inspect_ai/agent/_react.py	470	# check for context window overflow	COMMENT
MEDIUM	src/inspect_ai/agent/_react.py	724	# exceed the context window; _handle_overflow's forced	COMMENT
MEDIUM⚡	src/inspect_ai/agent/_acp/inspect_ext.py	132	# (claude_code, codex, …): their tools are run by the bridged scaffold, not by	STRING
MEDIUM⚡	src/inspect_ai/agent/_acp/connection.py	1165	# Bind / unbind orchestration	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/event_mapping.py	723	# and a known context window. ACP semantics: "Tokens currently in	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/event_mapping.py	724	# context" / "Total context window size". We do NOT gate this on	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/tui/session_screen.py	1042	# mid-turn — during scaffold startup / ``--resume`` relaunch or	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/header.py	235	# Used-only — the context window denominator was visual noise	STRING
MEDIUM⚡	src/inspect_ai/agent/_bridge/util.py	39	# Generation-tuning fields a scaffold may set on a bridged request that describe	COMMENT
MEDIUM⚡	src/inspect_ai/agent/_bridge/util.py	41	# (the scaffold computes them for its assumed --model, not the model actually	COMMENT
MEDIUM⚡	src/inspect_ai/agent/_bridge/util.py	43	# config / provider defaults govern generation. Structural fields the scaffold	COMMENT
MEDIUM	src/inspect_ai/agent/_bridge/util.py	242	# restore operator provenance lost to a bridged scaffold's round-trip (e.g.	COMMENT
MEDIUM	src/inspect_ai/agent/_bridge/util.py	323	# Approve the tool calls the scaffold is about to run. A rejection comes back	COMMENT
MEDIUM	src/inspect_ai/agent/_bridge/util.py	325	# every call in the response) so it can propose something else; the scaffold	COMMENT
MEDIUM	src/inspect_ai/agent/_bridge/types.py	59	# the scaffold a restored, mid-turn (assistant-terminated) conversation,	COMMENT
MEDIUM	src/inspect_ai/agent/_bridge/types.py	325	# verdicts keep the previous-call comparison — a scaffold that	COMMENT
37 more matches not shown…

Cross-File Repetition40 hits · 200 pts

Severity	File	Snippet	Context
HIGH	tests/tools/test_tools.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/agent/test_agent_react.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/solver/test_basic_agent.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/model/test_reasoning_claude.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/test_helpers/tools.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/approval/test_approval.py	add two numbers. args: x (int): first number to add. y (int): second number to add. returns: the sum of the two numbers.	STRING
HIGH	tests/agent/test_acp/test_server_forwarding.py	short /tmp data dir so af_unix paths fit in 104 chars on macos.	STRING
HIGH	tests/agent/test_acp/test_action_methods.py	short /tmp data dir so af_unix paths fit in 104 chars on macos.	STRING
HIGH	tests/agent/test_acp/test_elicitation_e2e.py	short /tmp data dir so af_unix paths fit in 104 chars on macos.	STRING
HIGH	tests/agent/test_acp/test_stdio_bridge.py	short /tmp data dir so af_unix paths fit in 104 chars on macos.	STRING
HIGH	tests/agent/test_acp/test_approval.py	short /tmp data dir so af_unix paths fit in 104 chars on macos.	STRING
HIGH	tests/_cli/test_token_limit_flag.py	run the test command and eval its echoed repr back to a python value.	STRING
HIGH	tests/_cli/test_ctl_server_flag.py	run the test command and eval its echoed repr back to a python value.	STRING
HIGH	tests/_cli/test_acp_server_flag.py	run the test command and eval its echoed repr back to a python value.	STRING
HIGH	tests/model/providers/test_google.py	add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.	STRING
HIGH	docs/_tools-basics.md	add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.	STRING
HIGH	examples/tool_use.py	add two numbers. args: x: first number to add. y: second number to add. returns: the sum of the two numbers.	STRING
HIGH	…ort/src/inspect_tool_support/_util/json_rpc_helpers.py	check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.go	STRING
HIGH	…ls/src/inspect_sandbox_tools/_util/json_rpc_helpers.py	check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.go	STRING
HIGH	src/inspect_ai/_util/httpx.py	check whether an exception raised from httpx should be retried. implements the strategy described here: https://cloud.go	STRING
HIGH	src/inspect_ai/util/_store_model.py	pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: opti	STRING
HIGH	src/inspect_ai/solver/_task_state.py	pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: opti	STRING
HIGH	src/inspect_ai/log/_log.py	pydantic model interface to the store. args: model_cls: pydantic model type (must derive from storemodel) instance: opti	STRING
HIGH	src/inspect_ai/model/_model.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/mistral.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/grok.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/google.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/openai.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/anthropic.py	canonical model name for model info database lookup.	STRING
HIGH	src/inspect_ai/model/_providers/mistral.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/grok.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/openai_compatible.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/google.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/groq.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/openai.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/anthropic.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/together.py	scope adaptive concurrency per (key, model). a pool shared across models lets the faster model's signals push the adapti	STRING
HIGH	src/inspect_ai/model/_providers/grok.py	model name used for looking up model input tokens (context window).	STRING
HIGH	src/inspect_ai/model/_providers/google.py	model name used for looking up model input tokens (context window).	STRING
HIGH	src/inspect_ai/model/_providers/openai.py	model name used for looking up model input tokens (context window).	STRING

Modern Structural Boilerplate206 hits · 197 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/input/test_input_panel.py	129	def _set_input_value(input_widget: Input, value: str) -> None:	CODE
LOW	tests/input/test_input_panel.py	53	def set_title(self, title: str) -> None:	CODE
LOW⚡	tests/agent/test_acp/test_tui/test_elicitation_card.py	68	def _set_input(input_widget: Input, value: str) -> None:	CODE
LOW	tests/test_package/inspect_package/__init__.py	3	__all__ = ["simple_score"]	CODE
LOW	…ridianlabs-ai/inspect-docs/filters/reference/filter.py	230	def set_default_title(elem: pf.Element, doc: pf.Doc) -> None:	CODE
LOW	examples/hooks/mlflow_tracking.py	44	logger = logging.getLogger(__name__)	CODE
LOW	examples/hooks/trackio_tracking.py	41	logger = logging.getLogger(__name__)	CODE
LOW	examples/hooks/mlflow_tracing.py	51	logger = logging.getLogger(__name__)	CODE
LOW	…pect_tool_support/src/inspect_tool_support/__init__.py	12	__all__ = ["__version__"]	CODE
LOW	…sandbox_tools/tests/test_text_editor_validated_path.py	14	def _set_history_path(monkeypatch: pytest.MonkeyPatch, history_path: Path) -> None:	CODE
LOW	src/inspect_sandbox_tools/tests/conftest.py	18	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_sandbox_tools/tests/test_exec_remote.py	90	def _setup_and_teardown(sandbox_server_cleanup: None) -> None:	CODE
LOW	…ct_sandbox_tools/src/inspect_sandbox_tools/__init__.py	12	__all__ = ["__version__"]	CODE
LOW	…ox_tools/_in_process_tools/_text_editor/text_editor.py	21	logger = logging.getLogger(__name__)	CODE
LOW	…x_tools/src/inspect_sandbox_tools/_util/user_switch.py	14	def set_oom_score_adj() -> None:	CODE
LOW	src/inspect_ai/__init__.py	32	__all__ = [	CODE
LOW	src/inspect_ai/_control/eval_state.py	626	def set_sample_requeue(eval_id: str, handle: "SampleRequeue \| None") -> None:	CODE
LOW	src/inspect_ai/_util/zipfile.py	25	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/zipfile.py	183	__all__ = ["zipfile_compress_kwargs"]	CODE
LOW	src/inspect_ai/_util/log_context.py	8	def set_run_shape(task_names: Iterable[str], max_epochs: int) -> None:	CODE
LOW	src/inspect_ai/_util/error.py	116	def set_exception_hook() -> None:	CODE
LOW	src/inspect_ai/_util/transcript.py	69	def set_transcript_markdown_options(markdown: Markdown) -> None:	CODE
LOW	src/inspect_ai/_util/registry.py	83	def set_annotations(wrapper: Callable[..., Any], annotations: dict[str, Any]) -> None:	CODE
LOW	src/inspect_ai/_util/registry.py	114	def set_return_annotation(wrapper: Callable[..., Any], return_type: type[Any]) -> None:	CODE
LOW	src/inspect_ai/_util/httpx.py	15	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/background.py	50	def set_background_task_group(tg: TaskGroup \| None) -> None:	CODE
LOW	src/inspect_ai/_util/local_server.py	14	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/file.py	632	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/event_loop_monitor.py	38	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/asyncfiles.py	50	logger = logging.getLogger(__name__)	CODE
LOW	src/inspect_ai/_util/future.py	14	def set_result(self, result: T) -> None:	CODE
LOW	src/inspect_ai/_util/future.py	18	def set_exception(self, ex: Exception) -> None:	CODE
LOW	src/inspect_ai/_util/future.py	32	def set_future_result(future: "Future[T]", result: T) -> None:	CODE
LOW	src/inspect_ai/_util/future.py	36	def set_future_exception(future: "Future[T]", error: Exception) -> None:	CODE
LOW	src/inspect_ai/_util/strenum.py	49	__all__ = ["StrEnum"]	CODE
LOW	src/inspect_ai/_util/textual/__init__.py	3	__all__ = ["ElicitationForm"]	CODE
LOW	src/inspect_ai/_display/__init__.py	14	__all__ = [	CODE
LOW	src/inspect_ai/_display/core/results.py	37	def set_retry_args_suffix(suffix: str) -> None:	CODE
LOW	src/inspect_ai/_display/core/display.py	156	def update_metrics(self, scores: list[TaskDisplayMetric]) -> None: ...	CODE
LOW	src/inspect_ai/_display/core/display.py	184	def update_task_count(self, n: int) -> None:	CODE
LOW	src/inspect_ai/_display/core/progress.py	70	def update_count(self, complete: int, total: int) -> None:	CODE
LOW	src/inspect_ai/_display/core/progress.py	77	def update_score(self, score: str) -> None:	CODE
LOW	src/inspect_ai/_display/core/panel.py	23	def set_eval_set_id_display(value: str \| None) -> None:	CODE
LOW	src/inspect_ai/_display/plain/display.py	75	def update_task_count(self, n: int) -> None:	CODE
LOW	src/inspect_ai/_display/plain/display.py	222	def update_metrics(self, metrics: list[TaskDisplayMetric]) -> None:	CODE
LOW	src/inspect_ai/_display/textual/display.py	85	def update_task_count(self, n: int) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	59	def set_flow_content(content: RenderableType) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	244	def update_task_count(self, n: int) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	300	def update_display(self) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	315	def update_scan(self) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	340	def update_title(self) -> None:	CODE
LOW⚡	src/inspect_ai/_display/textual/app.py	365	def update_tasks(self) -> None:	CODE
LOW⚡	src/inspect_ai/_display/textual/app.py	374	def update_samples(self) -> None:	CODE
LOW⚡	src/inspect_ai/_display/textual/app.py	381	def update_footer(self) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	394	def set_unread(unread: int \| None) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	408	async def set_active_tab(active: str) -> None:	CODE
LOW	src/inspect_ai/_display/textual/app.py	469	def set_title(self, title: str) -> None:	CODE
LOW⚡	src/inspect_ai/_display/textual/widgets/tasks.py	175	def set_display_metrics(self, display_metrics: bool) -> None:	STRING
LOW⚡	src/inspect_ai/_display/textual/widgets/tasks.py	178	def update_count_width(self, samples: int) -> None:	STRING
LOW⚡	src/inspect_ai/_display/textual/widgets/tasks.py	184	def update_progress_widths(self) -> None:	STRING
146 more matches not shown…

Over-Commented Block231 hits · 189 pts

Severity	File	Line	Snippet	Context
LOW	tests/conftest.py	21	# Automatically mark every async test function with @pytest.mark.anyio so	COMMENT
LOW	tests/checkpoint/resume_kill_harness.py	141		COMMENT
LOW	tests/util/test_text.py	21	# Stripping it to face value would make match(numeric=True)	COMMENT
LOW	tests/agent/test_agent_bridge.py	881	assert log.error	COMMENT
LOW	tests/agent/test_acp/test_session.py	221		COMMENT
LOW	tests/agent/test_acp/test_standard_acp_client.py	241	):	COMMENT
LOW	tests/agent/test_acp/test_approval.py	561	)	COMMENT
LOW	tests/agent/test_acp/test_approval.py	1201	for block in req.tool_call.content:	COMMENT
LOW	tests/agent/test_acp/test_tui/test_client.py	61	active.agent_name = agent_name	COMMENT
LOW	tests/model/test_reasoning_claude.py	21	@pytest.mark.anyio	COMMENT
LOW	tests/model/test_adaptive_connections.py	901	# should be the AdaptiveConcurrency() default `start=20` clamped to max=50.	COMMENT
LOW	tests/view/test_openapi_schema_generation.py	241	"""Without our override, str = 'foo' would NOT be required."""	COMMENT
LOW	tests/log/test_log_attachments.py	41	"data:image/png;base64"	COMMENT
LOW	tests/log/test_log_attachments.py	61	# tool_choice="auto",	COMMENT
LOW	tests/log/test_log_attachments.py	81		COMMENT
LOW	tests/log/test_log_attachments.py	101	# # Simulate what happens in _record_model_interaction's complete() callback:	COMMENT
LOW	tests/log/test_log_attachments.py	121	# assert transcript.attachments[attachment_hash] == long_response	COMMENT
LOW	tests/log/test_log_attachments.py	141	# )	COMMENT
LOW	tests/log/test_log_attachments.py	161	# assert isinstance(stored_event2, ModelEvent)	COMMENT
LOW	examples/checkpoint_ctf.py	181	# ---------------------------------------------------------------------------	COMMENT
LOW	.github/workflows/changelog-lint.yml	1	# Lints CHANGELOG.md placement on PRs: new entries must land under the	COMMENT
LOW	.github/workflows/build_sandbox_tools.yml	1	# This workflow builds container tool injectable executables (e.g. inspect-sandbox-tools-amd64-v666)	COMMENT
LOW	.github/workflows/build_sandbox_tools.yml	21	# pull_request:	COMMENT
LOW	.github/workflows/build_sandbox_tools.yml	41	# 📋 Step-by-Step Instructions	COMMENT
LOW	.github/workflows/build.yml	181		COMMENT
LOW	…ool_support/_remote_tools/_web_browser/scale_factor.py	1	import subprocess	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	41	# AXNode,	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	61	# def test_getitem(self):	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	81	# }	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	101	# }	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	121		COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	141	# node_data = {	COMMENT
LOW	…ote_tools/_web_browser/test_accessibility_tree_node.py	161	# }	COMMENT
LOW	…t_sandbox_tools/_remote_tools/_exec_remote/__init__.py	1	# TODO: Consider renaming _remote_tools directory to _remote_services or _json_rpc_services	COMMENT
LOW	…sandbox_tools/_remote_tools/_mcp/mcp_server_session.py	21	# line longer than the limit makes readline() raise ValueError ("Separator is not	COMMENT
LOW	…sandbox_tools/_remote_tools/_mcp/mcp_server_session.py	221	message = JSONRPCMessage.model_validate_json(line)	COMMENT
LOW	…sandbox_tools/src/inspect_sandbox_tools/_cli/server.py	21	# 1. LD_LIBRARY_PATH — restore it to its pre-bootloader value, otherwise the	COMMENT
LOW	src/inspect_ai/_control/server.py	161	raise PrerequisiteError(	COMMENT
LOW	src/inspect_ai/_control/server.py	501	@app.get("/evals/{eval_id}/samples")	COMMENT
LOW	src/inspect_ai/_control/server.py	721	status_code=404,	COMMENT
LOW	src/inspect_ai/_control/server.py	1021	# Pause / resume the whole run (the eval-set spelling — one	COMMENT
LOW	src/inspect_ai/_control/server.py	1081		COMMENT
LOW	src/inspect_ai/_control/cancel.py	161	}	COMMENT
LOW	src/inspect_ai/_control/events.py	201	# Slicing the raw sequence under-delivered badly with the default	COMMENT
LOW	src/inspect_ai/_control/__init__.py	1	"""Control channel — HTTP server embedded in each running eval process.	COMMENT
LOW	src/inspect_ai/_control/__init__.py	21	# it rejects unknown query params on any non-GET route with a 400	COMMENT
LOW	src/inspect_ai/_control/__init__.py	41	# 1 — max_subprocesses knob.	COMMENT
LOW	src/inspect_ai/_control/pause.py	141	# same run boundary) — so a pause survives an in-run task retry, matching how	COMMENT
LOW	src/inspect_ai/_control/pause.py	161	# (run_task_retry_attempts registers them via note_dispatch_models),	COMMENT
LOW	src/inspect_ai/_control/state.py	41	from inspect_ai._util.error import is_cancellation_message	COMMENT
LOW	src/inspect_ai/_control/state.py	181	samples_by_eval: dict[str, list[ActiveSample]] = defaultdict(list)	COMMENT
LOW	src/inspect_ai/_control/state.py	341	# supersede a still-running one (a sample that has since finished).	COMMENT
LOW	src/inspect_ai/_control/state.py	1121	# attempts* (completed_at set, retry pending) is still holdable (the	COMMENT
LOW	src/inspect_ai/_util/asyncfiles.py	1021	# fsspec write buffer size for cloud storage backends (GCS, Azure, etc.).	COMMENT
LOW	src/inspect_ai/_util/atomic_write.py	161		COMMENT
LOW	src/inspect_ai/_display/textual/widgets/transcript.py	281	# treatment — the adjacent InterruptEvent already says what	COMMENT
LOW	src/inspect_ai/_display/textual/widgets/samples.py	361	# has a bound, live agent loop (``is_interactive``) and has to be	COMMENT
LOW	src/inspect_ai/_display/textual/widgets/samples.py	1061	# Resolve the pending state from the transcript's sidecar.	COMMENT
LOW	src/inspect_ai/analysis/_dataframe/util.py	201	arrow_type = pa.string()	COMMENT
LOW	src/inspect_ai/analysis/_dataframe/util.py	221	# metadata_prompt string[pyarrow]	COMMENT
171 more matches not shown…

AI Slop Vocabulary40 hits · 96 pts

Severity	File	Line	Snippet	Context
MEDIUM⚡	tests/checkpoint/test_current_checkpointer_post_exit.py	224	# harness faithfully reproduces the normal span/checkpoint pairing — and that	COMMENT
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	234	"content": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tack	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	257	"content": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tack	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	1545	"result": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tackl	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	1577	"output": "{\"jsonrpc\": \"2.0\", \"result\": {\"web_url\": \"https://www.aisi.gov.uk/work\", \"main_content\"	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	2102	"result": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news/tackl	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	2134	"output": "{\"jsonrpc\": \"2.0\", \"result\": {\"web_url\": \"https://www.aisi.gov.uk/work\", \"main_content\"	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	2732	"replaced": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.uk/government/news	CODE
MEDIUM	…lysis/test_logs/2025-05-12T20-27-36-04-00_browser.json	3577	"cb6d9b3382c23a2729d389c44be2fb26": "[793] link \"We are now the AI Security Institute\" [url: https://www.gov.	CODE
MEDIUM	tests/agent/deepagent/test_deepagent_background.py	970	# deterministically without a scheduling barrier the harness lacks.)	COMMENT
MEDIUM	tests/agent/test_acp/test_deepagent_integration.py	479	# ACP harness. The cancel pathway itself is tested in	COMMENT
MEDIUM	tests/agent/test_acp/test_server_dispatch.py	102	# mirrors what the eval harness produces from the default	COMMENT
MEDIUM	tests/agent/test_acp/test_action_methods.py	899	# Set up the cancel-propagation harness in a real task group.	COMMENT
MEDIUM	tests/agent/test_acp/test_approval.py	1198	# Scan all blocks for the view markdown — robust to layout	COMMENT
MEDIUM	tests/agent/test_acp/test_tui/test_cancel_card.py	242	# Don't call original_notify — the harness can't render toasts.	COMMENT
MEDIUM⚡	tests/agent/test_acp/test_tui/test_widgets.py	63	# Single-widget harness	COMMENT
MEDIUM	tests/agent/test_acp/test_tui/test_widgets.py	1099	# comprehensive fingerprint should detect the change and the	COMMENT
MEDIUM	examples/skills/skills/system-info/scripts/sysinfo.sh	3	# Outputs comprehensive system details in a structured format	COMMENT
MEDIUM	.github/workflows/test.yml	1	# Test harness shell. GitHub only exposes workflow_dispatch for workflows that	COMMENT
LOW	src/inspect_ai/_util/notebook.py	53	# notebook cells, if not just return an empty module	COMMENT
LOW	src/inspect_ai/analysis/_dataframe/record.py	389	# For now, I'll just return an empty dict	COMMENT
LOW	src/inspect_ai/util/_limit.py	220	# gone out of scope for the sample so we just return that snapshot	COMMENT
MEDIUM	src/inspect_ai/util/_checkpoint/checkpointer_impl.py	591	# checkpoint span. Skip on the harness-driven final fire —	COMMENT
MEDIUM	src/inspect_ai/util/_sandbox/service.py	489	# under an eval harness -- a sandbox-awareness leak that was	COMMENT
MEDIUM	src/inspect_ai/agent/_react.py	209	# to clean exit and the harness will run scoring next. State	COMMENT
MEDIUM	src/inspect_ai/agent/_react.py	436	# to clean exit and the harness will run scoring next. State	COMMENT
LOW	src/inspect_ai/agent/_acp/tui/session_screen.py	184	# …); duplicating them in the footer would just add noise.	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/cancel_card.py	232	# ``fails_on_error`` is True, but stay robust if a future	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/cancel_card.py	267	# but stay robust if a future refactor changes the	COMMENT
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/transcript.py	158	# Gate on fingerprint change — the comprehensive	COMMENT
LOW	src/inspect_ai/_view/common.py	552	# then just return that file	COMMENT
MEDIUM	src/inspect_ai/_view/common.py	714	# Attempt existence check with robust handling for Azure-style auth issues.	COMMENT
MEDIUM	src/inspect_ai/model/_call_tools.py	763	# harness injects the real AgentState below)	COMMENT
LOW	src/inspect_ai/model/_call_tools.py	986	# if the function takes **kwargs: Any then just pass the tool arguments through	COMMENT
LOW	src/inspect_ai/model/_providers/openai.py	328	# side step that complexity and just use two different batchers.	COMMENT
LOW	src/inspect_ai/log/_recorders/buffer/database.py	579	# this will just return no events and no attachments until the	STRING
LOW	src/inspect_ai/log/_recorders/buffer/filestore.py	470	# we'll just return None	COMMENT
LOW	src/inspect_ai/_eval/loader.py	639	# if there is no solver file then just create from the registry by name	COMMENT
MEDIUM	src/inspect_ai/tool/_sandbox_tools_utils/sandbox.py	288	# TODO: once we get the github CI/CD actions robust, this should be fatal	COMMENT
LOW	src/inspect_ai/tool/_mcp/_local.py	301	# to the session, we can just return nit	COMMENT

Redundant / Tautological Comments56 hits · 78 pts

Severity	File	Line	Snippet	Context
LOW	tests/model/test_message_ids.py	177	# Output message should have an ID	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	117	# Check if the model used server-side tools	COMMENT
LOW	tests/model/test_stable_message_ids.py	157	# Output message should have ID	COMMENT
LOW	tests/model/providers/test_perplexity.py	74	# Check if citations are present	COMMENT
LOW	tests/model/providers/util/test_batch.py	99	# Check if batch should fail	COMMENT
LOW	tests/model/providers/util/test_batch.py	105	# Check if batch is "complete" based on elapsed time	COMMENT
LOW	tests/log/test_eval_log.py	181	# Set timestamp to a timezone-aware datetime object because when serializing to	COMMENT
LOW	examples/structured.py	170	# Check if the output matches the expected RGB format	STRING
LOW	scripts/pypi-release.py	222	# Check if there are any other versions present	COMMENT
LOW	scripts/pypi-release.py	282	# Check if .pypirc exists or environment variables are set	COMMENT
LOW	scripts/pypi-release.py	406	# Check if we're ahead of remote	COMMENT
LOW	scripts/pypi-release.py	449	# Check if we're in a git repository	COMMENT
LOW	scripts/pypi-release.py	496	# Check if branch is up to date	COMMENT
LOW	scripts/pypi-release.py	507	# Check if tag already exists	COMMENT
LOW	scripts/pypi-release.py	617	# Check if correct version already exists	COMMENT
LOW	…t_tool_support/src/inspect_tool_support/_cli/server.py	27	# Set umask to handle dynamic user switching scenarios:	COMMENT
LOW	…ox_tools/_in_process_tools/_text_editor/text_editor.py	89	# Read the file content	COMMENT
LOW	…ox_tools/_in_process_tools/_text_editor/text_editor.py	94	# Check if old_str is unique in the file	COMMENT
LOW	…ox_tools/_in_process_tools/_text_editor/text_editor.py	195	# Check if path exists	COMMENT
LOW	…ox_tools/_in_process_tools/_text_editor/text_editor.py	204	# Check if the path points to a directory	COMMENT
LOW	…ols/src/inspect_sandbox_tools/_util/pseudo_terminal.py	64	# Close the file descriptors	COMMENT
LOW	…spect_sandbox_tools/_remote_tools/_exec_remote/_job.py	137	# Check if process has finished	COMMENT
LOW	src/inspect_ai/_util/platform.py	53	# Check if running in VS Code Jupyter notebook or interactive window	COMMENT
LOW	src/inspect_ai/_util/platform.py	61	# Check if running in a VS Code terminal	COMMENT
LOW	src/inspect_ai/_util/local_server.py	162	os.kill(pid, 0) # Check if process exists	CODE
LOW	src/inspect_ai/_util/local_server.py	254	# Check if the process is still alive	COMMENT
LOW	src/inspect_ai/_util/file.py	429	# Check if it's already a URI	COMMENT
LOW	src/inspect_ai/_util/file.py	539	# Open the file using fsspec and retrieve the file's information	COMMENT
LOW	src/inspect_ai/analysis/_dataframe/samples/table.py	220	# Check if it's specifically the pyarrow offset overflow error	COMMENT
LOW	src/inspect_ai/analysis/_dataframe/samples/table.py	615	# Check if this is a PyArrow-backed column	COMMENT
LOW	src/inspect_ai/util/_json.py	433	# Check if it's already Optional/Union with None	COMMENT
LOW	src/inspect_ai/util/_sandbox/self_check.py	431	# Write the file byte-by-byte from ASCII-only octal escapes, then cat it, so	COMMENT
LOW	src/inspect_ai/scorer/_math.py	934	# Check if the result contains unexpected free symbols (variables)	STRING
LOW	src/inspect_ai/_cli/eval.py	2298	# Check if it's a file path	COMMENT
LOW	src/inspect_ai/_cli/eval.py	2318	# Check if it looks like a file path that doesn't exist	COMMENT
LOW	src/inspect_ai/_cli/score.py	289	# Print the results panel	COMMENT
LOW	src/inspect_ai/model/_openai_responses.py	310	# Check if this is a compaction marker message	COMMENT
LOW	src/inspect_ai/model/_openai_responses.py	1292	# Check if this is a valid OpenAI action (correct type + required fields)	COMMENT
LOW	src/inspect_ai/model/_openai_responses.py	1455	# Check if result was cleared during compaction	COMMENT
LOW	src/inspect_ai/model/_model_info.py	153	# Check if the second-to-last part is a known service prefix	COMMENT
LOW	src/inspect_ai/model/_model_info.py	159	# Check if this is a hosting provider that needs org detection	COMMENT
LOW	src/inspect_ai/model/_providers/mockllm.py	51	# Check if it's a callable function	COMMENT
LOW	src/inspect_ai/model/_providers/google.py	1892	# Check if this block has an associated thought_signature and	COMMENT
LOW	src/inspect_ai/model/_providers/anthropic.py	739	# Check if model doesn't support compaction	COMMENT
LOW	src/inspect_ai/model/_providers/anthropic.py	4234	# Check if result was cleared during compaction	COMMENT
LOW	src/inspect_ai/model/_providers/anthropic.py	4445	# Check if previous message has corresponding tool_use blocks	COMMENT
LOW	src/inspect_ai/model/_providers/anthropic.py	4487	# Check if next message has corresponding tool_result blocks	COMMENT
LOW	src/inspect_ai/model/_providers/sglang.py	181	# Check if process is still alive	COMMENT
LOW	src/inspect_ai/log/_bundle.py	186	# Open the file for writing to save the updated content	STRING
LOW	src/inspect_ai/event/_timeline_repr.py	202	# Check if span overlaps with any span in the current cluster	COMMENT
LOW	…spect_ai/tool/_sandbox_tools_utils/validate_distros.py	109	# Check if it's for a supported architecture	COMMENT
LOW	src/inspect_ai/tool/_sandbox_tools_utils/sandbox.py	461	# Check if we're in a git repo	COMMENT
LOW	src/inspect_ai/tool/_tools/_memory.py	281	# Read file	COMMENT
LOW	src/inspect_ai/tool/_tools/_memory.py	349	# Read file content	COMMENT
LOW	src/inspect_ai/tool/_tools/_memory.py	354	# Check if old_str is unique	COMMENT
LOW	…_tools/_computer/_resources/entrypoint/xvfb_startup.sh	30	# Check if Xvfb is already running	COMMENT

Hallucination Indicators6 hits · 60 pts

Severity	File	Line	Snippet	Context
CRITICAL	tests/_view/test_view_server.py	969	buffer = inspect_ai.log._recorders.buffer.filestore.SampleBufferFilestore(	CODE
CRITICAL	tests/_view/test_view_server.py	1252	inspect_ai.log._recorders.buffer.filestore.SampleManifest(	CODE
CRITICAL	tests/_view/test_view_server.py	1263	inspect_ai.log._recorders.buffer.filestore.Segment(	CODE
CRITICAL	tests/_view/test_view_server.py	1270	manifest = inspect_ai.log._recorders.buffer.filestore.Manifest(	CODE
CRITICAL	tests/model/test_should_retry_classification.py	329	from azure.core.exceptions import HttpResponseError	CODE
CRITICAL	tests/model/test_should_retry_classification.py	344	from azure.core.exceptions import HttpResponseError	CODE

AI Structural Patterns60 hits · 58 pts

Severity	File	Line	Context
LOW	…/_remote_tools/_web_browser/accessibility_tree_node.py	224	CODE
LOW	…sandbox_tools/_remote_tools/_mcp/mcp_server_session.py	117	CODE
LOW	src/inspect_ai/_control/server.py	575	CODE
LOW	src/inspect_ai/_control/server.py	851	CODE
LOW	src/inspect_ai/_control/server.py	928	CODE
LOW	src/inspect_ai/_control/limits.py	119	CODE
LOW	src/inspect_ai/_control/limits.py	196	CODE
LOW	src/inspect_ai/_control/eval_state.py	370	CODE
LOW	src/inspect_ai/_control/eval_state.py	427	CODE
LOW	src/inspect_ai/util/_subprocess.py	48	CODE
LOW	src/inspect_ai/util/_subprocess.py	62	CODE
LOW	src/inspect_ai/util/_subprocess.py	75	CODE
LOW	src/inspect_ai/util/_subtask.py	155	CODE
LOW	src/inspect_ai/util/_checkpoint/hydrate.py	354	CODE
LOW	src/inspect_ai/util/_sandbox/local.py	79	CODE
LOW	src/inspect_ai/util/_sandbox/docker/compose.py	58	CODE
LOW	src/inspect_ai/util/_sandbox/docker/compose.py	281	CODE
LOW	src/inspect_ai/dataset/_dataset.py	32	CODE
LOW	src/inspect_ai/dataset/_sources/file.py	14	CODE
LOW	src/inspect_ai/dataset/_sources/csv.py	20	CODE
LOW	src/inspect_ai/dataset/_sources/json.py	22	CODE
LOW	src/inspect_ai/dataset/_sources/hf.py	117	CODE
LOW	src/inspect_ai/agent/_react.py	51	CODE
LOW	src/inspect_ai/agent/_bridge/bridge.py	100	CODE
LOW	src/inspect_ai/agent/_bridge/types.py	37	CODE
LOW	src/inspect_ai/agent/_bridge/sandbox/bridge.py	45	CODE
LOW	src/inspect_ai/agent/_deepagent/agent_tool.py	207	CODE
LOW	src/inspect_ai/agent/_deepagent/plan.py	35	CODE
LOW	src/inspect_ai/agent/_deepagent/subagent.py	56	CODE
LOW	src/inspect_ai/agent/_deepagent/research.py	35	CODE
LOW	src/inspect_ai/agent/_deepagent/prompt.py	234	CODE
LOW	src/inspect_ai/agent/_deepagent/deepagent.py	50	CODE
LOW	src/inspect_ai/agent/_deepagent/general.py	32	CODE
LOW	src/inspect_ai/solver/_basic_agent.py	52	CODE
LOW	src/inspect_ai/solver/_task_state.py	150	CODE
LOW	src/inspect_ai/_view/fastapi_server.py	729	CODE
LOW	src/inspect_ai/_view/view.py	24	CODE
LOW	src/inspect_ai/_cli/_scanner.py	23	CODE
LOW	src/inspect_ai/_cli/_scanner.py	91	CODE
LOW	src/inspect_ai/_cli/ctl.py	4942	CODE
LOW	src/inspect_ai/_cli/eval.py	1748	CODE
LOW	src/inspect_ai/model/_model.py	1838	CODE
LOW	src/inspect_ai/model/_providers/vllm.py	141	CODE
LOW	src/inspect_ai/model/_providers/nnterp.py	53	CODE
LOW	src/inspect_ai/model/_providers/openai_compatible.py	63	CODE
LOW	src/inspect_ai/model/_providers/openai.py	97	CODE
LOW	src/inspect_ai/log/_condense.py	144	CODE
LOW	src/inspect_ai/_eval/run.py	121	CODE
LOW	src/inspect_ai/_eval/eval.py	116	CODE
LOW	src/inspect_ai/_eval/eval.py	406	CODE
LOW	src/inspect_ai/_eval/eval.py	671	CODE
LOW	src/inspect_ai/_eval/eval.py	1233	CODE
LOW	src/inspect_ai/_eval/eval.py	1421	CODE
LOW	src/inspect_ai/_eval/eval.py	1829	CODE
LOW	src/inspect_ai/_eval/evalset.py	133	CODE
LOW	src/inspect_ai/_eval/score.py	211	CODE
LOW	src/inspect_ai/_eval/task/task.py	286	CODE
LOW	src/inspect_ai/_eval/task/task.py	82	CODE
LOW	src/inspect_ai/tool/_tools/_memory.py	61	CODE
LOW	src/inspect_ai/tool/_tools/_computer/_computer.py	93	CODE

Structural Annotation Overuse33 hits · 56 pts

Severity	File	Line	Snippet	Context
LOW⚡	design/recover.md	434	### Step 1: Read crashed .eval files (done: `162628a3f`)	COMMENT
LOW⚡	design/recover.md	444	### Step 2: Read recovery data from sample buffer database (done: `664a9a43b`)	COMMENT
LOW⚡	design/recover.md	454	### Step 3: Reconstruct EvalSample from buffer DB data (done: `a05ce7fda`)	COMMENT
LOW	design/recover.md	468	### Step 4: Write recovered .eval file (done: `493de0a1d`)	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	101	# Step 1: Initial request with server-side tools	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	129	# Step 2: Follow-up to build more history	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	143	# Step 3: Another follow-up	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	158	# Step 4: Compact history - clear older tool results	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	181	# Step 5: Final request with compacted history - THIS SHOULD NOT CAUSE 400 ERROR	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	90	# Step 2: Execute tool calls if present	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	95	# Step 3: Ask follow-up question	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	112	# Step 4: Add thank you message	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	115	# Step 5: Compact history - remove ALL thinking blocks	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	145	# Step 6: Call model with compacted history	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	152	# Step 7: This should NOT raise an error	COMMENT
LOW	tests/model/test_compaction_edit_thinking.py	61	# Step 1: System prompt that STRONGLY requires tool use + user prompt	COMMENT
LOW	tests/model/test_web_search_exclude_none.py	97	# Step 1: Convert OpenAI Response to ChatMessageAssistant	COMMENT
LOW	tests/model/test_web_search_exclude_none.py	136	# Step 2: Convert ChatMessageAssistant back to OpenAI Input Items	COMMENT
LOW⚡	tests/model/providers/test_google.py	1192	# Step 1: Simulate model returning an image via inline_data	COMMENT
LOW	tests/model/providers/test_google.py	1215	# Step 2: Build assistant message and convert back to Google Content	COMMENT
LOW⚡	tests/model/providers/test_openai_responses.py	1227	# Step 2: Build an assistant message with the image content	COMMENT
LOW⚡	tests/model/providers/test_openai_responses.py	1234	# Step 3: Convert back to input items (simulating the next turn)	COMMENT
LOW	tests/model/providers/test_openai_responses.py	1210	# Step 1: Model returns an image generation call	COMMENT
LOW⚡	src/inspect_ai/scorer/_math.py	789	# Step 1: Try simple integer parsing	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	794	# Step 2: Try float parsing	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	799	# Step 3: Try sympify with LaTeX preprocessing (primary method)	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	805	# Step 4: Fall back to latex2sympy_fixed (backup method)	STRING
LOW	src/inspect_ai/model/_providers/_anthropic_citations.py	109	# NOTE: This function cannot assume that this module was responsible for the	COMMENT
LOW	src/inspect_ai/log/_log.py	1128	# WARNING: The order of these fields is important for the log file format.	COMMENT
LOW	src/inspect_ai/log/_recover/_api.py	109	# Step 1: Read the crashed .eval file metadata	COMMENT
LOW⚡	src/inspect_ai/log/_recover/_api.py	146	# Step 2: Read buffer DB metadata (lightweight — just summaries)	COMMENT
LOW⚡	src/inspect_ai/log/_recover/_api.py	154	# Step 3: Determine recovery path and write	COMMENT
LOW	src/inspect_ai/log/_recover/_api.py	197	# Step 4: Stream all samples into the recovered file.	COMMENT

Verbosity Indicators27 hits · 44 pts

Severity	File	Line	Snippet	Context
LOW	tests/model/test_compaction_edit_server_tools.py	101	# Step 1: Initial request with server-side tools	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	129	# Step 2: Follow-up to build more history	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	143	# Step 3: Another follow-up	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	158	# Step 4: Compact history - clear older tool results	COMMENT
LOW	tests/model/test_compaction_edit_server_tools.py	181	# Step 5: Final request with compacted history - THIS SHOULD NOT CAUSE 400 ERROR	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	90	# Step 2: Execute tool calls if present	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	95	# Step 3: Ask follow-up question	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	112	# Step 4: Add thank you message	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	115	# Step 5: Compact history - remove ALL thinking blocks	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	145	# Step 6: Call model with compacted history	COMMENT
LOW⚡	tests/model/test_compaction_edit_thinking.py	152	# Step 7: This should NOT raise an error	COMMENT
LOW	tests/model/test_compaction_edit_thinking.py	61	# Step 1: System prompt that STRONGLY requires tool use + user prompt	COMMENT
LOW	tests/model/test_web_search_exclude_none.py	97	# Step 1: Convert OpenAI Response to ChatMessageAssistant	COMMENT
LOW	tests/model/test_web_search_exclude_none.py	136	# Step 2: Convert ChatMessageAssistant back to OpenAI Input Items	COMMENT
LOW⚡	tests/model/providers/test_google.py	1192	# Step 1: Simulate model returning an image via inline_data	COMMENT
LOW	tests/model/providers/test_google.py	1215	# Step 2: Build assistant message and convert back to Google Content	COMMENT
LOW⚡	tests/model/providers/test_openai_responses.py	1227	# Step 2: Build an assistant message with the image content	COMMENT
LOW⚡	tests/model/providers/test_openai_responses.py	1234	# Step 3: Convert back to input items (simulating the next turn)	COMMENT
LOW	tests/model/providers/test_openai_responses.py	1210	# Step 1: Model returns an image generation call	COMMENT
LOW⚡	src/inspect_ai/scorer/_math.py	789	# Step 1: Try simple integer parsing	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	794	# Step 2: Try float parsing	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	799	# Step 3: Try sympify with LaTeX preprocessing (primary method)	STRING
LOW⚡	src/inspect_ai/scorer/_math.py	805	# Step 4: Fall back to latex2sympy_fixed (backup method)	STRING
LOW	src/inspect_ai/log/_recover/_api.py	109	# Step 1: Read the crashed .eval file metadata	COMMENT
LOW⚡	src/inspect_ai/log/_recover/_api.py	146	# Step 2: Read buffer DB metadata (lightweight — just summaries)	COMMENT
LOW⚡	src/inspect_ai/log/_recover/_api.py	154	# Step 3: Determine recovery path and write	COMMENT
LOW	src/inspect_ai/log/_recover/_api.py	197	# Step 4: Stream all samples into the recovered file.	COMMENT

Fake / Example Data34 hits · 40 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/tools/test_web_search.py	280	"fake-key" if key == "TAVILY_API_KEY" else default	CODE
LOW⚡	tests/tools/test_web_search.py	289	"fake-key" if key == "TAVILY_API_KEY" else default	CODE
LOW⚡	tests/tools/test_web_search.py	298	"fake-key" if key == "TAVILY_API_KEY" else default	CODE
LOW⚡	tests/tools/test_web_search.py	307	return_value=("fake-key", "fake-cse-id"),	CODE
LOW⚡	tests/tools/test_web_search.py	314	return_value=("fake-key", "fake-cse-id"),	CODE
LOW⚡	tests/tools/test_web_search.py	322	"fake-key" if key == "EXA_API_KEY" else default	CODE
LOW⚡	tests/tools/test_web_search.py	331	"fake-key" if key == "EXA_API_KEY" else default	CODE
LOW⚡	tests/tools/test_web_search.py	340	"fake-key" if key == "EXA_API_KEY" else default	CODE
LOW	tests/tools/test_web_search.py	365	return_value=("fake-key", "fake-cse-id"),	CODE
LOW	tests/tools/test_web_search.py	396	return_value=("fake-key", "fake-cse-id"),	CODE
LOW⚡	tests/tools/test_web_search_tavily.py	59	with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):	CODE
LOW	tests/tools/test_web_search_tavily.py	113	with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):	CODE
LOW⚡	tests/tools/test_web_search_tavily.py	127	with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):	CODE
LOW⚡	tests/tools/test_web_search_tavily.py	144	with patch.dict("os.environ", {"TAVILY_API_KEY": "dummy-key"}):	CODE
LOW	tests/tools/test_web_search_google.py	94	mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")	CODE
LOW	tests/tools/test_web_search_google.py	141	mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")	CODE
LOW	tests/tools/test_web_search_google.py	174	mock_get_keys.return_value = ("dummy-key", "dummy-cse-id")	CODE
LOW⚡	tests/util/test_store_nested_types.py	97	"street": "123 Main St",	CODE
LOW	tests/util/test_format_template.py	171	address=Address(street="123 Main St", city="Anytown"),	CODE
LOW⚡	tests/util/test_json_schema.py	169	instance = Model(name="Alice", address={"street": "123 Main St", "city": "NYC"})	CODE
LOW⚡	tests/util/test_json_schema.py	171	assert instance.address.street == "123 Main St"	CODE
LOW	tests/util/test_json_schema.py	200	"contact": {"email": "bob@example.com", "phone": "555-1234"},	CODE
LOW	tests/model/test_token_counting_integration.py	72	assert result[0]["name"] == "placeholder"	CODE
LOW⚡	tests/model/providers/test_vllm.py	30	message = ChatMessageUser(content="Lorem ipsum dolor")	CODE
LOW	tests/model/providers/test_vllm.py	49	message = ChatMessageUser(content="Lorem ipsum dolor")	CODE
LOW	tests/model/providers/test_hf.py	61	message = ChatMessageUser(content="Lorem ipsum dolor")	CODE
LOW	tests/model/providers/test_hf.py	87	message = ChatMessageUser(content="Lorem ipsum dolor")	CODE
LOW⚡	tests/model/providers/test_hf.py	256	message = ChatMessageUser(content="Lorem ipsum dolor")	CODE
LOW⚡	tests/model/providers/test_hf.py	258	assert chat == "user: Lorem ipsum dolor\n"	CODE
LOW	examples/http_proxy/claude.py	56	"FUTUREMODEL_API_KEY": "fm-fake-key-for-demo",	CODE
LOW	…ct_tool_support/_remote_tools/_web_browser/cdp/a11y.py	94	"attribute", "implicit", "stylet", "contents", "placeholder", "relatedElement"	CODE
LOW⚡	src/inspect_ai/model/_openai_responses.py	1124	param["id"] = "dummy-id"	CODE
LOW	src/inspect_ai/model/_openai_responses.py	2463	"name": "placeholder",	CODE
LOW	src/inspect_ai/model/_providers/anthropic.py	4466	name="placeholder",	CODE

Dead Code7 hits · 14 pts

Severity	File	Line	Context
MEDIUM	tests/tools/test_tool_error.py	75	CODE
MEDIUM	…ts/agent/test_acp/test_tui/test_inline_request_card.py	129	CODE
MEDIUM	tests/model/providers/test_sagemaker.py	1419	CODE
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/cancel_card.py	162	CODE
MEDIUM	src/inspect_ai/agent/_acp/tui/widgets/approval_card.py	113	CODE
MEDIUM	…spect_ai/agent/_acp/tui/widgets/inline_request_card.py	115	CODE
MEDIUM	…spect_ai/agent/_acp/tui/widgets/inline_request_card.py	126	CODE

Overly Generic Function Names5 hits · 5 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_flaky_retry.py	111	def test_function():	CODE
LOW	…t_tool_support/src/inspect_tool_support/_cli/server.py	16	async def handle_request(request: Request) -> Response:	CODE
LOW	…sandbox_tools/src/inspect_sandbox_tools/_cli/server.py	56	async def handle_request(request: Request) -> Response:	CODE
LOW	src/inspect_ai/_util/_async.py	64	async def run_task(func: Callable[[], Awaitable[T]], index: int) -> None:	CODE
LOW	src/inspect_ai/util/_collect.py	38	async def run_task(index: int, task: Awaitable[T]) -> None:	CODE

Analysis Overview

What These Metrics Mean

Score History

Severity Breakdown

Directory Score Breakdown

Pattern Findings