PrimeIntellect-ai/verifiers

29.3

Adjusted Score

29.3

Raw Score

100%

Time Factor

2026-07-14

Last Push

4.3K

Stars

Python

Language

106.0K

Lines of Code

596

Files

2.4K

Pattern Hits

2026-07-14

Scan Date

0.15

HC Hit Rate

What These Metrics Mean

Adjusted Score: Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score: The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor: The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits: Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate: High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files: Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 1HIGH 88MEDIUM 164LOW 2116

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 2369 distinct pattern matches across 22 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers914 hits · 894 pts

Severity	File	Line	Snippet	Context
LOW	environments/sentence_repeater/sentence_repeater.py	98	def compare_answers_reward_func(parser, completion, info, **kwargs) -> float:	CODE
LOW	environments/alphabet_sort/alphabet_sort.py	186	def _count_tag_instances_and_contents(text: str, tag: str) -> tuple[int, List[str]]:	CODE
LOW	tests/test_gepa_utils.py	15	def test_save_gepa_results_writes_best_system_prompt_verbatim(tmp_path):	CODE
LOW	tests/test_gepa_utils.py	29	def test_save_gepa_results_handles_none_best_score(tmp_path):	CODE
LOW	tests/test_gepa_utils.py	46	def test_save_gepa_results_writes_upload_schema_without_task_fields(tmp_path):	CODE
LOW	tests/test_gepa_utils.py	101	def test_save_gepa_results_writes_candidate_diffs_and_frontier(tmp_path):	CODE
LOW	tests/test_eval_utils.py	13	def test_print_results_rollout_indexing(capsys, make_metadata, make_state, make_input):	CODE
LOW	tests/test_eval_utils.py	62	def test_print_results_single_rollout(capsys, make_metadata, make_state, make_input):	CODE
LOW	tests/test_eval_utils.py	90	def test_print_results_includes_eval_name(capsys, make_metadata, make_output):	CODE
LOW	tests/test_eval_utils.py	106	def test_print_results_three_rollouts(capsys, make_metadata, make_state, make_input):	CODE
LOW	tests/test_eval_utils.py	139	def test_print_results_includes_usage(capsys, make_metadata, make_output):	CODE
LOW	tests/test_eval_utils.py	159	def test_attach_metadata_cost_uses_total_output_usage(make_metadata, make_output):	CODE
LOW	tests/test_eval_utils.py	188	def test_print_results_labels_cost_as_all(capsys, make_metadata, make_output):	CODE
LOW	tests/test_eval_utils.py	208	def test_print_results_handles_heterogeneous_metrics(	CODE
LOW⚡	tests/test_parser.py	9	def test_parser_initialization(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	14	def test_parse_returns_text_as_is(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	20	def test_get_assistant_messages(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	33	def test_parse_answer_with_string(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	39	def test_parse_answer_with_completion(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	48	def test_parse_answer_with_empty_completion(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	53	def test_parse_answer_with_no_assistant_messages(self, basic_parser):	CODE
LOW⚡	tests/test_parser.py	59	def test_get_format_reward_func(self, basic_parser):	CODE
LOW	tests/test_per_turn_timing.py	11	async def test_single_turn_records_one_model_span(self, mock_client, make_input):	CODE
LOW	tests/test_per_turn_timing.py	36	async def test_multi_turn_records_model_and_env_spans(	CODE
LOW⚡	tests/test_composable_env.py	147	def test_task_repr_no_sandbox():	CODE
LOW⚡	tests/test_composable_env.py	218	def test_composable_mini_swe_agent_install_script_uses_pip_requirement():	CODE
LOW⚡	tests/test_composable_env.py	225	def test_composable_opencode_install_script_owns_release_download():	CODE
LOW⚡	tests/test_composable_env.py	235	async def test_composable_env_exports_task_workdir():	CODE
LOW⚡	tests/test_composable_env.py	314	async def test_composable_env_install_env_passes_to_execute():	CODE
LOW⚡	tests/test_composable_env.py	556	def test_discover_sibling_dir_finds_skills(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	563	def test_discover_sibling_dir_returns_none_without_skills(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	573	def test_get_skills_dir_auto_discovers(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	580	def test_get_skills_dir_returns_none_without_skills(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	587	def test_get_upload_dirs_includes_skills_automatically(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	595	def test_get_upload_dirs_empty_without_skills(tmp_path, monkeypatch):	CODE
LOW⚡	tests/test_composable_env.py	606	async def test_composable_env_collects_harness_metrics():	CODE
LOW	tests/test_composable_env.py	95	def test_sandbox_spec_defaults():	CODE
LOW	tests/test_composable_env.py	190	def test_composable_env_eval_inputs_can_shuffle_taskset_dataset():	CODE
LOW	tests/test_composable_env.py	254	async def test_composable_env_quotes_paths_in_mkdir_command():	CODE
LOW	tests/test_composable_env.py	282	async def test_composable_env_quotes_log_path_when_collecting_logs():	CODE
LOW	tests/test_composable_env.py	343	async def test_composable_env_install_env_none_by_default():	CODE
LOW	tests/test_composable_env.py	371	def _make_temp_taskset_package(tmp_path, monkeypatch, *, with_skills: bool):	CODE
LOW	tests/test_composable_env.py	409	async def test_composable_env_uploads_task_dirs(tmp_path, monkeypatch):	CODE
LOW	tests/test_composable_env.py	449	async def test_composable_env_no_upload_when_no_dirs(tmp_path, monkeypatch):	CODE
LOW	tests/test_composable_env.py	477	async def test_composable_env_uploads_harness_dirs(tmp_path):	CODE
LOW	tests/test_composable_env.py	518	async def test_composable_env_rejects_duplicate_task_and_harness_upload_names(	CODE
LOW	tests/test_composable_env.py	655	async def test_composable_env_metrics_with_key_whitelist():	CODE
LOW	tests/test_composable_env.py	692	async def test_composable_env_no_metrics_when_path_not_set():	CODE
LOW⚡	tests/test_opencode_rlm_env.py	48	def test_default_plugin_branch(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	52	def test_default_plugin_install_path(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	60	def test_default_sub_llm_max_turns(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	64	def test_default_sub_timeout_ms(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	93	def test_config_includes_plugin_reference(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	100	def test_config_custom_install_path(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	105	def test_config_has_schema_and_provider(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	112	def test_config_renders_valid_json_after_shell_expansion(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	154	def test_run_command_installs_jq(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	161	def test_run_command_installs_bun(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	165	def test_run_command_clones_plugin(self):	CODE
LOW⚡	tests/test_opencode_rlm_env.py	172	def test_run_command_custom_branch(self):	CODE
854 more matches not shown…

Unused Imports523 hits · 452 pts

Severity	File	Line	Context
LOW	environments/deepwiki_v1/deepwiki_v1/__init__.py	1	CODE
LOW	environments/compact/compact/__init__.py	4	CODE
LOW	environments/compact/compact/__init__.py	4	CODE
LOW	environments/code_golf_v1/code_golf_v1/__init__.py	1	CODE
LOW	environments/glossary_v1/glossary_v1/__init__.py	1	CODE
LOW	environments/wiki_search_v1/wiki_search_v1/__init__.py	1	CODE
LOW	…vironments/reverse_text_v1/reverse_text_v1/__init__.py	1	CODE
LOW	…nments/color_codeword_v1/color_codeword_v1/__init__.py	1	CODE
LOW	environments/gsm8k_v1/gsm8k_v1/__init__.py	1	CODE
LOW	…ronments/alphabet_sort_v1/alphabet_sort_v1/__init__.py	1	CODE
LOW	environments/scratchpad_v1/scratchpad_v1/__init__.py	1	CODE
LOW	environments/mmmu_v1/mmmu_v1/__init__.py	1	CODE
LOW	environments/wordle_v1/wordle_v1/__init__.py	1	CODE
LOW	verifiers/__init__.py	14	CODE
LOW	verifiers/__init__.py	15	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	16	CODE
LOW	verifiers/__init__.py	26	CODE
LOW	verifiers/__init__.py	26	CODE
LOW	verifiers/__init__.py	26	CODE
LOW	verifiers/__init__.py	26	CODE
LOW	verifiers/__init__.py	27	CODE
LOW	verifiers/__init__.py	28	CODE
LOW	verifiers/__init__.py	31	CODE
LOW	verifiers/__init__.py	32	CODE
LOW	verifiers/__init__.py	33	CODE
LOW	verifiers/__init__.py	34	CODE
LOW	verifiers/__init__.py	35	CODE
LOW	verifiers/__init__.py	35	CODE
LOW	verifiers/__init__.py	36	CODE
LOW	verifiers/__init__.py	36	CODE
LOW	verifiers/__init__.py	36	CODE
LOW	verifiers/__init__.py	41	CODE
LOW	verifiers/__init__.py	41	CODE
LOW	verifiers/__init__.py	41	CODE
LOW	verifiers/__init__.py	161	CODE
LOW	verifiers/__init__.py	162	CODE
LOW	verifiers/__init__.py	163	CODE
LOW	verifiers/__init__.py	166	CODE
LOW	verifiers/__init__.py	167	CODE
LOW	verifiers/__init__.py	168	CODE
LOW	verifiers/__init__.py	169	CODE
LOW	verifiers/__init__.py	170	CODE
LOW	verifiers/__init__.py	171	CODE
LOW	verifiers/__init__.py	172	CODE
LOW	verifiers/__init__.py	173	CODE
LOW	verifiers/__init__.py	174	CODE
LOW	verifiers/__init__.py	175	CODE
LOW	verifiers/__init__.py	176	CODE
LOW	verifiers/__init__.py	177	CODE
LOW	verifiers/__init__.py	178	CODE
LOW	verifiers/__init__.py	179	CODE
LOW	verifiers/__init__.py	180	CODE
LOW	verifiers/__init__.py	181	CODE
463 more matches not shown…

Cross-Language Confusion63 hits · 365 pts

Severity	File	Line	Snippet	Context
HIGH⚡	tests/test_composable_env.py	304	"cat '/tmp/log dir/agent.log' 2>/dev/null \|\| echo '<no logs>'",	CODE
HIGH	tests/test_composable_env.py	443	"mkdir -p /task && tar -xzf /tmp/_upload_task_skills.tar.gz -C / && rm -f /tmp/_upload_task_skills.tar.gz",	CODE
HIGH	tests/test_composable_env.py	512	"mkdir -p /tmp && tar -xzf /tmp/_upload_tmp_agent-src.tar.gz -C / && rm -f /tmp/_upload_tmp_agent-src.tar.gz",	CODE
HIGH⚡	tests/test_opencode_rlm_env.py	178	assert "cd /tmp/opencode-rlm && bun install" in env.run_command	CODE
HIGH	tests/test_envs.py	111	f"cd {tmp_venv_dir} && uv venv --clear && source .venv/bin/activate && "	CODE
HIGH	tests/test_envs.py	139	import_cmd = f"cd {tmp_venv_dir} && source .venv/bin/activate && uv run python -c 'import {env_dir.name}'"	CODE
HIGH	tests/test_envs.py	156	load_cmd = f"""cd {tmp_venv_dir} && source .venv/bin/activate && uv run python -c 'import verifiers as vf; vf.load_e	CODE
HIGH⚡	tests/test_harbor_env_mcp.py	245	env = _DummyEnv(mcp_launch_commands={"svc": "cd /opt && python server.py"})	CODE
HIGH⚡	tests/test_harbor_env_mcp.py	255	assert "'cd /opt && python server.py'" in start_cmd	CODE
HIGH	tests/test_rlm_composable_env.py	532	"mkdir -p /task && tar -xzf /tmp/_upload_task_rlm-skills.tar.gz -C / && rm -f /tmp/_upload_task_rlm-skills.tar.g	CODE
HIGH	tests/test_rlm_composable_env.py	701	"cat '/tmp/log dir/agent.log' 2>/dev/null \|\| echo '<no logs>'",	CODE
HIGH⚡	tests/v1/conftest.py	113	pytest.param("null", marks=pytest.mark.null, id="null"),	CODE
HIGH⚡	verifiers/v1/harnesses/kimi_code/harness.py	20	if [ -x "$bin" ] && [ "$("$bin" --version 2>/dev/null)" = "{version}" ]; then	CODE
HIGH⚡	verifiers/v1/harnesses/kimi_code/harness.py	23	command -v curl >/dev/null \|\| { apt-get update -qq && apt-get install -y -qq curl ca-certificates >/dev/null; }	CODE
HIGH⚡	verifiers/v1/harnesses/kimi_code/harness.py	23	command -v curl >/dev/null \|\| { apt-get update -qq && apt-get install -y -qq curl ca-certificates >/dev/null; }	CODE
HIGH	verifiers/v1/harnesses/codex/harness.py	29	command -v curl >/dev/null 2>&1 \|\| { apt-get update -qq && apt-get install -y -qq curl >/dev/null; }	CODE
HIGH	verifiers/v1/harnesses/codex/harness.py	29	command -v curl >/dev/null 2>&1 \|\| { apt-get update -qq && apt-get install -y -qq curl >/dev/null; }	CODE
HIGH	verifiers/v1/harnesses/rlm/harness.py	79	"{ apt-get update -qq && apt-get install -y -qq git; } && "	CODE
HIGH	verifiers/v1/harnesses/rlm/harness.py	80	f"rm -rf /tmp/rlm && git clone https://{RLM_REPO} /tmp/rlm && "	CODE
HIGH	verifiers/v1/harnesses/terminus_2/harness.py	76	'tmux kill-server >/dev/null 2>&1 \|\| true; rm -rf "$TMUX_TMPDIR"',	CODE
HIGH⚡	verifiers/v1/runtimes/base.py	30	"{ command -v curl >/dev/null 2>&1 \|\| command -v wget >/dev/null 2>&1; } "	CODE
HIGH⚡	verifiers/v1/runtimes/base.py	31	"\|\| { apt-get update -qq && apt-get install -y -qq curl ca-certificates; } "	CODE
HIGH⚡	verifiers/v1/runtimes/base.py	35	"{ command -v curl >/dev/null 2>&1 && curl -LsSf https://astral.sh/uv/install.sh \| sh; } "	CODE
HIGH⚡	verifiers/v1/runtimes/base.py	36	"\|\| { command -v wget >/dev/null 2>&1 && wget -qO- https://astral.sh/uv/install.sh \| sh; }"	CODE
HIGH	verifiers/v1/tasksets/harbor/taskset.py	103	"mkdir -p /logs/verifier /tests && tar -xzf /tmp/tests.tgz -C /tests",	CODE
HIGH	verifiers/v1/tasksets/harbor/taskset.py	108	["sh", "-c", "cd /tests && bash test.sh"], verifier_env(self.data)	CODE
HIGH	verifiers/envs/experimental/opencode_rlm_env.py	101	apt-get -o Acquire::Retries=3 update && apt-get -o Acquire::Retries=3 install -y curl git unzip jq	CODE
HIGH	verifiers/envs/experimental/opencode_env.py	59	apt-get -o Acquire::Retries=3 update && apt-get -o Acquire::Retries=3 install -y curl	CODE
HIGH	verifiers/envs/experimental/opencode_env.py	334	f"cat {self.remote_logs_path} 2>/dev/null \|\| echo '<no logs>'",	CODE
HIGH	…rifiers/envs/experimental/composable/composable_env.py	294	f"cat {log_path} 2>/dev/null \|\| echo '<no logs>'",	CODE
HIGH	verifiers/envs/experimental/composable/harnesses/rlm.py	119	export RLM_APPEND_TO_SYSTEM_PROMPT="$(cat {shlex.quote(DEFAULT_APPEND_TO_SYSTEM_PROMPT_PATH)} 2>/dev/null \|\| true)"	CODE
HIGH⚡	…ers/envs/experimental/composable/harnesses/opencode.py	53	"apt-get -o Acquire::Retries=3 install -y -qq ripgrep > /dev/null 2>&1 \|\| true"	CODE
HIGH⚡	…ers/envs/experimental/composable/harnesses/opencode.py	62	apt-get -o Acquire::Retries=3 update -qq && apt-get -o Acquire::Retries=3 install -y -qq curl tar > /dev/null 2>&1	CODE
HIGH	…envs/experimental/composable/tasksets/harbor/harbor.py	139	f"mkdir -p /task /app && tar -xzf {remote_tar} -C / && rm {remote_tar}",	CODE
HIGH	…envs/experimental/composable/tasksets/harbor/harbor.py	176	f"mkdir -p /tests /logs/verifier && tar -xzf {remote_tar} -C / && rm {remote_tar}",	CODE
HIGH	…envs/experimental/composable/tasksets/harbor/harbor.py	252	f"mkdir -p /oracle && tar -xzf {remote_tar} -C / && rm {remote_tar}",	CODE
HIGH	…perimental/composable/tasksets/swe/swe_lego/taskset.py	113	conda activate testbed 2>/dev/null \|\| true	CODE
HIGH	…erimental/composable/tasksets/swe/shared/test_patch.py	20	``git add && git commit`` mid-rollout, ``HEAD`` points at the agent's	STRING
HIGH	…erimental/composable/tasksets/swe/shared/test_patch.py	174	an agent that runs ``git add && git commit`` mid-rollout moves	STRING
HIGH	…erimental/composable/tasksets/swe/swe_bench/taskset.py	437	"cd /testbed && git -c core.fileMode=false diff --name-only HEAD"	CODE
HIGH	…erimental/composable/tasksets/swe/multi_swe/taskset.py	214	# (e.g. skipped_tests) are empty get List(null) instead of List(string), causing	COMMENT
HIGH	…erimental/composable/tasksets/swe/multi_swe/taskset.py	263	"command -v patch \|\| (apt-get -o Acquire::Retries=3 update && apt-get -o Acquire::Retries=3 install -y patch	CODE
HIGH⚡	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	297	"timeout 30 bash -c 'shopt -s globstar; rm -rf */.pyc **/__pycache__' 2>/dev/null \|\| timeout 30 fi	CODE
HIGH⚡	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	301	"timeout 30 bash -c 'shopt -s globstar; rm -rf **/__pycache__' 2>/dev/null \|\| timeout 30 find . -nam	CODE
HIGH⚡	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	305	"timeout 30 bash -c 'shopt -s globstar; rm -rf /r2e_tests/*/.pyc /r2e_tests/**/__pycache__' 2>/dev	CODE
HIGH⚡	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	309	"timeout 30 bash -c 'shopt -s globstar; rm -rf /r2e_tests/**/__pycache__' 2>/dev/null \|\| timeout 30	CODE
HIGH	…erimental/composable/tasksets/swe/scale_swe/taskset.py	224	email=$(git config user.email 2>/dev/null)	CODE
HIGH	…erimental/composable/tasksets/swe/scale_swe/taskset.py	229	name=$(git config user.name 2>/dev/null)	CODE
HIGH	…erimental/composable/tasksets/swe/scale_swe/taskset.py	322	"cat /tmp/_awe_test_output.txt 2>/dev/null \|\| true",	STRING
HIGH	…erimental/composable/tasksets/swe/scale_swe/taskset.py	327	"cat /tmp/_awe_test_results.xml 2>/dev/null \|\| true",	STRING
HIGH⚡	…erimental/composable/tasksets/swe/scale_swe/taskset.py	349	git checkout "$base" -- tests/ test/ Test/ Tests/ 2>/dev/null \|\| true	CODE
HIGH⚡	…erimental/composable/tasksets/swe/scale_swe/taskset.py	353	git checkout "$base" -- "$path" 2>/dev/null \|\| true	CODE
HIGH⚡	…erimental/composable/tasksets/swe/scale_swe/taskset.py	360	if ! git cat-file -e "$base:$path" 2>/dev/null; then	CODE
HIGH⚡	…erimental/composable/tasksets/swe/scale_swe/taskset.py	362	git rm -q --cached -- "$path" 2>/dev/null \|\| true	CODE
HIGH	…ental/composable/tasksets/search/openseeker/taskset.py	465	f"cat {self.answer_file} 2>/dev/null \|\| true",	CODE
HIGH	…ntal/composable/tasksets/search/redsearcher/taskset.py	461	f"cat {self.answer_file} 2>/dev/null \|\| true",	CODE
HIGH	…perimental/composable/tasksets/search/quest/taskset.py	632	f"cat {self.answer_file} 2>/dev/null \|\| true",	CODE
HIGH	verifiers/envs/experimental/harbor_env/env.py	192	f"mkdir -p /oracle /tests && tar -xzf {remote_tar} -C / && rm {remote_tar}",	CODE
HIGH	verifiers/envs/experimental/harbor_env/mcp.py	185	f'kill -9 -"$(cat {pid_file} 2>/dev/null)" 2>/dev/null; rm -f {pid_file}',	CODE
HIGH⚡	verifiers/envs/integrations/openenv_env.py	622	cmd = f'sh -lc "curl -sS -m 2 http://localhost:{int(port)}/health 2>&1 \|\| true"'	CODE
3 more matches not shown…

Decorative Section Separators85 hits · 315 pts

Severity	File	Line	Snippet	Context
MEDIUM⚡	tests/test_composable_env.py	137	# ── Task from plain TaskSet ────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_composable_env.py	153	# ── TaskSet ─────────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_composable_env.py	310	# ── install_env ──────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_composable_env.py	553	# ── discover_sibling_dir ─────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_composable_env.py	570	# ── get_skills_dir / auto-discovery ──────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_composable_env.py	602	# ── Harness metrics collection ───────────────────────────────────────────	COMMENT
MEDIUM	tests/test_composable_env.py	28	# ── Mock Rubrics ──────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_composable_env.py	49	# ── Mock TaskSets ───────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_composable_env.py	92	# ── SandboxSpec ─────────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_composable_env.py	107	# ── Task from SandboxTaskSet ───────────────────────────────────────────	COMMENT
MEDIUM	tests/test_composable_env.py	368	# ── get_upload_dirs ──────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	38	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	40	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	87	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	89	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	148	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	150	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	185	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	187	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	244	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	246	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	280	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	282	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	340	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_opencode_rlm_env.py	342	# =============================================================================	COMMENT
MEDIUM	tests/test_opencode_rlm_env.py	18	# =============================================================================	COMMENT
MEDIUM	tests/test_opencode_rlm_env.py	20	# =============================================================================	COMMENT
MEDIUM⚡	tests/test_renderer_client.py	820	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_renderer_client.py	822	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_renderer_client.py	864	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_renderer_client.py	866	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_renderer_client.py	597	# ── Parity across real renderers: truncated most-recent step ──────────	COMMENT
MEDIUM⚡	tests/test_rlm_composable_env.py	175	# ── RLM harness ──────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_rlm_composable_env.py	652	# ── RLM metrics via harness fields ──────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_rlm_composable_env.py	804	# ── keep_trajectory_step end-to-end (header-stash ordering) ──────────────	COMMENT
MEDIUM	tests/test_rlm_composable_env.py	448	# ── install_env ──────────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_rlm_composable_env.py	495	# ── Skills upload ────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_browser_env.py	43	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_browser_env.py	45	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_browser_env.py	384	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_browser_env.py	386	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_browser_env.py	455	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_browser_env.py	457	# ============================================================================	COMMENT
MEDIUM	tests/test_browser_env.py	16	# ============================================================================	COMMENT
MEDIUM	tests/test_browser_env.py	18	# ============================================================================	COMMENT
MEDIUM	tests/test_browser_env.py	547	# ============================================================================	COMMENT
MEDIUM	tests/test_browser_env.py	549	# ============================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	14	# =========================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	16	# =========================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	61	# =========================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	63	# =========================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	177	# =========================================================================	COMMENT
MEDIUM	tests/test_context_token_metrics.py	179	# =========================================================================	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	77	# ── create_sandbox ───────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	178	# ── post_sandbox_setup ───────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	186	# ── delete_sandbox ───────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	206	# ── bulk_delete_sandboxes ────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	229	# ── run_background_job ───────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	273	# ── teardown_sandboxes ──────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_sandbox_mixin.py	320	# ── teardown_sandbox_client ──────────────────────────────────────────	COMMENT
25 more matches not shown…

Excessive Try-Catch Wrapping235 hits · 239 pts

Severity	File	Line	Snippet	Context
LOW	environments/alphabet_sort/alphabet_sort.py	178	except Exception as e:	CODE
LOW	environments/gsm8k_v1/gsm8k_v1/verify.py	29	except Exception:	CODE
LOW	verifiers/v1/rollout.py	230	except Exception as e:	CODE
LOW	verifiers/v1/rollout.py	246	except Exception:	CODE
LOW	verifiers/v1/errors.py	91	except Exception as e:	CODE
LOW	verifiers/v1/push.py	205	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	271	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	322	except Exception as e: # surface to the program as an API error	CODE
LOW	verifiers/v1/interception/server.py	354	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	388	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	424	except Exception as e: # surface to the program as an API error	CODE
LOW	verifiers/v1/interception/server.py	468	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	477	except Exception as e:	CODE
LOW	verifiers/v1/interception/server.py	529	except Exception as e:	CODE
LOW	verifiers/v1/interception/tunnel/prime.py	48	except Exception as e:	CODE
LOW	verifiers/v1/harnesses/codex/harness.py	174	except Exception:	CODE
LOW	verifiers/v1/harnesses/default/program.py	127	except Exception as e:	CODE
LOW	verifiers/v1/harnesses/default/program.py	137	except Exception as e:	CODE
LOW	verifiers/v1/harnesses/default/program.py	159	except Exception as e:	CODE
LOW	verifiers/v1/harnesses/default/program.py	166	except Exception as e:	CODE
LOW	verifiers/v1/harnesses/terminus_2/harness.py	80	except Exception:	CODE
LOW	verifiers/v1/runtimes/prime.py	184	except Exception as e: # surface prime's exposure constraints actionably	CODE
LOW	verifiers/v1/runtimes/prime.py	218	except Exception as e:	CODE
LOW	verifiers/v1/runtimes/prime.py	240	except Exception as e:	CODE
LOW	verifiers/v1/runtimes/prime.py	265	except Exception as e:	CODE
LOW	verifiers/v1/runtimes/modal.py	124	except Exception as e:	CODE
LOW⚡	verifiers/v1/runtimes/modal.py	171	except Exception as e:	CODE
LOW⚡	verifiers/v1/runtimes/modal.py	182	except Exception as e:	CODE
LOW	verifiers/v1/runtimes/modal.py	206	except Exception as e:	CODE
MEDIUM	verifiers/v1/runtimes/modal.py	74	def start(self) -> None:	CODE
LOW	verifiers/v1/mcp/launch.py	56	except Exception:	CODE
LOW	verifiers/v1/mcp/launch.py	459	except Exception as e:	CODE
LOW	verifiers/v1/cli/validate.py	105	except Exception as e:	CODE
LOW	verifiers/v1/cli/validate.py	110	except Exception:	CODE
LOW	verifiers/v1/cli/validate.py	140	except Exception as e:	CODE
LOW	verifiers/v1/cli/validate.py	145	except Exception:	CODE
LOW	verifiers/v1/cli/replay.py	95	except Exception:	CODE
LOW	verifiers/v1/cli/replay.py	157	except Exception as exc:	CODE
LOW	verifiers/v1/cli/debug.py	177	except Exception as e:	CODE
LOW	verifiers/v1/cli/debug.py	255	except Exception as e:	CODE
LOW	verifiers/v1/cli/debug.py	267	except Exception:	CODE
LOW	verifiers/v1/serve/pool.py	346	except Exception:	CODE
LOW	verifiers/clients/client.py	146	except Exception as e:	CODE
LOW	…ifiers/clients/openai_chat_completions_token_client.py	329	except Exception:	CODE
LOW	verifiers/utils/message_utils.py	432	except Exception:	CODE
LOW	verifiers/utils/version_utils.py	31	except Exception:	CODE
LOW	verifiers/utils/version_utils.py	42	except Exception:	CODE
LOW	verifiers/utils/eval_display.py	351	except Exception:	CODE
LOW	verifiers/utils/display_utils.py	107	except Exception:	CODE
LOW	verifiers/utils/interception_utils.py	253	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	307	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	342	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	355	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	371	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	389	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	400	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	421	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	439	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	453	except Exception as e:	CODE
LOW	verifiers/utils/interception_utils.py	478	except Exception as e:	CODE
175 more matches not shown…

AI Slop Vocabulary60 hits · 197 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/test_envs.py	50	# tasksets + the `compact` harness) are id-referenced and covered by tests/v1/test_envs.py.	COMMENT
MEDIUM⚡	tests/test_rlm_composable_env.py	175	# ── RLM harness ──────────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_rlm_composable_env.py	652	# ── RLM metrics via harness fields ──────────────────────────────────────	COMMENT
MEDIUM	tests/v1/conftest.py	18	uv run pytest tests/v1 -n auto -m default # only the default harness	STRING
MEDIUM⚡	tests/v1/conftest.py	43	# The harness runtime. Each value carries its runtime mark so a subset can be selected	COMMENT
MEDIUM⚡	tests/v1/conftest.py	46	# not modal"`. The `id`s make a test read like `<harness>-harness-in-<rt>` /	COMMENT
MEDIUM⚡	tests/v1/conftest.py	47	# `harness-in-<rt>-with-<user\|tool>-...`.	COMMENT
MEDIUM	tests/v1/conftest.py	63	# The user simulator's runtime: inside the harness's runtime (`colocated`) or its own runtime; this	COMMENT
MEDIUM⚡	tests/v1/conftest.py	106	# Harnesses, composed with the runtime fixtures, each carrying its harness mark (`-m default`, ...).	COMMENT
MEDIUM⚡	tests/v1/conftest.py	108	# install their dependencies at rollout. `compact` (an example harness) and `terminus-2` (drives	COMMENT
MEDIUM	tests/v1/test_e2e.py	29	# A user sim in a prime sandbox — its own, or colocated in a prime harness — must be reached by	COMMENT
MEDIUM	tests/v1/test_e2e.py	78	# the null harness offered the task's MCP tool as `echo_back`, schema included.	COMMENT
MEDIUM	verifiers/v1/harness.py	105	return # a @stop refused a turn mid-rollout; the harness's exit is expected	CODE
MEDIUM	verifiers/v1/env.py	92	# env-server subconfig the orchestrator writes would lose taskset/harness-specific knobs.	COMMENT
MEDIUM	verifiers/v1/env.py	158	# A taskset that bundles its own harness runs with it by default; an explicit	COMMENT
MEDIUM	verifiers/v1/env.py	159	# `--harness.id` / toml id (already on the field) takes precedence.	COMMENT
MEDIUM	verifiers/v1/env.py	259	# The warning is about the agent running arbitrary code on the host: every harness hands	COMMENT
MEDIUM	verifiers/v1/__init__.py	195	# taskset / harness / runtime / environment	COMMENT
MEDIUM	verifiers/v1/rollout.py	135	# Task setup and harness provisioning share one setup-stage deadline.	COMMENT
MEDIUM	verifiers/v1/rollout.py	155	# harness reaches the model at `{base_url}/v1`; tool/user servers reach this	COMMENT
MEDIUM	verifiers/v1/rollout.py	187	# Prefer an intercepted model/tool/user error to the harness exit it caused.	COMMENT
MEDIUM	verifiers/v1/clients/eval.py	32	# The harness uses this rollout secret to authenticate with the localhost server.	COMMENT
MEDIUM	verifiers/v1/clients/eval.py	101	# failure — map these to a retryable 502 so the harness SDK retries the	COMMENT
MEDIUM	verifiers/v1/clients/eval.py	166	# relay the provider's status (and body) so the harness SDK retries 5xx/429 and not	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	57	# Each session proxies one rollout's own harness requests, so aiohttp's default 1 MiB body	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	59	# and the harness gets a 413. Allow large bodies; the upstream provider and the model's	COMMENT
MEDIUM	verifiers/v1/interception/server.py	255	# `refused()` below halts the harness before any model call — no special-casing here.	COMMENT
MEDIUM	verifiers/v1/interception/server.py	278	# Refuse the first model call to halt the harness; once a simulated	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	300	# the harness (same shape as `refused` above).	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	310	# Stash the real cause; the rollout re-raises it after the harness returns. Relay	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	311	# the provider's status so the harness SDK retries 5xx/429 and not 4xx.	COMMENT
MEDIUM	verifiers/v1/interception/base.py	30	# bearer the harness/tool/user servers authenticate with. The harness reaches the model at	COMMENT
MEDIUM	verifiers/v1/harnesses/default/harness.py	14	# harness-injected prompt). The edit clause is appended only when the `edit` tool is enabled.	COMMENT
MEDIUM	verifiers/v1/harnesses/default/harness.py	78	# inheriting it via $SERPER_API_KEY / /proc/self/environ. Prefer a key set in the harness	COMMENT
MEDIUM	verifiers/v1/harnesses/default/harness.py	79	# env (--harness.env / forward_env); fall back to the host env only when the key is	COMMENT
MEDIUM	verifiers/v1/runtimes/modal.py	90	"infinity", # keep-alive entrypoint; the harness runs via `exec`	CODE
MEDIUM⚡	verifiers/v1/runtimes/base.py	22	# Ensure `uv` is available to run our PEP 723 scripts (the harness + tool servers): use it	COMMENT
MEDIUM	verifiers/v1/mcp/server.py	252	# These servers are reached by the harness through localhost or a tunnel, never a browser.	COMMENT
MEDIUM⚡	verifiers/v1/mcp/launch.py	275	# (`for_host`, always local, never colocated with it); a tool by the harness — colocated	COMMENT
MEDIUM⚡	verifiers/v1/mcp/launch.py	276	# when it shares the harness's runtime, reached with the harness's locality (read off the	COMMENT
MEDIUM⚡	verifiers/v1/mcp/launch.py	277	# harness runtime when there is one, else `harness_is_local` for an eval-level shared tool).	COMMENT
MEDIUM	verifiers/v1/mcp/launch.py	452	# Errors thrown into the yield belong to the harness body, not the connection.	COMMENT
MEDIUM	verifiers/v1/cli/init.py	238	uv run eval {dash} -n 3 # evaluate a few tasks with the default harness	STRING
MEDIUM	verifiers/v1/cli/dashboard/eval.py	208	# parsed as styling and dropped. `id` is in the `env` row; harness `runtime.type` too (hidden	COMMENT
MEDIUM	verifiers/v1/cli/dashboard/eval.py	209	# here), but only for the harness — `taskset.task.user.runtime.type` has no other display.	COMMENT
MEDIUM	verifiers/v1/dialects/chat.py	127	# `message_to_wire` (chat-only): used by `extend` (user-sim turn injection), the default harness	COMMENT
LOW	verifiers/gepa/adapter.py	187	# Empty prompt list - just add system message	COMMENT
MEDIUM	verifiers/utils/eval_display.py	958	"""Print a comprehensive summary after the display closes."""	STRING
MEDIUM	…rifiers/envs/experimental/composable/composable_env.py	63	# the harness ships up (e.g. for an agent checkout, .venv alone can dominate	COMMENT
MEDIUM	verifiers/envs/experimental/composable/harnesses/rlm.py	214	# surface at harness-build time, not per-rollout inside the closure.	STRING
MEDIUM	configs/gsm8k.toml	1	# gsm8k-v1 — grade-school math, default harness; a few tasks with several rollouts each.	COMMENT
MEDIUM	configs/textarena.toml	3	# WordSearch-v0. The framework's interception server plays the game; the harness never sees	COMMENT
MEDIUM	configs/textarena.toml	4	# it. The user sim runs colocated in the harness's runtime.	COMMENT
MEDIUM	configs/harbor.toml	2	# The `default` harness gives the agent the bash (+ edit) tools it drives the terminal with.	COMMENT
MEDIUM	configs/wiki_search_compact.toml	1	# wiki-search-v1 with the compact harness — the transcript is rewritten (branched) each turn,	COMMENT
MEDIUM⚡	configs/gsm8k_rlm.toml	1	# The gsm8k-v1 taskset run with the rlm harness, configured entirely from TOML.	COMMENT
MEDIUM⚡	configs/gsm8k_rlm.toml	5	# The discriminator ids (which taskset, which harness) come from this file's [taskset] /	COMMENT
MEDIUM⚡	configs/gsm8k_rlm.toml	6	# [harness] `id` — they select the schema, and the rest of the file fills the typed config,	COMMENT
MEDIUM	configs/glossary.toml	1	# glossary-v1 — a colocated MCP tool server (runs inside the harness's own runtime).	COMMENT
MEDIUM	.github/workflows/publish-envs.yml	25	# and the `compact` harness example) aren't hub-published yet.	COMMENT

Modern Structural Boilerplate179 hits · 184 pts

Severity	File	Line	Snippet	Context
LOW	environments/deepwiki_v1/deepwiki_v1/__init__.py	3	__all__ = ["DeepWikiTaskset"]	CODE
LOW	environments/compact/compact/__init__.py	6	__all__ = ["CompactingHarness", "CompactingHarnessConfig"]	CODE
LOW	environments/code_golf_v1/code_golf_v1/__init__.py	3	__all__ = ["CodeGolfTaskset"]	CODE
LOW	environments/glossary_v1/glossary_v1/__init__.py	3	__all__ = ["GlossaryTaskset"]	CODE
LOW	environments/wiki_search_v1/wiki_search_v1/__init__.py	3	__all__ = ["WikiSearchTaskset"]	CODE
LOW	…vironments/reverse_text_v1/reverse_text_v1/__init__.py	3	__all__ = ["ReverseTextTaskset"]	CODE
LOW	…nments/color_codeword_v1/color_codeword_v1/__init__.py	3	__all__ = ["ColorCodewordTaskset"]	CODE
LOW	environments/alphabet_sort/alphabet_sort.py	12	logger = logging.getLogger(__name__)	CODE
LOW	environments/gsm8k_v1/gsm8k_v1/__init__.py	3	__all__ = ["GSM8KTaskset"]	CODE
LOW	…ronments/alphabet_sort_v1/alphabet_sort_v1/__init__.py	3	__all__ = ["AlphabetSortTaskset"]	CODE
LOW	environments/scratchpad_v1/scratchpad_v1/__init__.py	3	__all__ = ["ScratchpadTaskset"]	CODE
LOW	environments/mmmu_v1/mmmu_v1/__init__.py	3	__all__ = ["MMMUTaskset"]	CODE
LOW	environments/wordle_v1/wordle_v1/__init__.py	3	__all__ = ["WordleTaskset"]	CODE
LOW	verifiers/__init__.py	53	__all__ = [	CODE
LOW⚡	verifiers/types.py	524	def _set_internal(self, key: str, value: Any) -> None:	CODE
LOW⚡	verifiers/types.py	529	def _set_completed(self, value: bool = True) -> None:	CODE
LOW⚡	verifiers/types.py	532	def _set_error(self, value: Error \| None) -> None:	CODE
LOW	verifiers/types.py	543	def _set_truncated(self, value: bool = True, *, overwrite: bool = False) -> None:	CODE
LOW	verifiers/types.py	710	def _set_timing_total(timing: dict[str, Any], end_time: float) -> None:	CODE
LOW	verifiers/v1/retries.py	33	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/trace.py	34	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/task.py	67	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/harness.py	24	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/env.py	176	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/legacy.py	50	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/taskset.py	47	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/session.py	24	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/__init__.py	135	__all__ = [	CODE
LOW	verifiers/v1/rollout.py	35	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/push.py	22	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/clients/client.py	12	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/clients/__init__.py	14	__all__ = [	CODE
LOW⚡	verifiers/v1/interception/server.py	54	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/interception/__init__.py	81	__all__ = [	CODE
LOW	verifiers/v1/interception/pool.py	31	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/interception/tunnel/__init__.py	31	__all__ = [	CODE
LOW	verifiers/v1/gepa/runner.py	27	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/gepa/__init__.py	12	__all__ = ["GEPAAdapter", "GEPAConfig", "run_gepa"]	CODE
LOW	verifiers/v1/harnesses/__init__.py	12	__all__ = [	CODE
LOW⚡	verifiers/v1/harnesses/kimi_code/harness.py	12	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/harnesses/kimi_code/__init__.py	6	__all__ = ["KimiCodeHarness", "KimiCodeHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/codex/harness.py	17	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/harnesses/codex/__init__.py	3	__all__ = ["CodexHarness", "CodexHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/default/__init__.py	6	__all__ = ["DefaultHarness", "DefaultHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/null/__init__.py	3	__all__ = ["NullHarness", "NullHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/rlm/harness.py	17	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/harnesses/rlm/__init__.py	3	__all__ = ["RLMHarness", "RLMHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/mini_swe_agent/__init__.py	6	__all__ = ["MiniSWEAgentHarness", "MiniSWEAgentHarnessConfig"]	CODE
LOW	verifiers/v1/harnesses/terminus_2/harness.py	10	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/harnesses/terminus_2/__init__.py	6	__all__ = ["Terminus2Harness", "Terminus2HarnessConfig"]	CODE
LOW	verifiers/v1/runtimes/prime.py	31	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/runtimes/__init__.py	56	__all__ = [	CODE
LOW	verifiers/v1/runtimes/docker.py	21	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/runtimes/modal.py	28	logger = logging.getLogger(__name__)	CODE
LOW⚡	verifiers/v1/runtimes/base.py	20	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/utils/interrupt.py	9	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/utils/install.py	13	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/mcp/server.py	22	logger = logging.getLogger(__name__)	CODE
LOW	verifiers/v1/mcp/server.py	246	async def _setup() -> None:	CODE
LOW	verifiers/v1/mcp/__init__.py	16	__all__ = [	CODE
119 more matches not shown…

Deep Nesting164 hits · 140 pts

Severity	File	Line	Context
LOW	…ronments/wiki_search_v1/wiki_search_v1/servers/wiki.py	77	CODE
LOW	environments/wiki_search/wiki_search.py	27	CODE
LOW	environments/wiki_search/wiki_search.py	186	CODE
LOW	environments/alphabet_sort/alphabet_sort.py	39	CODE
LOW	environments/alphabet_sort/alphabet_sort.py	50	CODE
LOW	…ironments/alphabet_sort_v1/alphabet_sort_v1/taskset.py	113	CODE
LOW	environments/mmmu_v1/mmmu_v1/taskset.py	102	CODE
LOW	tests/conftest.py	313	CODE
LOW	tests/conftest.py	325	CODE
LOW	tests/test_renderer_client.py	631	CODE
LOW	tests/test_gym_env.py	71	CODE
LOW	verifiers/v1/legacy.py	112	CODE
LOW	verifiers/v1/scoring.py	142	CODE
LOW	verifiers/v1/interception/server.py	372	CODE
LOW	verifiers/v1/gepa/adapter.py	83	CODE
LOW	verifiers/v1/harnesses/codex/harness.py	66	CODE
LOW	verifiers/v1/harnesses/default/program.py	254	CODE
LOW	verifiers/v1/utils/generic.py	8	CODE
LOW	verifiers/v1/cli/dashboard/eval.py	417	CODE
LOW	verifiers/v1/cli/dashboard/base.py	81	CODE
LOW	verifiers/v1/cli/eval/resume.py	65	CODE
LOW	verifiers/v1/tasksets/lean/scoring.py	102	CODE
LOW	verifiers/v1/dialects/responses.py	182	CODE
LOW	verifiers/v1/dialects/responses.py	277	CODE
LOW	verifiers/v1/dialects/chat.py	217	CODE
LOW	verifiers/v1/dialects/anthropic.py	44	CODE
LOW	verifiers/v1/dialects/anthropic.py	120	CODE
LOW	verifiers/v1/dialects/anthropic.py	185	CODE
LOW	verifiers/v1/serve/server.py	143	CODE
LOW	verifiers/v1/serve/pool.py	169	CODE
LOW	verifiers/clients/openai_responses_client.py	266	CODE
LOW	verifiers/clients/openai_responses_client.py	313	CODE
LOW	verifiers/clients/openai_responses_client.py	326	CODE
LOW	verifiers/clients/anthropic_messages_client.py	102	CODE
LOW	verifiers/clients/anthropic_messages_client.py	380	CODE
LOW	verifiers/clients/anthropic_messages_client.py	405	CODE
LOW	verifiers/clients/anthropic_messages_client.py	105	CODE
LOW	verifiers/clients/anthropic_messages_client.py	167	CODE
LOW	verifiers/clients/anthropic_messages_client.py	228	CODE
LOW	verifiers/clients/anthropic_messages_client.py	406	CODE
LOW	verifiers/clients/openai_completions_client.py	49	CODE
LOW	verifiers/clients/client.py	106	CODE
LOW	verifiers/clients/openai_chat_completions_client.py	109	CODE
LOW	verifiers/clients/openai_chat_completions_client.py	189	CODE
LOW	verifiers/clients/openai_chat_completions_client.py	271	CODE
LOW	verifiers/clients/openai_chat_completions_client.py	204	CODE
LOW	…ifiers/clients/openai_chat_completions_token_client.py	27	CODE
LOW	verifiers/clients/renderer_client.py	150	CODE
LOW	verifiers/clients/renderer_client.py	187	CODE
LOW	verifiers/parsers/xml_parser.py	10	CODE
LOW	verifiers/parsers/xml_parser.py	57	CODE
LOW	verifiers/parsers/xml_parser.py	128	CODE
LOW	verifiers/parsers/xml_parser.py	249	CODE
LOW	verifiers/parsers/xml_parser.py	138	CODE
LOW	verifiers/parsers/parser.py	28	CODE
LOW	verifiers/gepa/adapter.py	169	CODE
LOW	verifiers/gepa/display.py	130	CODE
LOW	verifiers/gepa/display.py	154	CODE
LOW	verifiers/gepa/display.py	232	CODE
LOW	verifiers/gepa/gepa_utils.py	117	CODE
104 more matches not shown…

Cross-File Repetition19 hits · 95 pts

Severity	File	Snippet	Context
HIGH	environments/AGENTS.md	question: {question} answer: {answer} response: {response} correct? reply yes or no.	STRING
HIGH	docs/v1/environments.md	question: {question} answer: {answer} response: {response} correct? reply yes or no.	STRING
HIGH	assets/lab/environments/AGENTS.md	question: {question} answer: {answer} response: {response} correct? reply yes or no.	STRING
HIGH	environments/compact/compact/program.py	connect to each configured mcp server (a streamable-http `url`); return (tool schemas, dispatch mapping `<server>_<tool>	STRING
HIGH	verifiers/v1/harnesses/default/program.py	connect to each configured mcp server (a streamable-http `url`); return (tool schemas, dispatch mapping `<server>_<tool>	STRING
HIGH	verifiers/v1/harnesses/null/program.py	connect to each configured mcp server (a streamable-http `url`); return (tool schemas, dispatch mapping `<server>_<tool>	STRING
HIGH	verifiers/v1/gepa/config.py	resolve + validate the config and dump it, then exit.	STRING
HIGH	verifiers/v1/configs/serve.py	resolve + validate the config and dump it, then exit.	STRING
HIGH	verifiers/v1/configs/eval.py	resolve + validate the config and dump it, then exit.	STRING
HIGH	…envs/experimental/composable/tasksets/harbor/harbor.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…erimental/composable/tasksets/swe/swe_bench/taskset.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…xperimental/composable/tasksets/swe/openswe/taskset.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…erimental/composable/tasksets/swe/multi_swe/taskset.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…perimental/composable/tasksets/swe/swe_lego/taskset.py	apply gold patch, run tests, and check if reward > 0. exceptions propagate to the caller (``taskset.validate``) so that	STRING
HIGH	…perimental/composable/tasksets/swe/swe_lego/taskset.py	args: filter_fn: optional python expression string forwarded to :class:`taskset` — see its docstring. applied to post-``	STRING
HIGH	…erimental/composable/tasksets/swe/swe_bench/taskset.py	args: filter_fn: optional python expression string forwarded to :class:`taskset` — see its docstring. applied to post-``	STRING
HIGH	…xperimental/composable/tasksets/swe/openswe/taskset.py	args: filter_fn: optional python expression string forwarded to :class:`taskset` — see its docstring. applied to post-``	STRING
HIGH	…erimental/composable/tasksets/swe/multi_swe/taskset.py	args: filter_fn: optional python expression string forwarded to :class:`taskset` — see its docstring. applied to post-``	STRING

AI Structural Patterns45 hits · 44 pts

Severity	File	Line	Context
LOW	environments/math_python/math_python.py	5	CODE
LOW	environments/wiki_search/wiki_search.py	27	CODE
LOW	environments/alphabet_sort/alphabet_sort.py	193	CODE
LOW	environments/browser_cua_example/browser_cua_example.py	118	CODE
LOW	tests/conftest.py	479	CODE
LOW	tests/conftest.py	541	CODE
LOW	tests/v1/conftest.py	161	CODE
LOW	verifiers/clients/openai_completions_client.py	96	CODE
LOW	verifiers/gepa/display.py	73	CODE
LOW	verifiers/utils/eval_display.py	249	CODE
LOW	verifiers/utils/display_utils.py	555	CODE
LOW	verifiers/rubrics/experimental/hybrid_math_rubric.py	102	CODE
LOW	verifiers/serve/server/env_server.py	28	CODE
LOW	verifiers/serve/server/env_router.py	94	CODE
LOW	verifiers/envs/sandbox_env.py	56	CODE
LOW	verifiers/envs/environment.py	97	CODE
LOW	verifiers/envs/environment.py	767	CODE
LOW	verifiers/envs/environment.py	821	CODE
LOW	verifiers/envs/environment.py	1218	CODE
LOW	verifiers/envs/environment.py	1276	CODE
LOW	verifiers/envs/environment.py	1363	CODE
LOW	verifiers/envs/experimental/cli_agent_env.py	92	CODE
LOW	verifiers/envs/experimental/opencode_rlm_env.py	178	CODE
LOW	verifiers/envs/experimental/opencode_env.py	171	CODE
LOW	verifiers/envs/experimental/sandbox_mixin.py	165	CODE
LOW	verifiers/envs/experimental/gym_env.py	56	CODE
LOW	…iers/envs/experimental/composable/sandbox_debug_env.py	44	CODE
LOW	verifiers/envs/experimental/composable/harnesses/rlm.py	127	CODE
LOW	…vs/experimental/composable/harnesses/mini_swe_agent.py	82	CODE
LOW	…vs/experimental/composable/harnesses/mini_swe_agent.py	170	CODE
LOW	…ers/envs/experimental/composable/harnesses/opencode.py	90	CODE
LOW	…ers/envs/experimental/composable/harnesses/opencode.py	149	CODE
LOW	…ers/envs/experimental/composable/harnesses/opencode.py	230	CODE
LOW	…xperimental/composable/tasksets/swe/r2e_gym/taskset.py	175	CODE
LOW	…nvs/experimental/composable/tasksets/math/math_task.py	36	CODE
LOW	…ental/composable/tasksets/search/openseeker/taskset.py	279	CODE
LOW	…ntal/composable/tasksets/search/redsearcher/taskset.py	271	CODE
LOW	…perimental/composable/tasksets/search/quest/taskset.py	382	CODE
LOW	…perimental/composable/tasksets/search/quest/taskset.py	543	CODE
LOW	…rs/envs/experimental/composable/tasksets/cp/cp_task.py	141	CODE
LOW	verifiers/envs/integrations/textarena_env.py	42	CODE
LOW	verifiers/envs/integrations/openenv_env.py	92	CODE
LOW	verifiers/envs/integrations/openenv_env.py	167	CODE
LOW	verifiers/envs/integrations/browser_env/browser_env.py	48	CODE
LOW	…ifiers/envs/integrations/browser_env/modes/cua_mode.py	67	CODE

Modern AI Meta-Vocabulary9 hits · 29 pts

Severity	File	Line	Snippet	Context
MEDIUM⚡	tests/v1/conftest.py	39	# Fixture tasksets/envs (echo-v1, echo-agentic-v1, echo-v0, echo-multi-v0) live in	COMMENT
MEDIUM	verifiers/v1/graph.py	501	# and its re-rendered input form place the turn-close scaffold in different nodes but at the	COMMENT
MEDIUM	verifiers/v1/graph.py	554	# Assistant node: trailing scaffold (the generation prompt) + the sampled completion.	COMMENT
MEDIUM	verifiers/v1/clients/eval.py	75	# No timeout: agentic completions are slow and the rollout timeout is the real backstop.	COMMENT
MEDIUM⚡	verifiers/v1/interception/server.py	60	# context window are the real limits, this is just a host-OOM backstop.	COMMENT
MEDIUM	verifiers/v1/harnesses/codex/harness.py	146	# Preserve any user-supplied multi-agent v2 tool and limit settings.	COMMENT
MEDIUM	verifiers/v1/judges/rubric.py	30	# criterion carries a one-sentence `reason` written before the verdict (chain-of-thought), so the	COMMENT
MEDIUM	…le/tasksets/search/quest/obj_task_eval/eval_toolkit.py	936	# Truncate message_content if it exceeds the model context window.truncate.	COMMENT
MEDIUM	docs/v0/training.md	108	Our [`prime-rl`](https://github.com/PrimeIntellect-ai/prime-rl) trainer is a production-ready async RL training framewor	CODE

Self-Referential Comments7 hits · 22 pts

Severity	File	Line	Snippet	Context
MEDIUM	…ironments/toxicity_explanation/toxicity_explanation.py	105	# Create the JudgeRubric	COMMENT
MEDIUM	tests/test_stateful_tool_env.py	166	# Create a tool call with invalid JSON arguments	COMMENT
MEDIUM	tests/test_tool_env.py	158	# Create a tool call with invalid JSON arguments	COMMENT
MEDIUM	tests/test_cli_agent_env.py	404	# Create a second task	COMMENT
MEDIUM	tests/test_environment.py	25	# Create a concrete implementation for testing the abstract base class	COMMENT
MEDIUM⚡	verifiers/v1/runtimes/modal.py	175	# Create the parent first (Modal's write does not mkdir).	COMMENT
MEDIUM	…/composable/tasksets/swe/swe_rebench_v2/log_parsers.py	2892	# Create a unique key for the test	COMMENT

Over-Commented Block23 hits · 22 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_renderer_client.py	601	# The anchor loop in _get_incremental_prompt_ids used to skip every	COMMENT
LOW	verifiers/v1/__init__.py	261	# The library logs via stdlib logging (per-module `getLogger(__name__)`), but is	COMMENT
LOW	verifiers/v1/types.py	201	completion_logprobs: list[float] = Field(default_factory=list)	COMMENT
LOW	verifiers/utils/save_utils.py	301	# add state columns (must be serializable)	COMMENT
LOW	…/composable/tasksets/swe/swe_rebench_v2/log_parsers.py	1	# ruff: noqa	COMMENT
LOW	configs/gepa/nemotron-3.toml	1	# Nemotron 3 models. Update both fields when switching sizes.	COMMENT
LOW	configs/gepa/qwen-3-5.toml	1	# Qwen3.5 dense models. Update both fields when switching sizes.	COMMENT
LOW	configs/gepa/gpt-oss.toml	1	# gpt-oss models. Update both fields when switching sizes.	COMMENT
LOW	configs/gepa/llama-3.toml	1	# Llama 3.2 models. Update both fields when switching sizes.	COMMENT
LOW	configs/gepa/qwen-3-5-moe.toml	1	# Qwen3.5 MoE models. Update both fields when switching sizes.	COMMENT
LOW	configs/eval/nemotron-3.toml	21	# [[eval]]	COMMENT
LOW	configs/eval/qwen-3-5.toml	21	# env_id = "primeintellect/dspy-flights"	COMMENT
LOW	configs/eval/gpt-oss.toml	21	# [[eval]]	COMMENT
LOW	configs/eval/llama-3.toml	21	# [[eval]]	COMMENT
LOW	configs/rl/qwen.toml	21		COMMENT
LOW	configs/rl/qwen-moe.toml	21		COMMENT
LOW	configs/rl/gpt-oss.toml	21	# id = "primeintellect/wiki-search"	COMMENT
LOW	configs/rl/llama.toml	21		COMMENT
LOW	configs/rl/nemotron.toml	21	# id = "primeintellect/wiki-search"	COMMENT
LOW	assets/templates/browserbase/cua/setup.sh	1	#!/bin/bash	COMMENT
LOW	assets/templates/browserbase/cua/setup-binary.sh	1	#!/bin/bash	COMMENT
LOW	assets/templates/browserbase/cua/build-and-push.sh	1	#!/bin/bash	COMMENT
LOW	…sets/templates/browserbase/cua/scripts/build-binary.sh	1	#!/bin/bash	COMMENT

Redundant / Tautological Comments15 hits · 22 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_imports.py	39	# Check if this is an expected optional dependency error	COMMENT
LOW	verifiers/parsers/xml_parser.py	155	# Check if the message has at least one valid field	COMMENT
LOW	verifiers/parsers/xml_parser.py	180	# Check if field exists in non-stripped version too (proper spacing)	COMMENT
LOW	verifiers/parsers/xml_parser.py	201	# Check if any field from the first field set starts the message	COMMENT
LOW	verifiers/parsers/xml_parser.py	211	# Check if any field from the last field set ends the message	COMMENT
LOW	verifiers/gepa/display.py	173	# Check if this is a valset eval by matching example_ids	COMMENT
LOW	verifiers/utils/display_utils.py	451	# Check if more chars follow (escape sequence vs standalone Esc)	COMMENT
LOW	verifiers/envs/environment.py	320	# Check if a system message already exists (first message)	COMMENT
LOW	…/composable/tasksets/swe/swe_rebench_v2/log_parsers.py	2497	# Check if Pass column has a value (look for leading spaces pattern)	COMMENT
LOW	…/composable/tasksets/swe/swe_rebench_v2/log_parsers.py	2907	# Check if counters are consecutive (indicating pass)	COMMENT
LOW	…/composable/tasksets/swe/swe_rebench_v2/log_parsers.py	3121	# Check if this is a test line (starts with "- ")	COMMENT
LOW	…le/tasksets/search/quest/obj_task_eval/eval_toolkit.py	896	# Check if cancellation has occurred	COMMENT
LOW	…sable/tasksets/search/quest/obj_task_eval/evaluator.py	930	# Check if there are failed preceding conditions	COMMENT
LOW	verifiers/envs/experimental/utils/git_checkout_cache.py	20	# Open file handles that hold a process-lifetime shared lock on each	COMMENT
LOW	.github/workflows/publish-envs.yml	74	# Check if this is just an unchanged content hash (not a real failure)	COMMENT

Verbosity Indicators9 hits · 19 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_save_utils.py	1208	# Step 2: prior was ["A"], current is ["A", "A"]. Multiset budget	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	31	# Step 1: Login to Docker Hub	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	36	# Step 2: Build the binary	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	41	# Step 3: Extract binary from builder	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	55	# Step 4: Build runtime image	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	60	# Step 5: Push to Docker Hub	COMMENT
LOW	…sets/templates/browserbase/cua/scripts/build-binary.sh	27	# Step 1: Bundle with esbuild	COMMENT
LOW⚡	…sets/templates/browserbase/cua/scripts/build-binary.sh	40	# Step 2: Generate SEA blob	COMMENT
LOW⚡	…sets/templates/browserbase/cua/scripts/build-binary.sh	46	# Step 3: Create the binary	COMMENT

Structural Annotation Overuse9 hits · 19 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_save_utils.py	1208	# Step 2: prior was ["A"], current is ["A", "A"]. Multiset budget	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	31	# Step 1: Login to Docker Hub	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	36	# Step 2: Build the binary	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	41	# Step 3: Extract binary from builder	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	55	# Step 4: Build runtime image	COMMENT
LOW⚡	assets/templates/browserbase/cua/build-and-push.sh	60	# Step 5: Push to Docker Hub	COMMENT
LOW	…sets/templates/browserbase/cua/scripts/build-binary.sh	27	# Step 1: Bundle with esbuild	COMMENT
LOW⚡	…sets/templates/browserbase/cua/scripts/build-binary.sh	40	# Step 2: Generate SEA blob	COMMENT
LOW⚡	…sets/templates/browserbase/cua/scripts/build-binary.sh	46	# Step 3: Create the binary	COMMENT

Magic Placeholder Names3 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	environments/browser_dom_example/README.md	33	export BROWSERBASE_API_KEY="your-api-key"	CODE
HIGH	environments/browser_cua_example/README.md	34	export BROWSERBASE_API_KEY="your-api-key"	CODE
HIGH	verifiers/envs/integrations/browser_env/README.md	46	export BROWSERBASE_API_KEY="your-api-key"	CODE

Docstring Block Structure3 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	verifiers/utils/save_utils.py	200	Convert a State to a serializable RolloutOutput. Args: state: The State object to convert. state_co	STRING
HIGH	verifiers/utils/install_utils.py	27	Parse environment ID into (owner, name, version). Args: env_id: Environment ID like 'owner/name' or 'owner/	STRING
HIGH	…sable/tasksets/search/quest/obj_task_eval/evaluator.py	477	Add custom judgment node - directly pass judgment result Args: result: Judgment result (Tr	STRING

Hallucination Indicators1 hit · 10 pts

Severity	File	Line	Snippet	Context
CRITICAL	…perimental/composable/tasksets/search/quest/taskset.py	183	response = await self._client.beta.chat.completions.parse(	CODE

Slop Phrases1 hit · 2 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/v1/test_judges.py	440	response = JudgeResponse(text="as an AI language model I cannot grade this")	CODE

Example Usage Blocks1 hit · 2 pts

Severity	File	Line	Snippet	Context
LOW	assets/templates/browserbase/cua/build-and-push.sh	9	# Usage:	COMMENT

Overly Generic Function Names1 hit · 1 pts

Severity	File	Line	Snippet	Context
LOW	verifiers/v1/interception/server.py	214	async def handle_request(	CODE

Analysis Overview

What These Metrics Mean

Score History

Severity Breakdown

Directory Score Breakdown

Pattern Findings