HolmesGPT/holmesgpt

Pattern Findings

The scanner identified 4855 distinct pattern matches across 24 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers3135 hits · 3024 pts

Severity	File	Line	Snippet	Context
LOW	conftest.py	306	def pytest_collection_modifyitems(config, items):	CODE
LOW	server.py	159	def open_experiment_from_request(http_request: Request) -> None:	CODE
LOW	server.py	211	def _warm_skill_repos_and_mirror():	CODE
LOW	server.py	280	def _get_next_refresh_interval(	CODE
LOW	server.py	294	def _toolset_status_refresh_loop():	CODE
LOW	server.py	427	def stop_conversation_runtime():	CODE
LOW	server.py	538	def extract_passthrough_headers(request: Request) -> dict:	CODE
LOW	server.py	566	def _stream_with_storage_cleanup(storage, stream_generator, req_info):	CODE
LOW	server.py	575	def _stream_with_trace_cleanup(storage, stream_generator, req_info, trace_span):	CODE
LOW	experimental/ag-ui/server-agui.py	310	def _should_execute_suggested_query(	CODE
LOW	experimental/ag-ui/server-agui.py	334	def _should_graph_timeseries_data(tool_name: str) -> bool:	CODE
LOW	experimental/ag-ui/server-agui.py	424	async def _stream_agui_text_message_event(message: str):	CODE
LOW	experimental/ag-ui/server-agui.py	439	def _agui_input_to_holmes_chat_request(input_data: RunAgentInput) -> ChatRequest:	CODE
LOW⚡	tests/test_server_ssl.py	35	def test_no_tls_config_returns_empty(monkeypatch):	CODE
LOW⚡	tests/test_server_ssl.py	39	def test_only_certfile_fails_fast(monkeypatch, cert_and_key):	CODE
LOW⚡	tests/test_server_ssl.py	46	def test_only_keyfile_fails_fast(monkeypatch, cert_and_key):	CODE
LOW⚡	tests/test_server_ssl.py	53	def test_missing_cert_file_fails_fast(monkeypatch, tmp_path, cert_and_key):	CODE
LOW	tests/test_server_ssl.py	64	def test_unreadable_cert_file_fails_fast(monkeypatch, cert_and_key):	CODE
LOW	tests/test_server_ssl.py	82	def test_keyfile_password_included(monkeypatch, cert_and_key):	CODE
LOW	tests/test_server_ssl.py	91	def test_mtls_ca_certs_enables_client_verification(monkeypatch, tmp_path, cert_and_key):	CODE
LOW⚡	tests/test_server_ssl.py	103	def test_missing_ca_certs_file_fails_fast(monkeypatch, tmp_path, cert_and_key):	CODE
LOW⚡	tests/test_server_ssl.py	112	def test_ca_certs_without_cert_and_key_fails_fast(monkeypatch, tmp_path):	CODE
LOW⚡	tests/test_server_ssl.py	121	def test_keyfile_password_without_cert_and_key_fails_fast(monkeypatch):	CODE
LOW	tests/test_robusta_client.py	39	def test_returns_models_on_first_success(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	50	def test_recovers_from_transient_gateway_errors(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	62	def test_gives_up_after_max_attempts(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	73	def test_does_not_retry_client_errors(mocked_responses, status):	CODE
LOW	tests/test_robusta_client.py	84	def test_retries_connection_errors(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	106	def test_retries_rate_limiting(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	116	def test_parses_an_opted_out_account(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	140	def test_parses_an_enabled_account(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	150	def test_ignores_the_envelope_fields_holmes_does_not_read(mocked_responses):	CODE
LOW	tests/test_robusta_client.py	162	def test_ignores_unknown_response_fields(mocked_responses):	CODE
LOW	tests/test_mcp_refresh_backoff.py	6	def test_backoff_schedule_when_mcp_failed(self):	CODE
LOW⚡	tests/test_mcp_refresh_backoff.py	20	def test_falls_back_to_default_after_schedule_exhausted(self):	CODE
LOW⚡	tests/test_mcp_refresh_backoff.py	29	def test_default_interval_when_no_failures(self):	CODE
LOW⚡	tests/test_mcp_refresh_backoff.py	35	def test_resets_backoff_when_mcp_recovers(self):	CODE
LOW	tests/test_mcp_refresh_backoff.py	47	def test_backoff_restarts_on_new_failure(self):	CODE
LOW⚡	tests/test_header_propagation.py	42	def test_request_context_header(self):	CODE
LOW⚡	tests/test_header_propagation.py	50	def test_case_insensitive_request_context(self):	CODE
LOW⚡	tests/test_header_propagation.py	58	def test_missing_header_renders_empty(self):	CODE
LOW⚡	tests/test_header_propagation.py	96	def test_command_renders_request_context_header(self):	CODE
LOW⚡	tests/test_header_propagation.py	181	def test_model_dump_redacts_request_context_headers(self):	CODE
LOW⚡	tests/test_header_propagation.py	204	def test_extra_headers_merged_into_request(self, mock_request):	CODE
LOW⚡	tests/test_header_propagation.py	298	def test_config_level_extra_headers_rendered(self):	CODE
LOW⚡	tests/test_header_propagation.py	512	def test_documented_curl_pattern_with_real_token(self):	CODE
LOW	tests/test_header_propagation.py	112	def test_command_renders_env_var(self, monkeypatch):	CODE
LOW	tests/test_header_propagation.py	128	def test_command_works_without_request_context(self):	CODE
LOW	tests/test_header_propagation.py	143	def test_script_renders_request_context_header(self):	CODE
LOW	tests/test_header_propagation.py	159	def test_command_renders_case_insensitive_header(self):	CODE
LOW	tests/test_header_propagation.py	249	def test_extra_headers_override_defaults(self, mock_request):	CODE
LOW	tests/test_header_propagation.py	316	def test_extra_headers_with_request_context(self):	CODE
LOW	tests/test_header_propagation.py	337	def test_static_headers_and_extra_headers_merged(self):	CODE
LOW	tests/test_header_propagation.py	357	def test_extra_headers_override_static_headers(self):	CODE
LOW	tests/test_header_propagation.py	396	def test_command_header_metacharacters_are_rejected(self, payload):	CODE
LOW	tests/test_header_propagation.py	414	def test_script_header_metacharacters_are_rejected(self, payload):	CODE
LOW	tests/test_header_propagation.py	428	def test_documented_curl_auth_token_example_is_safe(self):	CODE
LOW	tests/test_header_propagation.py	448	def test_top_level_request_context_value_is_rejected(self):	CODE
LOW	tests/test_header_propagation.py	461	def test_build_context_raises_on_metacharacter_header(self):	CODE
LOW	tests/test_header_propagation.py	474	def test_benign_header_with_whitespace_is_allowed(self):	CODE
3075 more matches not shown…

Decorative Section Separators283 hits · 1025 pts

Severity	File	Line	Snippet	Context
MEDIUM⚡	tests/test_header_propagation.py	91	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	93	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	176	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	178	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	198	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	200	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	293	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	295	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	499	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_header_propagation.py	502	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_header_propagation.py	26	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_header_propagation.py	28	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_header_propagation.py	377	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_header_propagation.py	379	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_check_prerequisites.py	400	# ============================================================================	COMMENT
MEDIUM	tests/test_check_prerequisites.py	402	# ============================================================================	COMMENT
MEDIUM⚡	tests/test_bash_session_prefix_forgery.py	167	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_bash_session_prefix_forgery.py	170	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	9	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	11	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	17	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	19	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	75	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	77	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	169	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	171	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	219	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	221	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	270	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_json_schema_coerce.py	272	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_json_schema_coerce.py	109	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	tests/test_json_schema_coerce.py	111	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	174	# ── build_tree_from_schema ────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	374	# ── save_config_to_file ──────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	488	# ── run_config_test ───────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	538	# ── select_toolset ────────────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	548	# ── _get_existing_config ──────────────────────────────────────────────	COMMENT
MEDIUM⚡	tests/test_toolset_config_tui.py	702	# ── Integration: real Pydantic configs ────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	40	# ── Fixtures ──────────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	133	# ── _resolve_primitive_type ───────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	274	# ── _flatten_tree ─────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	304	# ── tree_to_dict ──────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	579	# ── ConfigFieldNode manipulation ──────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	811	# ── Enum support ─────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	922	# ── MCP save/load ────────────────────────────────────────────────────	COMMENT
MEDIUM	tests/test_toolset_config_tui.py	1007	# ── Per-class config cache (mode cycling) ────────────────────────────	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	431	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	433	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	471	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	473	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	537	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	539	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	592	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	594	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	634	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	636	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	804	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	806	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	874	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	tests/test_tool_calling_llm.py	876	# ---------------------------------------------------------------------------	COMMENT
223 more matches not shown…

Excessive Try-Catch Wrapping498 hits · 490 pts

Severity	File	Line	Snippet	Context
LOW	conftest.py	133	except Exception as e:	CODE
MEDIUM	server.py	990	def readiness_check():	CODE
LOW⚡	server.py	245	except Exception:	CODE
LOW⚡	server.py	249	except Exception:	CODE
LOW⚡	server.py	254	except Exception:	CODE
LOW⚡	server.py	261	except Exception:	CODE
LOW	server.py	222	except Exception:	CODE
LOW	server.py	228	except Exception:	CODE
LOW	server.py	325	except Exception:	CODE
LOW	server.py	334	except Exception:	CODE
LOW	server.py	347	except Exception:	CODE
LOW	server.py	367	except Exception:	CODE
LOW	server.py	378	except Exception:	CODE
LOW	server.py	388	except Exception:	CODE
LOW	server.py	441	except Exception:	CODE
LOW	server.py	523	except Exception as e:	CODE
LOW	server.py	852	except Exception as e:	CODE
LOW	server.py	874	except Exception as e:	CODE
LOW	server.py	994	except Exception as e:	CODE
LOW	tempo_cli.py	95	except Exception as e:	CODE
LOW	tempo_cli.py	124	except Exception as e:	CODE
LOW	tempo_cli.py	171	except Exception as e:	CODE
LOW	tempo_cli.py	206	except Exception as e:	CODE
LOW	tempo_cli.py	253	except Exception as e:	CODE
LOW	tempo_cli.py	298	except Exception as e:	CODE
LOW	tempo_cli.py	330	except Exception as e:	CODE
LOW	tempo_cli.py	370	except Exception as e:	CODE
LOW	experimental/ag-ui/server-agui.py	280	except Exception as e:	CODE
LOW	experimental/ag-ui/server-agui.py	393	except Exception as e:	CODE
MEDIUM	experimental/ag-ui/server-agui.py	138	def event_generator(message_history):	CODE
MEDIUM	experimental/ag-ui/server-agui.py	342	def _parse_timeseries_data(data) -> dict:	CODE
LOW	tests/conftest.py	54	except Exception:	CODE
MEDIUM	tests/test_mcp_toolset.py	2272	print(f"Error: {result[1]}")	CODE
LOW	tests/test_mcp_toolset.py	122	except Exception as e:	CODE
LOW	tests/test_mcp_toolset.py	2327	except Exception as e:	CODE
LOW	tests/test_mcp_oauth.py	1361	except Exception:	CODE
LOW	tests/test_mcp_oauth.py	1405	except Exception:	CODE
LOW	tests/test_mcp_oauth.py	1449	except Exception:	CODE
LOW	tests/test_mcp_oauth.py	1502	except Exception:	CODE
LOW	tests/llm/conftest.py	122	except Exception as e: # pragma: no cover - defensive error propagation	CODE
LOW	tests/llm/conftest.py	314	except Exception as e:	CODE
LOW	tests/llm/conftest.py	409	except Exception:	CODE
LOW	tests/llm/conftest.py	472	except Exception as e:	CODE
LOW	tests/llm/conftest.py	501	except Exception as e:	CODE
LOW	tests/llm/test_holmes_checks.py	230	except Exception as e:	CODE
LOW	tests/llm/test_holmes_checks.py	241	except Exception:	CODE
LOW	tests/llm/test_compaction.py	243	except Exception as e:	CODE
LOW	tests/llm/test_ask_holmes.py	141	except Exception as e:	CODE
LOW	tests/llm/test_ask_holmes.py	387	except Exception as e:	CODE
LOW	tests/llm/test_ask_holmes.py	533	except Exception as e:	CODE
LOW	tests/llm/test_ask_holmes.py	642	except Exception as e:	CODE
LOW	tests/llm/utils/property_manager.py	330	except Exception:	CODE
LOW	tests/llm/utils/mock_dal.py	65	except Exception as e:	CODE
LOW⚡	tests/llm/utils/mock_dal.py	78	except Exception as e:	CODE
LOW⚡	tests/llm/utils/mock_dal.py	92	except Exception as e:	CODE
LOW	tests/llm/utils/port_forward.py	62	except Exception as e:	CODE
LOW	tests/llm/utils/port_forward.py	111	except Exception as e:	CODE
LOW	tests/llm/utils/port_forward.py	119	except Exception:	CODE
LOW	tests/llm/utils/port_forward.py	389	except Exception as e:	CODE
LOW	tests/llm/utils/braintrust.py	210	except Exception:	CODE
438 more matches not shown…

Empty Error Handlers59 hits · 295 pts

Severity	File	Line	Snippet	Context
HIGH	conftest.py	98	except ImportError:	CODE
HIGH	conftest.py	114	except ImportError:	CODE
HIGH	tests/conftest.py	54	except Exception:	CODE
HIGH	tests/test_approval_workflow.py	182	except json.JSONDecodeError:	CODE
HIGH	tests/test_approval_workflow.py	339	except json.JSONDecodeError:	CODE
HIGH	tests/test_approval_workflow.py	492	except json.JSONDecodeError:	CODE
HIGH	tests/test_http_docs.py	106	except ValueError:	CODE
HIGH	tests/test_bash_session_prefix_flow.py	180	except json.JSONDecodeError:	CODE
HIGH	tests/test_mcp_toolset.py	2016	except RuntimeError:	CODE
HIGH	tests/test_mcp_oauth.py	1361	except Exception:	CODE
HIGH	tests/test_mcp_oauth.py	1405	except Exception:	CODE
HIGH	tests/test_mcp_oauth.py	1449	except Exception:	CODE
HIGH	tests/test_mcp_oauth.py	1502	except Exception:	CODE
HIGH	tests/llm/conftest.py	409	except Exception:	CODE
HIGH	tests/llm/test_holmes_checks.py	241	except Exception:	CODE
HIGH	tests/llm/utils/property_manager.py	330	except Exception:	CODE
HIGH	tests/llm/utils/port_forward.py	119	except Exception:	CODE
HIGH	tests/llm/utils/port_forward.py	387	except (ValueError, ProcessLookupError):	CODE
HIGH	tests/llm/utils/answer_dump.py	84	except Exception: # noqa: BLE001 - never break a run over a debug aid	CODE
HIGH	tests/core/test_usage_recorder.py	91	except Exception:	CODE
HIGH	…ersations_worker/integration/broadcast_health_check.py	175	except Exception:	CODE
HIGH	…ersations_worker/integration/broadcast_health_check.py	180	except Exception:	CODE
HIGH	tests/core/conversations_worker/integration/__init__.py	176	except asyncio.TimeoutError:	CODE
HIGH	tests/plugins/toolsets/database/test_database.py	134	except Exception:	CODE
HIGH	tests/utils/curl_parser.py	293	except json.JSONDecodeError:	CODE
HIGH	tests/utils/curl_parser.py	111	except json.JSONDecodeError:	CODE
HIGH	tests/utils/curl_parser.py	181	except ValueError:	CODE
HIGH	holmes/version.py	137	except Exception:	CODE
HIGH	holmes/version.py	85	except Exception:	CODE
HIGH	holmes/interactive.py	893	except (TypeError, AttributeError):	CODE
HIGH	holmes/interactive.py	918	except (TypeError, AttributeError):	CODE
HIGH	holmes/core/tracing.py	84	except Exception:	CODE
HIGH	holmes/core/tools.py	715	except FileNotFoundError:	CODE
HIGH	holmes/core/json_schema_coerce.py	146	except (ValueError, OverflowError):	CODE
HIGH	holmes/core/json_schema_coerce.py	156	except (ValueError, OverflowError):	CODE
HIGH	holmes/core/tool_calling_llm.py	1426	except (AttributeError, IndexError, TypeError):	CODE
HIGH	holmes/core/conversations_worker/processor.py	367	except ValueError:	CODE
HIGH	holmes/core/conversations_worker/realtime_manager.py	427	except asyncio.TimeoutError:	CODE
HIGH	holmes/core/conversations_worker/realtime_manager.py	399	except asyncio.TimeoutError:	CODE
HIGH	holmes/plugins/toolsets/kubernetes_logs.py	492	except Exception:	CODE
HIGH	holmes/plugins/toolsets/kubernetes_logs.py	500	except Exception:	CODE
HIGH	holmes/plugins/toolsets/kafka.py	224	except OSError:	CODE
HIGH	holmes/plugins/toolsets/database/database.py	645	except Exception:	CODE
HIGH	holmes/plugins/toolsets/mcp/oauth_token_store.py	245	except (ValueError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/mcp/oauth_token_manager.py	106	except (ValueError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	609	except (json.JSONDecodeError, ValueError):	CODE
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	1264	except Exception:	CODE
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	1270	except Exception:	CODE
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	552	except (binascii.Error, ValueError):	CODE
HIGH	holmes/plugins/toolsets/bash/common/cli_prefixes.py	80	except Exception:	CODE
HIGH	holmes/plugins/toolsets/coralogix/api.py	102	except (json.JSONDecodeError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/internet/ssrf.py	99	except ValueError:	CODE
HIGH	…es/plugins/toolsets/datadog/toolset_datadog_general.py	582	except (ValueError, TypeError):	CODE
HIGH	…es/plugins/toolsets/datadog/toolset_datadog_general.py	587	except (ValueError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/datadog/datadog_api.py	538	except (ValueError, AttributeError):	CODE
HIGH	holmes/plugins/toolsets/datadog/datadog_api.py	546	except (ValueError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/datadog/datadog_api.py	701	except (json.JSONDecodeError, TypeError):	CODE
HIGH	holmes/plugins/toolsets/prometheus/prometheus.py	1713	except json.JSONDecodeError:	CODE
HIGH	holmes/plugins/toolsets/prometheus/prometheus.py	1970	except json.JSONDecodeError:	CODE

Self-Referential Comments72 hits · 226 pts

Severity	File	Line	Snippet	Context
MEDIUM⚡	tests/test_header_propagation.py	208	# Create an HTTP toolset with extra_headers in config	COMMENT
MEDIUM	tests/test_toolset_auto_enable.py	48	# Create a per-test subclass so config_classes doesn't leak between tests	COMMENT
MEDIUM	tests/stdio_server.py	7	# Create the MCP server	COMMENT
MEDIUM	tests/test_approval_workflow.py	87	# Create the actual ToolCallingLLM instance	COMMENT
MEDIUM	tests/test_approval_workflow.py	132	# Create a proper toolset mock with status	COMMENT
MEDIUM	tests/test_approval_workflow.py	229	# Create the actual ToolCallingLLM instance	COMMENT
MEDIUM	tests/test_approval_workflow.py	273	# Create a proper toolset mock with status	COMMENT
MEDIUM	tests/test_approval_workflow.py	382	# Create the actual ToolCallingLLM instance	COMMENT
MEDIUM	tests/test_approval_workflow.py	426	# Create a proper toolset mock with status	COMMENT
MEDIUM	tests/generate_eval_report.py	299	# Create the filter string for span_attributes.name	COMMENT
MEDIUM	tests/test_holmes_sync_toolsets.py	97	# Create a toolset without config_classes - should have null schema	COMMENT
MEDIUM	tests/test_bash_session_prefix_flow.py	624	# Create a shared config instance (simulates how toolset config is shared)	COMMENT
MEDIUM	tests/test_otel_tracing.py	373	# Create a child (simulates gen_ai.chat)	COMMENT
MEDIUM	tests/test_interactive.py	435	# Create a temporary directory for history file	COMMENT
MEDIUM	tests/test_mcp_oauth.py	2153	# Create a RemoteMCPTool with proper .toolset reference	COMMENT
MEDIUM	tests/test_mcp_oauth.py	2234	# Create a placeholder connect tool	COMMENT
MEDIUM	tests/llm/utils/constants.py	34	# Create a Literal type with all discovered tags	COMMENT
MEDIUM	tests/llm/utils/port_forward.py	177	# Create a hashable key for deduplication	COMMENT
MEDIUM	tests/llm/utils/port_forward.py	207	# Create a unique key for the service	COMMENT
MEDIUM	tests/core/test_supabase_dal.py	549	"""This function is typed `-> str`, and every path must honour that.	STRING
MEDIUM⚡	tests/core/test_tool_transformers.py	49	# Create a concrete tool for testing	COMMENT
MEDIUM	tests/core/test_tool_transformers.py	407	# Create a transformer that will fail	COMMENT
MEDIUM	tests/core/test_tool_transformers.py	793	# Create a transformer config that will fail to initialize	COMMENT
MEDIUM⚡	tests/core/transformers/test_llm_summarize.py	532	# Create a mock expanding transformer response	COMMENT
MEDIUM⚡	tests/core/transformers/test_llm_summarize.py	537	# Create a concrete tool class for testing	COMMENT
MEDIUM⚡	tests/core/transformers/test_llm_summarize.py	579	# Create a mock expanding transformer response that's much larger than original	COMMENT
MEDIUM⚡	tests/core/transformers/test_llm_summarize.py	586	# Create a concrete tool class for testing	COMMENT
MEDIUM	tests/core/transformers/test_llm_summarize.py	650	# Create a concrete tool class for testing	COMMENT
MEDIUM⚡	tests/plugins/toolsets/test_tool_kafka.py	323	# Create a new topic for this test to ensure clean state	COMMENT
MEDIUM⚡	tests/plugins/toolsets/test_tool_kafka.py	327	# Create the topic	COMMENT
MEDIUM	tests/plugins/toolsets/test_tool_kafka.py	454	# Create a consumer group and commit offsets	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	33	# Create the tool	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	71	# Create the tool	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	102	# Create the tool	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	130	# Create the tool	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	152	# Create the tool	COMMENT
MEDIUM	tests/plugins/toolsets/test_logging_api.py	187	# Create a mock LLM that simulates token counting	COMMENT
MEDIUM	…sts/plugins/toolsets/grafana/test_grafana_tempo_api.py	360	# Create an HTTPError with a response attribute	COMMENT
MEDIUM	tests/plugins/toolsets/bash/test_read_image_file.py	91	# Create a file just over the 20MB limit	COMMENT
MEDIUM	tests/checks/test_checks_cli.py	36	# Create a temporary checks config file	COMMENT
MEDIUM	…s/integration/test_kubernetes_transformer_execution.py	209	# Create a failing transformer	COMMENT
MEDIUM	…s/integration/test_kubernetes_transformer_execution.py	225	# Create a YAML toolset with the failing transformer	COMMENT
MEDIUM	…s/integration/test_kubernetes_transformer_execution.py	317	# Create a second transformer	COMMENT
MEDIUM	…s/integration/test_kubernetes_transformer_execution.py	425	# Create a YAML toolset for testing	COMMENT
MEDIUM⚡	tests/toolsets/mcp/test_approval_response_parsing.py	16	# Create a mock toolset	COMMENT
MEDIUM⚡	tests/toolsets/mcp/test_approval_response_parsing.py	20	# Create a tool instance	COMMENT
MEDIUM⚡	tests/toolsets/mcp/test_approval_response_parsing.py	29	# Create a mock MCP session and tool result. The status is the serialized	COMMENT
MEDIUM	tests/toolsets/mcp/test_approval_response_parsing.py	88	# Create a normal tool response	COMMENT
MEDIUM	tests/toolsets/mcp/test_approval_response_parsing.py	134	# Create a response that looks like JSON but isn't valid	COMMENT
MEDIUM	holmes/interactive.py	1208	# Create a temporary document with just the path part	COMMENT
MEDIUM	holmes/interactive.py	1239	# Create a temporary document with just the command part	COMMENT
MEDIUM	holmes/interactive.py	1717	# Create a temporary session without history for y/n prompts	COMMENT
MEDIUM	holmes/interactive.py	1997	# Create a temporary file to capture shell session	COMMENT
MEDIUM	holmes/interactive.py	2090	# Create a temporary session without history for feedback prompts	COMMENT
MEDIUM	holmes/core/openai_formatting.py	174	) # Create a copy to avoid modifying original	CODE
MEDIUM	holmes/core/tools.py	543	# Create a copy of the result with transformed data	COMMENT
MEDIUM	holmes/core/tool_calling_llm.py	95	# Create a named logger for cost tracking	COMMENT
MEDIUM	holmes/core/tool_calling_llm.py	1286	# Create a child gen_ai.chat span for each LLM call iteration.	COMMENT
MEDIUM	holmes/plugins/destinations/pagerduty/plugin.py	111	# Create the payload	COMMENT
MEDIUM	holmes/plugins/toolsets/kafka.py	207	# Create the file atomically with restrictive permissions.	COMMENT
12 more matches not shown…

Magic Placeholder Names34 hits · 180 pts

Severity	File	Line	Snippet	Context
HIGH	tempo_cli_README.md	112	export TEMPO_API_KEY=your-api-key-here	CODE
HIGH⚡	CONTRIBUTING.md	45	export OPENAI_API_KEY="your-api-key-here"	CODE
HIGH⚡	CONTRIBUTING.md	48	export ANTHROPIC_API_KEY="your-api-key-here"	CODE
HIGH⚡	CONTRIBUTING.md	51	export AZURE_API_KEY="your-api-key-here"	CODE
HIGH	tests/utils/curl_parser.py	278	- <your-api-key> -> test-api-key	STRING
HIGH	tests/utils/test_pydantic_utils.py	34	assert example["api_key"] == "your_api_key"	CODE
HIGH	docs/development/evaluations/reporting.md	30	export BRAINTRUST_API_KEY=sk-your-api-key-here	CODE
HIGH	docs/installation/cli-installation.md	76	-e OPENAI_API_KEY="your-api-key" \	CODE
HIGH	docs/installation/cli-installation.md	107	export ANTHROPIC_API_KEY="your-api-key"	CODE
HIGH	docs/installation/cli-installation.md	128	export OPENAI_API_KEY="your-api-key"	CODE
HIGH	docs/installation/docker-compose-installation.md	22	export OPENAI_API_KEY="your-api-key"	CODE
HIGH	docs/data-sources/builtin-toolsets/grafana-mcp.md	198	./test-grafana-api-key.sh '<your-api-key>' '<your-grafana-url>'	CODE
HIGH	…cs/data-sources/builtin-toolsets/atlassian-rovo-mcp.md	72	If your organization uses an Atlassian service account instead of a personal account, skip the base64 step and send	CODE
HIGH	…cs/data-sources/builtin-toolsets/atlassian-rovo-mcp.md	72	If your organization uses an Atlassian service account instead of a personal account, skip the base64 step and send	CODE
HIGH⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	66	- `<YOUR_TOKEN>` with the token generated in Prerequisites Step 4	CODE
HIGH⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	73	export SPLUNK_MCP_TOKEN=<YOUR_TOKEN>	CODE
HIGH	docs/data-sources/builtin-toolsets/splunk-mcp.md	103	--from-literal=SPLUNK_MCP_TOKEN=<YOUR_TOKEN> \	CODE
HIGH	docs/data-sources/builtin-toolsets/splunk-mcp.md	141	--from-literal=SPLUNK_MCP_TOKEN=<YOUR_TOKEN> \	CODE
HIGH	docs/data-sources/builtin-toolsets/servicenow.md	64	-H "x-sn-apikey: <your-api-key>"	CODE
HIGH	docs/data-sources/builtin-toolsets/servicenow.md	69	-H "x-sn-apikey: <your-api-key>"	CODE
HIGH	docs/data-sources/builtin-toolsets/elasticsearch.md	26	export ELASTICSEARCH_API_KEY=your-api-key	CODE
HIGH	docs/data-sources/builtin-toolsets/elasticsearch.md	60	--from-literal=ELASTICSEARCH_API_KEY=your-api-key \	CODE
HIGH	docs/data-sources/builtin-toolsets/elasticsearch.md	102	--from-literal=ELASTICSEARCH_API_KEY=your-api-key \	CODE
HIGH	docs/data-sources/builtin-toolsets/elasticsearch.md	148	api_key: "your-api-key"	CODE
HIGH	docs/ai-providers/anthropic.md	23	holmes ask "what pods are failing?" --model="anthropic/claude-sonnet-4-5" --api-key="your-api-key"	CODE
HIGH	docs/ai-providers/openai.md	28	holmes ask "what pods are failing?" --api-key="your-api-key"	CODE
HIGH	docs/ai-providers/using-multiple-providers.md	28	api_key: "your-api-key"	CODE
HIGH	docs/ai-providers/index.md	33	2. Set `export OPENAI_API_KEY="your-api-key"`	CODE
HIGH	docs/ai-providers/gemini.md	124	holmes ask "what pods are failing?" --model="gemini/<your-gemini-model>" --api-key="your-api-key"	CODE
HIGH	docs/ai-providers/openai-compatible.md	101	--from-literal=OPENAI_API_KEY="your-api-key" \	CODE
HIGH	docs/ai-providers/openai-compatible.md	140	--from-literal=OPENAI_API_KEY="your-api-key" \	CODE
HIGH	docs/ai-providers/azure-ai-foundry.md	135	holmes ask "what pods are failing?" --model="anthropic/claude-opus-4-7" --api-key="your-api-key"	CODE
HIGH	docs/reference/helm-configuration.md	73	value: "your-api-key"	CODE
HIGH	holmes/plugins/toolsets/grafana/common.py	52	examples=[{"Authorization": "Bearer YOUR_API_KEY"}],	CODE

Deep Nesting205 hits · 173 pts

Severity	File	Line	Context
LOW	conftest.py	88	CODE
LOW	server.py	294	CODE
LOW	server.py	591	CODE
LOW	server.py	304	CODE
LOW	experimental/ag-ui/server-agui.py	102	CODE
LOW	experimental/ag-ui/server-agui.py	138	CODE
LOW	tests/test_approval_workflow.py	71	CODE
LOW	tests/test_approval_workflow.py	213	CODE
LOW	tests/test_approval_workflow.py	366	CODE
LOW	tests/generate_eval_report.py	149	CODE
LOW	tests/generate_eval_report.py	431	CODE
LOW	tests/generate_eval_report.py	501	CODE
LOW	tests/generate_eval_report.py	597	CODE
LOW	tests/generate_eval_report.py	945	CODE
LOW	tests/generate_eval_report.py	1141	CODE
LOW	tests/generate_eval_report.py	1203	CODE
LOW	tests/generate_eval_report.py	1316	CODE
LOW	tests/test_http_docs.py	184	CODE
LOW	tests/test_bash_session_prefix_flow.py	169	CODE
LOW	tests/test_cache.py	44	CODE
LOW	tests/llm/conftest.py	131	CODE
LOW	tests/llm/conftest.py	376	CODE
LOW	tests/llm/conftest.py	603	CODE
LOW	tests/llm/conftest.py	680	CODE
LOW	tests/llm/conftest.py	744	CODE
LOW	tests/llm/test_holmes_checks.py	156	CODE
LOW	tests/llm/test_ask_holmes.py	74	CODE
LOW	tests/llm/test_ask_holmes.py	539	CODE
LOW	tests/llm/utils/test_case_utils.py	464	CODE
LOW	tests/llm/utils/property_manager.py	107	CODE
LOW	tests/llm/utils/mock_dal.py	55	CODE
LOW	tests/llm/utils/test_results.py	96	CODE
LOW	tests/llm/utils/test_results.py	113	CODE
LOW	tests/llm/utils/test_results.py	128	CODE
LOW	tests/llm/utils/env_config.py	29	CODE
LOW	tests/llm/utils/conversation_formatter.py	7	CODE
LOW	tests/llm/utils/port_forward.py	167	CODE
LOW	tests/llm/utils/port_forward.py	191	CODE
LOW	tests/llm/utils/port_forward.py	340	CODE
LOW	tests/llm/utils/port_forward.py	98	CODE
LOW	tests/llm/utils/braintrust.py	29	CODE
LOW	tests/llm/utils/setup_cleanup.py	21	CODE
LOW	tests/llm/utils/setup_cleanup.py	59	CODE
LOW	tests/llm/utils/classifiers.py	35	CODE
LOW	tests/llm/utils/test_toolset.py	132	CODE
LOW	tests/llm/utils/commands.py	21	CODE
LOW	tests/llm/utils/reporting/github_reporter.py	466	CODE
LOW	tests/llm/utils/reporting/terminal_reporter.py	504	CODE
LOW	tests/llm/utils/reporting/terminal_reporter.py	904	CODE
LOW	tests/llm/utils/reporting/terminal_reporter.py	1149	CODE
LOW	tests/llm/utils/reporting/terminal_reporter.py	1271	CODE
LOW	tests/plugins/toolsets/test_tool_kafka.py	452	CODE
LOW	…sts/plugins/toolsets/test_toolset_command_injection.py	61	CODE
LOW	…sts/plugins/toolsets/test_toolset_command_injection.py	91	CODE
LOW	…sts/plugins/toolsets/test_toolset_command_injection.py	183	CODE
LOW	…s/plugins/toolsets/grafana/test_grafana_tempo_tools.py	496	CODE
LOW	…/toolsets/datadog/metrics/test_datadog_metrics_live.py	101	CODE
LOW	…ins/toolsets/kubernetes/test_kubernetes_logs_mocked.py	9	CODE
LOW	…s/integration/test_kubernetes_transformer_execution.py	206	CODE
LOW	tests/holmes_operator/test_log.py	43	CODE
145 more matches not shown…

Structural Annotation Overuse82 hits · 137 pts

Severity	File	Line	Snippet	Context
LOW	experimental/ag-ui/server-agui.py	442	# IMPORTANT: Do not support front-end "tool" messages for now. Store them as assistant messages in conv history.	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1173	# Step 1: Discover OAuth endpoints	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1194	# Step 2: Generate PKCE	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1197	# Step 3: Start local callback server (port 0 = OS-assigned)	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1261	# Step 5: Wait for callback	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1270	# Step 6: Exchange code for token	COMMENT
LOW	tests/test_mcp_oauth.py	1244	# Step 4: Build authorization URL and open browser	COMMENT
LOW	tests/test_mcp_oauth.py	1292	# Step 7: Use token to list MCP tools	COMMENT
LOW	tests/llm/conftest.py	458	# Step 2: Environment is OK, now test if the API actually works	COMMENT
LOW	tests/core/conversations_worker/README.md	48	### Step 1: Start the Holmes server	COMMENT
LOW	tests/core/conversations_worker/README.md	63	### Step 2: Run the integration tests	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	309	# Step 1: Detect approval needed	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	317	# Step 2: Simulate user approval (in real flow, this happens via UI/CLI)	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	321	# Step 3: Execute with approval	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	17	## Step 1: Identify the CVE Source	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	31	## Step 2: Check if Upstream Fixes Exist	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	72	## Step 3: Apply the Fix	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	120	## Step 4: Build the Docker Image	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	130	## Step 5: Validate the Fix	COMMENT
LOW	.claude/skills/patch-cve/SKILL.md	158	## Step 6: Document and Clean Up	COMMENT
LOW⚡	.claude/skills/create-eval/SKILL.md	28	### Step 1: Choose Test Number and Namespace	COMMENT
LOW⚡	.claude/skills/create-eval/SKILL.md	38	### Step 2: Validate Tags	COMMENT
LOW⚡	.claude/skills/create-eval/SKILL.md	42	### Step 3: Write test_case.yaml	COMMENT
LOW	.claude/skills/create-eval/SKILL.md	68	### Step 4: Write toolsets.yaml (if needed)	COMMENT
LOW	.claude/skills/create-eval/SKILL.md	84	### Step 5: Write Setup Scripts	COMMENT
LOW	.claude/skills/create-eval/SKILL.md	106	### Step 6: Design Anti-Hallucination Measures	COMMENT
LOW	docs/development/evaluations/benchmarking-new-models.md	10	## Step 1: Create Model List File	COMMENT
LOW	docs/development/evaluations/benchmarking-new-models.md	40	## Step 2: Run Initial Test	COMMENT
LOW	docs/development/evaluations/benchmarking-new-models.md	55	## Step 3: Known Issues and Troubleshooting	COMMENT
LOW	docs/development/evaluations/benchmarking-new-models.md	64	## Step 4: Run Benchmarks	COMMENT
LOW	docs/development/evaluations/benchmarking-new-models.md	77	## Step 5: Review Results	COMMENT
LOW	docs/walkthrough/investigating-using-aks-mcp-server.md	17	### Step 1: Setup the MCP Server	COMMENT
LOW	docs/walkthrough/investigating-using-aks-mcp-server.md	23	### Step 2: Update Configuration for SSE	COMMENT
LOW	docs/walkthrough/investigating-using-aks-mcp-server.md	40	### Step 3: Configure HolmesGPT	COMMENT
LOW	docs/walkthrough/investigating-prometheus-alerts.md	16	### Step 1: Forward AlertManager	COMMENT
LOW	docs/walkthrough/investigating-prometheus-alerts.md	24	### Step 2: Create a Test Alert	COMMENT
LOW	docs/walkthrough/investigating-prometheus-alerts.md	57	### Step 3: Investigate Alerts	COMMENT
LOW	docs/data-sources/builtin-toolsets/splunk-mcp.md	18	### Step 1: Install Splunk MCP Server	COMMENT
LOW⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	32	### Step 2: Create MCP Role	COMMENT
LOW⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	41	### Step 3: Create MCP User	COMMENT
LOW⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	48	### Step 4: Generate Authentication Token	COMMENT
LOW⚡	docs/data-sources/builtin-toolsets/splunk-mcp.md	56	### Step 5: Get MCP Endpoint URL	COMMENT
LOW	docs/data-sources/builtin-toolsets/azure-mcp.md	89	### Step 1: Set Up Azure RBAC Roles	COMMENT
LOW	docs/data-sources/builtin-toolsets/azure-mcp.md	134	### Step 2: Deploy with Helm	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	20	### Step 1: Deploy	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	91	### Step 2: Verify	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	101	### Step 1: Generate a kubeconfig for Holmes	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	184	### Step 2: Set the default context In the Kubeconfig file	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	198	### Step 3: Create the kubeconfig Secret	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	216	### Step 4: Deploy	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	439	### Step 5: Route chats to the right cluster (Robusta UI)	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	460	### Step 1: Enable Azure AD on your AKS cluster	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	464	### Step 2: Create an Entra ID App Registration	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	481	### Step 3: Store the client secret	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	491	### Step 4: Deploy	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	600	### Step 5: Verify	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	612	### Step 1: Register Holmes as a Dex client	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	644	### Step 2: Store the client secret	COMMENT
LOW	docs/data-sources/builtin-toolsets/kubernetes-mcp.md	655	### Step 3: Deploy	COMMENT
LOW	docs/data-sources/builtin-toolsets/aws.md	12	### Step 1: Set Up IAM Permissions	COMMENT
22 more matches not shown…

Redundant / Tautological Comments82 hits · 125 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_bash_session_prefix_flow.py	140	# Check if all prefixes are in session_approved_prefixes	COMMENT
LOW	tests/test_bash_session_prefix_flow.py	366	# Check if we got approval_required or answer_end	COMMENT
LOW	tests/llm/conftest.py	200	# Check if any failures are not in the allowed list	COMMENT
LOW	tests/llm/conftest.py	265	# Check if we're in --only-setup mode	COMMENT
LOW	tests/llm/conftest.py	565	# Check if LLM marker is being excluded	COMMENT
LOW	tests/llm/conftest.py	606	# Check if we're in collect-only mode	COMMENT
LOW	tests/llm/conftest.py	612	# Check if LLM marker is being excluded	COMMENT
LOW	tests/llm/conftest.py	661	# Check if Braintrust is enabled	COMMENT
LOW	tests/llm/conftest.py	718	# Check if we're in a worker process	COMMENT
LOW	tests/llm/conftest.py	819	# Check if we have error information	COMMENT
LOW	tests/llm/conftest.py	970	# Check if Braintrust is enabled	COMMENT
LOW	tests/llm/test_holmes_checks.py	168	# Check if test should be skipped	COMMENT
LOW	tests/llm/utils/test_case_utils.py	340	# Check if test should be skipped	COMMENT
LOW	tests/llm/utils/test_case_utils.py	362	# Check if --only-setup is set (AFTER checking for setup failures)	COMMENT
LOW	tests/llm/utils/test_case_utils.py	367	# Check if --only-cleanup is set	COMMENT
LOW	tests/llm/utils/test_case_utils.py	378	# Check if test should be skipped due to port conflicts	COMMENT
LOW	tests/llm/utils/property_manager.py	160	# Check if we should include intermediate outputs (based on CLI flag, defaults to True)	COMMENT
LOW	tests/llm/utils/property_manager.py	347	# Check if this is a ThrottledError	COMMENT
LOW	tests/llm/utils/property_manager.py	358	# Check if this is a ToolsetPrerequisiteError (toolset infrastructure not available)	COMMENT
LOW	tests/llm/utils/test_results.py	40	# Check if removing this digit would leave us with underscore or nothing	COMMENT
LOW	tests/llm/utils/port_forward.py	29	# Check if port is already in use	COMMENT
LOW	tests/llm/utils/port_forward.py	44	# Check if process is still running	COMMENT
LOW	tests/llm/utils/port_forward.py	48	# Check if port is in use to provide better error message	COMMENT
LOW	tests/llm/utils/port_forward.py	259	# Check if ports are already in use on the system	COMMENT
LOW	tests/llm/utils/port_forward.py	373	# Check if it's a kubectl process before killing	COMMENT
LOW	tests/llm/utils/retry_handler.py	116	# Check if this is a throttle error	COMMENT
LOW	tests/llm/utils/retry_handler.py	127	# Check if retries are disabled	COMMENT
LOW	tests/llm/utils/retry_handler.py	138	# Check if we have retries left	COMMENT
LOW	tests/llm/utils/reporting/github_reporter.py	479	# Check if running on a specific branch (for cross-branch comparison)	COMMENT
LOW	tests/llm/utils/reporting/terminal_reporter.py	1276	# Check if any tests had retries	COMMENT
LOW⚡	tests/core/transformers/test_transformers.py	280	# Check if transformation should apply	COMMENT
LOW	holmes_operator/handlers/healthcheck.py	50	# Set status to Pending	COMMENT
LOW	scripts/publish-pypi.sh	19	# Check if PYPI_TOKEN is set	COMMENT
LOW	scripts/publish-pypi.sh	26	# Check if Poetry is installed	COMMENT
LOW	.github/workflows/build-and-test.yaml	78	# Check if file matches any skip pattern	COMMENT
LOW	.github/workflows/eval-regression.yaml	1306	# Check if results were actually produced. If not, it's a collection error.	COMMENT
LOW	.github/workflows/eval-benchmarks.yaml	184	# Check if there are changes to commit	COMMENT
LOW	.github/workflows/eval-benchmarks.yaml	202	# Check if PR already exists	COMMENT
LOW⚡	holmes/version.py	78	# Check if Git substitution didn't happen (placeholders are still present)	COMMENT
LOW⚡	holmes/version.py	91	# Set ceiling to prevent walking up beyond the project root	COMMENT
LOW	holmes/version.py	122	# Check if there are uncommitted changes	COMMENT
LOW	holmes/version.py	177	# Check if current version starts with latest version	COMMENT
LOW	holmes/interactive.py	2656	# Check if we got an approval-required event	COMMENT
LOW	holmes/core/tracing.py	64	# Check if .git is a file (worktree case)	COMMENT
LOW	holmes/core/tools.py	490	# Check if transformer should be applied	COMMENT
LOW	holmes/core/tools.py	504	# Check if this is llm_summarize and revert if summary is not smaller	COMMENT
LOW	holmes/core/tools.py	928	# Check if transformer is registered	COMMENT
LOW	holmes/core/tools.py	959	# Check if transformer is registered	COMMENT
LOW	holmes/core/tool_calling_llm.py	778	# Check if there are frontend tool calls — can't execute in sync mode	COMMENT
LOW	holmes/core/tool_calling_llm.py	1457	# Check if any tools require approval or are frontend-defined	COMMENT
LOW	holmes/core/toolset_manager.py	520	# Check if any datasource config file has changed since the last run.	COMMENT
LOW	holmes/core/transformers/llm_summarize.py	116	# Check if input exceeds threshold	COMMENT
LOW	holmes/plugins/toolsets/kubernetes_logs.py	71	# Check if kubectl is available	COMMENT
LOW	holmes/plugins/toolsets/kubernetes_logs.py	192	# Check if we have any logs to return	COMMENT
LOW	holmes/plugins/toolsets/kubernetes.yaml	166	# Output results as a JSON object with metadata	COMMENT
LOW	holmes/plugins/toolsets/kafka.py	805	# Check if any partition has a valid committed offset	COMMENT
LOW	holmes/plugins/toolsets/kafka.py	1284	# Check if partition is under-replicated	COMMENT
LOW	holmes/plugins/toolsets/kafka.py	1288	# Check if partition is offline	COMMENT
LOW	holmes/plugins/toolsets/database/database.py	649	# Check if the result returns rows (SELECT, SHOW, etc.) or not (INSERT, UPDATE, etc.)	COMMENT
LOW	holmes/plugins/toolsets/logging_utils/logging_api.py	234	# Check if it's already in RFC3339 format	COMMENT
22 more matches not shown…

Cross-Language Confusion23 hits · 121 pts

Severity	File	Line	Snippet	Context
HIGH⚡	tests/test_bash_toolset_validation.py	154	segments, has_compound = parse_command_segments("mkdir test && cd test")	CODE
HIGH⚡	tests/test_bash_toolset_validation.py	160	segments, has_compound = parse_command_segments("test -f file.txt \|\| touch file.txt")	CODE
HIGH⚡	tests/test_bash_toolset_validation.py	783	"echo $HOME && ls ${HOME}/projects",	CODE
HIGH	tests/test_bash_toolset_validation.py	735	"kubectl get pods && kubectl get services",	CODE
HIGH	tests/test_bash_toolset_validation.py	747	"kubectl get pods \|\| echo 'no pods'",	CODE
HIGH⚡	tests/test_bash_toolset_validation.py	153	"""Test parsing && operator."""	STRING
HIGH⚡	tests/test_bash_toolset_validation.py	159	"""Test parsing \|\| operator."""	STRING
HIGH	tests/test_approval_token_security.py	87	"arguments": json.dumps({"command": "id && pwd"}),	CODE
HIGH	tests/test_mcp_toolset.py	3513	Strict tool schemas advertise optional params as anyOf[<type>, null] and	STRING
HIGH	tests/test_mcp_oauth.py	1792	is null), a static Authorization header from `headers` must NOT be sent —	STRING
HIGH	tests/core/test_tool_memory_limit.py	21	assert result == f"ulimit -v {expected_kb} 2>/dev/null \|\| true; "	CODE
HIGH⚡	…ns_worker/integration/test_conversation_integration.py	160	"https://example.invalid/no-op \|\| echo done` to confirm a simple "	CODE
HIGH⚡	tests/plugins/toolsets/bash/test_argv_security.py	88	("mkdir -p /tmp/x && echo hi > /tmp/x/f", ["mkdir", "echo"]),	CODE
HIGH⚡	tests/plugins/toolsets/bash/test_shell_parser.py	45	p = parse_command("a && b \|\| c; d & e")	CODE
HIGH⚡	tests/plugins/toolsets/bash/test_shell_parser.py	45	p = parse_command("a && b \|\| c; d & e")	CODE
HIGH	tests/integration/test_tool_execution_pipeline.py	40	"ulimit -v 1048576 2>/dev/null \|\| true",	CODE
HIGH	holmes/main.py	130	help="Slack API key if --destination=slack (experimental). Can generate with `pip install robusta-cli && robusta int	CODE
HIGH	holmes/core/openai_formatting.py	145	# requires optional params to accept null, or when the source schema explicitly marks	COMMENT
HIGH	holmes/core/tools.py	282	"""Return the primary (non-null) type as a string."""	STRING
HIGH	holmes/core/json_schema_coerce.py	66	"""Extract the primary (non-null) type from a schema-like object.	STRING
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	785	# Nullable shorthand: anyOf[<type>, null] → collapse to the single type.	COMMENT
HIGH	holmes/plugins/toolsets/mcp/toolset_mcp.py	567	anyOf[<type>, null] and (in strict mode) also lists it in "required",	STRING
HIGH	holmes/utils/memory_limit.py	25	return f"ulimit -v {memory_limit_kb} 2>/dev/null \|\| true; "	CODE

Unused Imports137 hits · 114 pts

Severity	File	Line	Context
LOW	server.py	19	CODE
LOW	server.py	27	CODE
LOW	tests/test_header_propagation.py	12	CODE
LOW	tests/test_header_propagation.py	13	CODE
LOW	tests/test_header_propagation.py	13	CODE
LOW	tests/test_header_propagation.py	13	CODE
LOW	tests/test_header_propagation.py	13	CODE
LOW	tests/test_header_propagation.py	206	CODE
LOW	tests/test_header_propagation.py	251	CODE
LOW	tests/test_check_prerequisites.py	4	CODE
LOW	tests/test_check_prerequisites.py	7	CODE
LOW	tests/test_toolset_auto_enable.py	3	CODE
LOW	tests/test_toolset_auto_enable.py	3	CODE
LOW	tests/test_toolset_auto_enable.py	6	CODE
LOW	tests/test_toolset_auto_enable.py	9	CODE
LOW	tests/test_bash_toolset_validation.py	11	CODE
LOW	tests/test_toolset_config_tui.py	3	CODE
LOW	tests/test_toolset_config_tui.py	4	CODE
LOW	tests/test_toolset_config_tui.py	8	CODE
LOW	tests/test_toolset_config_tui.py	23	CODE
LOW	tests/test_toolset_prereq_timeout.py	3	CODE
LOW	tests/test_tool_calling_llm.py	18	CODE
LOW	tests/test_tool_calling_llm.py	18	CODE
LOW	tests/test_tool_calling_llm.py	18	CODE
LOW	tests/test_otel_tracing.py	13	CODE
LOW	tests/test_otel_tracing.py	13	CODE
LOW	tests/test_interactive.py	6	CODE
LOW	tests/test_mcp_oauth.py	30	CODE
LOW	tests/test_mcp_oauth.py	2221	CODE
LOW	tests/llm/utils/property_manager.py	10	CODE
LOW	tests/llm/utils/skill_suggestions.py	17	CODE
LOW	tests/llm/utils/test_toolset.py	5	CODE
LOW	tests/llm/utils/commands.py	14	CODE
LOW	tests/core/test_llm_completion_temperature.py	1	CODE
LOW	tests/core/test_tool_output_deduplication.py	8	CODE
LOW	tests/core/test_azure_token.py	1	CODE
LOW	…sts/core/tools_utils/test_filesystem_result_storage.py	3	CODE
LOW	…s/core/tools_utils/test_tool_context_window_limiter.py	2	CODE
LOW	…sts/core/conversations_worker/test_realtime_manager.py	4	CODE
LOW	…sts/core/conversations_worker/test_tool_call_worker.py	22	CODE
LOW	tests/core/conversations_worker/test_event_publisher.py	2	CODE
LOW	tests/core/conversations_worker/test_dal_contract.py	8	CODE
LOW	tests/core/conversations_worker/test_dal_contract.py	8	CODE
LOW	tests/core/conversations_worker/test_dal_contract.py	8	CODE
LOW	tests/core/conversations_worker/integration/conftest.py	12	CODE
LOW	…ersations_worker/integration/broadcast_health_check.py	14	CODE
LOW	…versations_worker/integration/test_retry_resilience.py	20	CODE
LOW	…ations_worker/integration/test_claiming_concurrency.py	27	CODE
LOW	tests/core/conversations_worker/integration/__init__.py	13	CODE
LOW	…ns_worker/integration/test_conversation_integration.py	15	CODE
LOW	…ns_worker/integration/test_conversation_integration.py	17	CODE
LOW	tests/plugins/sources/test_pagerduty_source.py	3	CODE
LOW	tests/plugins/toolsets/test_tool_kafka.py	7	CODE
LOW	tests/plugins/toolsets/test_logging_api.py	3	CODE
LOW	tests/plugins/toolsets/test_logging_api.py	6	CODE
LOW	tests/plugins/toolsets/test_logging_api.py	8	CODE
LOW	…ts/plugins/toolsets/grafana/test_grafana_tempo_unit.py	6	CODE
LOW	tests/config_class/test_config_anchors.py	1	CODE
LOW	tests/config_class/test_source_factory.py	14	CODE
LOW	tests/config_class/test_source_factory.py	15	CODE
77 more matches not shown…

Docstring Block Structure17 hits · 85 pts

Severity	File	Line	Snippet	Context
HIGH	tests/llm/utils/retry_handler.py	69	Retry a function when encountering throttling/overload errors. Args: func: The function to call	STRING
HIGH	tests/core/test_prompt.py	79	Extract user message content from messages list. Args: messages: List of message dictionaries with 'role' a	STRING
HIGH	holmes_operator/scheduler/manager.py	70	Register a cron job with APScheduler. Args: name: ScheduledHealthCheck name na	STRING
HIGH	holmes_operator/client/holmes_api_client.py	55	Execute a health check via Holmes API. Endpoint: POST /api/checks/execute Args: c	STRING
HIGH	holmes/core/otel_tracing.py	549	Create the OTLP span and metric exporters for the given protocol. Args: protocol: ``"grpc"`` or ``"http/pro	STRING
HIGH	holmes/core/prompt.py	60	Build content array for vision models with text and images. Args: text: The text content image	STRING
HIGH	holmes/core/transformers/registry.py	67	Create a transformer instance by name. Args: name: The name of the transformer to create	STRING
HIGH	holmes/core/transformers/llm_summarize.py	131	Transform the input text by summarizing it with the fast model. Args: input_text: The tool	STRING
HIGH	holmes/core/transformers/base.py	28	Transform the input text and return the transformed output. Args: input_text: The raw tool	STRING
HIGH	holmes/core/tools_utils/oauth_tool_connector.py	48	Try to process a tool approval decision as an OAuth code exchange. If the decision contains an OAuth authorizat	STRING
HIGH	…lugins/toolsets/servicenow_tables/servicenow_tables.py	196	Make a GET request to ServiceNow API and return JSON data and headers. Args: endpoint: API endpoint	STRING
HIGH	holmes/plugins/toolsets/grafana/grafana_tempo_api.py	71	Make HTTP request to Tempo API with retry logic. Args: endpoint: API endpoint path (e.g., "/api/ech	STRING
HIGH	holmes/plugins/toolsets/grafana/toolset_grafana.py	587	Make a GET request to Grafana render API and return PNG bytes. Args: render_path: Render URL path (	STRING
HIGH	holmes/plugins/toolsets/newrelic/newrelic.py	286	Create a NewRelicAPI client instance. Args: override_account_id: Account ID to use. If None, uses t	STRING
HIGH	holmes/plugins/toolsets/newrelic/new_relic_api.py	55	Make HTTP POST request to NewRelic GraphQL API. Args: graphql_query: The GraphQL query as a diction	STRING
HIGH	holmes/plugins/toolsets/newrelic/new_relic_api.py	94	Execute an NRQL query via the NewRelic GraphQL API. Args: nrql_query: The NRQL query string to exec	STRING
HIGH	holmes/plugins/toolsets/elasticsearch/elasticsearch.py	231	Make HTTP request to Elasticsearch. Args: method: HTTP method (GET, POST, etc.) endpoin	STRING

Modern Structural Boilerplate65 hits · 66 pts

Severity	File	Line	Snippet	Context
LOW	tests/test_cache.py	22	logger = logging.getLogger(__name__)	CODE
LOW	tests/llm/utils/property_manager.py	75	def set_trace_properties(request, eval_span) -> None:	CODE
LOW	tests/llm/utils/property_manager.py	97	def update_property(request, key: str, value: Any) -> None:	CODE
LOW	tests/llm/utils/braintrust_history.py	28	__all__ = [	CODE
LOW	tests/core/conversations_worker/integration/__init__.py	140	async def _setup() -> None:	CODE
LOW	holmes_operator/trigger_executor.py	23	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/operator.py	32	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/context.py	13	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/utils.py	17	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/scheduler/job_executor.py	12	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/scheduler/__init__.py	7	__all__ = ["SchedulerManager"]	CODE
LOW	holmes_operator/scheduler/manager.py	14	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/client/__init__.py	5	__all__ = ["HolmesAPIClient"]	CODE
LOW	holmes_operator/client/holmes_api_client.py	11	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/handlers/triggeredhealthcheck.py	23	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/handlers/healthcheck.py	25	logger = logging.getLogger(__name__)	CODE
LOW	holmes_operator/handlers/scheduledhealthcheck.py	25	logger = logging.getLogger(__name__)	CODE
LOW⚡	holmes/toolset_config_tui.py	34	logger = logging.getLogger(__name__)	CODE
LOW	holmes/clients/robusta_client.py	32	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/oauth_utils.py	29	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/oauth_utils.py	52	def set_oauth_dal(dal: Any) -> None:	CODE
LOW	holmes/core/otel_tracing.py	82	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/otel_tracing.py	336	def set_attributes(self, name: Optional[str] = None, span_type: Optional[str] = None, span_attributes: Optional[Dict	CODE
LOW	holmes/core/oauth_config.py	13	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/feedback.py	68	def update_llm(self, llm: LLM) -> None:	CODE
LOW	holmes/core/feedback.py	119	def set_user_feedback(self, user_feedback: UserFeedback) -> None:	CODE
LOW	holmes/core/tools.py	58	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/json_schema_coerce.py	52	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/oauth_server_callbacks.py	16	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/usage_recorder.py	579	__all__ = [	CODE
LOW	holmes/core/azure_token.py	11	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/transformers/__init__.py	16	__all__ = [	CODE
LOW	holmes/core/transformers/llm_summarize.py	20	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/transformers/llm_summarize.py	77	def set_default_fast_model(cls, model: str) -> None:	CODE
LOW	holmes/core/transformers/base.py	5	__all__ = ["BaseTransformer", "TransformerError"]	CODE
LOW	holmes/core/tools_utils/oauth_tool_connector.py	23	logger = logging.getLogger(__name__)	CODE
LOW	holmes/core/scheduled_prompts/__init__.py	7	__all__ = [	CODE
LOW	holmes/core/conversations_worker/__init__.py	3	__all__ = ["ConversationRuntime"]	CODE
LOW	holmes/core/conversations_worker/runtime.py	43	__all__ = ["ConversationRuntime", "SHUTDOWN_RETIRE_BUDGET_SECONDS"]	CODE
LOW	holmes/plugins/toolsets/multi_instance.py	51	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/json_filter_mixin.py	14	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/database/database.py	35	logger = logging.getLogger(__name__)	CODE
LOW⚡	holmes/plugins/toolsets/database/database.py	536	def _update_tool_descriptions(self) -> None:	CODE
LOW	holmes/plugins/toolsets/database/__init__.py	3	__all__ = ["DatabaseToolset"]	CODE
LOW	holmes/plugins/toolsets/grafana/grafana_tempo_api.py	17	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/grafana/toolset_grafana.py	29	logger = logging.getLogger(__name__)	CODE
LOW	…/toolsets/robusta_platform_mcp/robusta_platform_mcp.py	44	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/mcp/oauth_token_store.py	22	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/mcp/oauth_token_manager.py	25	logger = logging.getLogger(__name__)	CODE
LOW⚡	holmes/plugins/toolsets/mcp/oauth_token_manager.py	68	def set_dal(self, dal: Any) -> None:	CODE
LOW	holmes/plugins/toolsets/mcp/toolset_mcp.py	54	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/mongodb/mongodb.py	26	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/mongodb/__init__.py	3	__all__ = ["MongoDBToolset"]	CODE
LOW	holmes/plugins/toolsets/bash/validation.py	39	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/http/http_toolset.py	43	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/datadog/datadog_models.py	10	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/confluence/confluence.py	19	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/confluence/confluence.py	417	def _setup_http_tools(self) -> None:	STRING
LOW	holmes/plugins/toolsets/newrelic/new_relic_api.py	8	logger = logging.getLogger(__name__)	CODE
LOW	holmes/plugins/toolsets/prometheus/prometheus.py	2157	def _set_meta_from_config(self) -> None:	CODE
5 more matches not shown…

Modern AI Meta-Vocabulary24 hits · 62 pts

Severity	File	Line	Snippet	Context
MEDIUM	config.example.yaml	9	# Helps reduce context window usage, improve performance, and reduce costs	COMMENT
MEDIUM	tests/llm/test_compaction.py	125	# calls, like the agentic loop does in production (ROB-424).	COMMENT
MEDIUM	…s/core/tools_utils/test_tool_context_window_limiter.py	34	2048 # Default to 50% of context window	CODE
MEDIUM	…s/core/tools_utils/test_tool_context_window_limiter.py	196	# Test with smaller context window	COMMENT
MEDIUM	…ns_worker/integration/test_conversation_integration.py	634	# it by actually invoking the noop tool, ruling out hallucination.	COMMENT
MEDIUM	docs/ai-providers/other.md	38	export OVERRIDE_MAX_CONTENT_SIZE=98304 # 98k context window	CODE
MEDIUM	docs/ai-providers/other.md	78	max_context_size: 98304 # Override context window to 98k tokens	CODE
MEDIUM	docs/ai-providers/other.md	109	max_context_size: 98304 # Override context window to 98k tokens	CODE
MEDIUM	docs/ai-providers/other.md	137	max_context_size: 98304 # Override context window to 98k tokens	CODE
MEDIUM	docs/ai-providers/other.md	161	max_context_size: 98304 # Override context window to 98k tokens	CODE
MEDIUM	docs/ai-providers/other.md	202	# Override context window size for any model	COMMENT
MEDIUM	docs/ai-providers/other.md	204	max_context_size: 1000000 # Override the model's context window	CODE
MEDIUM	docs/reference/environment-variables.md	221	# Allow each tool response to use up to 25% of context window	COMMENT
MEDIUM	holmes/core/llm.py	826	# Reserve output budget = max(64k, 12% of the context window). The 64k	COMMENT
MEDIUM	holmes/core/usage_recorder.py	349	# for an agentic loop with N-1 tool turns). Filled by the wrapper	COMMENT
MEDIUM	holmes/core/usage_recorder.py	484	# record partial cost when the agentic loop raises mid-loop —	COMMENT
MEDIUM	holmes/core/truncation/compaction.py	232	# context window, so it can describe what was in them. Otherwise strip them.	COMMENT
MEDIUM	holmes/core/truncation/input_context_window_limiter.py	125	# reuse the agentic prompt cache). Key for diagnosing zero	COMMENT
MEDIUM	holmes/core/tools_utils/tool_context_window_limiter.py	114	# Allocate remaining char budget to the preview so the final string fits the context window	COMMENT
MEDIUM	holmes/plugins/toolsets/openshift.yaml	12	# This reduces context window usage while preserving key information for debugging.	COMMENT
MEDIUM	holmes/plugins/toolsets/kubernetes.yaml	14	# This reduces context window usage while preserving key information for debugging.	COMMENT
MEDIUM	holmes/common/env_vars.py	17	200000 # Fallback context window size if it can't be determined from the model	CODE
MEDIUM	holmes/common/env_vars.py	134	# Limit each tool response to N% of the total context window.	COMMENT
MEDIUM	helm/holmes/values.yaml	690	# Provides access to Prefect workflow orchestration for monitoring and troubleshooting	COMMENT

Over-Commented Block64 hits · 60 pts

Severity	File	Line	Snippet	Context
LOW	conftest.py	141		COMMENT
LOW	conftest.py	161	# console_handler = logging.StreamHandler(sys.stderr) # pytest only prints error logs	COMMENT
LOW	config.example.yaml	1	#model: "gpt-4.1"	COMMENT
LOW	config.example.yaml	21	#jira_query: "project = 'Natan Test Project' and Status = 'To Do'"	COMMENT
LOW	tests/llm/test_ask_holmes.py	301	# The primary pass succeeded — all primary assertions above passed. We	COMMENT
LOW	tests/llm/test_ask_holmes.py	361	try:	COMMENT
LOW	tests/llm/test_ask_holmes.py	381	# replay's numbers would overwrite the PRIMARY	COMMENT
LOW	tests/llm/utils/test_case_utils.py	161	# - False → the test fails if the LLM emits any skill suggestions	COMMENT
LOW	tests/llm/utils/test_case_utils.py	181	# When set, asserts that the number of SKILL.md files written from the	COMMENT
LOW	tests/llm/utils/test_case_utils.py	221	frontend_tools: Optional[Union[str, List[Dict[str, Any]]]] = (	COMMENT
LOW	tests/llm/utils/reporting/github_reporter.py	701		COMMENT
LOW	tests/core/test_supabase_dal_usage.py	201	assert payload["cached_tokens"] is None	COMMENT
LOW	tests/core/conversations_worker/test_runtime_polling.py	41	# When realtime is connected, we still poll as a safety net for missed	COMMENT
LOW	…/conversations_worker/test_processor_usage_recorder.py	441		COMMENT
LOW	…ugins/toolsets/kubernetes/test_kubernetes_logs_unit.py	21	# init-config-reloader alertmanager config-reloader	COMMENT
LOW	docs/installation/kubernetes-installation.md	81	value: "https://your-resource.openai.azure.com/"	COMMENT
LOW	docs/data-sources/builtin-toolsets/zabbix.md	181	## Troubleshooting	COMMENT
LOW	docs/data-sources/builtin-toolsets/zabbix.md	201	# - Ensure all required parameters are included in the request	COMMENT
LOW	docs/operator/destinations.md	221	- If the check has an associated URL, it's included as a link in the incident	COMMENT
LOW	examples/azure-mcp-additional-instance.yaml	1	# =============================================================================	COMMENT
LOW	examples/azure-mcp-additional-instance.yaml	21	#	COMMENT
LOW	examples/azure-mcp-additional-instance.yaml	261	# mcp_servers:	COMMENT
LOW	scripts/build_go_binaries.sh	1	#!/bin/bash	COMMENT
LOW	scripts/build_go_binaries.sh	21	# in 0.52.0; 0.55.0 adds the CVE-2026-56854	COMMENT
LOW	scripts/build_go_binaries.sh	41	# go-billy >= 5.9.0 (blocked on go-git/go-git#1551 upstream).	COMMENT
LOW	scripts/build_go_binaries.sh	61	# was reverted — see git history if it ever needs to come back.	COMMENT
LOW	scripts/setup-sandbox-k8s.sh	1	#!/usr/bin/env bash	COMMENT
LOW	scripts/setup-sandbox-k8s.sh	21	# before invoking the real runc.	COMMENT
LOW	holmes/core/tools.py	161	return {k: sanitize(str(v)) for k, v in params.items()}	COMMENT
LOW	holmes/core/llm.py	741	# Always send an explicit output-token limit: without one, litellm falls	COMMENT
LOW	holmes/core/llm.py	1241	if model_params is not None:	COMMENT
LOW	holmes/core/relay_refusal.py	21	# HTTP status before the LLM call runs, so this is where the client reads the	COMMENT
LOW	holmes/core/usage_recorder.py	221	# values dashboards group by:	COMMENT
LOW	holmes/core/usage_recorder.py	241	# True when the LLM call hit a Robusta-managed model (Robusta paid	COMMENT
LOW	holmes/core/usage_recorder.py	261		COMMENT
LOW	holmes/core/usage_recorder.py	281	# 'chat_history' — direct /api/chat path (defaulted by chat() when	COMMENT
LOW	holmes/core/usage_recorder.py	301	# here. NULL for CLI / ad-hoc flows that have no cluster context.	COMMENT
LOW	holmes/core/usage_recorder.py	321		COMMENT
LOW	holmes/core/usage_recorder.py	341	# Aggregate token / cost counters. Filled by `stream_with_usage_recording`	COMMENT
LOW	holmes/core/issue.py	41	# Controls whether to append status to the title in Slack messages (e.g., " - open")	COMMENT
LOW	holmes/core/conversations_worker/processor.py	161	# task.metadata when the per-turn event omits the key entirely.	COMMENT
LOW	holmes/core/conversations_worker/event_publisher.py	21	}	COMMENT
LOW	holmes/plugins/skills/git_skill_repos.py	41		COMMENT
LOW	holmes/plugins/skills/git_skill_repos.py	601	"--detach",	COMMENT
LOW	holmes/plugins/toolsets/connectivity_check.py	201	# destination from the internal-IP block.	COMMENT
LOW	holmes/plugins/toolsets/bash/validation.py	41		COMMENT
LOW	holmes/plugins/toolsets/skills/skills_fetcher.py	101	return self._get_robusta_skill(skill_id, params)	COMMENT
LOW	holmes/plugins/toolsets/prometheus/prometheus.py	2161	subtype = getattr(type(self.config), "_subtype", None) if self.config else None	COMMENT
LOW	holmes/utils/holmes_sync_skills.py	101	# Stable published path, not the sha-bearing worktree the	COMMENT
LOW	helm/holmes/values.yaml	1	certificate: "" # base64 encoded	COMMENT
LOW	helm/holmes/values.yaml	161	# - /etc/holmes/my-skills	COMMENT
LOW	helm/holmes/values.yaml	181	# for entries with tokenSecret, injects the token env var from that Secret.	COMMENT
LOW	helm/holmes/values.yaml	221	# - name: fetch-secret	COMMENT
LOW	helm/holmes/values.yaml	241	# single env-var namespace — same-name keys shadow each other in list order.	COMMENT
LOW	helm/holmes/values.yaml	321	# Extra volume mounts to add to the AWS MCP container	COMMENT
LOW	helm/holmes/values.yaml	341	# --from-file=key.json=key.json -n YOUR_NAMESPACE	COMMENT
LOW	helm/holmes/values.yaml	481	# Option 1 - PAT: Deploys a self-hosted MCP server using the standard github-mcp image	COMMENT
LOW	helm/holmes/values.yaml	501	# Authentication - choose ONE of the two methods below	COMMENT
LOW	helm/holmes/values.yaml	741	# Holmes's human-approval gate. The chart generates a random per-release	COMMENT
LOW	helm/holmes/values.yaml	761	annotations: {}	COMMENT
4 more matches not shown…

Verbosity Indicators25 hits · 44 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/test_mcp_oauth.py	1173	# Step 1: Discover OAuth endpoints	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1194	# Step 2: Generate PKCE	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1197	# Step 3: Start local callback server (port 0 = OS-assigned)	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1261	# Step 5: Wait for callback	COMMENT
LOW⚡	tests/test_mcp_oauth.py	1270	# Step 6: Exchange code for token	COMMENT
LOW	tests/test_mcp_oauth.py	1244	# Step 4: Build authorization URL and open browser	COMMENT
LOW	tests/test_mcp_oauth.py	1292	# Step 7: Use token to list MCP tools	COMMENT
LOW	tests/llm/conftest.py	458	# Step 2: Environment is OK, now test if the API actually works	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	309	# Step 1: Detect approval needed	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	317	# Step 2: Simulate user approval (in real flow, this happens via UI/CLI)	COMMENT
LOW⚡	tests/toolsets/bash/test_bash_approval_flow.py	321	# Step 3: Execute with approval	COMMENT
LOW	holmes/core/llm.py	997	# so we need to check if the user has set an OPENAI_API_KEY to load the config model.	COMMENT
LOW⚡	…lmes/plugins/toolsets/grafana/toolset_grafana_tempo.py	360	# Step 2: Sort traces by duration	COMMENT
LOW⚡	…lmes/plugins/toolsets/grafana/toolset_grafana_tempo.py	363	# Step 3: Calculate basic statistics	COMMENT
LOW⚡	…lmes/plugins/toolsets/grafana/toolset_grafana_tempo.py	384	# Step 4: Select representative traces to fetch	COMMENT
LOW⚡	…lmes/plugins/toolsets/grafana/toolset_grafana_tempo.py	393	# Step 5: Fetch full trace details	COMMENT
LOW	…lmes/plugins/toolsets/grafana/toolset_grafana_tempo.py	332	# Step 1: Get all trace summaries	COMMENT
LOW	holmes/plugins/toolsets/grafana/trace_parser.py	35	# Step 1: Extract all spans and create span objects	COMMENT
LOW	holmes/plugins/toolsets/grafana/trace_parser.py	78	# Step 2: Build the hierarchy by connecting parents and children	COMMENT
LOW⚡	holmes/plugins/toolsets/mcp/toolset_mcp.py	1318	# Step 1: Find auth server via Protected Resource Metadata (RFC 9728)	COMMENT
LOW⚡	holmes/plugins/toolsets/mcp/toolset_mcp.py	1325	# Step 2: Fetch OAuth/OIDC metadata	COMMENT
LOW⚡	holmes/plugins/toolsets/bash/validation.py	522	# Step 1: Check hardcoded blocks	COMMENT
LOW⚡	holmes/plugins/toolsets/bash/validation.py	531	# Step 2: Check deny list (using stricter matching)	COMMENT
LOW⚡	holmes/plugins/toolsets/bash/validation.py	540	# Step 3: Check allow list	COMMENT
LOW⚡	holmes/plugins/toolsets/bash/validation.py	545	# Step 4: Not in any list -> needs approval	COMMENT

AI Slop Vocabulary15 hits · 28 pts

Severity	File	Line	Snippet	Context
MEDIUM	run_benchmarks_local.py	341	%(prog)s --models gpt-4o --benchmark-type full # Full comprehensive benchmark	CODE
MEDIUM	tests/test_bash_session_prefix_forgery.py	133	"""End-to-end reproduction, robust to the chosen fix: a forged approval note	STRING
LOW	tests/llm/utils/test_case_utils.py	101	# TODO: do we ever use this? or do we always just use float below	COMMENT
MEDIUM	tests/llm/utils/braintrust.py	92	# Build comprehensive metadata	COMMENT
MEDIUM	tests/core/test_supabase_dal_transport.py	39	# Start from a clean CA-env slate; the test harness/sandbox may set these.	COMMENT
LOW	tests/checks/test_checks_cli.py	53	# Note: checks_app already has "check" as the command name, so we just pass the options	COMMENT
LOW	…s/integration/test_kubernetes_transformer_execution.py	50	# If we don't have much space, just use a short summary	COMMENT
MEDIUM⚡	examples/azure-mcp-additional-instance.yaml	263	# description: "Azure API MCP Server (replaceme account) - comprehensive Azure service access. Execute any Azure CLI	COMMENT
LOW	holmes/core/tools.py	729	check=False, # do not throw error, we just return the error code	CODE
LOW	holmes/core/llm.py	513	# litellm.validate_environment simply set all AZURE_* variables to missing_keys for azure models when any	COMMENT
LOW	holmes/plugins/toolsets/kubernetes_logs.py	246	check=False, # do not throw error, we just return the error code	CODE
LOW	holmes/plugins/toolsets/kubernetes_logs.py	434	# If we can't parse it, just return the original	COMMENT
MEDIUM	holmes/plugins/toolsets/kafka.py	1245	"""Provides comprehensive health metrics and status of a Kafka cluster."""	STRING
LOW	holmes/plugins/toolsets/prometheus/prometheus.py	1721	# For other status codes, just return the status code and content	COMMENT
MEDIUM	helm/holmes/values.yaml	311	# Leave empty to use default comprehensive instructions	COMMENT

AI Structural Patterns21 hits · 20 pts

Severity	File	Line	Context
LOW	tempo_cli.py	129	CODE
LOW	tempo_cli.py	176	CODE
LOW	tempo_cli.py	211	CODE
LOW	tempo_cli.py	258	CODE
LOW	tempo_cli.py	303	CODE
LOW	tempo_cli.py	335	CODE
LOW	holmes_operator/utils.py	20	CODE
LOW	holmes/config.py	667	CODE
LOW	holmes/interactive.py	2274	CODE
LOW	holmes/interactive.py	2619	CODE
LOW	holmes/main.py	194	CODE
LOW	holmes/main.py	456	CODE
LOW	holmes/main.py	591	CODE
LOW	holmes/main.py	684	CODE
LOW	holmes/main.py	804	CODE
LOW	holmes/main.py	889	CODE
LOW	holmes/main.py	974	CODE
LOW	holmes/core/tool_calling_llm.py	1160	CODE
LOW	holmes/core/toolset_manager.py	88	CODE
LOW	holmes/plugins/toolsets/mcp/oauth_token_store.py	103	CODE
LOW	holmes/checks/checks_cli.py	40	CODE

Cross-File Repetition3 hits · 15 pts

Severity	File	Snippet	Context
HIGH	holmes/plugins/toolsets/datadog/toolset_datadog_logs.py	get a one-liner description of the tool invocation.	STRING
HIGH	…es/plugins/toolsets/datadog/toolset_datadog_general.py	get a one-liner description of the tool invocation.	STRING
HIGH	…mes/plugins/toolsets/datadog/toolset_datadog_traces.py	get a one-liner description of the tool invocation.	STRING

Hallucination Indicators1 hit · 10 pts

Severity	File	Line	Snippet	Context
CRITICAL	.github/workflows/eval-regression.yaml	537	const sha = context.payload.pull_request.head.sha.substring(0, 7);	CODE

Fake / Example Data7 hits · 8 pts

Severity	File	Line	Snippet	Context
LOW⚡	tests/test_server_ssl.py	31	key.write_text("dummy-key")	CODE
LOW	tests/core/test_supabase_dal.py	36	dal.email = "user@example.com"	CODE
LOW	tests/core/test_image_token_counting.py	69	return DefaultLLM(model=model, api_key="fake-key")	CODE
LOW	tests/plugins/toolsets/test_notion.py	11	"Authorization": "Bearer fake_token",	CODE
LOW⚡	tests/plugins/toolsets/test_confluence_tools.py	25	api_key="fake-token",	CODE
LOW	tests/plugins/toolsets/bash/test_read_image_file.py	102	img_file.write_bytes(b"fake-data")	CODE
LOW	docs/ai-providers/ollama.md	101	export OPENAI_API_KEY="dummy-key" # Required but can be any value	CODE

Dead Code1 hit · 2 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/test_server_endpoints.py	72		CODE

Overly Generic Function Names1 hit · 1 pts

Severity	File	Line	Snippet	Context
LOW	holmes/core/supabase_dal.py	244	def handle_request(self, request: httpx.Request) -> httpx.Response:	CODE

Slop Phrases1 hit · 1 pts

Severity	File	Line	Snippet	Context
LOW	holmes/plugins/sources/opsgenie/__init__.py	11	OPSGENIE_TEAM_INTEGRATION_KEY_HELP = "OpsGenie Team Integration key for writing back results. (NOT a normal API Key.) Ge	CODE

Maintenance and Governance Review

Risk Review · v3

What These Metrics Mean

Pattern-Signal History

Severity Breakdown

Directory Score Breakdown

Pattern Findings