Full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit
This report presents the forensic synthetic code analysis of JoshuaC215/agent-service-toolkit, a Python project with 4,377 GitHub stars. SynthScan v2.0 examined 11,978 lines of code across 102 source files, recording 223 pattern matches distributed across 14 syntactic categories. The overall adjusted score of 27.6 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 223 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/conftest.py | 17 | def pytest_collection_modifyitems(config, items): | CODE |
| LOW | tests/smoke/test_persistence.py | 14 | def test_checkpointer_persists_history(): | CODE |
| LOW | tests/core/test_llm.py | 56 | def test_get_model_groq_guard(): | CODE |
| LOW | tests/core/test_llm.py | 64 | def test_get_model_groq_gpt_oss(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 31 | def test_settings_default_values(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 39 | def test_settings_no_api_keys(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 46 | def test_settings_with_openai_key(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 54 | def test_settings_with_anthropic_key(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 62 | def test_settings_with_vertexai_credentials_file(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 70 | def test_settings_with_multiple_api_keys(): | CODE |
| LOW | tests/core/test_settings.py | 90 | def test_settings_use_fake_model(): | CODE |
| LOW | tests/core/test_settings.py | 97 | def test_settings_use_fake_model_wins_over_ambient_real_keys(): | CODE |
| LOW | tests/core/test_settings.py | 116 | def test_settings_explicit_default_model_overrides_fake(): | CODE |
| LOW | tests/core/test_settings.py | 140 | def test_settings_with_azure_openai_key(): | CODE |
| LOW | tests/core/test_settings.py | 156 | def test_settings_with_both_openai_and_azure(): | CODE |
| LOW | tests/core/test_settings.py | 178 | def test_settings_azure_deployment_names(): | CODE |
| LOW | tests/core/test_settings.py | 183 | def test_settings_azure_missing_deployment_names(): | CODE |
| LOW | tests/core/test_settings.py | 196 | def test_settings_azure_deployment_map(): | CODE |
| LOW | tests/core/test_settings.py | 213 | def test_settings_azure_invalid_deployment_map(): | CODE |
| LOW | tests/core/test_settings.py | 227 | def test_settings_azure_openai(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 253 | def test_settings_log_level_default(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 261 | def test_settings_log_level_from_env(): | CODE |
| LOW⚡ | tests/core/test_settings.py | 269 | def test_settings_log_level_invalid(): | CODE |
| LOW | tests/app/test_streamlit_app.py | 12 | def test_app_simple_non_streaming(mock_agent_client): | CODE |
| LOW | tests/app/test_streamlit_app.py | 75 | def test_app_thread_id_history(mock_agent_client): | CODE |
| LOW | tests/app/test_streamlit_app.py | 100 | def test_app_resume_with_agent_param(mock_agent_client): | CODE |
| LOW | tests/app/test_streamlit_app.py | 339 | async def test_app_streaming_single_sub_agent(mock_agent_client, multi_agent_messages): | CODE |
| LOW | tests/app/test_streamlit_app.py | 409 | async def test_app_streaming_sequential_sub_agents(mock_agent_client, multi_agent_messages): | CODE |
| LOW | tests/app/test_streamlit_app.py | 499 | async def test_app_streaming_nested_sub_agents(mock_agent_client, multi_agent_messages): | CODE |
| LOW | tests/integration/test_docker_e2e.py | 8 | def test_service_with_fake_model(): | CODE |
| LOW⚡ | tests/agents/test_lazy_agent.py | 43 | def test_get_graph_before_load(self): | CODE |
| LOW⚡ | tests/agents/test_lazy_agent.py | 49 | def test_get_graph_after_load(self): | CODE |
| LOW⚡ | tests/agents/test_lazy_agent.py | 58 | def test_get_graph_no_graph_created(self): | CODE |
| LOW | tests/agents/test_agent_loading.py | 15 | async def test_load_agent_static_agent(self): | CODE |
| LOW | tests/agents/test_agent_loading.py | 22 | async def test_load_agent_lazy_agent(self): | CODE |
| LOW⚡ | tests/agents/test_agent_loading.py | 34 | async def test_load_agent_nonexistent(self): | CODE |
| LOW⚡ | tests/agents/test_agent_loading.py | 39 | def test_get_agent_static_agent(self): | CODE |
| LOW⚡ | tests/agents/test_agent_loading.py | 44 | def test_get_agent_lazy_agent_not_loaded(self): | CODE |
| LOW | tests/agents/test_agent_loading.py | 55 | def test_get_agent_lazy_agent_loaded(self): | CODE |
| LOW | tests/agents/test_agent_loading.py | 68 | def test_get_agent_nonexistent(self): | CODE |
| LOW | tests/agents/test_github_mcp_agent.py | 22 | async def test_load_without_github_pat(self): | CODE |
| LOW | tests/agents/test_github_mcp_agent.py | 35 | async def test_load_with_github_pat(self): | CODE |
| LOW | tests/agents/test_github_mcp_agent.py | 118 | def test_get_graph_not_loaded(self): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 11 | def test_init_with_openai_provider(mock_openai_client): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 19 | def test_init_with_invalid_provider(): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 26 | def test_init_with_unimplemented_provider(): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 33 | def test_from_env_provider_not_set(): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 41 | def test_from_env_valid_provider(mock_openai_client): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 51 | def test_from_env_invalid_provider_returns_none(): | CODE |
| LOW⚡ | tests/voice/test_tts.py | 59 | def test_get_api_key_from_param(mock_openai_client): | CODE |
| LOW | tests/voice/test_tts.py | 70 | def test_get_api_key_from_env(mock_openai_client): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 11 | def test_init_with_openai_provider(mock_openai_client): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 19 | def test_init_with_invalid_provider(): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 26 | def test_init_with_unimplemented_provider(): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 33 | def test_from_env_provider_not_set(): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 41 | def test_from_env_valid_provider(mock_openai_client): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 51 | def test_from_env_invalid_provider_returns_none(): | CODE |
| LOW⚡ | tests/voice/test_stt.py | 59 | def test_get_api_key_from_param(mock_openai_client): | CODE |
| LOW | tests/voice/test_stt.py | 70 | def test_get_api_key_from_env(mock_openai_client): | CODE |
| LOW | tests/voice/test_manager.py | 8 | def test_init_with_both_stt_and_tts(): | CODE |
| 39 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/service/test_service.py | 38 | # Create a separate mock for the default agent | COMMENT |
| MEDIUM | tests/client/test_client.py | 190 | # Create an async iterator for the events | COMMENT |
| MEDIUM | scripts/create_chroma_db.py | 71 | # Create the Chroma database | COMMENT |
| MEDIUM | src/streamlit_app.py | 400 | # Create a status container for each tool call and store the | COMMENT |
| MEDIUM | src/streamlit_app.py | 575 | # Create a nested status container for the sub-agent | COMMENT |
| MEDIUM | src/agents/rag_assistant.py | 95 | # Define the graph | STRING |
| MEDIUM | src/agents/interrupt_agent.py | 220 | # Define the graph | COMMENT |
| MEDIUM | src/agents/safeguard.py | 103 | # Create a formatted history string to evaluate | STRING |
| MEDIUM | src/agents/tools.py | 51 | # Create the embedding function for our project description database | COMMENT |
| MEDIUM | src/agents/command_agent.py | 16 | # Define the nodes | COMMENT |
| MEDIUM⚡ | src/agents/knowledge_base_agent.py | 18 | # Define the state | COMMENT |
| MEDIUM⚡ | src/agents/knowledge_base_agent.py | 27 | # Create the retriever | COMMENT |
| MEDIUM⚡ | src/agents/knowledge_base_agent.py | 35 | # Create the retriever with the specified Knowledge Base ID | COMMENT |
| MEDIUM | src/agents/knowledge_base_agent.py | 157 | # Define the graph | COMMENT |
| MEDIUM | src/agents/research_assistant.py | 99 | # Define the graph | STRING |
| MEDIUM | src/agents/github_mcp_agent/github_mcp_agent.py | 101 | # Create the agent instance | COMMENT |
| MEDIUM | src/agents/bg_task_agent/bg_task_agent.py | 53 | # Define the graph | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/check_live_models.py | 88 | except Exception as e: | CODE |
| LOW | scripts/e2e_ui_tests.py | 90 | except Exception: | CODE |
| LOW | scripts/e2e_ui_tests.py | 437 | except Exception as e: | CODE |
| LOW | scripts/e2e_ui_tests.py | 461 | except Exception: | CODE |
| MEDIUM | scripts/e2e_ui_tests.py | 87 | def launch_browser(p) -> Browser: | CODE |
| LOW | scripts/smoke_live_app.py | 50 | except Exception: | CODE |
| LOW | scripts/smoke_live_app.py | 73 | except Exception: | CODE |
| LOW | src/core/settings.py | 242 | except Exception as e: | CODE |
| LOW | src/agents/interrupt_agent.py | 117 | except Exception as e: | CODE |
| LOW | src/agents/interrupt_agent.py | 163 | except Exception as e: | CODE |
| LOW | src/agents/tools.py | 34 | except Exception as e: | CODE |
| LOW | src/agents/tools.py | 54 | except Exception as e: | CODE |
| LOW | src/agents/knowledge_base_agent.py | 119 | except Exception as e: | CODE |
| LOW | src/agents/github_mcp_agent/github_mcp_agent.py | 80 | except Exception as e: | CODE |
| LOW | src/voice/tts.py | 150 | except Exception as e: | CODE |
| LOW | src/voice/stt.py | 142 | except Exception as e: | CODE |
| LOW | src/voice/providers/openai_tts.py | 112 | except Exception as e: | CODE |
| LOW | src/voice/providers/openai_stt.py | 54 | except Exception as e: | CODE |
| LOW | src/service/service.py | 95 | except Exception as e: | CODE |
| LOW | src/service/service.py | 105 | except Exception as e: | CODE |
| LOW | src/service/service.py | 221 | except Exception as e: | CODE |
| LOW | src/service/service.py | 307 | except Exception as e: | CODE |
| LOW | src/service/service.py | 332 | except Exception as e: | CODE |
| LOW | src/service/service.py | 420 | except Exception as e: | CODE |
| LOW | src/service/service.py | 435 | except Exception as e: | CODE |
| LOW | src/client/client.py | 185 | except Exception as e: | CODE |
| LOW | src/client/client.py | 192 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/core/__init__.py | 1 | CODE | |
| LOW | src/core/__init__.py | 2 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/agents/__init__.py | 1 | CODE | |
| LOW | src/schema/__init__.py | 1 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/schema/__init__.py | 2 | CODE | |
| LOW | src/voice/__init__.py | 33 | CODE | |
| LOW | src/voice/__init__.py | 34 | CODE | |
| LOW | src/voice/__init__.py | 35 | CODE | |
| LOW | src/voice/providers/__init__.py | 3 | CODE | |
| LOW | src/voice/providers/__init__.py | 4 | CODE | |
| LOW | src/service/__init__.py | 1 | CODE | |
| LOW | src/client/__init__.py | 1 | CODE | |
| LOW | src/client/__init__.py | 1 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | src/agents/rag_assistant.py | 0 | `total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totality | STRING |
| HIGH | src/agents/interrupt_agent.py | 0 | `total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totality | STRING |
| HIGH | src/agents/command_agent.py | 0 | `total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totality | STRING |
| HIGH | src/agents/research_assistant.py | 0 | `total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totality | STRING |
| HIGH | src/agents/bg_task_agent/bg_task_agent.py | 0 | `total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totality | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/core/__init__.py | 4 | __all__ = ["settings", "get_model"] | CODE |
| LOW | src/memory/mongodb.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/memory/__init__.py | 40 | __all__ = ["initialize_database", "initialize_store"] | CODE |
| LOW | src/memory/postgres.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/agents/interrupt_agent.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/agents/__init__.py | 10 | __all__ = [ | CODE |
| LOW⚡ | src/agents/knowledge_base_agent.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/agents/github_mcp_agent/github_mcp_agent.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/schema/__init__.py | 14 | __all__ = [ | CODE |
| LOW | src/voice/tts.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/voice/__init__.py | 37 | __all__ = ["VoiceManager", "SpeechToText", "TextToSpeech"] | CODE |
| LOW | src/voice/manager.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/voice/stt.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/voice/providers/openai_tts.py | 7 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/voice/providers/__init__.py | 10 | __all__ = ["OpenAISTT", "OpenAITTS"] | CODE |
| LOW | src/voice/providers/openai_stt.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/service/service.py | 46 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/service/agui.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/service/__init__.py | 3 | __all__ = ["app"] | CODE |
| LOW | src/client/client.py | 75 | def update_agent(self, agent: str, verify: bool = True) -> None: | CODE |
| LOW | src/client/__init__.py | 3 | __all__ = ["AgentClient", "AgentClientError"] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .claude/skills/maintainer-response/SKILL.md | 156 | ### Be skeptical of automated / AI-generated PRs | COMMENT |
| HIGH | .claude/skills/maintainer-response/SKILL.md | 249 | layer that *unconditionally* adds a "with Claude" badge and a "Generated by Claude Code" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/create_chroma_db.py | 81 | # Display results | COMMENT |
| LOW | src/streamlit_app.py | 34 | # Check if user_id exists in session state | COMMENT |
| LOW | src/streamlit_app.py | 573 | # Check if this is a nested transfer/delegate | COMMENT |
| LOW⚡ | src/agents/interrupt_agent.py | 93 | # Check if we already have the birthdate in the store for this user | COMMENT |
| LOW⚡ | src/agents/interrupt_agent.py | 98 | if result: # Check if anything was returned | CODE |
| LOW⚡ | src/agents/interrupt_agent.py | 100 | if result: # Check if list is not empty | CODE |
| LOW | src/agents/knowledge_base_agent.py | 66 | # Check if documents were retrieved | STRING |
| LOW | src/voice/manager.py | 99 | # Check if transcription succeeded | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | src/agents/safeguard.py | 60 | Answer: {"violation": 0, "category": null, "rationale": "Legitimate question about capabilities"} | CODE |
| HIGH | src/agents/safeguard.py | 64 | Answer: {"violation": 0, "category": null, "rationale": "Normal programming assistance request"} | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | src/voice/tts.py | 78 | Load the appropriate TTS provider implementation. Args: provider: Provider name api_key | STRING |
| HIGH | src/voice/stt.py | 76 | Load the appropriate STT provider implementation. Args: provider: Provider name api_key | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/e2e_ui_tests.py | 145 | CODE | |
| LOW | scripts/e2e_ui_tests.py | 400 | CODE | |
| LOW | scripts/smoke_live_app.py | 66 | CODE | |
| LOW | src/streamlit_app.py | 56 | CODE | |
| LOW | src/streamlit_app.py | 319 | CODE | |
| LOW | src/streamlit_app.py | 508 | CODE | |
| LOW | src/agents/interrupt_agent.py | 77 | CODE | |
| LOW | src/service/service.py | 226 | CODE | |
| LOW | src/client/client.py | 202 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | docs/RAG_Assistant.md | 1 | # Creating a RAG assistant | COMMENT |
| MEDIUM | docs/RAG_Assistant.md | 20 | ## Configuring the RAG assistant | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/smoke_test.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | scripts/smoke_test.sh | 21 | # Only databases run in Docker; the service itself runs on the host via uv, | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/smoke_test.sh | 7 | # Usage: | COMMENT |