Repository Analysis

JoshuaC215/agent-service-toolkit

Full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit

27.6 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of JoshuaC215/agent-service-toolkit, a Python project with 4,377 GitHub stars. SynthScan v2.0 examined 11,978 lines of code across 102 source files, recording 223 pattern matches distributed across 14 syntactic categories. The overall adjusted score of 27.6 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

27.6
Adjusted Score
27.6
Raw Score
100%
Time Factor
2026-07-14
Last Push
4.4K
Stars
Python
Language
12.0K
Lines of Code
102
Files
223
Pattern Hits
2026-07-14
Scan Date
0.11
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 11MEDIUM 20LOW 192

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 223 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hyper-Verbose Identifiers99 hits · 118 pts
SeverityFileLineSnippetContext
LOWtests/conftest.py17def pytest_collection_modifyitems(config, items):CODE
LOWtests/smoke/test_persistence.py14def test_checkpointer_persists_history():CODE
LOWtests/core/test_llm.py56def test_get_model_groq_guard():CODE
LOWtests/core/test_llm.py64def test_get_model_groq_gpt_oss():CODE
LOWtests/core/test_settings.py31def test_settings_default_values():CODE
LOWtests/core/test_settings.py39def test_settings_no_api_keys():CODE
LOWtests/core/test_settings.py46def test_settings_with_openai_key():CODE
LOWtests/core/test_settings.py54def test_settings_with_anthropic_key():CODE
LOWtests/core/test_settings.py62def test_settings_with_vertexai_credentials_file():CODE
LOWtests/core/test_settings.py70def test_settings_with_multiple_api_keys():CODE
LOWtests/core/test_settings.py90def test_settings_use_fake_model():CODE
LOWtests/core/test_settings.py97def test_settings_use_fake_model_wins_over_ambient_real_keys():CODE
LOWtests/core/test_settings.py116def test_settings_explicit_default_model_overrides_fake():CODE
LOWtests/core/test_settings.py140def test_settings_with_azure_openai_key():CODE
LOWtests/core/test_settings.py156def test_settings_with_both_openai_and_azure():CODE
LOWtests/core/test_settings.py178def test_settings_azure_deployment_names():CODE
LOWtests/core/test_settings.py183def test_settings_azure_missing_deployment_names():CODE
LOWtests/core/test_settings.py196def test_settings_azure_deployment_map():CODE
LOWtests/core/test_settings.py213def test_settings_azure_invalid_deployment_map():CODE
LOWtests/core/test_settings.py227def test_settings_azure_openai():CODE
LOWtests/core/test_settings.py253def test_settings_log_level_default():CODE
LOWtests/core/test_settings.py261def test_settings_log_level_from_env():CODE
LOWtests/core/test_settings.py269def test_settings_log_level_invalid():CODE
LOWtests/app/test_streamlit_app.py12def test_app_simple_non_streaming(mock_agent_client):CODE
LOWtests/app/test_streamlit_app.py75def test_app_thread_id_history(mock_agent_client):CODE
LOWtests/app/test_streamlit_app.py100def test_app_resume_with_agent_param(mock_agent_client):CODE
LOWtests/app/test_streamlit_app.py339async def test_app_streaming_single_sub_agent(mock_agent_client, multi_agent_messages):CODE
LOWtests/app/test_streamlit_app.py409async def test_app_streaming_sequential_sub_agents(mock_agent_client, multi_agent_messages):CODE
LOWtests/app/test_streamlit_app.py499async def test_app_streaming_nested_sub_agents(mock_agent_client, multi_agent_messages):CODE
LOWtests/integration/test_docker_e2e.py8def test_service_with_fake_model():CODE
LOWtests/agents/test_lazy_agent.py43 def test_get_graph_before_load(self):CODE
LOWtests/agents/test_lazy_agent.py49 def test_get_graph_after_load(self):CODE
LOWtests/agents/test_lazy_agent.py58 def test_get_graph_no_graph_created(self):CODE
LOWtests/agents/test_agent_loading.py15 async def test_load_agent_static_agent(self):CODE
LOWtests/agents/test_agent_loading.py22 async def test_load_agent_lazy_agent(self):CODE
LOWtests/agents/test_agent_loading.py34 async def test_load_agent_nonexistent(self):CODE
LOWtests/agents/test_agent_loading.py39 def test_get_agent_static_agent(self):CODE
LOWtests/agents/test_agent_loading.py44 def test_get_agent_lazy_agent_not_loaded(self):CODE
LOWtests/agents/test_agent_loading.py55 def test_get_agent_lazy_agent_loaded(self):CODE
LOWtests/agents/test_agent_loading.py68 def test_get_agent_nonexistent(self):CODE
LOWtests/agents/test_github_mcp_agent.py22 async def test_load_without_github_pat(self):CODE
LOWtests/agents/test_github_mcp_agent.py35 async def test_load_with_github_pat(self):CODE
LOWtests/agents/test_github_mcp_agent.py118 def test_get_graph_not_loaded(self):CODE
LOWtests/voice/test_tts.py11def test_init_with_openai_provider(mock_openai_client):CODE
LOWtests/voice/test_tts.py19def test_init_with_invalid_provider():CODE
LOWtests/voice/test_tts.py26def test_init_with_unimplemented_provider():CODE
LOWtests/voice/test_tts.py33def test_from_env_provider_not_set():CODE
LOWtests/voice/test_tts.py41def test_from_env_valid_provider(mock_openai_client):CODE
LOWtests/voice/test_tts.py51def test_from_env_invalid_provider_returns_none():CODE
LOWtests/voice/test_tts.py59def test_get_api_key_from_param(mock_openai_client):CODE
LOWtests/voice/test_tts.py70def test_get_api_key_from_env(mock_openai_client):CODE
LOWtests/voice/test_stt.py11def test_init_with_openai_provider(mock_openai_client):CODE
LOWtests/voice/test_stt.py19def test_init_with_invalid_provider():CODE
LOWtests/voice/test_stt.py26def test_init_with_unimplemented_provider():CODE
LOWtests/voice/test_stt.py33def test_from_env_provider_not_set():CODE
LOWtests/voice/test_stt.py41def test_from_env_valid_provider(mock_openai_client):CODE
LOWtests/voice/test_stt.py51def test_from_env_invalid_provider_returns_none():CODE
LOWtests/voice/test_stt.py59def test_get_api_key_from_param(mock_openai_client):CODE
LOWtests/voice/test_stt.py70def test_get_api_key_from_env(mock_openai_client):CODE
LOWtests/voice/test_manager.py8def test_init_with_both_stt_and_tts():CODE
39 more matches not shown…
Self-Referential Comments17 hits · 50 pts
SeverityFileLineSnippetContext
MEDIUMtests/service/test_service.py38 # Create a separate mock for the default agentCOMMENT
MEDIUMtests/client/test_client.py190 # Create an async iterator for the eventsCOMMENT
MEDIUMscripts/create_chroma_db.py71 # Create the Chroma databaseCOMMENT
MEDIUMsrc/streamlit_app.py400 # Create a status container for each tool call and store theCOMMENT
MEDIUMsrc/streamlit_app.py575 # Create a nested status container for the sub-agentCOMMENT
MEDIUMsrc/agents/rag_assistant.py95# Define the graphSTRING
MEDIUMsrc/agents/interrupt_agent.py220# Define the graphCOMMENT
MEDIUMsrc/agents/safeguard.py103 # Create a formatted history string to evaluateSTRING
MEDIUMsrc/agents/tools.py51 # Create the embedding function for our project description databaseCOMMENT
MEDIUMsrc/agents/command_agent.py16# Define the nodesCOMMENT
MEDIUMsrc/agents/knowledge_base_agent.py18# Define the stateCOMMENT
MEDIUMsrc/agents/knowledge_base_agent.py27# Create the retrieverCOMMENT
MEDIUMsrc/agents/knowledge_base_agent.py35 # Create the retriever with the specified Knowledge Base IDCOMMENT
MEDIUMsrc/agents/knowledge_base_agent.py157# Define the graphCOMMENT
MEDIUMsrc/agents/research_assistant.py99# Define the graphSTRING
MEDIUMsrc/agents/github_mcp_agent/github_mcp_agent.py101# Create the agent instanceCOMMENT
MEDIUMsrc/agents/bg_task_agent/bg_task_agent.py53# Define the graphCOMMENT
Excessive Try-Catch Wrapping27 hits · 28 pts
SeverityFileLineSnippetContext
LOWscripts/check_live_models.py88 except Exception as e:CODE
LOWscripts/e2e_ui_tests.py90 except Exception:CODE
LOWscripts/e2e_ui_tests.py437 except Exception as e:CODE
LOWscripts/e2e_ui_tests.py461 except Exception:CODE
MEDIUMscripts/e2e_ui_tests.py87def launch_browser(p) -> Browser:CODE
LOWscripts/smoke_live_app.py50 except Exception:CODE
LOWscripts/smoke_live_app.py73 except Exception:CODE
LOWsrc/core/settings.py242 except Exception as e:CODE
LOWsrc/agents/interrupt_agent.py117 except Exception as e:CODE
LOWsrc/agents/interrupt_agent.py163 except Exception as e:CODE
LOWsrc/agents/tools.py34 except Exception as e:CODE
LOWsrc/agents/tools.py54 except Exception as e:CODE
LOWsrc/agents/knowledge_base_agent.py119 except Exception as e:CODE
LOWsrc/agents/github_mcp_agent/github_mcp_agent.py80 except Exception as e:CODE
LOWsrc/voice/tts.py150 except Exception as e:CODE
LOWsrc/voice/stt.py142 except Exception as e:CODE
LOWsrc/voice/providers/openai_tts.py112 except Exception as e:CODE
LOWsrc/voice/providers/openai_stt.py54 except Exception as e:CODE
LOWsrc/service/service.py95 except Exception as e:CODE
LOWsrc/service/service.py105 except Exception as e:CODE
LOWsrc/service/service.py221 except Exception as e:CODE
LOWsrc/service/service.py307 except Exception as e:CODE
LOWsrc/service/service.py332 except Exception as e:CODE
LOWsrc/service/service.py420 except Exception as e:CODE
LOWsrc/service/service.py435 except Exception as e:CODE
LOWsrc/client/client.py185 except Exception as e:CODE
LOWsrc/client/client.py192 except Exception as e:CODE
Unused Imports26 hits · 26 pts
SeverityFileLineSnippetContext
LOWsrc/core/__init__.py1CODE
LOWsrc/core/__init__.py2CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/agents/__init__.py1CODE
LOWsrc/schema/__init__.py1CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/schema/__init__.py2CODE
LOWsrc/voice/__init__.py33CODE
LOWsrc/voice/__init__.py34CODE
LOWsrc/voice/__init__.py35CODE
LOWsrc/voice/providers/__init__.py3CODE
LOWsrc/voice/providers/__init__.py4CODE
LOWsrc/service/__init__.py1CODE
LOWsrc/client/__init__.py1CODE
LOWsrc/client/__init__.py1CODE
Cross-File Repetition5 hits · 25 pts
SeverityFileLineSnippetContext
HIGHsrc/agents/rag_assistant.py0`total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totalitySTRING
HIGHsrc/agents/interrupt_agent.py0`total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totalitySTRING
HIGHsrc/agents/command_agent.py0`total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totalitySTRING
HIGHsrc/agents/research_assistant.py0`total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totalitySTRING
HIGHsrc/agents/bg_task_agent/bg_task_agent.py0`total=false` is pep589 specs. documentation: https://typing.readthedocs.io/en/latest/spec/typeddict.html#totalitySTRING
Modern Structural Boilerplate21 hits · 22 pts
SeverityFileLineSnippetContext
LOWsrc/core/__init__.py4__all__ = ["settings", "get_model"]CODE
LOWsrc/memory/mongodb.py11logger = logging.getLogger(__name__)CODE
LOWsrc/memory/__init__.py40__all__ = ["initialize_database", "initialize_store"]CODE
LOWsrc/memory/postgres.py11logger = logging.getLogger(__name__)CODE
LOWsrc/agents/interrupt_agent.py18logger = logging.getLogger(__name__)CODE
LOWsrc/agents/__init__.py10__all__ = [CODE
LOWsrc/agents/knowledge_base_agent.py15logger = logging.getLogger(__name__)CODE
LOWsrc/agents/github_mcp_agent/github_mcp_agent.py15logger = logging.getLogger(__name__)CODE
LOWsrc/schema/__init__.py14__all__ = [CODE
LOWsrc/voice/tts.py11logger = logging.getLogger(__name__)CODE
LOWsrc/voice/__init__.py37__all__ = ["VoiceManager", "SpeechToText", "TextToSpeech"]CODE
LOWsrc/voice/manager.py15logger = logging.getLogger(__name__)CODE
LOWsrc/voice/stt.py11logger = logging.getLogger(__name__)CODE
LOWsrc/voice/providers/openai_tts.py7logger = logging.getLogger(__name__)CODE
LOWsrc/voice/providers/__init__.py10__all__ = ["OpenAISTT", "OpenAITTS"]CODE
LOWsrc/voice/providers/openai_stt.py8logger = logging.getLogger(__name__)CODE
LOWsrc/service/service.py46logger = logging.getLogger(__name__)CODE
LOWsrc/service/agui.py27logger = logging.getLogger(__name__)CODE
LOWsrc/service/__init__.py3__all__ = ["app"]CODE
LOWsrc/client/client.py75 def update_agent(self, agent: str, verify: bool = True) -> None:CODE
LOWsrc/client/__init__.py3__all__ = ["AgentClient", "AgentClientError"]CODE
Synthetic Comment Markers2 hits · 12 pts
SeverityFileLineSnippetContext
HIGH.claude/skills/maintainer-response/SKILL.md156### Be skeptical of automated / AI-generated PRsCOMMENT
HIGH.claude/skills/maintainer-response/SKILL.md249 layer that *unconditionally* adds a "with Claude" badge and a "Generated by Claude Code"CODE
Redundant / Tautological Comments8 hits · 12 pts
SeverityFileLineSnippetContext
LOWscripts/create_chroma_db.py81 # Display resultsCOMMENT
LOWsrc/streamlit_app.py34 # Check if user_id exists in session stateCOMMENT
LOWsrc/streamlit_app.py573 # Check if this is a nested transfer/delegateCOMMENT
LOWsrc/agents/interrupt_agent.py93 # Check if we already have the birthdate in the store for this userCOMMENT
LOWsrc/agents/interrupt_agent.py98 if result: # Check if anything was returnedCODE
LOWsrc/agents/interrupt_agent.py100 if result: # Check if list is not emptyCODE
LOWsrc/agents/knowledge_base_agent.py66 # Check if documents were retrievedSTRING
LOWsrc/voice/manager.py99 # Check if transcription succeededCOMMENT
Cross-Language Confusion2 hits · 10 pts
SeverityFileLineSnippetContext
HIGHsrc/agents/safeguard.py60Answer: {"violation": 0, "category": null, "rationale": "Legitimate question about capabilities"}CODE
HIGHsrc/agents/safeguard.py64Answer: {"violation": 0, "category": null, "rationale": "Normal programming assistance request"}CODE
Docstring Block Structure2 hits · 10 pts
SeverityFileLineSnippetContext
HIGHsrc/voice/tts.py78Load the appropriate TTS provider implementation. Args: provider: Provider name api_keySTRING
HIGHsrc/voice/stt.py76Load the appropriate STT provider implementation. Args: provider: Provider name api_keySTRING
Deep Nesting9 hits · 9 pts
SeverityFileLineSnippetContext
LOWscripts/e2e_ui_tests.py145CODE
LOWscripts/e2e_ui_tests.py400CODE
LOWscripts/smoke_live_app.py66CODE
LOWsrc/streamlit_app.py56CODE
LOWsrc/streamlit_app.py319CODE
LOWsrc/streamlit_app.py508CODE
LOWsrc/agents/interrupt_agent.py77CODE
LOWsrc/service/service.py226CODE
LOWsrc/client/client.py202CODE
Modern AI Meta-Vocabulary2 hits · 6 pts
SeverityFileLineSnippetContext
MEDIUMdocs/RAG_Assistant.md1# Creating a RAG assistantCOMMENT
MEDIUMdocs/RAG_Assistant.md20## Configuring the RAG assistantCOMMENT
Over-Commented Block2 hits · 2 pts
SeverityFileLineSnippetContext
LOWscripts/smoke_test.sh1#!/usr/bin/env bashCOMMENT
LOWscripts/smoke_test.sh21# Only databases run in Docker; the service itself runs on the host via uv,COMMENT
Example Usage Blocks1 hit · 2 pts
SeverityFileLineSnippetContext
LOWscripts/smoke_test.sh7# Usage:COMMENT