Agent S: an open agentic framework that uses computers like a human
This report presents the forensic synthetic code analysis of simular-ai/Agent-S, a Python project with 12,105 GitHub stars. SynthScan v2.0 examined 22,233 lines of code across 109 source files, recording 432 pattern matches distributed across 17 syntactic categories. The overall adjusted score of 50.5 places this repository in the Strong AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 432 distinct pattern matches across 17 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | gui_agents/s2/cli_app.py | 0 | get a single character from stdin without pressing enter | STRING |
| HIGH | gui_agents/s3/cli_app.py | 0 | get a single character from stdin without pressing enter | STRING |
| HIGH | gui_agents/s1/cli_app.py | 0 | get a single character from stdin without pressing enter | STRING |
| HIGH | gui_agents/s2_5/cli_app.py | 0 | get a single character from stdin without pressing enter | STRING |
| HIGH | gui_agents/s2/cli_app.py | 0 | handle ctrl+c signal for debugging during agent execution | STRING |
| HIGH | gui_agents/s3/cli_app.py | 0 | handle ctrl+c signal for debugging during agent execution | STRING |
| HIGH | gui_agents/s1/cli_app.py | 0 | handle ctrl+c signal for debugging during agent execution | STRING |
| HIGH | gui_agents/s2_5/cli_app.py | 0 | handle ctrl+c signal for debugging during agent execution | STRING |
| HIGH | gui_agents/s2/cli_app.py | 0 | show a platform-specific permission dialog and return true if approved. | STRING |
| HIGH | gui_agents/s3/cli_app.py | 0 | show a platform-specific permission dialog and return true if approved. | STRING |
| HIGH | gui_agents/s1/cli_app.py | 0 | show a platform-specific permission dialog and return true if approved. | STRING |
| HIGH | gui_agents/s2_5/cli_app.py | 0 | show a platform-specific permission dialog and return true if approved. | STRING |
| HIGH | gui_agents/s2/core/mllm.py | 0 | generate the next response based on previous messages | STRING |
| HIGH | gui_agents/s3/core/mllm.py | 0 | generate the next response based on previous messages | STRING |
| HIGH | gui_agents/s1/mllm/MultimodalAgent.py | 0 | generate the next response based on previous messages | STRING |
| HIGH | gui_agents/s2_5/core/mllm.py | 0 | generate the next response based on previous messages | STRING |
| HIGH | gui_agents/s2/core/knowledge.py | 0 | summarize the narrative experience for lifelong learning reflection args: trajectory: str: the narrative experience to b | STRING |
| HIGH | gui_agents/s2/agents/manager.py | 0 | summarize the narrative experience for lifelong learning reflection args: trajectory: str: the narrative experience to b | STRING |
| HIGH | gui_agents/s1/core/Manager.py | 0 | summarize the narrative experience for lifelong learning reflection args: trajectory: str: the narrative experience to b | STRING |
| HIGH | gui_agents/s2/memory/procedural_memory.py | 0 | your response should be formatted like this: (previous action verification) carefully analyze based on the screenshot if | STRING |
| HIGH | gui_agents/s3/memory/procedural_memory.py | 0 | your response should be formatted like this: (previous action verification) carefully analyze based on the screenshot if | STRING |
| HIGH | gui_agents/s2_5/memory/procedural_memory.py | 0 | your response should be formatted like this: (previous action verification) carefully analyze based on the screenshot if | STRING |
| HIGH | gui_agents/s2/memory/procedural_memory.py | 0 | you are an expert in graphical user interfaces. your task is to process a phrase of text, and identify the most relevant | STRING |
| HIGH | gui_agents/s2_5/memory/procedural_memory.py | 0 | you are an expert in graphical user interfaces. your task is to process a phrase of text, and identify the most relevant | STRING |
| HIGH | gui_agents/s3/memory/procedural_memory.py | 0 | you are an expert in graphical user interfaces. your task is to process a phrase of text, and identify the most relevant | STRING |
| HIGH | gui_agents/s2/agents/worker.py | 0 | predict the next action(s) based on the current observation. | STRING |
| HIGH | gui_agents/s3/agents/worker.py | 0 | predict the next action(s) based on the current observation. | STRING |
| HIGH | gui_agents/s1/core/Worker.py | 0 | predict the next action(s) based on the current observation. | STRING |
| HIGH | gui_agents/s2_5/agents/worker.py | 0 | predict the next action(s) based on the current observation. | STRING |
| HIGH | gui_agents/s2/agents/agent_s.py | 0 | generate next action prediction args: instruction: natural language instruction observation: current ui state observatio | STRING |
| HIGH | gui_agents/s3/agents/agent_s.py | 0 | generate next action prediction args: instruction: natural language instruction observation: current ui state observatio | STRING |
| HIGH | gui_agents/s1/core/AgentS.py | 0 | generate next action prediction args: instruction: natural language instruction observation: current ui state observatio | STRING |
| HIGH | gui_agents/s2_5/agents/agent_s.py | 0 | generate next action prediction args: instruction: natural language instruction observation: current ui state observatio | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | import subprocess; import difflib; import pyautogui; import time; pyautogui.press('escape'); time.sleep(0.5); output = s | STRING |
| HIGH | gui_agents/s1/aci/LinuxOSACI.py | 0 | import subprocess; import difflib; import pyautogui; import time; pyautogui.press('escape'); time.sleep(0.5); output = s | STRING |
| HIGH | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 0 | import subprocess; import difflib; import pyautogui; import time; pyautogui.press('escape'); time.sleep(0.5); output = s | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | import subprocess; import difflib; import pyautogui; import time; pyautogui.press('escape'); time.sleep(0.5); output = s | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | import subprocess; import difflib; import pyautogui; import time; pyautogui.press('escape'); time.sleep(0.5); output = s | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | click on the element args: element_description:str, a detailed descriptions of which element to click on. this descripti | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | click on the element args: element_description:str, a detailed descriptions of which element to click on. this descripti | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | click on the element args: element_description:str, a detailed descriptions of which element to click on. this descripti | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s1/aci/LinuxOSACI.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s1/aci/MacOSACI.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | import pyautogui; pyautogui.click({x}, {y}, clicks={num_clicks}, button={repr(button_type)}); | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | switch to a different application that is already open args: app_code:str the code name of the application to switch to | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | switch to a different application that is already open args: app_code:str the code name of the application to switch to | STRING |
| HIGH | gui_agents/s1/aci/LinuxOSACI.py | 0 | switch to a different application that is already open args: app_code:str the code name of the application to switch to | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | switch to a different application that is already open args: app_code:str the code name of the application to switch to | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | type text/unicode into a specific element args: element_description:str, a detailed description of which element to ente | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | type text/unicode into a specific element args: element_description:str, a detailed description of which element to ente | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | type text/unicode into a specific element args: element_description:str, a detailed description of which element to ente | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | save facts, elements, texts, etc. to a long-term knowledge bank for reuse during this task. can be used for copy-pasting | STRING |
| HIGH | gui_agents/s3/agents/grounding.py | 0 | save facts, elements, texts, etc. to a long-term knowledge bank for reuse during this task. can be used for copy-pasting | STRING |
| HIGH | gui_agents/s1/aci/LinuxOSACI.py | 0 | save facts, elements, texts, etc. to a long-term knowledge bank for reuse during this task. can be used for copy-pasting | STRING |
| HIGH | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 0 | save facts, elements, texts, etc. to a long-term knowledge bank for reuse during this task. can be used for copy-pasting | STRING |
| HIGH | gui_agents/s2_5/agents/grounding.py | 0 | save facts, elements, texts, etc. to a long-term knowledge bank for reuse during this task. can be used for copy-pasting | STRING |
| HIGH | gui_agents/s2/agents/grounding.py | 0 | drag from the starting description to the ending description args: starting_description:str, a very detailed description | STRING |
| 49 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/agents/worker.py | 225 | except Exception as e: | STRING |
| LOW | gui_agents/s2/agents/agent_s.py | 361 | except Exception as e: | CODE |
| LOW | gui_agents/s2/agents/grounding.py | 305 | except Exception as e: | STRING |
| LOW | gui_agents/s2/utils/common_utils.py | 36 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 79 | print("Error: Invalid JSON") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 82 | print("Error: 'dag' key not found in JSON") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 85 | print(f"Error: Invalid data structure - {e}") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 88 | print("Error: JSON not found") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 109 | print("Error: JSON not found in either <json> tags or ```json``` fence") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 115 | print(f"Error: Invalid JSON ({e})") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 119 | print("Error: 'dag' key not found in JSON") | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 125 | print(f"Error: Invalid data structure - {e}") | CODE |
| LOW⚡ | gui_agents/s2/utils/common_utils.py | 204 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 205 | print(f"Error loading knowledge base: {e}") | CODE |
| LOW⚡ | gui_agents/s2/utils/common_utils.py | 213 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 214 | print(f"Error loading embeddings: {e}") | CODE |
| LOW⚡ | gui_agents/s2/utils/common_utils.py | 222 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s2/utils/common_utils.py | 223 | print(f"Error saving embeddings: {e}") | CODE |
| LOW | gui_agents/s3/bbon/comparative_judge.py | 57 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s3/bbon/comparative_judge.py | 58 | print(f"Error loading image {image_path}: {e}") | CODE |
| LOW | gui_agents/s3/agents/worker.py | 329 | except Exception as e: | STRING |
| LOW | gui_agents/s3/agents/code_agent.py | 47 | except Exception as e: | CODE |
| LOW | gui_agents/s3/agents/code_agent.py | 329 | except Exception as e: | STRING |
| LOW | gui_agents/s3/agents/grounding.py | 74 | except Exception: | CODE |
| LOW | gui_agents/s3/agents/grounding.py | 78 | except Exception: | CODE |
| LOW | gui_agents/s3/utils/formatters.py | 26 | except Exception as e: | CODE |
| LOW | gui_agents/s3/utils/local_env.py | 40 | except Exception as e: | CODE |
| LOW | gui_agents/s3/utils/local_env.py | 64 | except Exception as e: | CODE |
| LOW | gui_agents/s3/utils/common_utils.py | 50 | except Exception as e: | CODE |
| LOW | gui_agents/s3/utils/common_utils.py | 139 | except Exception as e: | CODE |
| LOW | gui_agents/s1/core/AgentS.py | 318 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 437 | print("Error: Invalid JSON") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 440 | print("Error: 'dag' key not found in JSON") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 443 | print(f"Error: Invalid data structure - {e}") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 446 | print("Error: JSON not found") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 845 | print(f"Error loading knowledge base: {e}") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 854 | print(f"Error loading embeddings: {e}") | CODE |
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 863 | print(f"Error saving embeddings: {e}") | CODE |
| LOW⚡ | gui_agents/s1/utils/common_utils.py | 844 | except Exception as e: | CODE |
| LOW⚡ | gui_agents/s1/utils/common_utils.py | 853 | except Exception as e: | CODE |
| LOW⚡ | gui_agents/s1/utils/common_utils.py | 862 | except Exception as e: | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 67 | except Exception as e: | CODE |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 249 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s1/aci/LinuxOSACI.py | 250 | print(f"Error: {e}") | CODE |
| LOW | gui_agents/s1/aci/WindowsOSACI.py | 133 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s1/aci/WindowsOSACI.py | 134 | print(f"Error: {e}") | CODE |
| LOW⚡ | gui_agents/s1/aci/WindowsOSACI.py | 194 | except Exception as e: | CODE |
| MEDIUM⚡ | gui_agents/s1/aci/WindowsOSACI.py | 195 | print(f"Error accessing foreground window: {e}") | CODE |
| LOW | gui_agents/s1/aci/WindowsOSACI.py | 471 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s1/aci/WindowsOSACI.py | 472 | print(f"Error accessing children: {e}") | CODE |
| MEDIUM | gui_agents/s1/aci/WindowsOSACI.py | 468 | def children(self): | CODE |
| LOW | gui_agents/s1/aci/MacOSACI.py | 154 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s1/aci/MacOSACI.py | 155 | print(f"Error: {e}") | CODE |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 211 | except Exception as e: | CODE |
| MEDIUM | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 212 | print(f"Error: {e}") | CODE |
| LOW | gui_agents/s2_5/agents/worker.py | 186 | except Exception as e: | STRING |
| LOW | gui_agents/s2_5/agents/grounding.py | 305 | except Exception as e: | STRING |
| LOW | gui_agents/s2_5/utils/common_utils.py | 20 | except Exception as e: | CODE |
| LOW | gui_agents/s2_5/utils/common_utils.py | 40 | except Exception as e: | CODE |
| LOW | integrations/openclaw/agent_s_wrapper.py | 108 | except Exception as e: | CODE |
| 45 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | README.md | 124 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | README.md | 124 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | README.md | 132 | os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>" | CODE |
| HIGH⚡ | README.md | 132 | os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>" | CODE |
| HIGH⚡ | models.md | 6 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 6 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 12 | export ANTHROPIC_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 12 | export ANTHROPIC_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 18 | export GEMINI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 18 | export GEMINI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 26 | export AZURE_OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | models.md | 26 | export AZURE_OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH | models.md | 40 | export OPENROUTER_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH | models.md | 40 | export OPENROUTER_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | gui_agents/s1/README.md | 64 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | gui_agents/s1/README.md | 64 | export OPENAI_API_KEY=<YOUR_API_KEY> | CODE |
| HIGH⚡ | gui_agents/s1/README.md | 71 | os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>" | CODE |
| HIGH⚡ | gui_agents/s1/README.md | 71 | os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>" | CODE |
| HIGH⚡ | integrations/openclaw/README.md | 33 | export ANTHROPIC_API_KEY="your-api-key-here" | CODE |
| HIGH⚡ | integrations/openclaw/README.md | 38 | export OPENAI_API_KEY="your-api-key-here" | CODE |
| HIGH⚡ | integrations/openclaw/README.md | 43 | export GEMINI_API_KEY="your-api-key-here" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/cli_app.py | 48 | CODE | |
| LOW | gui_agents/s2/cli_app.py | 154 | CODE | |
| LOW | gui_agents/s2/core/mllm.py | 18 | CODE | |
| LOW | gui_agents/s2/core/mllm.py | 112 | CODE | |
| LOW | gui_agents/s2/agents/agent_s.py | 210 | CODE | |
| LOW | gui_agents/s2/utils/common_utils.py | 129 | CODE | |
| LOW | gui_agents/s3/cli_app.py | 49 | CODE | |
| LOW | gui_agents/s3/cli_app.py | 155 | CODE | |
| LOW | gui_agents/s3/core/mllm.py | 18 | CODE | |
| LOW | gui_agents/s3/core/mllm.py | 175 | CODE | |
| LOW | gui_agents/s3/agents/worker.py | 90 | CODE | |
| LOW | gui_agents/s3/agents/worker.py | 178 | CODE | |
| LOW | gui_agents/s1/cli_app.py | 46 | CODE | |
| LOW | gui_agents/s1/cli_app.py | 154 | CODE | |
| LOW | gui_agents/s1/core/Worker.py | 85 | CODE | |
| LOW | gui_agents/s1/core/AgentS.py | 187 | CODE | |
| LOW | gui_agents/s1/utils/common_utils.py | 193 | CODE | |
| LOW | gui_agents/s1/utils/common_utils.py | 466 | CODE | |
| LOW | gui_agents/s1/utils/common_utils.py | 515 | CODE | |
| LOW | gui_agents/s1/utils/common_utils.py | 556 | CODE | |
| LOW | gui_agents/s1/utils/common_utils.py | 595 | CODE | |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 581 | CODE | |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 140 | CODE | |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 225 | CODE | |
| LOW | gui_agents/s1/aci/WindowsOSACI.py | 26 | CODE | |
| LOW | gui_agents/s1/aci/WindowsOSACI.py | 104 | CODE | |
| LOW | gui_agents/s1/aci/MacOSACI.py | 125 | CODE | |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 105 | CODE | |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 187 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 22 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 108 | CODE | |
| LOW | gui_agents/s2_5/cli_app.py | 48 | CODE | |
| LOW | gui_agents/s2_5/cli_app.py | 154 | CODE | |
| LOW | gui_agents/s2_5/core/mllm.py | 18 | CODE | |
| LOW | gui_agents/s2_5/core/mllm.py | 112 | CODE | |
| LOW | gui_agents/s2_5/agents/worker.py | 75 | CODE | |
| LOW | gui_agents/s2_5/utils/common_utils.py | 44 | CODE | |
| LOW | osworld_setup/s2/run.py | 160 | CODE | |
| LOW | osworld_setup/s2/run.py | 305 | CODE | |
| LOW | osworld_setup/s2/run.py | 342 | CODE | |
| LOW | osworld_setup/s3/run.py | 80 | CODE | |
| LOW | osworld_setup/s3/run.py | 362 | CODE | |
| LOW | osworld_setup/s3/run.py | 462 | CODE | |
| LOW | osworld_setup/s3/run.py | 499 | CODE | |
| LOW | osworld_setup/s3/run_local.py | 176 | CODE | |
| LOW | osworld_setup/s3/run_local.py | 303 | CODE | |
| LOW | osworld_setup/s3/run_local.py | 340 | CODE | |
| LOW | osworld_setup/s3/bbon/run_judge.py | 78 | CODE | |
| LOW | osworld_setup/s3/bbon/generate_facts.py | 145 | CODE | |
| LOW | osworld_setup/s3/bbon/utils.py | 50 | CODE | |
| LOW | osworld_setup/s3/bbon/utils.py | 163 | CODE | |
| LOW | osworld_setup/s1/run.py | 121 | CODE | |
| LOW | osworld_setup/s1/run.py | 241 | CODE | |
| LOW | osworld_setup/s1/run.py | 278 | CODE | |
| LOW | osworld_setup/s2_5/run.py | 76 | CODE | |
| LOW | osworld_setup/s2_5/run.py | 356 | CODE | |
| LOW | osworld_setup/s2_5/run.py | 456 | CODE | |
| LOW | osworld_setup/s2_5/run.py | 493 | CODE | |
| LOW | osworld_setup/s2_5/run_local.py | 176 | CODE | |
| LOW | osworld_setup/s2_5/run_local.py | 306 | CODE | |
| 1 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/core/knowledge.py | 161 | def retrieve_narrative_experience(self, instruction: str) -> Tuple[str, str]: | CODE |
| LOW | gui_agents/s2/core/knowledge.py | 198 | def retrieve_episodic_experience(self, instruction: str) -> Tuple[str, str]: | CODE |
| LOW | gui_agents/s2/core/knowledge.py | 312 | def initialize_task_trajectory(self, instruction: str) -> None: | CODE |
| LOW | gui_agents/s2/core/knowledge.py | 338 | def handle_subtask_trajectory(self, meta_data: Dict) -> None: | CODE |
| LOW | gui_agents/s2/memory/procedural_memory.py | 8 | def construct_worker_procedural_memory(agent_class, skipped_actions): | CODE |
| LOW | gui_agents/s2/agents/worker.py | 247 | def clean_worker_generation_for_reflection(self, worker_generation: str) -> str: | STRING |
| LOW | gui_agents/s2/agents/manager.py | 100 | def _generate_step_by_step_plan( | CODE |
| LOW⚡ | gui_agents/s2/utils/common_utils.py | 129 | def parse_single_code_from_string(input_string): | CODE |
| LOW | gui_agents/s2/utils/common_utils.py | 189 | def extract_first_agent_function(code_string): | CODE |
| LOW | gui_agents/s3/memory/procedural_memory.py | 13 | def construct_simple_worker_procedural_memory(agent_class, skipped_actions): | STRING |
| LOW | gui_agents/s3/bbon/behavior_narrator.py | 85 | def get_mouse_action_representation(mouse_actions: list[str]) -> str: | CODE |
| LOW | gui_agents/s3/bbon/comparative_judge.py | 10 | def get_final_screenshot_file(task_dir: str) -> str: | CODE |
| LOW | gui_agents/s3/bbon/comparative_judge.py | 31 | def image_to_openai_message_format( | CODE |
| LOW | gui_agents/s1/core/ProceduralMemory.py | 7 | def construct_worker_procedural_memory(agent_class): | CODE |
| LOW | gui_agents/s1/core/Knowledge.py | 154 | def retrieve_narrative_experience(self, instruction: str) -> Tuple[str, str]: | CODE |
| LOW | gui_agents/s1/core/Knowledge.py | 190 | def retrieve_episodic_experience(self, instruction: str) -> Tuple[str, str]: | CODE |
| LOW | gui_agents/s1/core/Manager.py | 86 | def _generate_step_by_step_plan( | CODE |
| LOW | gui_agents/s1/utils/ocr_server.py | 19 | def text_cvt_orc_format_paddle(paddle_result): | CODE |
| LOW⚡ | gui_agents/s1/utils/common_utils.py | 595 | def parse_action_from_fixed_code(action_string, linearized_accessibility_tree): | CODE |
| LOW⚡ | gui_agents/s1/utils/common_utils.py | 599 | def parse_action_from_agent_code(action_str): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 360 | def linearize_accessibility_tree(accessibility_tree, platform="ubuntu", tag=False): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 466 | def parse_actions_from_string(input_string): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 500 | def parse_fixed_action_from_string(input_string): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 556 | def parse_single_code_from_string(input_string): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 695 | def parse_code_from_som_string(input_string, masks): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 769 | def text_cvt_orc_format_paddle(paddle_result): | CODE |
| LOW | gui_agents/s1/utils/common_utils.py | 829 | def extract_first_agent_function(code_string): | CODE |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 199 | def extract_elements_from_screenshot(self, screenshot) -> Dict: | STRING |
| LOW | gui_agents/s1/aci/LinuxOSACI.py | 301 | def linearize_and_annotate_tree(self, obs, show_all=False): | CODE |
| LOW | gui_agents/s1/aci/WindowsOSACI.py | 89 | def extract_elements_from_screenshot(self, screenshot: bytes) -> Dict[str, Any]: | CODE |
| LOW⚡ | gui_agents/s1/aci/WindowsOSACI.py | 186 | def linearize_and_annotate_tree( | CODE |
| LOW | gui_agents/s1/aci/ACI.py | 30 | def linearize_and_annotate_tree( | CODE |
| LOW | gui_agents/s1/aci/MacOSACI.py | 110 | def extract_elements_from_screenshot(self, screenshot: bytes) -> Dict[str, Any]: | CODE |
| LOW | gui_agents/s1/aci/MacOSACI.py | 207 | def linearize_and_annotate_tree( | CODE |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 163 | def extract_elements_from_screenshot(self, screenshot) -> Dict: | STRING |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 263 | def linearize_and_annotate_tree(self, obs, show_all=False): | CODE |
| LOW | gui_agents/s2_5/memory/procedural_memory.py | 7 | def construct_simple_worker_procedural_memory(agent_class, skipped_actions): | CODE |
| LOW | gui_agents/s2_5/utils/common_utils.py | 44 | def parse_single_code_from_string(input_string): | CODE |
| LOW | gui_agents/s2_5/utils/common_utils.py | 98 | def extract_first_agent_function(code_string): | CODE |
| LOW | tests/test_providers.py | 22 | def test_ollama_missing_config(self): | CODE |
| LOW | tests/test_providers.py | 28 | def test_ollama_valid_config_param(self): | CODE |
| LOW | tests/test_providers.py | 40 | def test_ollama_valid_config_env(self): | CODE |
| LOW | osworld_setup/s3/bbon/generate_facts.py | 14 | async def generate_single_fact_caption( | CODE |
| LOW | osworld_setup/s3/bbon/generate_facts.py | 59 | async def generate_fact_captions_parallel( | CODE |
| LOW | osworld_setup/s3/bbon/utils.py | 10 | def image_to_openai_message_format( | CODE |
| LOW | osworld_setup/s3/bbon/utils.py | 107 | def get_final_screenshot_file(result_dir: str) -> str: | CODE |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 163 | def get_new_tasks_classification(results_dirs: [str]): | CODE |
| LOW | osworld_setup/s3/bbon/utils.py | 229 | def check_selected_trajectory(results_dirs: [str], selected_trajectory: str, task: str): | CODE |
| LOW | osworld_setup/s3/bbon/utils.py | 268 | def evaluate_comparative_results(results_dirs: [str], json_path: str = None): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | osworld_setup/s2/OSWorld.md | 3 | # Step 1: Set up Agent S2 | COMMENT |
| LOW | osworld_setup/s2/OSWorld.md | 7 | # Step 2: Copying Over Run Files | COMMENT |
| LOW | osworld_setup/s3/run.sh | 1 | # Step 1: Complete 2 or more rollouts on either AWS or locally | COMMENT |
| LOW⚡ | osworld_setup/s3/run.sh | 36 | # Step 2: Generate Facts | COMMENT |
| LOW⚡ | osworld_setup/s3/run.sh | 45 | # Step 3: Run the Judge. Make sure the order of the results-dirs is the same as the order above. | COMMENT |
| LOW⚡ | osworld_setup/s3/OSWorld.md | 3 | # Step 1: Set up Agent S3 | COMMENT |
| LOW⚡ | osworld_setup/s3/OSWorld.md | 7 | # Step 2: Copying Over Run Files | COMMENT |
| LOW⚡ | osworld_setup/s3/OSWorld.md | 11 | # Step 3: Switch the AMI | COMMENT |
| LOW⚡ | osworld_setup/s3/OSWorld.md | 15 | # Step 4: Generating Facts | COMMENT |
| LOW | osworld_setup/s3/OSWorld.md | 31 | # Step 5: Run the Judge | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 164 | # Step 1: collect domain/task_ids for each trajectory | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 178 | # Step 2: find tasks common to all trajectories | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 187 | # Step 3: evaluate each common task | COMMENT |
| LOW⚡ | osworld_setup/s1/OSWorld.md | 3 | # Step 1: Set up Agent S | COMMENT |
| LOW⚡ | osworld_setup/s1/OSWorld.md | 7 | # Step 2: Copying Over Run Files | COMMENT |
| LOW⚡ | osworld_setup/s1/OSWorld.md | 14 | # Step 3: Best Practices | COMMENT |
| LOW | osworld_setup/s2_5/OSWorld.md | 3 | # Step 1: Set up Agent S2.5 | COMMENT |
| LOW | osworld_setup/s2_5/OSWorld.md | 7 | # Step 2: Copying Over Run Files | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/cli_app.py | 160 | # Check if we're in paused state and wait | COMMENT |
| LOW | gui_agents/s2/core/mllm.py | 149 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s2/core/mllm.py | 200 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s2/core/mllm.py | 247 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s3/cli_app.py | 161 | # Check if we're in paused state and wait | COMMENT |
| LOW | gui_agents/s3/core/mllm.py | 212 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s3/core/mllm.py | 263 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s3/core/mllm.py | 310 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s3/agents/code_agent.py | 155 | # Check if response is None or empty | COMMENT |
| LOW | gui_agents/s3/agents/grounding.py | 450 | # Check if text contains Unicode characters that pyautogui.write() can't handle | COMMENT |
| LOW | gui_agents/s1/cli_app.py | 160 | # Check if we're in paused state and wait | COMMENT |
| LOW | gui_agents/s1/utils/common_utils.py | 254 | # Check if the area only contains one color | COMMENT |
| LOW | gui_agents/s1/aci/MacOSACI.py | 546 | # Iterate through the list and print each application's name | COMMENT |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 130 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 175 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 222 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s2_5/cli_app.py | 160 | # Check if we're in paused state and wait | COMMENT |
| LOW | gui_agents/s2_5/core/mllm.py | 149 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s2_5/core/mllm.py | 200 | # Check if image_content is a list or a single image | COMMENT |
| LOW | gui_agents/s2_5/core/mllm.py | 247 | # Check if image_content is a list or a single image | COMMENT |
| LOW | integrations/openclaw/agent_s_wrapper.py | 165 | # Output result | COMMENT |
| LOW | osworld_setup/s3/bbon/utils.py | 134 | # Check if file exists and is valid | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | gui_agents/s2/core/knowledge.py | 55 | # All three agents share a generic RAG prompt that asks the agent to provide information for UI automation in CU | COMMENT |
| MEDIUM | gui_agents/s2/agents/worker.py | 114 | # Get RAG knowledge, only update system message at t=0 | COMMENT |
| MEDIUM | gui_agents/s3/agents/grounding.py | 573 | # This is a full task - use the original task instruction to prevent hallucination | COMMENT |
| MEDIUM | gui_agents/s1/core/Worker.py | 125 | # Get RAG knowledge, only update system message at t=0 | COMMENT |
| MEDIUM | gui_agents/s1/core/Worker.py | 127 | # TODO: uncomment and fix for subtask level RAG | COMMENT |
| MEDIUM | gui_agents/s1/core/Knowledge.py | 56 | # All three agent share a generic RAG prompt that ask agent to provide information for UI automation in CURRENT_ | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/agents/manager.py | 11 | CODE | |
| LOW | gui_agents/s1/core/Worker.py | 2 | CODE | |
| LOW | gui_agents/s1/utils/query_perplexica.py | 2 | CODE | |
| LOW | gui_agents/s1/aci/MacOSACI.py | 11 | CODE | |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 3 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalEngine.py | 6 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalEngine.py | 11 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalAgent.py | 6 | CODE | |
| LOW | tests/test_providers.py | 3 | CODE | |
| LOW | osworld_setup/s2/lib_run_single.py | 6 | CODE | |
| LOW | osworld_setup/s3/lib_run_single.py | 6 | CODE | |
| LOW | osworld_setup/s3/lib_run_single.py | 7 | CODE | |
| LOW | osworld_setup/s1/lib_run_single.py | 6 | CODE | |
| LOW | osworld_setup/s2_5/lib_run_single_local.py | 6 | CODE | |
| LOW | osworld_setup/s2_5/lib_run_single_local.py | 7 | CODE | |
| LOW | osworld_setup/s2_5/lib_run_single.py | 6 | CODE | |
| LOW | osworld_setup/s2_5/lib_run_single.py | 7 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | osworld_setup/s3/run.sh | 1 | # Step 1: Complete 2 or more rollouts on either AWS or locally | COMMENT |
| LOW⚡ | osworld_setup/s3/run.sh | 36 | # Step 2: Generate Facts | COMMENT |
| LOW⚡ | osworld_setup/s3/run.sh | 45 | # Step 3: Run the Judge. Make sure the order of the results-dirs is the same as the order above. | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 164 | # Step 1: collect domain/task_ids for each trajectory | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 178 | # Step 2: find tasks common to all trajectories | COMMENT |
| LOW⚡ | osworld_setup/s3/bbon/utils.py | 187 | # Step 3: evaluate each common task | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | gui_agents/s2/memory/procedural_memory.py | 88 | - If you feel some future subtasks are incorrect or unnecessary, feel free to modify or even remove them. | CODE |
| MEDIUM | gui_agents/s1/core/Worker.py | 159 | # Clear older messages - we keep full context. if you want to keep only the last n messages, you can use the flu | COMMENT |
| LOW | gui_agents/s1/core/ProceduralMemory.py | 60 | # MANAGER_PROMPT = """You are a planning agent for solving GUI navigation tasks. You will be provided the initial co | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/core/knowledge.py | 322 | def update_task_trajectory(self, meta_data: Dict) -> None: | CODE |
| LOW | gui_agents/s2/agents/agent_s.py | 66 | def update_narrative_memory(self, trajectory: str) -> None: | CODE |
| LOW | gui_agents/s2/agents/agent_s.py | 339 | def update_narrative_memory(self, trajectory: str) -> None: | CODE |
| LOW | gui_agents/s1/core/AgentS.py | 61 | def update_narrative_memory(self, trajectory: str) -> None: | CODE |
| LOW | gui_agents/s1/core/AgentS.py | 296 | def update_narrative_memory(self, trajectory: str) -> None: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | gui_agents/s3/utils/common_utils.py | 16 | Attempts to evaluate the code into a pyautogui code snippet with grounded actions using the observation screenshot. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | gui_agents/s1/utils/common_utils.py | 609 | # Define a regex pattern to extract the action type and parameters | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 421 | # if overwrite: | COMMENT |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 441 | COMMENT | |
| LOW | gui_agents/s1/aci/windowsagentarena/GroundingAgent.py | 461 | # node = self.find_element(0) | COMMENT |
| LOW | gui_agents/s1/mllm/MultimodalEngine.py | 21 | # tokenizer_image_token, | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | gui_agents/s2/agents/agent_s.py | 90 | CODE | |
| LOW | gui_agents/s3/bbon/behavior_narrator.py | 271 | CODE | |
| LOW | gui_agents/s1/mllm/MultimodalEngine.py | 40 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | gui_agents/s1/core/ProceduralMemory.py | 66 | # MANAGER_PROMPT = """You are an expert planning agent for GUI tasks. You will be provided with an initial state of | COMMENT |