Python scraper based on AI
This report presents the forensic synthetic code analysis of ScrapeGraphAI/Scrapegraph-ai, a Python project with 28,350 GitHub stars. SynthScan v2.0 examined 40,181 lines of code across 325 source files, recording 831 pattern matches distributed across 14 syntactic categories. The overall adjusted score of 42.7 places this repository in the Strong AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 831 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_fetch_node_timeout.py | 165 | # Create a mock loader that takes longer than timeout | COMMENT |
| MEDIUM | tests/test_json_scraper_graph.py | 46 | # Create a JSONScraperGraph instance | COMMENT |
| MEDIUM | tests/test_json_scraper_graph.py | 103 | # Create a JSONScraperGraph instance with a single JSON file | COMMENT |
| MEDIUM | tests/test_json_scraper_graph.py | 157 | # Create a JSONScraperGraph instance | COMMENT |
| MEDIUM | tests/test_json_scraper_graph.py | 203 | # Define a custom schema | COMMENT |
| MEDIUM | tests/test_json_scraper_graph.py | 220 | # Create a JSONScraperGraph instance with a custom schema | COMMENT |
| MEDIUM | tests/test_script_creator_multi_graph.py | 172 | # Create a dummy graph that raises an exception when execute is called. | COMMENT |
| MEDIUM⚡ | tests/test_chromium.py | 1026 | # Create a loader instance with a backend that does not have a corresponding scraping method. | COMMENT |
| MEDIUM⚡ | tests/test_chromium.py | 1038 | # Create a dummy undetected_chromedriver module with a dummy Chrome driver that always fails. | COMMENT |
| MEDIUM⚡ | tests/test_chromium.py | 2007 | # Create a loader instance with a retry limit of 2 and a short timeout. | COMMENT |
| MEDIUM | tests/test_chromium.py | 295 | # Create a dummy undetected_chromedriver module with a dummy Chrome driver. | COMMENT |
| MEDIUM | tests/test_chromium.py | 623 | # Create a loader with JS support and a retry_limit of 2 (so one failure is allowed) | COMMENT |
| MEDIUM | tests/test_chromium.py | 1232 | # Create a dummy undetected_chromedriver module where Chrome is defined but will not be used. | COMMENT |
| MEDIUM | tests/test_chromium.py | 1491 | # Define a dummy scraper that returns an integer (non‐string) | COMMENT |
| MEDIUM | tests/test_chromium.py | 1976 | # Create a dummy playwright so that evaluate and content can be called | COMMENT |
| MEDIUM | tests/test_omni_search_graph.py | 7 | # Create a dummy graph class to simulate graph execution | COMMENT |
| MEDIUM | tests/nodes/search_internet_node_test.py | 18 | # Define the model | COMMENT |
| MEDIUM⚡ | tests/nodes/search_link_node_test.py | 14 | # Define the configuration for the graph | COMMENT |
| MEDIUM⚡ | tests/nodes/search_link_node_test.py | 22 | # Define the SearchLinkNode with necessary configurations | COMMENT |
| MEDIUM⚡ | tests/nodes/search_link_node_test.py | 29 | # Define the initial state for the node | COMMENT |
| MEDIUM | tests/utils/copy_utils_test.py | 176 | original["self"] = original # Create a circular reference | CODE |
| MEDIUM | tests/graphs/screenshot_scraper_test.py | 13 | # Define a fixture for the graph configuration | COMMENT |
| MEDIUM | tests/graphs/code_generator_graph_openai_test.py | 17 | # Define the output schema for the graph | COMMENT |
| MEDIUM | tests/graphs/xml_scraper_openai_test.py | 16 | # Define the test fixtures and helpers | COMMENT |
| MEDIUM | tests/graphs/xml_scraper_openai_test.py | 49 | # Define the test cases | COMMENT |
| MEDIUM | tests/graphs/abstract_graph_test.py | 56 | # Create a mock graph with mock nodes | COMMENT |
| MEDIUM | tests/graphs/abstract_graph_test.py | 61 | # Create a TestGraph instance with the mock graph | COMMENT |
| MEDIUM | tests/graphs/abstract_graph_test.py | 254 | # Create a mock graph with mock nodes | COMMENT |
| MEDIUM | tests/graphs/abstract_graph_test.py | 260 | # Create a TestGraph instance with the mock graph | COMMENT |
| MEDIUM | tests/graphs/search_graph_openai_test.py | 15 | # Define the test fixtures and helpers | COMMENT |
| MEDIUM | tests/graphs/search_graph_openai_test.py | 36 | # Define the test cases | COMMENT |
| MEDIUM | examples/speech_graph/speech_graph_openai.py | 23 | # Define the configuration for the graph | COMMENT |
| MEDIUM | examples/speech_graph/speech_graph_openai.py | 39 | # Create the SpeechGraph instance and run it | COMMENT |
| MEDIUM | examples/custom_graph/ollama/custom_graph_ollama.py | 16 | # Define the configuration for the graph | COMMENT |
| MEDIUM | examples/custom_graph/ollama/custom_graph_ollama.py | 31 | # Define the graph nodes | COMMENT |
| MEDIUM | examples/custom_graph/ollama/custom_graph_ollama.py | 75 | # Create the graph by defining the connections | COMMENT |
| MEDIUM | examples/custom_graph/openai/custom_graph_openai.py | 22 | # Define the configuration for the graph | COMMENT |
| MEDIUM | examples/custom_graph/openai/custom_graph_openai.py | 34 | # Define the graph nodes | COMMENT |
| MEDIUM | examples/custom_graph/openai/custom_graph_openai.py | 86 | # Create the graph by defining the connections | COMMENT |
| MEDIUM | …mples/json_scraper_graph/ollama/json_scraper_ollama.py | 26 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …mples/json_scraper_graph/ollama/json_scraper_ollama.py | 41 | # Create the JSONScraperGraph instance and run it | COMMENT |
| MEDIUM | …mples/json_scraper_graph/openai/json_scraper_openai.py | 26 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …mples/json_scraper_graph/openai/json_scraper_openai.py | 39 | # Create the JSONScraperGraph instance and run it | COMMENT |
| MEDIUM | …mples/json_scraper_graph/openai/omni_scraper_openai.py | 16 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …mples/json_scraper_graph/openai/omni_scraper_openai.py | 32 | # Create the OmniScraperGraph instance and run it | COMMENT |
| MEDIUM | examples/json_scraper_graph/openai/md_scraper_openai.py | 26 | # Define the configuration for the graph | COMMENT |
| MEDIUM | examples/json_scraper_graph/openai/md_scraper_openai.py | 39 | # Create the DocumentScraperGraph instance and run it | COMMENT |
| MEDIUM | …_generator_graph/ollama/code_generator_graph_ollama.py | 15 | # Define the output schema for the graph | COMMENT |
| MEDIUM | …_generator_graph/ollama/code_generator_graph_ollama.py | 29 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …_generator_graph/ollama/code_generator_graph_ollama.py | 54 | # Create the SmartScraperGraph instance and run it | COMMENT |
| MEDIUM | …_generator_graph/openai/code_generator_graph_openai.py | 16 | # Define the output schema for the graph | COMMENT |
| MEDIUM | …_generator_graph/openai/code_generator_graph_openai.py | 30 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …_generator_graph/openai/code_generator_graph_openai.py | 54 | # Create the SmartScraperGraph instance and run it | COMMENT |
| MEDIUM | examples/search_graph/ollama/search_graph_ollama.py | 9 | # Define the configuration for the graph | COMMENT |
| MEDIUM | examples/search_graph/ollama/search_graph_ollama.py | 25 | # Create the SearchGraph instance and run it | COMMENT |
| MEDIUM | …ples/search_graph/ollama/search_graph_schema_ollama.py | 13 | # Define the output schema for the graph | COMMENT |
| MEDIUM | …ples/search_graph/ollama/search_graph_schema_ollama.py | 27 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …ples/search_graph/ollama/search_graph_schema_ollama.py | 42 | # Create the SearchGraph instance and run it | COMMENT |
| MEDIUM | …amples/search_graph/openai/search_link_graph_openai.py | 14 | # Define the configuration for the graph | COMMENT |
| MEDIUM | …amples/search_graph/openai/search_link_graph_openai.py | 29 | # Create the SearchLinkGraph instance and run it | COMMENT |
| 95 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …amples/search_graph/openai/search_link_graph_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …art_scraper_graph/ollama/smart_scraper_multi_ollama.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …les/smart_scraper_graph/ollama/smart_scraper_ollama.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …craper_graph/ollama/smart_scraper_multi_lite_ollama.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …aper_graph/ollama/smart_scraper_multi_concat_ollama.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …mart_scraper_graph/ollama/smart_scraper_lite_ollama.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …aper_graph/openai/smart_scraper_multi_concat_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …les/smart_scraper_graph/openai/smart_scraper_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …craper_graph/openai/smart_scraper_multi_lite_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …art_scraper_graph/openai/smart_scraper_multi_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …mart_scraper_graph/openai/smart_scraper_lite_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …ript_generator_graph/openai/script_generator_openai.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/slow_mo.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/custom_prompt.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/browser_base_integration.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/force_mode.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/proxy_rotation.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/undected_playwright.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/reasoning.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/rag_caching.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/load_yml.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | examples/extras/scrape_do.py | 0 | basic example of scraping pipeline using smartscraper | STRING |
| HIGH | …enerator_graph/ollama/script_multi_generator_ollama.py | 0 | basic example of scraping pipeline using scriptcreatorgraph | STRING |
| HIGH | …ript_generator_graph/ollama/script_generator_ollama.py | 0 | basic example of scraping pipeline using scriptcreatorgraph | STRING |
| HIGH | …enerator_graph/openai/script_generator_multi_openai.py | 0 | basic example of scraping pipeline using scriptcreatorgraph | STRING |
| HIGH | …nerator_graph/openai/script_generator_schema_openai.py | 0 | basic example of scraping pipeline using scriptcreatorgraph | STRING |
| HIGH | scrapegraphai/nodes/search_node_with_context.py | 0 | generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, | STRING |
| HIGH | scrapegraphai/nodes/generate_answer_omni_node.py | 0 | generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, | STRING |
| HIGH | scrapegraphai/nodes/search_internet_node.py | 0 | generates an answer by constructing a prompt from the user's input and the scraped content, querying the language model, | STRING |
| HIGH | scrapegraphai/nodes/merge_answers_node.py | 0 | executes the node's logic to concatenate the answers from multiple graph instances into a single answer. args: state (di | STRING |
| HIGH | scrapegraphai/nodes/merge_generated_scripts_node.py | 0 | executes the node's logic to concatenate the answers from multiple graph instances into a single answer. args: state (di | STRING |
| HIGH | scrapegraphai/nodes/concat_answers_node.py | 0 | executes the node's logic to concatenate the answers from multiple graph instances into a single answer. args: state (di | STRING |
| HIGH | scrapegraphai/nodes/rag_node.py | 0 | a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan | STRING |
| HIGH | scrapegraphai/nodes/generate_answer_node_k_level.py | 0 | a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan | STRING |
| HIGH | scrapegraphai/nodes/description_node.py | 0 | a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan | STRING |
| HIGH | scrapegraphai/nodes/parse_node_depth_k_node.py | 0 | executes the node's logic to index the content present in the state. args: state (dict): the current state of the graph. | STRING |
| HIGH | scrapegraphai/nodes/parse_node.py | 0 | executes the node's logic to index the content present in the state. args: state (dict): the current state of the graph. | STRING |
| HIGH | scrapegraphai/integrations/indexify_node.py | 0 | executes the node's logic to index the content present in the state. args: state (dict): the current state of the graph. | STRING |
| HIGH | scrapegraphai/models/minimax.py | 0 | a wrapper for the chatopenai class (xai uses an openai-compatible api) that provides default configuration and could be | STRING |
| HIGH | scrapegraphai/models/clod.py | 0 | a wrapper for the chatopenai class (xai uses an openai-compatible api) that provides default configuration and could be | STRING |
| HIGH | scrapegraphai/models/deepseek.py | 0 | a wrapper for the chatopenai class (xai uses an openai-compatible api) that provides default configuration and could be | STRING |
| HIGH | scrapegraphai/models/xai.py | 0 | a wrapper for the chatopenai class (xai uses an openai-compatible api) that provides default configuration and could be | STRING |
| HIGH | scrapegraphai/graphs/search_link_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/depth_search_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/smart_scraper_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/code_generator_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/document_scraper_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/smart_scraper_lite_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/omni_scraper_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/script_creator_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/xml_scraper_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/json_scraper_graph.py | 0 | creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing | STRING |
| HIGH | scrapegraphai/graphs/search_link_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/speech_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/smart_scraper_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/screenshot_scraper_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/document_scraper_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/omni_scraper_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/script_creator_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| HIGH | scrapegraphai/graphs/xml_scraper_graph.py | 0 | executes the web scraping process and returns the answer to the prompt. returns: str: the answer to the prompt. | STRING |
| 23 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | scrapegraphai/docloaders/chromium.py | 174 | Asynchronously scrape the content of a given URL using Playwright's sync API and scrolling. Notes: | STRING |
| HIGH | scrapegraphai/docloaders/chromium.py | 323 | Asynchronously scrape the content of a given URL using Playwright's async API. Args: url ( | STRING |
| HIGH | scrapegraphai/docloaders/chromium.py | 384 | Asynchronously scrape the content of a given URL by rendering JavaScript using Playwright. Args: | STRING |
| HIGH | scrapegraphai/builders/graph_builder.py | 14 | GraphBuilder is a dynamic tool for constructing web scraping graphs based on user prompts. It utilizes a natura | STRING |
| HIGH | scrapegraphai/nodes/search_link_node.py | 94 | Filter out relevant links from the webpage that are relavant to prompt. Out of the filtered links, also | STRING |
| HIGH | scrapegraphai/nodes/search_node_with_context.py | 51 | Generates an answer by constructing a prompt from the user's input and the scraped content, querying th | STRING |
| HIGH | scrapegraphai/nodes/merge_answers_node.py | 58 | Executes the node's logic to merge the answers from multiple graph instances into a single answer. | STRING |
| HIGH | scrapegraphai/nodes/search_internet_node.py | 61 | Generates an answer by constructing a prompt from the user's input and the scraped content, querying th | STRING |
| HIGH | scrapegraphai/nodes/reasoning_node.py | 57 | Generate a refined prompt for the reasoning task based on the user's input and the JSON schema. | STRING |
| HIGH | scrapegraphai/nodes/graph_iterator_node.py | 47 | Executes the node's logic to instantiate and run multiple graph instances in parallel. Args: | STRING |
| HIGH | scrapegraphai/nodes/graph_iterator_node.py | 82 | asynchronously executes the node's logic with multiple graph instances running in parallel, using a semaphore of | STRING |
| HIGH | scrapegraphai/nodes/generate_answer_csv_node.py | 71 | Generates an answer by constructing a prompt from the user's input and the scraped content, querying th | STRING |
| HIGH | scrapegraphai/nodes/generate_scraper_node.py | 56 | Generates a python script for scraping a website using the specified library. Args: state | STRING |
| HIGH | scrapegraphai/nodes/robots_node.py | 58 | Checks if a website is scrapeable based on the robots.txt file and updates the state with the scrapeabi | STRING |
| HIGH | scrapegraphai/nodes/parse_node_depth_k_node.py | 43 | Executes the node's logic to parse the HTML documents content. Args: state (dict): The cur | STRING |
| HIGH | scrapegraphai/nodes/fetch_node_level_k.py | 73 | Executes the node's logic to fetch the HTML content of a specified URL and its sub-links recursively, t | STRING |
| HIGH | scrapegraphai/nodes/merge_generated_scripts_node.py | 42 | Executes the node's logic to merge the answers from multiple graph instances into a single answer. | STRING |
| HIGH | scrapegraphai/nodes/fetch_node.py | 224 | Handles the local source by fetching HTML content, optionally converting it to Markdown, and updating t | STRING |
| HIGH | scrapegraphai/nodes/fetch_node.py | 267 | Handles the web source by fetching HTML content from a URL, optionally converting it to Markdown, and u | STRING |
| HIGH | scrapegraphai/nodes/concat_answers_node.py | 42 | Executes the node's logic to concatenate the answers from multiple graph instances into a single answer | STRING |
| HIGH | scrapegraphai/nodes/generate_code_node.py | 94 | Generates Python code for a function that extracts data from HTML based on a output schema. Args: | STRING |
| HIGH | scrapegraphai/nodes/generate_code_node.py | 146 | Executes the overall reasoning loop to generate and validate the code. Args: state (dict): | STRING |
| HIGH | scrapegraphai/nodes/text_to_speech_node.py | 40 | Converts text to speech using the specified text-to-speech model. Args: state (dict): The | STRING |
| HIGH | scrapegraphai/nodes/markdownify_node.py | 42 | Executes the node's logic to convert HTML content to Markdown. Args: state (dict): The cur | STRING |
| HIGH | scrapegraphai/nodes/get_probable_tags_node.py | 46 | Generates a list of probable HTML tags based on the user's input and updates the state with this list. | STRING |
| HIGH | scrapegraphai/nodes/base_node.py | 13 | An abstract base class for nodes in a graph-based workflow, designed to perform specific actions when executed. | STRING |
| HIGH | scrapegraphai/nodes/base_node.py | 99 | Determines the necessary state keys based on the input specification. Args: state (dict): | STRING |
| HIGH | scrapegraphai/nodes/base_node.py | 137 | Parses the input keys expression to extract relevant keys from the state based on logical conditions. | STRING |
| HIGH | scrapegraphai/nodes/html_analyzer_node.py | 59 | Generates an analysis of the provided HTML code based on the wanted infromations to be extracted. Args | STRING |
| HIGH | scrapegraphai/nodes/parse_node.py | 63 | Executes the node's logic to parse the HTML document content and split it into chunks. Args: | STRING |
| HIGH | scrapegraphai/nodes/prompt_refiner_node.py | 63 | Generate a refined prompt using the user's prompt, the schema, and additional context. Args: | STRING |
| HIGH | scrapegraphai/nodes/generate_answer_omni_node.py | 65 | Generates an answer by constructing a prompt from the user's input and the scraped content, querying th | STRING |
| HIGH | scrapegraphai/utils/batch_api.py | 118 | Create and submit an OpenAI Batch API job. Args: client: An initialized OpenAI client. requests: Li | STRING |
| HIGH | scrapegraphai/utils/batch_api.py | 196 | Poll a batch job until it completes, fails, or times out. Args: client: An initialized OpenAI client. | STRING |
| HIGH | scrapegraphai/utils/code_error_analysis.py | 107 | Analyzes the syntax errors in the generated code. Args: state (dict): Contains the 'generated_code' an | STRING |
| HIGH | scrapegraphai/utils/code_error_analysis.py | 160 | Analyzes the execution errors in the generated code and HTML code. Args: state (dict): Contains the 'g | STRING |
| HIGH | scrapegraphai/utils/code_error_analysis.py | 215 | Analyzes the validation errors in the generated code based on a JSON schema. Args: state (dict): Conta | STRING |
| HIGH | scrapegraphai/utils/code_error_analysis.py | 278 | Analyzes the semantic differences in the generated code based on a comparison result. Args: state (dic | STRING |
| HIGH | scrapegraphai/utils/code_error_correction.py | 89 | Generates corrected code based on syntax error analysis. Args: state (dict): Contains the 'generated_c | STRING |
| HIGH | scrapegraphai/utils/code_error_correction.py | 142 | Generates corrected code based on execution error analysis. Args: state (dict): Contains the 'generate | STRING |
| HIGH | scrapegraphai/utils/code_error_correction.py | 195 | Generates corrected code based on validation error analysis. Args: state (dict): Contains the 'generat | STRING |
| HIGH | scrapegraphai/utils/code_error_correction.py | 254 | Generates corrected code based on semantic error analysis. Args: state (dict): Contains the 'generated | STRING |
| HIGH | scrapegraphai/utils/copy.py | 36 | Safely create a deep copy of an object, handling special cases. Args: obj: Object to copy Returns | STRING |
| HIGH | scrapegraphai/utils/parse_state_keys.py | 9 | Parses a complex boolean expression involving state keys. Args: expression (str): The boolean expressi | STRING |
| HIGH | scrapegraphai/utils/research_web.py | 171 | Search web function with improved error handling, validation, and security features. Args: query (str) | STRING |
| HIGH | scrapegraphai/utils/sys_dynamic_import.py | 16 | imports a python module from its srcfile Args: modpath: The srcfile absolute path modname: The | STRING |
| HIGH | …apegraphai/utils/screenshot_scraping/text_detection.py | 7 | Detects and extracts text from a given image. Parameters: image (PIL Image): The input image to ext | STRING |
| HIGH | …ai/utils/screenshot_scraping/screenshot_preparation.py | 225 | Crop an image using the specified coordinates. Parameters: image (PIL.Image): The image to be cropped. | STRING |
| HIGH | scrapegraphai/integrations/indexify_node.py | 38 | Executes the node's logic to index the content present in the state. Args: state (dict): T | STRING |
| HIGH | scrapegraphai/graphs/abstract_graph.py | 119 | Create a large language model instance based on the configuration provided. Args: llm_conf | STRING |
| HIGH | scrapegraphai/graphs/base_graph.py | 19 | BaseGraph manages the execution flow of a graph composed of interconnected nodes. Attributes: nodes (l | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_fetch_node_timeout.py | 65 | def test_timeout_default_value(self): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 74 | def test_timeout_custom_value(self): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 102 | def test_requests_get_with_timeout(self, mock_requests): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 125 | def test_requests_get_without_timeout_when_none(self, mock_requests): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 147 | def test_pdf_parsing_with_timeout(self): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 163 | def test_pdf_parsing_timeout_exceeded(self): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 189 | def test_timeout_propagated_to_chromium_loader(self, mock_loader_class): | CODE |
| LOW | tests/test_fetch_node_timeout.py | 213 | def test_timeout_not_overridden_in_loader_kwargs(self, mock_loader_class): | CODE |
| LOW | tests/test_csv_scraper_multi_graph.py | 74 | def test_create_graph_structure(monkeypatch): | CODE |
| LOW | tests/test_csv_scraper_multi_graph.py | 119 | def test_run_argument_passing(): | CODE |
| LOW | tests/test_csv_scraper_multi_graph.py | 146 | def test_run_with_exception_in_execute(): | CODE |
| LOW | tests/test_cleanup_html.py | 13 | def test_extract_from_script_tags(): | CODE |
| LOW | tests/test_cleanup_html.py | 32 | def test_cleanup_html_success(): | STRING |
| LOW | tests/test_cleanup_html.py | 61 | def test_cleanup_html_no_body(): | STRING |
| LOW | tests/conftest.py | 388 | def pytest_collection_modifyitems(config, items): | CODE |
| LOW | tests/test_json_scraper_graph.py | 21 | def test_json_scraper_graph_with_directory( | CODE |
| LOW | tests/test_json_scraper_graph.py | 78 | def test_json_scraper_graph_with_single_file( | CODE |
| LOW | tests/test_json_scraper_graph.py | 135 | def test_json_scraper_graph_no_answer_found( | CODE |
| LOW | tests/test_json_scraper_graph.py | 189 | def test_json_scraper_graph_with_custom_schema( | CODE |
| LOW | tests/test_scrape_do.py | 9 | def test_scrape_do_fetch_without_proxy(): | CODE |
| LOW | tests/test_scrape_do.py | 36 | def test_scrape_do_fetch_with_proxy_no_geo(): | CODE |
| LOW | tests/test_scrape_do.py | 70 | def test_scrape_do_fetch_with_proxy_with_geo(): | CODE |
| LOW | tests/test_scrape_do.py | 107 | def test_scrape_do_fetch_without_proxy_custom_env(): | CODE |
| LOW | tests/test_scrape_do.py | 131 | def test_scrape_do_fetch_with_proxy_custom_env(): | CODE |
| LOW | tests/test_scrape_do.py | 161 | def test_scrape_do_fetch_exception_propagation(): | CODE |
| LOW | tests/test_scrape_do.py | 174 | def test_scrape_do_fetch_with_proxy_with_geo_and_super_false(): | CODE |
| LOW | tests/test_scrape_do.py | 208 | def test_scrape_do_fetch_empty_token_without_proxy(): | CODE |
| LOW | tests/test_scrape_do.py | 230 | def test_scrape_do_fetch_with_proxy_with_empty_geo(): | CODE |
| LOW | tests/test_scrape_do.py | 265 | def test_scrape_do_fetch_api_encoding_special_characters(): | CODE |
| LOW | tests/test_script_creator_multi_graph.py | 65 | def test_create_graph_structure(self): | CODE |
| LOW | tests/test_script_creator_multi_graph.py | 130 | def test_create_graph_node_configs(self): | CODE |
| LOW | tests/test_script_creator_multi_graph.py | 181 | def test_run_with_empty_prompt(self): | CODE |
| LOW | tests/test_batch_api.py | 45 | def test_to_jsonl_line_with_max_tokens(self): | CODE |
| LOW | tests/test_batch_api.py | 56 | def test_to_jsonl_line_with_response_format(self): | CODE |
| LOW | tests/test_batch_api.py | 67 | def test_to_jsonl_line_without_optional_fields(self): | CODE |
| LOW | tests/test_batch_api.py | 78 | def test_to_jsonl_line_custom_temperature(self): | CODE |
| LOW | tests/test_batch_api.py | 159 | def test_retrieve_no_output_file(self): | CODE |
| LOW | tests/test_batch_api.py | 173 | def test_results_sorted_by_custom_id(self): | CODE |
| LOW | tests/test_batch_api.py | 230 | def test_handles_partial_failures(self): | CODE |
| LOW | tests/test_batch_api.py | 280 | def test_rejects_non_openai_provider(self): | CODE |
| LOW | tests/test_batch_api.py | 293 | def test_rejects_groq_provider(self): | CODE |
| LOW | tests/test_batch_api.py | 313 | def test_empty_parsed_docs_raises(self): | CODE |
| LOW | tests/test_batch_api.py | 349 | def test_model_name_extraction(self): | CODE |
| LOW | tests/test_batch_api.py | 366 | def test_batch_model_override(self): | CODE |
| LOW | tests/test_batch_api.py | 386 | def test_format_instructions_without_schema(self): | CODE |
| LOW | tests/test_depth_search_graph.py | 19 | def test_depth_search_graph_initialization(self, source, expected_input_key): | CODE |
| LOW | tests/test_models_tokens.py | 41 | def test_non_existent_provider(self): | CODE |
| LOW | tests/test_models_tokens.py | 52 | def test_specific_token_value(self): | CODE |
| LOW | tests/test_models_tokens.py | 58 | def test_non_empty_model_keys(self): | CODE |
| LOW | tests/test_models_tokens.py | 88 | def test_specific_model_token_values(self): | CODE |
| LOW | tests/test_models_tokens.py | 121 | def test_no_whitespace_in_model_names(self): | CODE |
| LOW | tests/test_models_tokens.py | 130 | def test_specific_models_additional(self): | CODE |
| LOW | tests/test_generate_answer_node.py | 45 | def test_process_missing_content_and_user_prompt(dummy_node): | CODE |
| LOW | tests/test_generate_answer_node.py | 84 | def test_execute_multiple_chunks(dummy_node_with_pipe): | CODE |
| LOW | tests/test_generate_answer_node.py | 109 | def test_execute_single_chunk(dummy_node_with_pipe): | CODE |
| LOW | tests/test_generate_answer_node.py | 125 | def test_execute_merge_json_decode_error(dummy_node_with_pipe): | CODE |
| LOW | tests/test_generate_answer_node.py | 190 | def test_execute_timeout_single_chunk(dummy_node_with_pipe): | CODE |
| LOW | tests/test_generate_answer_node.py | 206 | def test_execute_script_creator_single_chunk(): | CODE |
| LOW | tests/test_generate_answer_node.py | 249 | def test_init_chat_ollama_format(): | CODE |
| LOW⚡ | tests/test_chromium.py | 486 | async def test_alazy_load_empty_urls(): | CODE |
| 162 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_minimax_models.py | 5 | CODE | |
| LOW | tests/utils/test_sys_dynamic_import.py | 71 | CODE | |
| LOW | tests/utils/test_sys_dynamic_import.py | 77 | CODE | |
| LOW | scrapegraphai/docloaders/__init__.py | 8 | CODE | |
| LOW | scrapegraphai/docloaders/__init__.py | 9 | CODE | |
| LOW | scrapegraphai/docloaders/plasmate.py | 37 | CODE | |
| LOW | scrapegraphai/builders/__init__.py | 5 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 5 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 6 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 7 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 8 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 9 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 10 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 11 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 12 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 13 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 14 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 15 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 16 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 17 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 18 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 19 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 20 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 21 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 22 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 23 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 24 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 25 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 26 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 27 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 28 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 29 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 30 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 31 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 32 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 33 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 34 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 35 | CODE | |
| LOW | scrapegraphai/nodes/__init__.py | 36 | CODE | |
| LOW | scrapegraphai/nodes/batch_generate_answer_node.py | 10 | CODE | |
| LOW | scrapegraphai/nodes/batch_generate_answer_node.py | 10 | CODE | |
| LOW | scrapegraphai/nodes/batch_generate_answer_node.py | 13 | CODE | |
| LOW | scrapegraphai/nodes/batch_generate_answer_node.py | 19 | CODE | |
| LOW | scrapegraphai/utils/batch_api.py | 15 | CODE | |
| LOW | scrapegraphai/utils/batch_api.py | 16 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 5 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 6 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 6 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 7 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 7 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 7 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 7 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 13 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 13 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 13 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 13 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 19 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 20 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 20 | CODE | |
| LOW | scrapegraphai/utils/__init__.py | 20 | CODE | |
| 128 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/conftest.py | 43 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 45 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 164 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 166 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 188 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 190 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 269 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 271 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 306 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 308 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 332 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 334 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 364 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 366 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 97 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 99 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 139 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 141 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 179 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 181 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 221 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 223 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 257 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_plasmate.py | 259 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 13 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 15 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 32 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 34 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 65 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_plasmate.py | 67 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/integration/test_smart_scraper_integration.py | 124 | except Exception as e: | CODE |
| LOW | examples/extras/chromium_selenium.py | 45 | except Exception as e: | CODE |
| LOW | examples/extras/chromium_selenium.py | 81 | except Exception as e: | CODE |
| LOW | examples/extras/chromium_selenium.py | 115 | except Exception as e: | CODE |
| LOW | examples/extras/chromium_selenium.py | 138 | except Exception as e: | CODE |
| LOW | examples/extras/chromium_selenium.py | 147 | except Exception as e: | CODE |
| LOW | scrapegraphai/docloaders/chromium.py | 81 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/search_link_node.py | 146 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/generate_answer_node.py | 86 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/generate_answer_node.py | 114 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/fetch_node_level_k.py | 229 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/fetch_node_level_k.py | 251 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/image_to_text_node.py | 83 | except Exception: | CODE |
| LOW | scrapegraphai/nodes/generate_code_node.py | 467 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/conditional_node.py | 109 | except Exception as e: | CODE |
| LOW | scrapegraphai/nodes/parse_node.py | 106 | except Exception: | CODE |
| LOW | scrapegraphai/utils/code_error_analysis.py | 155 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_analysis.py | 210 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_analysis.py | 271 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_analysis.py | 334 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_correction.py | 135 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_correction.py | 188 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_correction.py | 247 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/code_error_correction.py | 318 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/copy.py | 70 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/cleanup_html.py | 41 | except Exception: | CODE |
| LOW | scrapegraphai/utils/research_web.py | 281 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/research_web.py | 328 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/research_web.py | 369 | except Exception as e: | CODE |
| LOW | scrapegraphai/utils/research_web.py | 415 | except Exception as e: | CODE |
| LOW | scrapegraphai/graphs/abstract_graph.py | 281 | except Exception as e: | CODE |
| LOW | scrapegraphai/graphs/base_graph.py | 195 | except Exception: | CODE |
| LOW | scrapegraphai/graphs/base_graph.py | 293 | except Exception as e: | CODE |
| LOW | scrapegraphai/telemetry/telemetry.py | 23 | except Exception: | CODE |
| LOW⚡ | scrapegraphai/telemetry/telemetry.py | 34 | except Exception: | CODE |
| LOW⚡ | scrapegraphai/telemetry/telemetry.py | 44 | except Exception: | CODE |
| LOW⚡ | scrapegraphai/telemetry/telemetry.py | 52 | except Exception: | CODE |
| LOW | scrapegraphai/telemetry/telemetry.py | 141 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapegraphai/docloaders/__init__.py | 25 | __all__ = [ | CODE |
| LOW | scrapegraphai/builders/__init__.py | 7 | __all__ = [ | CODE |
| LOW | scrapegraphai/nodes/__init__.py | 38 | __all__ = [ | CODE |
| LOW | scrapegraphai/nodes/batch_generate_answer_node.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapegraphai/utils/logging.py | 35 | def _set_library_root_logger() -> None: | CODE |
| LOW | scrapegraphai/utils/logging.py | 90 | def set_verbosity(verbosity: int) -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 101 | def set_verbosity_debug() -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 108 | def set_verbosity_info() -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 115 | def set_verbosity_warning() -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 122 | def set_verbosity_error() -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 129 | def set_verbosity_fatal() -> None: | CODE |
| LOW⚡ | scrapegraphai/utils/logging.py | 136 | def set_handler(handler: logging.Handler) -> None: | CODE |
| LOW | scrapegraphai/utils/logging.py | 178 | def set_propagation() -> None: | CODE |
| LOW | scrapegraphai/utils/logging.py | 192 | def set_formatting() -> None: | CODE |
| LOW | scrapegraphai/utils/batch_api.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapegraphai/utils/__init__.py | 58 | __all__ = [ | CODE |
| LOW | scrapegraphai/utils/screenshot_scraping/__init__.py | 9 | __all__ = [ | CODE |
| LOW | scrapegraphai/models/__init__.py | 14 | __all__ = ["DeepSeek", "MiniMax", "OneApi", "OpenAIImageToText", "OpenAITextToSpeech", "CLoD", "XAI", "Nvidia"] | CODE |
| LOW | scrapegraphai/integrations/__init__.py | 8 | __all__ = [ | CODE |
| LOW | scrapegraphai/prompts/__init__.py | 58 | __all__ = [ | CODE |
| LOW | scrapegraphai/graphs/__init__.py | 32 | __all__ = [ | CODE |
| LOW | scrapegraphai/graphs/smart_scraper_graph.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapegraphai/telemetry/__init__.py | 7 | __all__ = [ | CODE |
| LOW | scrapegraphai/telemetry/telemetry.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapegraphai/helpers/__init__.py | 10 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapegraphai/docloaders/chromium.py | 86 | CODE | |
| LOW | scrapegraphai/docloaders/chromium.py | 165 | CODE | |
| LOW | scrapegraphai/docloaders/chromium.py | 322 | CODE | |
| LOW | scrapegraphai/docloaders/chromium.py | 381 | CODE | |
| LOW | scrapegraphai/nodes/fetch_node_level_k.py | 234 | CODE | |
| LOW | scrapegraphai/nodes/fetch_node.py | 171 | CODE | |
| LOW | scrapegraphai/nodes/fetch_node.py | 266 | CODE | |
| LOW | scrapegraphai/utils/llm_callback_manager.py | 37 | CODE | |
| LOW | scrapegraphai/utils/cleanup_html.py | 13 | CODE | |
| LOW | scrapegraphai/utils/dict_content_compare.py | 15 | CODE | |
| LOW | scrapegraphai/utils/research_web.py | 160 | CODE | |
| LOW | scrapegraphai/utils/research_web.py | 285 | CODE | |
| LOW | scrapegraphai/utils/research_web.py | 373 | CODE | |
| LOW | scrapegraphai/utils/schema_trasform.py | 6 | CODE | |
| LOW | scrapegraphai/utils/schema_trasform.py | 17 | CODE | |
| LOW | …ai/utils/screenshot_scraping/screenshot_preparation.py | 44 | CODE | |
| LOW | …ai/utils/screenshot_scraping/screenshot_preparation.py | 67 | CODE | |
| LOW | scrapegraphai/graphs/abstract_graph.py | 118 | CODE | |
| LOW | scrapegraphai/graphs/base_graph.py | 130 | CODE | |
| LOW | scrapegraphai/graphs/base_graph.py | 157 | CODE | |
| LOW | scrapegraphai/graphs/base_graph.py | 236 | CODE | |
| LOW | scrapegraphai/graphs/base_graph.py | 344 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | docs/timeout_configuration.md | 169 | "api_key": "your-api-key" | CODE |
| HIGH | examples/markdownify/readme.md | 22 | sgai_client = Client(api_key="your-api-key") | CODE |
| HIGH | examples/smart_scraper_graph/scrapegraphai/readme.md | 48 | SCRAPEGRAPH_API_KEY=your-api-key-here | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/conftest.py | 405 | # Check if any API key is available | COMMENT |
| LOW | tests/test_json_scraper_graph.py | 246 | # Check if the custom schema was passed to GenerateAnswerNode | COMMENT |
| LOW | tests/test_chromium.py | 1073 | # Set evaluate to always return constant height value (simulate constant page height) | COMMENT |
| LOW | tests/nodes/robot_node_test.py | 72 | # Set force_scraping to True | COMMENT |
| LOW | tests/nodes/search_link_node_test.py | 56 | # Check if the result is not None | COMMENT |
| LOW | scrapegraphai/utils/code_error_analysis.py | 134 | # Check if syntax errors exist | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapegraphai/docloaders/chromium.py | 26 | CODE | |
| LOW | scrapegraphai/utils/research_web.py | 160 | CODE | |
| LOW | scrapegraphai/models/openai_itt.py | 47 | CODE | |
| LOW | scrapegraphai/graphs/abstract_graph.py | 333 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_chromium.py | 535 | # If no failure, simply pass. | COMMENT |
| LOW | scrapegraphai/nodes/generate_scraper_node.py | 109 | # If there are more than one chunks returned by ParseNode we just use the first one | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .github/workflows/codeql.yml | 1 | # For most projects, this workflow file will not need changing; you simply need | COMMENT |
| LOW | .github/workflows/dependency-review.yml | 1 | # Dependency Review Action | COMMENT |