Home / ScrapeGraphAI / Scrapegraph-ai
Repository Analysis

ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

40.0 Strong AI signal View on GitHub
40.0
Adjusted Score
40.0
Raw Score
100%
Time Factor
2026-05-21
Last Push
26,469
Stars
Python
Language
39,987
Lines of Code
325
Files
785
Pattern Hits
2026-05-31
Scan Date

Score History

Severity Breakdown

CRITICAL 0HIGH 119MEDIUM 186LOW 480

Pattern Findings

785 matches across 12 categories. Click a row to expand file-level details.

Self-Referential Comments156 hits · 476 pts
SeverityFileLineSnippet
MEDIUMtests/test_fetch_node_timeout.py165 # Create a mock loader that takes longer than timeout
MEDIUMtests/test_json_scraper_graph.py46 # Create a JSONScraperGraph instance
MEDIUMtests/test_json_scraper_graph.py103 # Create a JSONScraperGraph instance with a single JSON file
MEDIUMtests/test_json_scraper_graph.py157 # Create a JSONScraperGraph instance
MEDIUMtests/test_json_scraper_graph.py203 # Define a custom schema
MEDIUMtests/test_json_scraper_graph.py220 # Create a JSONScraperGraph instance with a custom schema
MEDIUMtests/test_script_creator_multi_graph.py172 # Create a dummy graph that raises an exception when execute is called.
MEDIUMtests/test_chromium.py1026 # Create a loader instance with a backend that does not have a corresponding scraping method.
MEDIUMtests/test_chromium.py1038 # Create a dummy undetected_chromedriver module with a dummy Chrome driver that always fails.
MEDIUMtests/test_chromium.py2007 # Create a loader instance with a retry limit of 2 and a short timeout.
MEDIUMtests/test_chromium.py295 # Create a dummy undetected_chromedriver module with a dummy Chrome driver.
MEDIUMtests/test_chromium.py623 # Create a loader with JS support and a retry_limit of 2 (so one failure is allowed)
MEDIUMtests/test_chromium.py1232 # Create a dummy undetected_chromedriver module where Chrome is defined but will not be used.
MEDIUMtests/test_chromium.py1491 # Define a dummy scraper that returns an integer (non‐string)
MEDIUMtests/test_chromium.py1976 # Create a dummy playwright so that evaluate and content can be called
MEDIUMtests/test_omni_search_graph.py7# Create a dummy graph class to simulate graph execution
MEDIUMtests/nodes/search_internet_node_test.py18 # Define the model
MEDIUMtests/nodes/search_link_node_test.py14 # Define the configuration for the graph
MEDIUMtests/nodes/search_link_node_test.py22 # Define the SearchLinkNode with necessary configurations
MEDIUMtests/nodes/search_link_node_test.py29 # Define the initial state for the node
MEDIUMtests/utils/copy_utils_test.py176 original["self"] = original # Create a circular reference
MEDIUMtests/graphs/screenshot_scraper_test.py13# Define a fixture for the graph configuration
MEDIUMtests/graphs/code_generator_graph_openai_test.py17# Define the output schema for the graph
MEDIUMtests/graphs/xml_scraper_openai_test.py16# Define the test fixtures and helpers
MEDIUMtests/graphs/xml_scraper_openai_test.py49# Define the test cases
MEDIUMtests/graphs/abstract_graph_test.py56 # Create a mock graph with mock nodes
MEDIUMtests/graphs/abstract_graph_test.py61 # Create a TestGraph instance with the mock graph
MEDIUMtests/graphs/abstract_graph_test.py254 # Create a mock graph with mock nodes
MEDIUMtests/graphs/abstract_graph_test.py260 # Create a TestGraph instance with the mock graph
MEDIUMtests/graphs/search_graph_openai_test.py15# Define the test fixtures and helpers
MEDIUMtests/graphs/search_graph_openai_test.py36# Define the test cases
MEDIUMexamples/speech_graph/speech_graph_openai.py23# Define the configuration for the graph
MEDIUMexamples/speech_graph/speech_graph_openai.py39# Create the SpeechGraph instance and run it
MEDIUMexamples/custom_graph/ollama/custom_graph_ollama.py16# Define the configuration for the graph
MEDIUMexamples/custom_graph/ollama/custom_graph_ollama.py31# Define the graph nodes
MEDIUMexamples/custom_graph/ollama/custom_graph_ollama.py75# Create the graph by defining the connections
MEDIUMexamples/custom_graph/openai/custom_graph_openai.py22# Define the configuration for the graph
MEDIUMexamples/custom_graph/openai/custom_graph_openai.py34# Define the graph nodes
MEDIUMexamples/custom_graph/openai/custom_graph_openai.py86# Create the graph by defining the connections
MEDIUM…mples/json_scraper_graph/ollama/json_scraper_ollama.py26# Define the configuration for the graph
MEDIUM…mples/json_scraper_graph/ollama/json_scraper_ollama.py41# Create the JSONScraperGraph instance and run it
MEDIUM…mples/json_scraper_graph/openai/json_scraper_openai.py26# Define the configuration for the graph
MEDIUM…mples/json_scraper_graph/openai/json_scraper_openai.py39# Create the JSONScraperGraph instance and run it
MEDIUM…mples/json_scraper_graph/openai/omni_scraper_openai.py16# Define the configuration for the graph
MEDIUM…mples/json_scraper_graph/openai/omni_scraper_openai.py32# Create the OmniScraperGraph instance and run it
MEDIUMexamples/json_scraper_graph/openai/md_scraper_openai.py26# Define the configuration for the graph
MEDIUMexamples/json_scraper_graph/openai/md_scraper_openai.py39# Create the DocumentScraperGraph instance and run it
MEDIUM…_generator_graph/ollama/code_generator_graph_ollama.py15# Define the output schema for the graph
MEDIUM…_generator_graph/ollama/code_generator_graph_ollama.py29# Define the configuration for the graph
MEDIUM…_generator_graph/ollama/code_generator_graph_ollama.py54# Create the SmartScraperGraph instance and run it
MEDIUM…_generator_graph/openai/code_generator_graph_openai.py16# Define the output schema for the graph
MEDIUM…_generator_graph/openai/code_generator_graph_openai.py30# Define the configuration for the graph
MEDIUM…_generator_graph/openai/code_generator_graph_openai.py54# Create the SmartScraperGraph instance and run it
MEDIUMexamples/search_graph/ollama/search_graph_ollama.py9# Define the configuration for the graph
MEDIUMexamples/search_graph/ollama/search_graph_ollama.py25# Create the SearchGraph instance and run it
MEDIUM…ples/search_graph/ollama/search_graph_schema_ollama.py13# Define the output schema for the graph
MEDIUM…ples/search_graph/ollama/search_graph_schema_ollama.py27# Define the configuration for the graph
MEDIUM…ples/search_graph/ollama/search_graph_schema_ollama.py42# Create the SearchGraph instance and run it
MEDIUM…amples/search_graph/openai/search_link_graph_openai.py14# Define the configuration for the graph
MEDIUM…amples/search_graph/openai/search_link_graph_openai.py29# Create the SearchLinkGraph instance and run it
96 more matches not shown…
Cross-File Repetition65 hits · 325 pts
SeverityFileLineSnippet
HIGH…amples/search_graph/openai/search_link_graph_openai.py0basic example of scraping pipeline using smartscraper
HIGH…art_scraper_graph/ollama/smart_scraper_multi_ollama.py0basic example of scraping pipeline using smartscraper
HIGH…les/smart_scraper_graph/ollama/smart_scraper_ollama.py0basic example of scraping pipeline using smartscraper
HIGH…craper_graph/ollama/smart_scraper_multi_lite_ollama.py0basic example of scraping pipeline using smartscraper
HIGH…aper_graph/ollama/smart_scraper_multi_concat_ollama.py0basic example of scraping pipeline using smartscraper
HIGH…mart_scraper_graph/ollama/smart_scraper_lite_ollama.py0basic example of scraping pipeline using smartscraper
HIGH…aper_graph/openai/smart_scraper_multi_concat_openai.py0basic example of scraping pipeline using smartscraper
HIGH…les/smart_scraper_graph/openai/smart_scraper_openai.py0basic example of scraping pipeline using smartscraper
HIGH…craper_graph/openai/smart_scraper_multi_lite_openai.py0basic example of scraping pipeline using smartscraper
HIGH…art_scraper_graph/openai/smart_scraper_multi_openai.py0basic example of scraping pipeline using smartscraper
HIGH…mart_scraper_graph/openai/smart_scraper_lite_openai.py0basic example of scraping pipeline using smartscraper
HIGH…ript_generator_graph/openai/script_generator_openai.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/slow_mo.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/custom_prompt.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/browser_base_integration.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/force_mode.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/proxy_rotation.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/undected_playwright.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/reasoning.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/rag_caching.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/load_yml.py0basic example of scraping pipeline using smartscraper
HIGHexamples/extras/scrape_do.py0basic example of scraping pipeline using smartscraper
HIGH…enerator_graph/ollama/script_multi_generator_ollama.py0basic example of scraping pipeline using scriptcreatorgraph
HIGH…ript_generator_graph/ollama/script_generator_ollama.py0basic example of scraping pipeline using scriptcreatorgraph
HIGH…enerator_graph/openai/script_generator_multi_openai.py0basic example of scraping pipeline using scriptcreatorgraph
HIGH…nerator_graph/openai/script_generator_schema_openai.py0basic example of scraping pipeline using scriptcreatorgraph
HIGHscrapegraphai/nodes/rag_node.py0a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan
HIGHscrapegraphai/nodes/generate_answer_node_k_level.py0a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan
HIGHscrapegraphai/nodes/description_node.py0a node responsible for compressing the input tokens and storing the document in a vector database for retrieval. relevan
HIGHscrapegraphai/graphs/search_link_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/depth_search_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/smart_scraper_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/code_generator_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/document_scraper_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/smart_scraper_lite_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/omni_scraper_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/script_creator_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/xml_scraper_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/json_scraper_graph.py0creates the graph of nodes representing the workflow for web scraping. returns: basegraph: a graph instance representing
HIGHscrapegraphai/graphs/search_link_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/speech_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/smart_scraper_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/screenshot_scraper_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/document_scraper_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/omni_scraper_graph.py0executes the scraping process and returns the answer to the prompt. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/search_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/omni_search_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/xml_scraper_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/csv_scraper_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/smart_scraper_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/script_creator_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/json_scraper_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/document_scraper_multi_graph.py0creates the graph of nodes representing the workflow for web scraping and searching. returns: basegraph: a graph instanc
HIGHscrapegraphai/graphs/search_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGH…rapegraphai/graphs/smart_scraper_multi_concat_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/omni_search_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/xml_scraper_multi_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/csv_scraper_multi_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/smart_scraper_multi_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
HIGHscrapegraphai/graphs/script_creator_multi_graph.py0executes the web scraping and searching process. returns: str: the answer to the prompt.
5 more matches not shown…
Docstring Block Structure51 hits · 255 pts
SeverityFileLineSnippet
HIGHscrapegraphai/docloaders/chromium.py175 Asynchronously scrape the content of a given URL using Playwright's sync API and scrolling. Notes:
HIGHscrapegraphai/docloaders/chromium.py324 Asynchronously scrape the content of a given URL using Playwright's async API. Args: url (
HIGHscrapegraphai/docloaders/chromium.py385 Asynchronously scrape the content of a given URL by rendering JavaScript using Playwright. Args:
HIGHscrapegraphai/builders/graph_builder.py14 GraphBuilder is a dynamic tool for constructing web scraping graphs based on user prompts. It utilizes a natura
HIGHscrapegraphai/nodes/search_link_node.py94 Filter out relevant links from the webpage that are relavant to prompt. Out of the filtered links, also
HIGHscrapegraphai/nodes/search_node_with_context.py51 Generates an answer by constructing a prompt from the user's input and the scraped content, querying th
HIGHscrapegraphai/nodes/merge_answers_node.py58 Executes the node's logic to merge the answers from multiple graph instances into a single answer.
HIGHscrapegraphai/nodes/search_internet_node.py61 Generates an answer by constructing a prompt from the user's input and the scraped content, querying th
HIGHscrapegraphai/nodes/reasoning_node.py57 Generate a refined prompt for the reasoning task based on the user's input and the JSON schema.
HIGHscrapegraphai/nodes/graph_iterator_node.py47 Executes the node's logic to instantiate and run multiple graph instances in parallel. Args:
HIGHscrapegraphai/nodes/graph_iterator_node.py82asynchronously executes the node's logic with multiple graph instances running in parallel, using a semaphore of
HIGHscrapegraphai/nodes/generate_answer_csv_node.py71 Generates an answer by constructing a prompt from the user's input and the scraped content, querying th
HIGHscrapegraphai/nodes/generate_scraper_node.py56 Generates a python script for scraping a website using the specified library. Args: state
HIGHscrapegraphai/nodes/robots_node.py59 Checks if a website is scrapeable based on the robots.txt file and updates the state with the scrapeabi
HIGHscrapegraphai/nodes/parse_node_depth_k_node.py43 Executes the node's logic to parse the HTML documents content. Args: state (dict): The cur
HIGHscrapegraphai/nodes/fetch_node_level_k.py73 Executes the node's logic to fetch the HTML content of a specified URL and its sub-links recursively, t
HIGHscrapegraphai/nodes/merge_generated_scripts_node.py42 Executes the node's logic to merge the answers from multiple graph instances into a single answer.
HIGHscrapegraphai/nodes/fetch_node.py224 Handles the local source by fetching HTML content, optionally converting it to Markdown, and updating t
HIGHscrapegraphai/nodes/fetch_node.py267 Handles the web source by fetching HTML content from a URL, optionally converting it to Markdown, and u
HIGHscrapegraphai/nodes/concat_answers_node.py42 Executes the node's logic to concatenate the answers from multiple graph instances into a single answer
HIGHscrapegraphai/nodes/generate_code_node.py94 Generates Python code for a function that extracts data from HTML based on a output schema. Args:
HIGHscrapegraphai/nodes/generate_code_node.py146 Executes the overall reasoning loop to generate and validate the code. Args: state (dict):
HIGHscrapegraphai/nodes/text_to_speech_node.py40 Converts text to speech using the specified text-to-speech model. Args: state (dict): The
HIGHscrapegraphai/nodes/markdownify_node.py42 Executes the node's logic to convert HTML content to Markdown. Args: state (dict): The cur
HIGHscrapegraphai/nodes/get_probable_tags_node.py46 Generates a list of probable HTML tags based on the user's input and updates the state with this list.
HIGHscrapegraphai/nodes/base_node.py13 An abstract base class for nodes in a graph-based workflow, designed to perform specific actions when executed.
HIGHscrapegraphai/nodes/base_node.py99 Determines the necessary state keys based on the input specification. Args: state (dict):
HIGHscrapegraphai/nodes/base_node.py137 Parses the input keys expression to extract relevant keys from the state based on logical conditions.
HIGHscrapegraphai/nodes/html_analyzer_node.py59 Generates an analysis of the provided HTML code based on the wanted infromations to be extracted. Args
HIGHscrapegraphai/nodes/parse_node.py63 Executes the node's logic to parse the HTML document content and split it into chunks. Args:
HIGHscrapegraphai/nodes/prompt_refiner_node.py63 Generate a refined prompt using the user's prompt, the schema, and additional context. Args:
HIGHscrapegraphai/nodes/generate_answer_omni_node.py65 Generates an answer by constructing a prompt from the user's input and the scraped content, querying th
HIGHscrapegraphai/utils/batch_api.py118Create and submit an OpenAI Batch API job. Args: client: An initialized OpenAI client. requests: Li
HIGHscrapegraphai/utils/batch_api.py196Poll a batch job until it completes, fails, or times out. Args: client: An initialized OpenAI client.
HIGHscrapegraphai/utils/code_error_analysis.py107 Analyzes the syntax errors in the generated code. Args: state (dict): Contains the 'generated_code' an
HIGHscrapegraphai/utils/code_error_analysis.py160 Analyzes the execution errors in the generated code and HTML code. Args: state (dict): Contains the 'g
HIGHscrapegraphai/utils/code_error_analysis.py215 Analyzes the validation errors in the generated code based on a JSON schema. Args: state (dict): Conta
HIGHscrapegraphai/utils/code_error_analysis.py278 Analyzes the semantic differences in the generated code based on a comparison result. Args: state (dic
HIGHscrapegraphai/utils/code_error_correction.py89 Generates corrected code based on syntax error analysis. Args: state (dict): Contains the 'generated_c
HIGHscrapegraphai/utils/code_error_correction.py142 Generates corrected code based on execution error analysis. Args: state (dict): Contains the 'generate
HIGHscrapegraphai/utils/code_error_correction.py195 Generates corrected code based on validation error analysis. Args: state (dict): Contains the 'generat
HIGHscrapegraphai/utils/code_error_correction.py254 Generates corrected code based on semantic error analysis. Args: state (dict): Contains the 'generated
HIGHscrapegraphai/utils/copy.py36 Safely create a deep copy of an object, handling special cases. Args: obj: Object to copy Returns
HIGHscrapegraphai/utils/parse_state_keys.py9 Parses a complex boolean expression involving state keys. Args: expression (str): The boolean expressi
HIGHscrapegraphai/utils/research_web.py172 Search web function with improved error handling, validation, and security features. Args: query (str)
HIGHscrapegraphai/utils/sys_dynamic_import.py16 imports a python module from its srcfile Args: modpath: The srcfile absolute path modname: The
HIGH…apegraphai/utils/screenshot_scraping/text_detection.py7 Detects and extracts text from a given image. Parameters: image (PIL Image): The input image to ext
HIGH…ai/utils/screenshot_scraping/screenshot_preparation.py225 Crop an image using the specified coordinates. Parameters: image (PIL.Image): The image to be cropped.
HIGHscrapegraphai/integrations/indexify_node.py38 Executes the node's logic to index the content present in the state. Args: state (dict): T
HIGHscrapegraphai/graphs/abstract_graph.py119 Create a large language model instance based on the configuration provided. Args: llm_conf
HIGHscrapegraphai/graphs/base_graph.py19 BaseGraph manages the execution flow of a graph composed of interconnected nodes. Attributes: nodes (l
Hyper-Verbose Identifiers221 hits · 195 pts
SeverityFileLineSnippet
LOWtests/test_fetch_node_timeout.py65 def test_timeout_default_value(self):
LOWtests/test_fetch_node_timeout.py74 def test_timeout_custom_value(self):
LOWtests/test_fetch_node_timeout.py102 def test_requests_get_with_timeout(self, mock_requests):
LOWtests/test_fetch_node_timeout.py125 def test_requests_get_without_timeout_when_none(self, mock_requests):
LOWtests/test_fetch_node_timeout.py147 def test_pdf_parsing_with_timeout(self):
LOWtests/test_fetch_node_timeout.py163 def test_pdf_parsing_timeout_exceeded(self):
LOWtests/test_fetch_node_timeout.py189 def test_timeout_propagated_to_chromium_loader(self, mock_loader_class):
LOWtests/test_fetch_node_timeout.py213 def test_timeout_not_overridden_in_loader_kwargs(self, mock_loader_class):
LOWtests/test_csv_scraper_multi_graph.py74def test_create_graph_structure(monkeypatch):
LOWtests/test_csv_scraper_multi_graph.py119def test_run_argument_passing():
LOWtests/test_csv_scraper_multi_graph.py146def test_run_with_exception_in_execute():
LOWtests/test_cleanup_html.py13def test_extract_from_script_tags():
LOWtests/test_cleanup_html.py32def test_cleanup_html_success():
LOWtests/test_cleanup_html.py61def test_cleanup_html_no_body():
LOWtests/conftest.py373def pytest_collection_modifyitems(config, items):
LOWtests/test_json_scraper_graph.py21 def test_json_scraper_graph_with_directory(
LOWtests/test_json_scraper_graph.py78 def test_json_scraper_graph_with_single_file(
LOWtests/test_json_scraper_graph.py135 def test_json_scraper_graph_no_answer_found(
LOWtests/test_json_scraper_graph.py189 def test_json_scraper_graph_with_custom_schema(
LOWtests/test_scrape_do.py9def test_scrape_do_fetch_without_proxy():
LOWtests/test_scrape_do.py36def test_scrape_do_fetch_with_proxy_no_geo():
LOWtests/test_scrape_do.py70def test_scrape_do_fetch_with_proxy_with_geo():
LOWtests/test_scrape_do.py107def test_scrape_do_fetch_without_proxy_custom_env():
LOWtests/test_scrape_do.py131def test_scrape_do_fetch_with_proxy_custom_env():
LOWtests/test_scrape_do.py161def test_scrape_do_fetch_exception_propagation():
LOWtests/test_scrape_do.py174def test_scrape_do_fetch_with_proxy_with_geo_and_super_false():
LOWtests/test_scrape_do.py208def test_scrape_do_fetch_empty_token_without_proxy():
LOWtests/test_scrape_do.py230def test_scrape_do_fetch_with_proxy_with_empty_geo():
LOWtests/test_scrape_do.py265def test_scrape_do_fetch_api_encoding_special_characters():
LOWtests/test_script_creator_multi_graph.py65 def test_create_graph_structure(self):
LOWtests/test_script_creator_multi_graph.py130 def test_create_graph_node_configs(self):
LOWtests/test_script_creator_multi_graph.py181 def test_run_with_empty_prompt(self):
LOWtests/test_batch_api.py45 def test_to_jsonl_line_with_max_tokens(self):
LOWtests/test_batch_api.py56 def test_to_jsonl_line_with_response_format(self):
LOWtests/test_batch_api.py67 def test_to_jsonl_line_without_optional_fields(self):
LOWtests/test_batch_api.py78 def test_to_jsonl_line_custom_temperature(self):
LOWtests/test_batch_api.py159 def test_retrieve_no_output_file(self):
LOWtests/test_batch_api.py173 def test_results_sorted_by_custom_id(self):
LOWtests/test_batch_api.py230 def test_handles_partial_failures(self):
LOWtests/test_batch_api.py280 def test_rejects_non_openai_provider(self):
LOWtests/test_batch_api.py293 def test_rejects_groq_provider(self):
LOWtests/test_batch_api.py313 def test_empty_parsed_docs_raises(self):
LOWtests/test_batch_api.py349 def test_model_name_extraction(self):
LOWtests/test_batch_api.py366 def test_batch_model_override(self):
LOWtests/test_batch_api.py386 def test_format_instructions_without_schema(self):
LOWtests/test_depth_search_graph.py19 def test_depth_search_graph_initialization(self, source, expected_input_key):
LOWtests/test_models_tokens.py41 def test_non_existent_provider(self):
LOWtests/test_models_tokens.py52 def test_specific_token_value(self):
LOWtests/test_models_tokens.py58 def test_non_empty_model_keys(self):
LOWtests/test_models_tokens.py88 def test_specific_model_token_values(self):
LOWtests/test_models_tokens.py121 def test_no_whitespace_in_model_names(self):
LOWtests/test_models_tokens.py130 def test_specific_models_additional(self):
LOWtests/test_generate_answer_node.py45def test_process_missing_content_and_user_prompt(dummy_node):
LOWtests/test_generate_answer_node.py84def test_execute_multiple_chunks(dummy_node_with_pipe):
LOWtests/test_generate_answer_node.py109def test_execute_single_chunk(dummy_node_with_pipe):
LOWtests/test_generate_answer_node.py125def test_execute_merge_json_decode_error(dummy_node_with_pipe):
LOWtests/test_generate_answer_node.py190def test_execute_timeout_single_chunk(dummy_node_with_pipe):
LOWtests/test_generate_answer_node.py206def test_execute_script_creator_single_chunk():
LOWtests/test_generate_answer_node.py249def test_init_chat_ollama_format():
LOWtests/test_chromium.py486async def test_alazy_load_empty_urls():
161 more matches not shown…
Unused Imports190 hits · 156 pts
SeverityFileLineSnippet
LOWtests/test_minimax_models.py5
LOWtests/utils/test_sys_dynamic_import.py71
LOWtests/utils/test_sys_dynamic_import.py77
LOWscrapegraphai/docloaders/__init__.py5
LOWscrapegraphai/docloaders/__init__.py6
LOWscrapegraphai/docloaders/__init__.py7
LOWscrapegraphai/docloaders/__init__.py8
LOWscrapegraphai/docloaders/plasmate.py37
LOWscrapegraphai/builders/__init__.py5
LOWscrapegraphai/nodes/__init__.py5
LOWscrapegraphai/nodes/__init__.py6
LOWscrapegraphai/nodes/__init__.py7
LOWscrapegraphai/nodes/__init__.py8
LOWscrapegraphai/nodes/__init__.py9
LOWscrapegraphai/nodes/__init__.py10
LOWscrapegraphai/nodes/__init__.py11
LOWscrapegraphai/nodes/__init__.py12
LOWscrapegraphai/nodes/__init__.py13
LOWscrapegraphai/nodes/__init__.py14
LOWscrapegraphai/nodes/__init__.py15
LOWscrapegraphai/nodes/__init__.py16
LOWscrapegraphai/nodes/__init__.py17
LOWscrapegraphai/nodes/__init__.py18
LOWscrapegraphai/nodes/__init__.py19
LOWscrapegraphai/nodes/__init__.py20
LOWscrapegraphai/nodes/__init__.py21
LOWscrapegraphai/nodes/__init__.py22
LOWscrapegraphai/nodes/__init__.py23
LOWscrapegraphai/nodes/__init__.py24
LOWscrapegraphai/nodes/__init__.py25
LOWscrapegraphai/nodes/__init__.py26
LOWscrapegraphai/nodes/__init__.py27
LOWscrapegraphai/nodes/__init__.py28
LOWscrapegraphai/nodes/__init__.py29
LOWscrapegraphai/nodes/__init__.py30
LOWscrapegraphai/nodes/__init__.py31
LOWscrapegraphai/nodes/__init__.py32
LOWscrapegraphai/nodes/__init__.py33
LOWscrapegraphai/nodes/__init__.py34
LOWscrapegraphai/nodes/__init__.py35
LOWscrapegraphai/nodes/__init__.py36
LOWscrapegraphai/nodes/batch_generate_answer_node.py10
LOWscrapegraphai/nodes/batch_generate_answer_node.py10
LOWscrapegraphai/nodes/batch_generate_answer_node.py13
LOWscrapegraphai/nodes/batch_generate_answer_node.py19
LOWscrapegraphai/utils/batch_api.py15
LOWscrapegraphai/utils/batch_api.py16
LOWscrapegraphai/utils/__init__.py5
LOWscrapegraphai/utils/__init__.py6
LOWscrapegraphai/utils/__init__.py6
LOWscrapegraphai/utils/__init__.py7
LOWscrapegraphai/utils/__init__.py7
LOWscrapegraphai/utils/__init__.py7
LOWscrapegraphai/utils/__init__.py7
LOWscrapegraphai/utils/__init__.py13
LOWscrapegraphai/utils/__init__.py13
LOWscrapegraphai/utils/__init__.py13
LOWscrapegraphai/utils/__init__.py13
LOWscrapegraphai/utils/__init__.py19
LOWscrapegraphai/utils/__init__.py20
130 more matches not shown…
Decorative Section Separators30 hits · 105 pts
SeverityFileLineSnippet
MEDIUMtests/conftest.py28# ============================================================================
MEDIUMtests/conftest.py30# ============================================================================
MEDIUMtests/conftest.py149# ============================================================================
MEDIUMtests/conftest.py151# ============================================================================
MEDIUMtests/conftest.py173# ============================================================================
MEDIUMtests/conftest.py175# ============================================================================
MEDIUMtests/conftest.py254# ============================================================================
MEDIUMtests/conftest.py256# ============================================================================
MEDIUMtests/conftest.py291# ============================================================================
MEDIUMtests/conftest.py293# ============================================================================
MEDIUMtests/conftest.py317# ============================================================================
MEDIUMtests/conftest.py319# ============================================================================
MEDIUMtests/conftest.py349# ============================================================================
MEDIUMtests/conftest.py351# ============================================================================
MEDIUMtests/test_plasmate.py97# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py99# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py139# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py141# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py179# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py181# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py221# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py223# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py257# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py259# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py13# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py15# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py32# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py34# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py65# ---------------------------------------------------------------------------
MEDIUMtests/test_plasmate.py67# ---------------------------------------------------------------------------
Excessive Try-Catch Wrapping37 hits · 38 pts
SeverityFileLineSnippet
LOWtests/integration/test_smart_scraper_integration.py124 except Exception as e:
LOWexamples/extras/chromium_selenium.py45 except Exception as e:
LOWexamples/extras/chromium_selenium.py81 except Exception as e:
LOWexamples/extras/chromium_selenium.py115 except Exception as e:
LOWexamples/extras/chromium_selenium.py138 except Exception as e:
LOWexamples/extras/chromium_selenium.py147 except Exception as e:
LOWscrapegraphai/docloaders/chromium.py82 except Exception as e:
LOWscrapegraphai/nodes/search_link_node.py146 except Exception as e:
LOWscrapegraphai/nodes/generate_answer_node.py86 except Exception as e:
LOWscrapegraphai/nodes/generate_answer_node.py114 except Exception as e:
LOWscrapegraphai/nodes/fetch_node_level_k.py229 except Exception as e:
LOWscrapegraphai/nodes/fetch_node_level_k.py251 except Exception as e:
LOWscrapegraphai/nodes/image_to_text_node.py83 except Exception:
LOWscrapegraphai/nodes/generate_code_node.py467 except Exception as e:
LOWscrapegraphai/nodes/conditional_node.py109 except Exception as e:
LOWscrapegraphai/nodes/parse_node.py106 except Exception:
LOWscrapegraphai/utils/code_error_analysis.py155 except Exception as e:
LOWscrapegraphai/utils/code_error_analysis.py210 except Exception as e:
LOWscrapegraphai/utils/code_error_analysis.py271 except Exception as e:
LOWscrapegraphai/utils/code_error_analysis.py334 except Exception as e:
LOWscrapegraphai/utils/code_error_correction.py135 except Exception as e:
LOWscrapegraphai/utils/code_error_correction.py188 except Exception as e:
LOWscrapegraphai/utils/code_error_correction.py247 except Exception as e:
LOWscrapegraphai/utils/code_error_correction.py318 except Exception as e:
LOWscrapegraphai/utils/copy.py70 except Exception as e:
LOWscrapegraphai/utils/cleanup_html.py41 except Exception:
LOWscrapegraphai/utils/research_web.py293 except Exception as e:
LOWscrapegraphai/utils/research_web.py334 except Exception as e:
LOWscrapegraphai/utils/research_web.py380 except Exception as e:
LOWscrapegraphai/graphs/abstract_graph.py278 except Exception as e:
LOWscrapegraphai/graphs/base_graph.py195 except Exception:
LOWscrapegraphai/graphs/base_graph.py293 except Exception as e:
LOWscrapegraphai/telemetry/telemetry.py23 except Exception:
LOWscrapegraphai/telemetry/telemetry.py34 except Exception:
LOWscrapegraphai/telemetry/telemetry.py44 except Exception:
LOWscrapegraphai/telemetry/telemetry.py52 except Exception:
LOWscrapegraphai/telemetry/telemetry.py141 except Exception as e:
Deep Nesting22 hits · 22 pts
SeverityFileLineSnippet
LOWscrapegraphai/docloaders/chromium.py87
LOWscrapegraphai/docloaders/chromium.py166
LOWscrapegraphai/docloaders/chromium.py323
LOWscrapegraphai/docloaders/chromium.py382
LOWscrapegraphai/nodes/fetch_node_level_k.py234
LOWscrapegraphai/nodes/fetch_node.py172
LOWscrapegraphai/nodes/fetch_node.py266
LOWscrapegraphai/utils/llm_callback_manager.py37
LOWscrapegraphai/utils/cleanup_html.py13
LOWscrapegraphai/utils/dict_content_compare.py15
LOWscrapegraphai/utils/research_web.py161
LOWscrapegraphai/utils/research_web.py250
LOWscrapegraphai/utils/research_web.py338
LOWscrapegraphai/utils/schema_trasform.py6
LOWscrapegraphai/utils/schema_trasform.py17
LOW…ai/utils/screenshot_scraping/screenshot_preparation.py44
LOW…ai/utils/screenshot_scraping/screenshot_preparation.py67
LOWscrapegraphai/graphs/abstract_graph.py118
LOWscrapegraphai/graphs/base_graph.py130
LOWscrapegraphai/graphs/base_graph.py157
LOWscrapegraphai/graphs/base_graph.py236
LOWscrapegraphai/graphs/base_graph.py344
Magic Placeholder Names3 hits · 15 pts
SeverityFileLineSnippet
HIGHdocs/timeout_configuration.md169 "api_key": "your-api-key"
HIGHexamples/markdownify/readme.md22sgai_client = Client(api_key="your-api-key")
HIGHexamples/smart_scraper_graph/scrapegraphai/readme.md48SCRAPEGRAPH_API_KEY=your-api-key-here
Redundant / Tautological Comments6 hits · 9 pts
SeverityFileLineSnippet
LOWtests/conftest.py390 # Check if any API key is available
LOWtests/test_json_scraper_graph.py246 # Check if the custom schema was passed to GenerateAnswerNode
LOWtests/test_chromium.py1073 # Set evaluate to always return constant height value (simulate constant page height)
LOWtests/nodes/robot_node_test.py72 # Set force_scraping to True
LOWtests/nodes/search_link_node_test.py56 # Check if the result is not None
LOWscrapegraphai/utils/code_error_analysis.py134 # Check if syntax errors exist
AI Slop Vocabulary2 hits · 2 pts
SeverityFileLineSnippet
LOWtests/test_chromium.py535 # If no failure, simply pass.
LOWscrapegraphai/nodes/generate_scraper_node.py109 # If there are more than one chunks returned by ParseNode we just use the first one
Over-Commented Block2 hits · 2 pts
SeverityFileLineSnippet
LOW.github/workflows/codeql.yml1# For most projects, this workflow file will not need changing; you simply need
LOW.github/workflows/dependency-review.yml1# Dependency Review Action