Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
This report presents the forensic synthetic code analysis of apify/crawlee-python, a Python project with 9,472 GitHub stars. SynthScan v2.0 examined 110,510 lines of code across 1003 source files, recording 2280 pattern matches distributed across 19 syntactic categories. The overall adjusted score of 32.8 places this repository in the Strong AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 2280 distinct pattern matches across 19 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …d_docs/version-1.9/introduction/code_examples/02_bs.py | 17 | # Define a request handler and attach it to the crawler using the decorator. | COMMENT |
| MEDIUM | …ides/code_examples/request_router/router_middleware.py | 10 | # Create a custom router instance | COMMENT |
| MEDIUM | …_examples/request_router/custom_router_default_only.py | 8 | # Create a custom router instance | COMMENT |
| MEDIUM | …code_examples/request_router/failed_request_handler.py | 7 | # Create a crawler instance with retry settings | COMMENT |
| MEDIUM | …9/guides/code_examples/request_router/error_handler.py | 11 | # Create a crawler instance | COMMENT |
| MEDIUM | …code_examples/request_router/simple_default_handler.py | 7 | # Create a crawler instance | COMMENT |
| MEDIUM | …code_examples/request_router/basic_request_handlers.py | 9 | # Create a custom router instance | COMMENT |
| MEDIUM | …code_examples/request_router/basic_request_handlers.py | 12 | # Define the default handler (fallback for requests without specific labels) | COMMENT |
| MEDIUM | …code_examples/request_router/basic_request_handlers.py | 31 | # Define a handler for category pages | COMMENT |
| MEDIUM | …code_examples/request_router/basic_request_handlers.py | 53 | # Define a handler for product detail pages | COMMENT |
| MEDIUM | …ples/storages/dataset_with_crawler_explicit_example.py | 12 | # Create a new crawler (it can be any subclass of BasicCrawler). | COMMENT |
| MEDIUM | …ples/storages/dataset_with_crawler_explicit_example.py | 15 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …des/code_examples/storages/kvs_with_crawler_example.py | 7 | # Create a new Playwright crawler. | COMMENT |
| MEDIUM | …des/code_examples/storages/kvs_with_crawler_example.py | 10 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …code_examples/storages/dataset_with_crawler_example.py | 7 | # Create a new crawler (it can be any subclass of BasicCrawler). | COMMENT |
| MEDIUM | …code_examples/storages/dataset_with_crawler_example.py | 10 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …examples/storages/kvs_with_crawler_explicit_example.py | 12 | # Create a new Playwright crawler. | COMMENT |
| MEDIUM | …examples/storages/kvs_with_crawler_explicit_example.py | 15 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …ides/code_examples/storages/rq_with_crawler_example.py | 7 | # Create a new crawler (it can be any subclass of BasicCrawler). Request queue is | COMMENT |
| MEDIUM | …ides/code_examples/storages/rq_with_crawler_example.py | 11 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …_examples/storages/rq_with_crawler_explicit_example.py | 15 | # Create a new crawler (it can be any subclass of BasicCrawler) and pass the request | COMMENT |
| MEDIUM | …_examples/storages/rq_with_crawler_explicit_example.py | 19 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …s/code_examples/http_crawlers/beautifulsoup_example.py | 7 | # Create a BeautifulSoupCrawler instance | COMMENT |
| MEDIUM | …s/code_examples/http_crawlers/beautifulsoup_example.py | 13 | # Define the default request handler | COMMENT |
| MEDIUM | …1.9/guides/code_examples/http_crawlers/http_example.py | 8 | # Create an HttpCrawler instance - no automatic parsing | COMMENT |
| MEDIUM | …1.9/guides/code_examples/http_crawlers/http_example.py | 14 | # Define the default request handler | COMMENT |
| MEDIUM | …9/guides/code_examples/http_crawlers/parsel_example.py | 7 | # Create a ParselCrawler instance | COMMENT |
| MEDIUM | …9/guides/code_examples/http_crawlers/parsel_example.py | 13 | # Define the default request handler | COMMENT |
| MEDIUM⚡ | …reating_web_archive/manual_archiving_parsel_crawler.py | 37 | # Create a WARC archive file a prepare the writer. | COMMENT |
| MEDIUM⚡ | …reating_web_archive/manual_archiving_parsel_crawler.py | 42 | # Create a WARC info record to store metadata about the archive. | COMMENT |
| MEDIUM⚡ | …reating_web_archive/manual_archiving_parsel_crawler.py | 50 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …ing_web_archive/manual_archiving_playwright_crawler.py | 52 | # Create a WARC archive file a prepare the writer. | COMMENT |
| MEDIUM | …ing_web_archive/manual_archiving_playwright_crawler.py | 57 | # Create a WARC info record to store metadata about the archive. | COMMENT |
| MEDIUM | …n-1.9/guides/code_examples/login_crawler/http_login.py | 68 | # Create a POST request to the authentication API endpoint | COMMENT |
| MEDIUM | …des/code_examples/http_clients/parsel_impit_example.py | 21 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …des/code_examples/http_clients/parsel_httpx_example.py | 20 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …amples/http_clients/parsel_curl_impersonate_example.py | 20 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …des/code_examples/cookie_management/persist_cookies.py | 9 | # Define the pool outside the crawler so its state can be read before the run. | COMMENT |
| MEDIUM | …des/code_examples/error_handling/handle_proxy_error.py | 27 | # Create a new `Request` with a modified key to avoid deduplication | COMMENT |
| MEDIUM | …9/guides/code_examples/error_handling/disable_retry.py | 10 | # Create a parsing error for demonstration | COMMENT |
| MEDIUM | …void_blocking/playwright_with_fingerprint_generator.py | 10 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …xamples/avoid_blocking/playwright_with_cloakbrowser.py | 56 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …examples/playwright_crawler/multiple_launch_example.py | 8 | # Create a plugin for each required browser. | COMMENT |
| MEDIUM | …/storage_clients/redis_storage_client_basic_example.py | 4 | # Create a new instance of storage client using connection string. | COMMENT |
| MEDIUM | …ge_clients/file_system_storage_client_basic_example.py | 4 | # Create a new instance of storage client. | COMMENT |
| MEDIUM | …ts/file_system_storage_client_configuration_example.py | 5 | # Create a new instance of storage client. | COMMENT |
| MEDIUM | …ts/file_system_storage_client_configuration_example.py | 8 | # Create a configuration with custom settings. | COMMENT |
| MEDIUM | …_clients/redis_storage_client_configuration_example.py | 7 | # Create a new instance of storage client using a Redis client with custom settings. | COMMENT |
| MEDIUM | …_clients/redis_storage_client_configuration_example.py | 20 | # Create a configuration with custom settings. | COMMENT |
| MEDIUM | …es/storage_clients/sql_storage_client_basic_example.py | 6 | # Create a new instance of storage client. | COMMENT |
| MEDIUM | …ge_clients/sql_storage_client_configuration_example.py | 9 | # Create a new instance of storage client. | COMMENT |
| MEDIUM | …ge_clients/sql_storage_client_configuration_example.py | 24 | # Create a configuration with custom settings. | COMMENT |
| MEDIUM | …storage_clients/memory_storage_client_basic_example.py | 4 | # Create a new instance of storage client. | COMMENT |
| MEDIUM | …des/code_examples/proxy_management/tiers_bs_example.py | 8 | # Create a ProxyConfiguration object and pass it to the crawler. | COMMENT |
| MEDIUM | …des/code_examples/proxy_management/tiers_bs_example.py | 28 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …des/code_examples/proxy_management/tiers_pw_example.py | 8 | # Create a ProxyConfiguration object and pass it to the crawler. | COMMENT |
| MEDIUM | …des/code_examples/proxy_management/tiers_pw_example.py | 28 | # Define the default request handler, which will be called for every request. | COMMENT |
| MEDIUM | …s/code_examples/proxy_management/session_bs_example.py | 8 | # Create a ProxyConfiguration object and pass it to the crawler. | COMMENT |
| MEDIUM | …s/code_examples/proxy_management/session_pw_example.py | 8 | # Create a ProxyConfiguration object and pass it to the crawler. | COMMENT |
| MEDIUM | …de_examples/proxy_management/integration_pw_example.py | 8 | # Create a ProxyConfiguration object and pass it to the crawler. | COMMENT |
| 304 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ides/code_examples/http_crawlers/selectolax_context.py | 25 | def from_parsed_http_crawling_context( | CODE |
| LOW | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 13 | def stop_crawler_if_url_visited(context: BasicCrawlingContext) -> None: | CODE |
| LOW | …/examples/code_examples/adaptive_playwright_crawler.py | 22 | async def request_handler_for_label( | CODE |
| LOW | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 13 | def stop_crawler_if_url_visited(context: BasicCrawlingContext) -> None: | CODE |
| LOW | …/examples/code_examples/adaptive_playwright_crawler.py | 21 | async def request_handler_for_label( | CODE |
| LOW | website/src/theme/Navbar/MobileSidebar/Layout/index.js | 5 | export default function NavbarMobileSidebarLayout({ header, primaryMenu, secondaryMenu }) { | CODE |
| LOW | website/src/theme/Navbar/MobileSidebar/Header/index.js | 29 | export default function NavbarMobileSidebarHeader() { | CODE |
| LOW | …/src/theme/NavbarItem/DocsVersionDropdownNavbarItem.js | 93 | export default function DocsVersionDropdownNavbarItem({ | CODE |
| LOW | tests/unit/conftest.py | 85 | def _isolate_test_environment(prepare_test_env: Callable[[], None]) -> None: | CODE |
| LOW | tests/unit/server.py | 341 | async def generic_response_endpoint(_scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW | tests/unit/server.py | 349 | async def problematic_links_endpoint(scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW | tests/unit/server.py | 549 | async def resource_loading_endpoint(_scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW | tests/unit/server.py | 567 | async def start_enqueue_non_href_endpoint(_scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 23 | def test_configuration_rejects_out_of_range_max_used_cpu_ratio(invalid_value: float) -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 29 | def test_configuration_rejects_out_of_range_max_used_memory_ratio(invalid_value: float) -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 35 | def test_configuration_accepts_in_range_max_used_cpu_ratio(valid_value: float) -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 40 | def test_configuration_accepts_in_range_max_used_memory_ratio(valid_value: float) -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 44 | def test_global_configuration_works() -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 53 | def test_global_configuration_works_reversed() -> None: | CODE |
| LOW⚡ | tests/unit/test_configuration.py | 62 | async def test_storage_not_persisted_when_non_persistable_storage_used(tmp_path: Path, server_url: URL) -> None: | CODE |
| LOW | tests/unit/test_configuration.py | 78 | async def test_storage_persisted_with_explicit_statistics_with_persistable_storage( | CODE |
| LOW | tests/unit/test_configuration.py | 103 | async def test_storage_persisted_when_enabled(tmp_path: Path, server_url: URL) -> None: | CODE |
| LOW | tests/unit/test_request.py | 6 | def test_from_url_does_not_mutate_caller_user_data() -> None: | CODE |
| LOW⚡ | tests/unit/test_request.py | 17 | def test_from_url_can_reuse_same_user_data_dict() -> None: | CODE |
| LOW⚡ | tests/unit/test_request.py | 26 | def test_from_url_omits_crawlee_data_when_empty() -> None: | CODE |
| LOW⚡ | tests/unit/test_request.py | 33 | def test_from_url_keeps_crawlee_data_when_supplied() -> None: | CODE |
| LOW | tests/unit/test_router.py | 37 | async def test_router_no_default_handler() -> None: | CODE |
| LOW | tests/unit/test_router.py | 51 | async def test_router_default_handler_invoked() -> None: | CODE |
| LOW | tests/unit/test_router.py | 70 | async def test_router_specific_handler_invoked() -> None: | CODE |
| LOW | tests/unit/test_router.py | 95 | async def test_router_handler_not_nullified() -> None: | CODE |
| LOW | tests/unit/test_router.py | 105 | async def test_router_multi_labelled_handler() -> None: | CODE |
| LOW | tests/unit/test_router.py | 121 | async def test_router_use_middleware() -> None: | CODE |
| LOW | tests/unit/test_router.py | 149 | async def test_router_use_middleware_with_label() -> None: | CODE |
| LOW | tests/unit/test_router.py | 184 | async def test_router_middleware_order_execution() -> None: | CODE |
| LOW | tests/unit/test_router.py | 208 | async def test_router_middleware_exception_interrupts_chain() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 12 | def test_default_configuration() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 18 | def test_custom_configuration() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 25 | def test_configuration_overwrite_not_possible() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 34 | def test_configuration_conflict() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 42 | def test_default_event_manager() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 47 | def test_custom_event_manager() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 54 | def test_event_manager_overwrite_not_possible() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 63 | def test_event_manager_conflict() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 71 | def test_default_storage_client() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 76 | def test_custom_storage_client() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 83 | def test_storage_client_overwrite_not_possible() -> None: | CODE |
| LOW⚡ | tests/unit/test_service_locator.py | 92 | def test_storage_client_conflict() -> None: | CODE |
| LOW | tests/unit/test_log_config.py | 40 | def test_formatting_with_exception() -> None: | CODE |
| LOW | tests/unit/test_log_config.py | 54 | def test_formatter_without_name() -> None: | CODE |
| LOW | tests/unit/utils.py | 25 | def make_status_stream_client( | CODE |
| LOW | tests/unit/test_cli.py | 57 | def test_create_interactive_non_default_template(mock_cookiecutter: Mock, monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | tests/unit/test_cli.py | 91 | def test_create_non_interactive(mock_cookiecutter: Mock) -> None: | CODE |
| LOW | tests/unit/test_cli.py | 125 | def test_create_existing_folder( | CODE |
| LOW | tests/unit/test_cli.py | 174 | def test_create_existing_folder_interactive( | CODE |
| LOW | tests/unit/test_cli.py | 214 | def test_create_existing_folder_interactive_multiple_attempts( | CODE |
| LOW | tests/unit/test_cli.py | 266 | def test_import_error_handled(optional_module_name: str) -> None: | CODE |
| LOW⚡ | tests/unit/test_throttling_request_manager.py | 264 | async def test_record_domain_delay_returns_false_for_unconfigured_domain( | CODE |
| LOW⚡ | tests/unit/test_throttling_request_manager.py | 272 | async def test_different_domains_independent(manager: ThrottlingRequestManager[RequestQueue]) -> None: | CODE |
| LOW⚡ | tests/unit/test_throttling_request_manager.py | 357 | async def test_crawl_delay_integration(manager: ThrottlingRequestManager[RequestQueue]) -> None: | CODE |
| LOW⚡ | tests/unit/test_throttling_request_manager.py | 366 | async def test_crawl_delay_throttles_after_dispatch(manager: ThrottlingRequestManager[RequestQueue]) -> None: | CODE |
| 995 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …reating_web_archive/manual_archiving_parsel_crawler.py | 0 | helper function for archiving response in warc format. | STRING |
| HIGH | …ing_web_archive/manual_archiving_playwright_crawler.py | 0 | helper function for archiving response in warc format. | STRING |
| HIGH | …reating_web_archive/manual_archiving_parsel_crawler.py | 0 | helper function for archiving response in warc format. | STRING |
| HIGH | …ing_web_archive/manual_archiving_playwright_crawler.py | 0 | helper function for archiving response in warc format. | STRING |
| HIGH | …_examples/playwright_crawler_adaptive/pre_nav_hooks.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …_examples/playwright_crawler_adaptive/pre_nav_hooks.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …_examples/playwright_crawler_adaptive/pre_nav_hooks.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed both in static sub crawler and playwright sub crawler. trying to access `context.page` in this hook would | STRING |
| HIGH | …9/guides/code_examples/running_in_web_server/server.py | 0 | <!doctype html> <html> <body> <h1>scraper server</h1> <p>to scrape some page, visit "scrape" endpoint with url parameter | STRING |
| HIGH | …6/guides/code_examples/running_in_web_server/server.py | 0 | <!doctype html> <html> <body> <h1>scraper server</h1> <p>to scrape some page, visit "scrape" endpoint with url parameter | STRING |
| HIGH | …s/guides/code_examples/running_in_web_server/server.py | 0 | <!doctype html> <html> <body> <h1>scraper server</h1> <p>to scrape some page, visit "scrape" endpoint with url parameter | STRING |
| HIGH | …ples/pydantic_ai_crawler/selector_extractor_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …amples/pydantic_ai_crawler/custom_distiller_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …code_examples/pydantic_ai_crawler/debugging_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …des/code_examples/pydantic_ai_crawler/basic_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …de_examples/pydantic_ai_crawler/usage_limit_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …ples/pydantic_ai_crawler/selector_extractor_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …amples/pydantic_ai_crawler/custom_distiller_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …code_examples/pydantic_ai_crawler/debugging_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …des/code_examples/pydantic_ai_crawler/basic_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …de_examples/pydantic_ai_crawler/usage_limit_example.py | 0 | model representing the extracted data for an article. | STRING |
| HIGH | …9/deployment/code_examples/google/cloud_run_example.py | 0 | the crawler entry point that will be called when the http endpoint is accessed. | STRING |
| HIGH | …6/deployment/code_examples/google/cloud_run_example.py | 0 | the crawler entry point that will be called when the http endpoint is accessed. | STRING |
| HIGH | …s/deployment/code_examples/google/cloud_run_example.py | 0 | the crawler entry point that will be called when the http endpoint is accessed. | STRING |
| HIGH | …9/deployment/code_examples/google/cloud_run_example.py | 0 | default request handler that processes each page during crawling. | STRING |
| HIGH | …6/deployment/code_examples/google/cloud_run_example.py | 0 | default request handler that processes each page during crawling. | STRING |
| HIGH | …s/deployment/code_examples/google/cloud_run_example.py | 0 | default request handler that processes each page during crawling. | STRING |
| HIGH | …ples/code_examples/playwright_crawler_with_camoufox.py | 0 | example browser plugin that uses camoufox browser, but otherwise keeps the functionality of playwrightbrowserplugin. | STRING |
| HIGH | …ples/code_examples/playwright_crawler_with_camoufox.py | 0 | example browser plugin that uses camoufox browser, but otherwise keeps the functionality of playwrightbrowserplugin. | STRING |
| HIGH | …ples/code_examples/playwright_crawler_with_camoufox.py | 0 | example browser plugin that uses camoufox browser, but otherwise keeps the functionality of playwrightbrowserplugin. | STRING |
| HIGH | …project_template/templates/main_playwright_camoufox.py | 0 | example browser plugin that uses camoufox browser, but otherwise keeps the functionality of playwrightbrowserplugin. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | stop crawler once specific url is visited. example of guard condition to stop the crawler. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | stop crawler once specific url is visited. example of guard condition to stop the crawler. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | stop crawler once specific url is visited. example of guard condition to stop the crawler. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | add requests to the queue after some time. can be done by external code. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | add requests to the queue after some time. can be done by external code. | STRING |
| HIGH | …ples/code_examples/beautifulsoup_crawler_keep_alive.py | 0 | add requests to the queue after some time. can be done by external code. | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed only in playwright sub crawler. it is safe to access `page` object. | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed only in playwright sub crawler. it is safe to access `page` object. | STRING |
| HIGH | …/examples/code_examples/adaptive_playwright_crawler.py | 0 | hook executed only in playwright sub crawler. it is safe to access `page` object. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test that open() raises an error when both id and name are provided. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test that open() raises an error when both id and name are provided. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test that open() raises an error when both id and name are provided. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test that providing both alias and id raises error. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test that providing both alias and id raises error. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test that providing both alias and id raises error. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test that providing both alias and name raises error. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test that providing both alias and name raises error. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test that providing both alias and name raises error. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test that conflicts between named and alias storages are detected. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test that conflicts between named and alias storages are detected. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test that conflicts between named and alias storages are detected. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test purge behavior when purge_on_start=true: named storages retain data, unnamed storages are purged. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test purge behavior when purge_on_start=true: named storages retain data, unnamed storages are purged. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test purge behavior when purge_on_start=true: named storages retain data, unnamed storages are purged. | STRING |
| HIGH | tests/unit/storages/test_request_queue.py | 0 | test purge behavior when purge_on_start=false: all storages retain data regardless of type. | STRING |
| HIGH | tests/unit/storages/test_dataset.py | 0 | test purge behavior when purge_on_start=false: all storages retain data regardless of type. | STRING |
| HIGH | tests/unit/storages/test_key_value_store.py | 0 | test purge behavior when purge_on_start=false: all storages retain data regardless of type. | STRING |
| 51 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | website/generate_module_shortcuts.py | 3 | CODE | |
| LOW | …1.9/introduction/code_examples/03_transform_request.py | 1 | CODE | |
| LOW | …uides/code_examples/http_crawlers/selectolax_parser.py | 1 | CODE | |
| LOW | …ides/code_examples/http_crawlers/selectolax_crawler.py | 1 | CODE | |
| LOW | …de_examples/cookie_management/retry_restore_cookies.py | 7 | CODE | |
| LOW | …/playwright_crawler/browser_pool_page_hooks_example.py | 1 | CODE | |
| LOW | …laywright_crawler/browser_pool_launch_hooks_example.py | 1 | CODE | |
| LOW | …mples/storage_clients/custom_storage_client_example.py | 1 | CODE | |
| LOW | …9/guides/code_examples/running_in_web_server/server.py | 1 | CODE | |
| LOW | …n-1.9/examples/code_examples/configure_json_logging.py | 1 | CODE | |
| LOW | …0.6/introduction/code_examples/03_transform_request.py | 1 | CODE | |
| LOW | …6/guides/code_examples/running_in_web_server/server.py | 1 | CODE | |
| LOW | …n-0.6/examples/code_examples/configure_json_logging.py | 1 | CODE | |
| LOW | tests/unit/conftest.py | 1 | CODE | |
| LOW | tests/unit/server.py | 1 | CODE | |
| LOW | tests/unit/test_configuration.py | 1 | CODE | |
| LOW | tests/unit/test_request.py | 1 | CODE | |
| LOW | tests/unit/test_router.py | 1 | CODE | |
| LOW | tests/unit/test_service_locator.py | 1 | CODE | |
| LOW | tests/unit/test_log_config.py | 1 | CODE | |
| LOW | tests/unit/utils.py | 1 | CODE | |
| LOW | tests/unit/test_cli.py | 1 | CODE | |
| LOW | tests/unit/test_throttling_request_manager.py | 3 | CODE | |
| LOW | tests/unit/fingerprint_suite/test_header_generator.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_measure_time.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_retry.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_crypto.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_console.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_time.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_recurring_task.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_robots.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_log.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_sitemap.py | 34 | CODE | |
| LOW | tests/unit/_utils/test_sitemap.py | 34 | CODE | |
| LOW | tests/unit/_utils/test_system.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_requests.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_file.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_globs.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_byte_size.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_timedelta_ms.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_urls.py | 1 | CODE | |
| LOW | tests/unit/_utils/test_html_to_text.py | 1 | CODE | |
| LOW | tests/unit/_statistics/test_persistence.py | 1 | CODE | |
| LOW | tests/unit/_statistics/test_request_max_duration.py | 1 | CODE | |
| LOW | tests/unit/_statistics/test_periodic_logging.py | 1 | CODE | |
| LOW | tests/unit/storages/test_request_manager_tandem.py | 1 | CODE | |
| LOW | tests/unit/storages/conftest.py | 1 | CODE | |
| LOW | tests/unit/storages/test_request_queue.py | 1 | CODE | |
| LOW | tests/unit/storages/test_dataset.py | 1 | CODE | |
| LOW | tests/unit/storages/test_key_value_store.py | 1 | CODE | |
| LOW | tests/unit/crawlers/_basic/test_context_pipeline.py | 1 | CODE | |
| LOW | tests/unit/crawlers/_basic/test_basic_crawler.py | 2 | CODE | |
| LOW | …/crawlers/_beautifulsoup/test_beautifulsoup_crawler.py | 1 | CODE | |
| LOW | tests/unit/crawlers/_parsel/test_parsel_crawler.py | 1 | CODE | |
| LOW | …ts/unit/crawlers/_pydantic_ai/test_direct_extractor.py | 1 | CODE | |
| LOW | …nit/crawlers/_pydantic_ai/test_clean_html_distiller.py | 1 | CODE | |
| LOW | …/unit/crawlers/_pydantic_ai/test_skeleton_distiller.py | 1 | CODE | |
| LOW | …/unit/crawlers/_pydantic_ai/test_selector_extractor.py | 1 | CODE | |
| LOW | …unit/crawlers/_pydantic_ai/test_pydantic_ai_crawler.py | 1 | CODE | |
| LOW | …s/unit/crawlers/_playwright/test_playwright_crawler.py | 1 | CODE | |
| 383 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | src/crawlee/_utils/file.py | 67 | Resolve a storage subdirectory inside a base directory. Joins `subdirectory` onto `base_dir` and verifies that the | STRING |
| HIGH | src/crawlee/_utils/context.py | 12 | Ensure the (async) context manager is initialized before executing the method. This decorator checks if the calling | STRING |
| HIGH | src/crawlee/storages/_storage_instance_manager.py | 93 | Open a storage instance with caching support. Args: cls: The storage class to instantiate. | STRING |
| HIGH | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 353 | Execute an HTTP request utilizing the `BrowserPool` and the `Playwright` library. Args: context: Th | STRING |
| HIGH | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 497 | Validate the HTTP status code and raise appropriate exceptions if needed. Args: context: The curren | STRING |
| HIGH | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 525 | Try to detect if the request is blocked based on the response content. Args: context: The current c | STRING |
| HIGH | …wlee/crawlers/_abstract_http/_abstract_http_crawler.py | 292 | Validate the HTTP status code and raise appropriate exceptions if needed. Args: context: The curren | STRING |
| HIGH | …wlee/crawlers/_abstract_http/_abstract_http_crawler.py | 319 | Try to detect if the request is blocked based on the parsed response content. Args: context: The cu | STRING |
| HIGH | src/crawlee/http_clients/_base.py | 109 | Perform the crawling for a given request. This method is called from `crawler.run()`. Args: | STRING |
| HIGH | src/crawlee/http_clients/_base.py | 139 | Send an HTTP request via the client. This method is called from `context.send_request()` helper. Args: | STRING |
| HIGH | src/crawlee/http_clients/_base.py | 171 | Stream an HTTP request via the client. This method should be used for downloading potentially large data where | STRING |
| HIGH | src/crawlee/http_clients/_curl_impersonate.py | 335 | Convert from Crawlee HTTP method to curl-cffi HTTP method. Args: method: Crawlee HTTP method. | STRING |
| HIGH | src/crawlee/http_clients/_impit.py | 284 | Perform a request, following redirects one hop at a time. Redirects are resolved here instead of by `impit`, so | STRING |
| HIGH | src/crawlee/browsers/_browser_controller.py | 78 | Create a new page with the given context options. Args: browser_new_context_options: Keyword argume | STRING |
| HIGH | src/crawlee/browsers/_playwright_browser_controller.py | 163 | Create a new page with the given context options. Args: browser_new_context_options: Keyword argume | STRING |
| HIGH | src/crawlee/browsers/_stagehand_types.py | 119 | Extract structured data from the page using natural language. Argument `page` is automatically set. Args: | STRING |
| HIGH | src/crawlee/browsers/_stagehand_types.py | 131 | Run an autonomous multi-step AI agent on the page. Argument `page` is automatically set. Args: **kw | STRING |
| HIGH | …/crawlee/storage_clients/_sql/_request_queue_client.py | 133 | Open an existing request queue or create a new one. This method first tries to find an existing queue by ID or | STRING |
| HIGH | …rawlee/storage_clients/_sql/_key_value_store_client.py | 95 | Open or create a SQL key-value store client. This method attempts to open an existing key-value store from the | STRING |
| HIGH | src/crawlee/storage_clients/_sql/_dataset_client.py | 94 | Open an existing dataset or create a new one. Args: id: The ID of the dataset to open. If provided, | STRING |
| HIGH | …/storage_clients/_file_system/_request_queue_client.py | 161 | Open or create a file system request queue client. This method attempts to open an existing request queue from | STRING |
| HIGH | …torage_clients/_file_system/_key_value_store_client.py | 98 | Open or create a file system key-value store client. This method attempts to open an existing key-value store f | STRING |
| HIGH | …rawlee/storage_clients/_file_system/_dataset_client.py | 100 | Open or create a file system dataset client. This method attempts to open an existing dataset from the file sys | STRING |
| HIGH | …awlee/storage_clients/_memory/_request_queue_client.py | 72 | Open or create a new memory request queue client. This method creates a new in-memory request queue instance. U | STRING |
| HIGH | …lee/storage_clients/_memory/_key_value_store_client.py | 57 | Open or create a new memory key-value store client. This method creates a new in-memory key-value store instanc | STRING |
| HIGH | src/crawlee/storage_clients/_memory/_dataset_client.py | 62 | Open or create a new memory dataset client. This method creates a new in-memory dataset instance. Unlike persis | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …guides/code_examples/login_crawler/playwright_login.py | 47 | # Check if the session is available | COMMENT |
| LOW | …9/guides/code_examples/running_in_web_server/server.py | 57 | # Return the result | STRING |
| LOW⚡ | …code_examples/pydantic_ai_crawler/debugging_example.py | 40 | # Set concurrency to 1, which ensures only one request is processed at a time. | COMMENT |
| LOW⚡ | …code_examples/pydantic_ai_crawler/debugging_example.py | 44 | # Set abort_on_error to True to stop the crawl if an error occurs during | COMMENT |
| LOW | …les/code_examples/respect_robots_on_skipped_request.py | 22 | # Check if the request was skipped due to robots.txt rules | COMMENT |
| LOW | …guides/code_examples/login_crawler/playwright_login.py | 47 | # Check if the session is available | COMMENT |
| LOW | …6/guides/code_examples/running_in_web_server/server.py | 57 | # Return the result | STRING |
| LOW | …les/code_examples/respect_robots_on_skipped_request.py | 22 | # Check if the request was skipped due to robots.txt rules | COMMENT |
| LOW | tests/unit/crawlers/_basic/test_basic_crawler.py | 1157 | # Set max_concurrency to 1 to ensure testing max_requests_per_crawl accurately | COMMENT |
| LOW | tests/unit/crawlers/_basic/test_basic_crawler.py | 1181 | # Set max_concurrency to 1 to ensure testing max_requests_per_crawl accurately | COMMENT |
| LOW | tests/unit/crawlers/_basic/test_basic_crawler.py | 1431 | # Set max_concurrency to 1 to ensure testing urls are visited one by one in order. | COMMENT |
| LOW | tests/unit/crawlers/_basic/test_basic_crawler.py | 1459 | # Set concurrency to 2 to ensure two urls are being visited in parallel. | COMMENT |
| LOW | …/crawlers/_beautifulsoup/test_beautifulsoup_crawler.py | 113 | # Set max_concurrency to 1 to ensure testing max_requests_per_crawl accurately | COMMENT |
| LOW | tests/unit/crawlers/_parsel/test_parsel_crawler.py | 132 | # Set max_concurrency to 1 to ensure testing max_requests_per_crawl accurately | COMMENT |
| LOW⚡ | tests/unit/sessions/test_session_pool.py | 136 | # Check if the state is correctly persisted | COMMENT |
| LOW⚡ | tests/unit/sessions/test_session_pool.py | 141 | # Check if all the sessions are correctly persisted | COMMENT |
| LOW | …s/unit/storage_clients/_redis/test_redis_kvs_client.py | 60 | # Check if the records were created | COMMENT |
| LOW⚡ | …s/unit/storage_clients/_redis/test_redis_kvs_client.py | 121 | # Check if record content is valid JSON | COMMENT |
| LOW | …nit/storage_clients/_file_system/test_fs_kvs_client.py | 85 | # Check if the files were created | COMMENT |
| LOW⚡ | …nit/storage_clients/_file_system/test_fs_kvs_client.py | 130 | # Check if file content is valid JSON | COMMENT |
| LOW | …guides/code_examples/login_crawler/playwright_login.py | 47 | # Check if the session is available | COMMENT |
| LOW | …s/guides/code_examples/running_in_web_server/server.py | 57 | # Return the result | STRING |
| LOW⚡ | …code_examples/pydantic_ai_crawler/debugging_example.py | 40 | # Set concurrency to 1, which ensures only one request is processed at a time. | COMMENT |
| LOW⚡ | …code_examples/pydantic_ai_crawler/debugging_example.py | 44 | # Set abort_on_error to True to stop the crawl if an error occurs during | COMMENT |
| LOW | …les/code_examples/respect_robots_on_skipped_request.py | 22 | # Check if the request was skipped due to robots.txt rules | COMMENT |
| LOW | src/crawlee/_utils/sitemap.py | 428 | # Check if the first chunk is a valid gzip header | COMMENT |
| LOW | src/crawlee/_utils/file.py | 204 | # Set lineterminator to '\n' if not explicitly provided. This prevents double line endings on Windows. The | COMMENT |
| LOW | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 541 | # Check if the session is blocked based on the response content | COMMENT |
| LOW | src/crawlee/http_clients/_curl_impersonate.py | 307 | # Check if a session for the given proxy URL has already been created. | COMMENT |
| LOW⚡ | src/crawlee/project_template/hooks/pre_gen_project.py | 17 | # Check if package manager is available in PATH | COMMENT |
| LOW⚡ | src/crawlee/project_template/hooks/pre_gen_project.py | 22 | # Check if the package manager is executable | COMMENT |
| LOW⚡ | src/crawlee/project_template/hooks/pre_gen_project.py | 29 | # Check if the version matches the required regex | COMMENT |
| LOW | …/crawlee/storage_clients/_sql/_request_queue_client.py | 604 | # Check if there are any unhandled requests that are not blocked. | COMMENT |
| LOW | …/crawlee/storage_clients/_sql/_request_queue_client.py | 643 | # Check if there are any buffered updates that might change the pending count | COMMENT |
| LOW | …rawlee/storage_clients/_sql/_key_value_store_client.py | 292 | # Check if record exists | COMMENT |
| LOW | …rawlee/storage_clients/_redis/_request_queue_client.py | 508 | # Check if there are any requests in the queue. | COMMENT |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 342 | # Check if the request has already been handled. | COMMENT |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 351 | # Check if the request is already in progress. | COMMENT |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 500 | # Check if the request is in progress. | COMMENT |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 552 | # Check if the request is in progress. | COMMENT |
| LOW | …torage_clients/_file_system/_key_value_store_client.py | 372 | # Check if the KVS directory exists | COMMENT |
| LOW | …awlee/storage_clients/_memory/_request_queue_client.py | 234 | # Check if the request is in progress. | COMMENT |
| LOW | …awlee/storage_clients/_memory/_request_queue_client.py | 268 | # Check if the request is in progress. | COMMENT |
| LOW | src/crawlee/_autoscaling/snapshotter.py | 423 | # Check if the warning has been logged recently to avoid spamming | COMMENT |
| LOW⚡ | src/crawlee/request_loaders/_sitemap_request_loader.py | 416 | # Check if URL should be included | COMMENT |
| LOW⚡ | src/crawlee/request_loaders/_sitemap_request_loader.py | 423 | # Check if we have capacity in the queue | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …/playwright_crawler/browser_pool_page_hooks_example.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | …laywright_crawler/browser_pool_launch_hooks_example.py | 14 | logger = logging.getLogger(__name__) | CODE |
| LOW | …des/code_examples/http_headers/browser_page_headers.py | 15 | async def set_page_headers(context: PlaywrightPreNavCrawlingContext) -> None: | CODE |
| LOW | …mples/request_loaders/rl_basic_example_with_persist.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | …amples/request_loaders/sitemap_example_with_persist.py | 9 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/unit/conftest.py | 98 | def _set_crawler_log_level(pytestconfig: pytest.Config, monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | tests/unit/server.py | 183 | async def set_cookies(scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW | tests/unit/server.py | 426 | async def set_complex_cookies(_scope: dict[str, Any], _receive: Receive, send: Send) -> None: | CODE |
| LOW | …/playwright_crawler/browser_pool_page_hooks_example.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | …laywright_crawler/browser_pool_launch_hooks_example.py | 14 | logger = logging.getLogger(__name__) | CODE |
| LOW | …des/code_examples/http_headers/browser_page_headers.py | 15 | async def set_page_headers(context: PlaywrightPreNavCrawlingContext) -> None: | CODE |
| LOW | …mples/request_loaders/rl_basic_example_with_persist.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | …amples/request_loaders/sitemap_example_with_persist.py | 9 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/crawlee/configuration.py | 16 | __all__ = ['Configuration'] | CODE |
| LOW | src/crawlee/__init__.py | 10 | __all__ = [ | CODE |
| LOW | src/crawlee/proxy_configuration.py | 21 | __all__ = ['ProxyConfiguration', 'ProxyInfo'] | CODE |
| LOW | src/crawlee/errors.py | 10 | __all__ = [ | CODE |
| LOW | src/crawlee/router.py | 11 | __all__ = ['Router'] | CODE |
| LOW | src/crawlee/_service_locator.py | 46 | def set_configuration(self, configuration: Configuration) -> None: | CODE |
| LOW | src/crawlee/_service_locator.py | 76 | def set_event_manager(self, event_manager: EventManager) -> None: | CODE |
| LOW | src/crawlee/_service_locator.py | 106 | def set_storage_client(self, storage_client: StorageClient) -> None: | CODE |
| LOW | src/crawlee/statistics/__init__.py | 4 | __all__ = ['FinalStatistics', 'Statistics', 'StatisticsState'] | CODE |
| LOW | src/crawlee/fingerprint_suite/__init__.py | 6 | __all__ = [ | CODE |
| LOW | src/crawlee/storages/__init__.py | 5 | __all__ = [ | CODE |
| LOW | src/crawlee/storages/_dataset.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/crawlee/crawlers/__init__.py | 100 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_basic/__init__.py | 5 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_beautifulsoup/__init__.py | 15 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_parsel/__init__.py | 13 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_pydantic_ai/_types.py | 95 | def set_ai_usage(self, value: PydanticAiUsageStats) -> None: | CODE |
| LOW | src/crawlee/crawlers/_pydantic_ai/__init__.py | 29 | __all__ = [ | CODE |
| LOW | …c/crawlee/crawlers/_pydantic_ai/_selector_extractor.py | 342 | def set_ai_usage(self, value: PydanticAiUsageStats) -> None: | CODE |
| LOW | src/crawlee/crawlers/_pydantic_ai/_base_extractor.py | 65 | def set_ai_usage(self, value: PydanticAiUsageStats) -> None: | CODE |
| LOW | src/crawlee/crawlers/_playwright/__init__.py | 17 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 587 | async def _update_cookies(self, page: Page, cookies: list[PlaywrightCookieParam]) -> None: | CODE |
| LOW | src/crawlee/crawlers/_stagehand/__init__.py | 20 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_http/__init__.py | 6 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_file_download/__init__.py | 4 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_adaptive_playwright/__init__.py | 22 | __all__ = [ | CODE |
| LOW | src/crawlee/crawlers/_abstract_http/__init__.py | 5 | __all__ = [ | CODE |
| LOW | src/crawlee/otel/__init__.py | 3 | __all__ = [ | CODE |
| LOW⚡ | src/crawlee/http_clients/_curl_impersonate.py | 55 | def update_cookies_from_curl(self, morsels: list[CurlMorsel]) -> None: | CODE |
| LOW⚡ | src/crawlee/http_clients/_curl_impersonate.py | 59 | def update_cookies_from_curl_changes(self, changes: list[bytes]) -> None: | CODE |
| LOW | src/crawlee/http_clients/__init__.py | 19 | __all__ = [ | CODE |
| LOW | src/crawlee/browsers/__init__.py | 28 | __all__ = [ | CODE |
| LOW | src/crawlee/sessions/_cookies.py | 224 | def set_cookies(self, cookie_dicts: list[CookieParam]) -> None: | CODE |
| LOW⚡ | src/crawlee/sessions/_cookies.py | 268 | def set_cookies_from_playwright_format(self, pw_cookies: list[PlaywrightCookieParam]) -> None: | CODE |
| LOW | src/crawlee/sessions/__init__.py | 5 | __all__ = ['CookieParam', 'Session', 'SessionCookies', 'SessionPool'] | CODE |
| LOW | src/crawlee/storage_clients/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …rawlee/storage_clients/_sql/_key_value_store_client.py | 149 | async def set_value(self, *, key: str, value: Any, content_type: str | None = None) -> None: | CODE |
| LOW | src/crawlee/storage_clients/_sql/__init__.py | 6 | __all__ = ['SqlDatasetClient', 'SqlKeyValueStoreClient', 'SqlRequestQueueClient', 'SqlStorageClient'] | CODE |
| LOW | …wlee/storage_clients/_redis/_key_value_store_client.py | 125 | async def set_value(self, *, key: str, value: Any, content_type: str | None = None) -> None: | CODE |
| LOW | src/crawlee/storage_clients/_redis/__init__.py | 6 | __all__ = ['RedisDatasetClient', 'RedisKeyValueStoreClient', 'RedisRequestQueueClient', 'RedisStorageClient'] | CODE |
| LOW | …torage_clients/_file_system/_key_value_store_client.py | 302 | async def set_value(self, *, key: str, value: Any, content_type: str | None = None) -> None: | CODE |
| LOW | src/crawlee/storage_clients/_file_system/__init__.py | 6 | __all__ = [ | CODE |
| LOW | …lee/storage_clients/_memory/_key_value_store_client.py | 112 | async def set_value(self, *, key: str, value: Any, content_type: str | None = None) -> None: | CODE |
| LOW | src/crawlee/storage_clients/_memory/__init__.py | 6 | __all__ = [ | CODE |
| LOW | …awlee/storage_clients/_base/_key_value_store_client.py | 53 | async def set_value(self, *, key: str, value: Any, content_type: str | None = None) -> None: | CODE |
| LOW | src/crawlee/storage_clients/_base/__init__.py | 6 | __all__ = [ | CODE |
| LOW | src/crawlee/events/__init__.py | 15 | __all__ = [ | CODE |
| 3 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ing_web_archive/manual_archiving_playwright_crawler.py | 28 | except Exception as e: | CODE |
| LOW | tests/unit/_statistics/test_error_tracker.py | 34 | except Exception as e: # noqa:PERF203 | CODE |
| LOW | tests/unit/_statistics/test_error_tracker.py | 39 | except Exception as e: | CODE |
| LOW | tests/unit/_statistics/test_error_tracker.py | 69 | except Exception as e: # noqa:PERF203 | CODE |
| LOW⚡ | tests/unit/_statistics/test_error_tracker.py | 98 | except Exception as e: | CODE |
| LOW⚡ | tests/unit/_statistics/test_error_tracker.py | 110 | except Exception as e: | CODE |
| LOW⚡ | tests/unit/_statistics/test_error_tracker.py | 113 | except Exception as e: | CODE |
| LOW | tests/unit/storages/test_request_queue.py | 707 | except Exception: | CODE |
| LOW⚡ | tests/unit/crawlers/_parsel/test_parsel_crawler.py | 101 | except Exception as e: | CODE |
| LOW⚡ | …s/unit/crawlers/_playwright/test_playwright_crawler.py | 152 | except Exception as e: | CODE |
| LOW | …ing_web_archive/manual_archiving_playwright_crawler.py | 28 | except Exception as e: | CODE |
| LOW | src/crawlee/_cli.py | 258 | except Exception as exc: | CODE |
| LOW | src/crawlee/proxy_configuration.py | 186 | except Exception as e: | CODE |
| LOW | src/crawlee/statistics/_error_tracker.py | 104 | except Exception: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 224 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 235 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 374 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 503 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 658 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 677 | except Exception: | CODE |
| LOW | src/crawlee/_utils/sitemap.py | 749 | except Exception: | CODE |
| LOW | src/crawlee/_utils/wait.py | 40 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/wait.py | 85 | except Exception as e: | CODE |
| LOW | src/crawlee/_utils/recurring_task.py | 56 | except Exception: | CODE |
| LOW | src/crawlee/_utils/file.py | 61 | except Exception: | CODE |
| LOW | src/crawlee/_utils/requests.py | 82 | except Exception as exc: | CODE |
| LOW | src/crawlee/_utils/robots.py | 71 | except Exception as e: | CODE |
| LOW | src/crawlee/storages/_request_queue.py | 70 | except Exception: | STRING |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1181 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1246 | except Exception as secondary_error: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1268 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1283 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1550 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1574 | except Exception as internal_error: | CODE |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1586 | except Exception: # noqa: PERF203 | CODE |
| LOW | src/crawlee/crawlers/_basic/_context_pipeline.py | 50 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_context_pipeline.py | 107 | except Exception as e: | CODE |
| LOW | src/crawlee/crawlers/_basic/_context_pipeline.py | 118 | except Exception as e: | CODE |
| LOW | …rs/_playwright/_playwright_pre_nav_crawling_context.py | 39 | except Exception: | CODE |
| LOW | …rs/_playwright/_playwright_pre_nav_crawling_context.py | 44 | except Exception: | CODE |
| LOW | …s/_adaptive_playwright/_adaptive_playwright_crawler.py | 340 | except Exception as e: | CODE |
| LOW | src/crawlee/browsers/_browser_pool.py | 374 | except Exception: | CODE |
| LOW | src/crawlee/browsers/_stagehand_browser_controller.py | 187 | except Exception: | CODE |
| LOW | …rawlee/storage_clients/_redis/_request_queue_client.py | 437 | except Exception: | CODE |
| LOW | src/crawlee/events/_event_manager.py | 275 | except Exception: | CODE |
| LOW⚡ | src/crawlee/request_loaders/_sitemap_request_loader.py | 433 | except Exception: | CODE |
| LOW | src/crawlee/request_loaders/_sitemap_request_loader.py | 450 | except Exception: | CODE |
| LOW | src/crawlee/request_loaders/_request_manager_tandem.py | 87 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …rsion-1.9/examples/code_examples/playwright_crawler.py | 56 | # browser page among other things. In this example, we log the URL being | COMMENT |
| HIGH | …-1.9/examples/code_examples/respect_robots_txt_file.py | 19 | # In this example, 'https://news.ycombinator.com/login' will be skipped | COMMENT |
| HIGH | …rsion-0.6/examples/code_examples/playwright_crawler.py | 56 | # browser page among other things. In this example, we log the URL being | COMMENT |
| HIGH | …-0.6/examples/code_examples/respect_robots_txt_file.py | 19 | # In this example, 'https://news.ycombinator.com/login' will be skipped | COMMENT |
| HIGH | docs/examples/code_examples/playwright_crawler.py | 56 | # browser page among other things. In this example, we log the URL being | COMMENT |
| HIGH | docs/examples/code_examples/respect_robots_txt_file.py | 19 | # In this example, 'https://news.ycombinator.com/login' will be skipped | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/e2e/project_template/utils.py | 57 | CODE | |
| LOW | src/crawlee/_cli.py | 133 | CODE | |
| LOW | src/crawlee/fingerprint_suite/_browserforge_adapter.py | 56 | CODE | |
| LOW | src/crawlee/_utils/system.py | 36 | CODE | |
| LOW | src/crawlee/_utils/globs.py | 19 | CODE | |
| LOW | src/crawlee/_utils/globs.py | 76 | CODE | |
| LOW | src/crawlee/_utils/sitemap.py | 271 | CODE | |
| LOW | src/crawlee/_utils/sitemap.py | 380 | CODE | |
| LOW | src/crawlee/_utils/sitemap.py | 576 | CODE | |
| LOW | src/crawlee/_utils/sitemap.py | 698 | CODE | |
| LOW | src/crawlee/_utils/sitemap.py | 142 | CODE | |
| LOW | src/crawlee/_utils/wait.py | 49 | CODE | |
| LOW | src/crawlee/_utils/urls.py | 42 | CODE | |
| LOW | src/crawlee/storages/_storage_instance_manager.py | 40 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1075 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1435 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1624 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_context_pipeline.py | 82 | CODE | |
| LOW | src/crawlee/crawlers/_beautifulsoup/_utils.py | 20 | CODE | |
| LOW | src/crawlee/crawlers/_beautifulsoup/_utils.py | 38 | CODE | |
| LOW | src/crawlee/crawlers/_parsel/_utils.py | 16 | CODE | |
| LOW | src/crawlee/crawlers/_parsel/_utils.py | 34 | CODE | |
| LOW | …crawlee/crawlers/_pydantic_ai/_clean_html_distiller.py | 176 | CODE | |
| LOW | …c/crawlee/crawlers/_pydantic_ai/_selector_extractor.py | 656 | CODE | |
| LOW | src/crawlee/sessions/_cookies.py | 77 | CODE | |
| LOW | src/crawlee/sessions/_session.py | 94 | CODE | |
| LOW | src/crawlee/storage_clients/_sql/_storage_client.py | 112 | CODE | |
| LOW | …/crawlee/storage_clients/_sql/_request_queue_client.py | 207 | CODE | |
| LOW | …rawlee/storage_clients/_sql/_key_value_store_client.py | 149 | CODE | |
| LOW | …rawlee/storage_clients/_sql/_key_value_store_client.py | 191 | CODE | |
| LOW | …wlee/storage_clients/_redis/_key_value_store_client.py | 125 | CODE | |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 153 | CODE | |
| LOW | …/storage_clients/_file_system/_request_queue_client.py | 321 | CODE | |
| LOW | …torage_clients/_file_system/_key_value_store_client.py | 90 | CODE | |
| LOW | …torage_clients/_file_system/_key_value_store_client.py | 223 | CODE | |
| LOW | …torage_clients/_file_system/_key_value_store_client.py | 302 | CODE | |
| LOW | …rawlee/storage_clients/_file_system/_dataset_client.py | 92 | CODE | |
| LOW | src/crawlee/_autoscaling/autoscaled_pool.py | 105 | CODE | |
| LOW | src/crawlee/_autoscaling/autoscaled_pool.py | 213 | CODE | |
| LOW | src/crawlee/request_loaders/_sitemap_request_loader.py | 353 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | src/crawlee/_types.py | 373 | CODE | |
| LOW | src/crawlee/_types.py | 398 | CODE | |
| LOW | src/crawlee/_request.py | 244 | CODE | |
| LOW | src/crawlee/statistics/_statistics.py | 71 | CODE | |
| LOW | src/crawlee/statistics/_statistics.py | 127 | CODE | |
| LOW | src/crawlee/fingerprint_suite/_browserforge_adapter.py | 56 | CODE | |
| LOW | src/crawlee/storages/_dataset.py | 150 | CODE | |
| LOW | src/crawlee/storages/_dataset.py | 201 | CODE | |
| LOW | src/crawlee/storages/_dataset.py | 249 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 278 | CODE | |
| LOW | src/crawlee/crawlers/_basic/_basic_crawler.py | 1030 | CODE | |
| LOW | …crawlee/crawlers/_pydantic_ai/_clean_html_distiller.py | 71 | CODE | |
| LOW | …c/crawlee/crawlers/_pydantic_ai/_selector_extractor.py | 290 | CODE | |
| LOW | …c/crawlee/crawlers/_pydantic_ai/_skeleton_distiller.py | 64 | CODE | |
| LOW | src/crawlee/crawlers/_playwright/_playwright_crawler.py | 113 | CODE | |
| LOW | src/crawlee/crawlers/_stagehand/_stagehand_crawler.py | 94 | CODE | |
| LOW | src/crawlee/sessions/_session.py | 36 | CODE | |
| LOW | …/crawlee/storage_clients/_sql/_request_queue_client.py | 682 | CODE | |
| LOW | src/crawlee/storage_clients/_sql/_dataset_client.py | 163 | CODE | |
| LOW | src/crawlee/storage_clients/_sql/_dataset_client.py | 210 | CODE | |
| LOW | src/crawlee/storage_clients/_sql/_dataset_client.py | 243 | CODE | |
| LOW | src/crawlee/storage_clients/_redis/_dataset_client.py | 146 | CODE | |
| LOW | src/crawlee/storage_clients/_redis/_dataset_client.py | 232 | CODE | |
| LOW | …rawlee/storage_clients/_file_system/_dataset_client.py | 240 | CODE | |
| LOW | …rawlee/storage_clients/_file_system/_dataset_client.py | 339 | CODE | |
| LOW | src/crawlee/storage_clients/_memory/_dataset_client.py | 135 | CODE | |
| LOW | src/crawlee/storage_clients/_memory/_dataset_client.py | 194 | CODE | |
| LOW | src/crawlee/storage_clients/_base/_dataset_client.py | 53 | CODE | |
| LOW | src/crawlee/storage_clients/_base/_dataset_client.py | 74 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/unit/test_throttling_request_manager.py | 255 | # ── Core Throttling Tests ───────────────────────────────── | COMMENT |
| MEDIUM⚡ | tests/unit/test_throttling_request_manager.py | 354 | # ── Crawl-Delay Integration Tests ───────────────────────── | COMMENT |
| MEDIUM⚡ | tests/unit/test_throttling_request_manager.py | 376 | # ── Fetch Scheduling Tests ──────────────────────────── | COMMENT |
| MEDIUM⚡ | tests/unit/test_throttling_request_manager.py | 923 | # ── Utility Tests ────────────────────────────────────── | COMMENT |
| MEDIUM | tests/unit/test_throttling_request_manager.py | 102 | # ── Request Routing Tests ───────────────────────────────── | COMMENT |
| MEDIUM | tests/unit/test_throttling_request_manager.py | 493 | # ── Delegation Tests ──────────────────────────────────── | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ersion-1.9/introduction/code_examples/07_final_code.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | …/version-1.9/introduction/code_examples/06_scraping.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | …n-1.9/introduction/code_examples/05_crawling_detail.py | 13 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_context.py | 10 | # to facilitate working with the parsed document. | COMMENT |
| LOW | …ersion-0.6/introduction/code_examples/07_final_code.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | …/version-0.6/introduction/code_examples/06_scraping.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | …n-0.6/introduction/code_examples/05_crawling_detail.py | 13 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| MEDIUM | …daptive_playwright/test_adaptive_playwright_crawler.py | 48 | # Generous wait for the JS-injected element. Resolves as soon as it appears, so it stays robust on | COMMENT |
| LOW | docs/introduction/code_examples/07_final_code.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | docs/introduction/code_examples/06_scraping.py | 16 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| LOW | docs/introduction/code_examples/05_crawling_detail.py | 13 | # We're not processing detail pages yet, so we just pass. | COMMENT |
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_context.py | 10 | # to facilitate working with the parsed document. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_context.py | 9 | # Custom context for Selectolax parser, you can add your own methods here | COMMENT |
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_crawler.py | 20 | # Custom crawler using custom context, It is optional and you can use | COMMENT |
| LOW | …e2e/project_template/test_static_crawlers_templates.py | 15 | # To run these tests locally, make sure you have apify-cli installed and available in the path. | COMMENT |
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_context.py | 9 | # Custom context for Selectolax parser, you can add your own methods here | COMMENT |
| MEDIUM | …ides/code_examples/http_crawlers/selectolax_crawler.py | 20 | # Custom crawler using custom context, It is optional and you can use | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | website/docusaurus.config.js | 121 | }, | COMMENT |
| LOW | website/docusaurus.config.js | 141 | // to: '/docs/introduction', | COMMENT |
| LOW | website/sidebars.js | 81 | type: 'autogenerated', | COMMENT |
| LOW | website/src/components/Highlights.jsx | 61 | // or plug an adapter for your DB. Your URLs are <a href="https://crawlee.dev/docs/guides/request-storag | COMMENT |
| LOW | …emplate/{{cookiecutter.project_name}}/requirements.txt | 1 | # % if cookiecutter.crawler_type == 'playwright-camoufox' | COMMENT |
| LOW | …_template/{{cookiecutter.project_name}}/pyproject.toml | 1 | # % if cookiecutter.crawler_type.startswith('adaptive-') | COMMENT |
| LOW | …oject_name}}/{{cookiecutter.__package_name}}/routes.py | 1 | # % if cookiecutter.crawler_type.startswith('playwright') | COMMENT |
| LOW | src/crawlee/project_template/hooks/post_gen_project.py | 41 | COMMENT | |
| LOW | src/crawlee/project_template/templates/main.py | 1 | # % if cookiecutter.enable_apify_integration | COMMENT |
| LOW | src/crawlee/project_template/templates/main.py | 21 | http_client=HttpxHttpClient(), | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ples/code_examples/fill_and_submit_web_form_request.py | 15 | 'custname': 'John Doe', | CODE |
| LOW | …ples/code_examples/fill_and_submit_web_form_crawler.py | 25 | 'custname': 'John Doe', | CODE |
| LOW | …ples/code_examples/fill_and_submit_web_form_request.py | 15 | 'custname': 'John Doe', | CODE |
| LOW | …ples/code_examples/fill_and_submit_web_form_crawler.py | 25 | 'custname': 'John Doe', | CODE |
| LOW | tests/unit/crawlers/_http/test_http_crawler.py | 28 | 'custname': 'John Doe', | CODE |
| LOW | …ples/code_examples/fill_and_submit_web_form_request.py | 15 | 'custname': 'John Doe', | CODE |
| LOW | …ples/code_examples/fill_and_submit_web_form_crawler.py | 25 | 'custname': 'John Doe', | CODE |
| LOW | …crawlee/crawlers/_pydantic_ai/_clean_html_distiller.py | 35 | 'placeholder', | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/unit/crawlers/_basic/test_context_pipeline.py | 150 | CODE | |
| MEDIUM | …awlee/storage_clients/_base/_key_value_store_client.py | 79 | CODE | |
| MEDIUM | src/crawlee/storage_clients/_base/_dataset_client.py | 93 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/unit/server_endpoints.py | 127 | if (loading || page >= 3) return; | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …/code_examples/http_crawlers/selectolax_crawler_run.py | 12 | async def handle_request(context: SelectolaxLexborContext) -> None: | CODE |
| LOW | …code_examples/http_crawlers/selectolax_adaptive_run.py | 19 | async def handle_request(context: AdaptivePlaywrightCrawlingContext) -> None: | CODE |
| LOW | …/code_examples/http_crawlers/selectolax_crawler_run.py | 12 | async def handle_request(context: SelectolaxLexborContext) -> None: | CODE |
| LOW | …code_examples/http_crawlers/selectolax_adaptive_run.py | 19 | async def handle_request(context: AdaptivePlaywrightCrawlingContext) -> None: | CODE |