🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
This report presents the forensic synthetic code analysis of D4Vinci/Scrapling, a Python project with 69,441 GitHub stars. SynthScan v2.0 examined 44,664 lines of code across 211 source files, recording 875 pattern matches distributed across 14 syntactic categories. The overall adjusted score of 23.4 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 875 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/core/test_storage_core.py | 19 | def test_returns_default_when_url_is_none(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 23 | def test_returns_default_when_url_is_empty(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 27 | def test_returns_fld_for_valid_url(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 43 | def test_deterministic_output(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 48 | def test_different_input_different_output(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 53 | def test_strips_and_lowercases(self): | CODE |
| LOW⚡ | tests/core/test_storage_core.py | 58 | def test_includes_length_suffix(self): | CODE |
| LOW | tests/core/test_storage_core.py | 70 | def test_sqlite_storage_creation(self): | CODE |
| LOW | tests/core/test_storage_core.py | 75 | def test_sqlite_storage_with_file(self): | CODE |
| LOW | tests/core/test_storage_core.py | 91 | def test_sqlite_storage_initialization_args(self): | CODE |
| LOW | tests/core/test_storage_core.py | 125 | def test_retrieve_nonexistent_returns_none(self): | CODE |
| LOW | tests/core/test_storage_core.py | 129 | def test_save_overwrites_existing(self): | CODE |
| LOW | tests/core/test_storage_core.py | 167 | def test_element_path_is_stored(self): | CODE |
| LOW | tests/core/test_storage_core.py | 178 | def test_element_with_children_and_siblings(self): | CODE |
| LOW | tests/core/test_storage_core.py | 254 | def test_element_no_attributes(self): | CODE |
| LOW | tests/core/test_storage_core.py | 260 | def test_element_strips_whitespace_attributes(self): | CODE |
| LOW⚡ | tests/core/test_shell_core.py | 14 | def test_simple_cookie_parsing(self): | CODE |
| LOW⚡ | tests/core/test_shell_core.py | 21 | def test_multiple_cookies_parsing(self): | CODE |
| LOW⚡ | tests/core/test_shell_core.py | 31 | def test_cookie_with_attributes(self): | CODE |
| LOW | tests/core/test_shell_core.py | 43 | def test_malformed_cookie_handling(self): | CODE |
| LOW | tests/core/test_shell_core.py | 67 | def test_headers_with_cookies(self): | CODE |
| LOW | tests/core/test_shell_core.py | 80 | def test_headers_without_colons(self): | CODE |
| LOW | tests/core/test_shell_core.py | 92 | def test_invalid_header_format(self): | CODE |
| LOW | tests/core/test_shell_core.py | 102 | def test_headers_with_multiple_colons(self): | CODE |
| LOW | tests/core/test_shell_core.py | 113 | def test_headers_with_whitespace(self): | CODE |
| LOW | tests/core/test_shell_core.py | 125 | def test_parse_cookies_disabled(self): | CODE |
| LOW | tests/core/test_shell_core.py | 187 | def test_request_field_access(self): | CODE |
| LOW | tests/core/test_shell_core.py | 212 | def test_known_logging_levels(self): | CODE |
| LOW | tests/core/test_shell_core.py | 220 | def test_logging_level_values(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 147 | def test_browser_encoding_unquoted_charset(): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 154 | def test_browser_encoding_quoted_charset(): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 162 | def test_browser_encoding_defaults_when_missing(): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 343 | def test_continues_non_blocked_domain(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 349 | def test_resource_blocking_takes_priority_over_domain(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 356 | def test_domain_blocking_with_resources_disabled(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 363 | def test_no_blocking_continues(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 369 | def test_does_not_block_partial_domain_match(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 376 | def test_multiple_blocked_domains(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 464 | def test_ad_domains_is_frozenset(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 469 | def test_ad_domains_has_entries(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 474 | def test_ad_domains_contains_known_entries(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 484 | def test_block_ads_populates_blocked_domains(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 492 | def test_block_ads_false_leaves_blocked_domains_none(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 498 | def test_block_ads_merges_with_user_domains(self): | CODE |
| LOW⚡ | tests/fetchers/test_utils.py | 507 | def test_block_ads_does_not_modify_original_set(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 132 | def test_parsing_response_status(status_map): | CODE |
| LOW | tests/fetchers/test_utils.py | 179 | def test_proxy_string_with_auth(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 201 | def test_invalid_proxy_string(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 223 | def test_generate_headers_basic(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 231 | def test_generate_headers_browser_mode(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 260 | def test_response_with_bytes_content(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 319 | def test_blocks_disabled_resource_types(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 325 | def test_continues_allowed_resource_types(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 393 | async def test_blocks_disabled_resource_types(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 407 | async def test_continues_non_blocked(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 421 | async def test_does_not_block_partial_domain_match(self): | CODE |
| LOW | tests/fetchers/test_utils.py | 439 | def test_deep_subdomain_match(self): | CODE |
| LOW⚡ | tests/fetchers/test_impersonate_list.py | 14 | def test_select_random_browser_with_single_string(self): | CODE |
| LOW⚡ | tests/fetchers/test_impersonate_list.py | 19 | def test_select_random_browser_with_none(self): | CODE |
| LOW⚡ | tests/fetchers/test_impersonate_list.py | 24 | def test_select_random_browser_with_list(self): | CODE |
| 529 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/spiders/test_engine.py | 174 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 176 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 190 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 192 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 227 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 229 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 268 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 270 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 304 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 306 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 326 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 328 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 524 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 526 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 554 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 556 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 587 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 589 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 627 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 629 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 787 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 789 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 808 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 810 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 862 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_engine.py | 864 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_engine.py | 18 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_engine.py | 20 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_engine.py | 925 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_engine.py | 927 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_force_stop_checkpoint.py | 25 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_force_stop_checkpoint.py | 27 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_force_stop_checkpoint.py | 121 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_force_stop_checkpoint.py | 123 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 296 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 298 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 390 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 392 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 469 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 471 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 10 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 12 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 40 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 42 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 356 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_robotstxt.py | 358 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 68 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 70 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 151 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 153 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 227 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | tests/spiders/test_robotstxt.py | 229 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM | tests/spiders/test_cache.py | 129 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/spiders/test_cache.py | 131 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/fetchers/test_impersonate_list.py | 4 | CODE | |
| LOW | tests/fetchers/test_impersonate_list.py | 4 | CODE | |
| LOW | tests/fetchers/test_merge_request_args.py | 6 | CODE | |
| LOW | tests/spiders/test_result.py | 7 | CODE | |
| LOW | tests/spiders/test_engine.py | 6 | CODE | |
| LOW | tests/spiders/test_engine.py | 14 | CODE | |
| LOW | tests/spiders/test_request.py | 5 | CODE | |
| LOW | tests/spiders/test_session.py | 3 | CODE | |
| LOW | tests/spiders/test_session.py | 5 | CODE | |
| LOW | tests/spiders/test_spider.py | 12 | CODE | |
| LOW | tests/spiders/test_force_stop_checkpoint.py | 22 | CODE | |
| LOW | tests/spiders/test_links.py | 5 | CODE | |
| LOW | tests/spiders/test_checkpoint.py | 8 | CODE | |
| LOW | tests/parser/test_parser_advanced.py | 5 | CODE | |
| LOW | scrapling/__init__.py | 8 | CODE | |
| LOW | scrapling/__init__.py | 8 | CODE | |
| LOW | scrapling/__init__.py | 9 | CODE | |
| LOW | scrapling/__init__.py | 9 | CODE | |
| LOW | scrapling/__init__.py | 10 | CODE | |
| LOW | scrapling/__init__.py | 10 | CODE | |
| LOW | scrapling/__init__.py | 10 | CODE | |
| LOW | scrapling/__init__.py | 10 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 5 | CODE | |
| LOW | scrapling/core/_types.py | 34 | CODE | |
| LOW | scrapling/core/_types.py | 34 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/core/utils/__init__.py | 1 | CODE | |
| LOW | scrapling/fetchers/__init__.py | 2 | CODE | |
| LOW | scrapling/fetchers/__init__.py | 5 | CODE | |
| LOW | scrapling/fetchers/__init__.py | 5 | CODE | |
| LOW | scrapling/fetchers/__init__.py | 5 | CODE | |
| 36 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | cleanup.py | 29 | except Exception as e: | CODE |
| LOW | cleanup.py | 37 | except Exception as e: | CODE |
| LOW | tests/core/test_storage_core.py | 213 | except Exception as e: | CODE |
| MEDIUM | tests/core/test_storage_core.py | 207 | def save_element(idx): | CODE |
| LOW | scrapling/cli.py | 74 | except Exception as err: | CODE |
| LOW | scrapling/core/ai.py | 323 | except Exception as exc: | CODE |
| LOW | scrapling/core/shell.py | 172 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/core/shell.py | 204 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/core/shell.py | 218 | except Exception as e: | CODE |
| LOW | scrapling/core/shell.py | 235 | except Exception as e: | CODE |
| LOW | scrapling/core/shell.py | 322 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/core/shell.py | 364 | except Exception: # pragma: no cover | CODE |
| LOW | scrapling/core/shell.py | 381 | except Exception as e: | CODE |
| LOW | scrapling/core/shell.py | 567 | except Exception as e: | CODE |
| LOW | scrapling/core/utils/_shell.py | 41 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/spiders/robotstxt.py | 31 | except Exception as e: | CODE |
| LOW | scrapling/spiders/robotstxt.py | 36 | except Exception as e: | CODE |
| LOW | scrapling/spiders/checkpoint.py | 56 | except Exception as e: | CODE |
| LOW | scrapling/spiders/checkpoint.py | 79 | except Exception as e: | CODE |
| LOW | scrapling/spiders/checkpoint.py | 89 | except Exception as e: | CODE |
| LOW | scrapling/spiders/cache.py | 42 | except Exception as e: | CODE |
| LOW | scrapling/spiders/cache.py | 68 | except Exception as e: | CODE |
| LOW | scrapling/spiders/engine.py | 167 | except Exception as e: | CODE |
| LOW | scrapling/spiders/engine.py | 210 | except Exception as e: | CODE |
| LOW | scrapling/spiders/spider.py | 140 | except Exception as e: | CODE |
| LOW | scrapling/spiders/templates/sitemap.py | 88 | except Exception as e: | CODE |
| LOW | scrapling/engines/static.py | 727 | except Exception: | CODE |
| LOW | scrapling/engines/static.py | 753 | except Exception: | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 71 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 76 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 128 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 188 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 193 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/toolbelt/convertor.py | 276 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 94 | except Exception: | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 160 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 173 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 181 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 196 | except Exception as e: | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 282 | except Exception: | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 349 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 362 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 370 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_controllers.py | 385 | except Exception as e: | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 99 | except Exception: | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 243 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 261 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 269 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 284 | except Exception as e: | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 374 | except Exception: | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 519 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 537 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 545 | except Exception as e: # pragma: no cover | CODE |
| LOW | scrapling/engines/_browsers/_stealth.py | 560 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | cleanup.py | 6 | CODE | |
| LOW | scrapling/parser.py | 519 | CODE | |
| LOW | scrapling/parser.py | 628 | CODE | |
| LOW | scrapling/parser.py | 698 | CODE | |
| LOW | scrapling/parser.py | 1096 | CODE | |
| LOW | scrapling/core/mixins.py | 15 | CODE | |
| LOW | scrapling/core/shell.py | 334 | CODE | |
| LOW | scrapling/core/shell.py | 150 | CODE | |
| LOW | scrapling/core/shell.py | 617 | CODE | |
| LOW | scrapling/core/shell.py | 658 | CODE | |
| LOW | scrapling/core/utils/_shell.py | 19 | CODE | |
| LOW | scrapling/spiders/request.py | 71 | CODE | |
| LOW | scrapling/spiders/links.py | 226 | CODE | |
| LOW | scrapling/spiders/engine.py | 141 | CODE | |
| LOW | scrapling/spiders/engine.py | 325 | CODE | |
| LOW | scrapling/spiders/spider.py | 34 | CODE | |
| LOW | scrapling/spiders/spider.py | 255 | CODE | |
| LOW | scrapling/spiders/spider.py | 297 | CODE | |
| LOW | scrapling/spiders/templates/sitemap.py | 94 | CODE | |
| LOW | scrapling/spiders/templates/sitemap.py | 105 | CODE | |
| LOW | scrapling/spiders/templates/sitemap.py | 125 | CODE | |
| LOW | scrapling/engines/static.py | 224 | CODE | |
| LOW | scrapling/engines/static.py | 439 | CODE | |
| LOW | scrapling/engines/_browsers/_controllers.py | 72 | CODE | |
| LOW | scrapling/engines/_browsers/_controllers.py | 102 | CODE | |
| LOW | scrapling/engines/_browsers/_controllers.py | 261 | CODE | |
| LOW | scrapling/engines/_browsers/_controllers.py | 290 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 76 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 107 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 184 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 352 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 382 | CODE | |
| LOW | scrapling/engines/_browsers/_stealth.py | 459 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/fetchers/async/test_dynamic.py | 0 | test doing a basic fetch request with multiple statuses | STRING |
| HIGH | tests/fetchers/async/test_stealth.py | 0 | test doing a basic fetch request with multiple statuses | STRING |
| HIGH | tests/fetchers/sync/test_dynamic.py | 0 | test doing a basic fetch request with multiple statuses | STRING |
| HIGH | scrapling/fetchers/stealth_chrome.py | 0 | a browser session manager with page pooling, it's using a persistent browser context by default with a temporary user pr | STRING |
| HIGH | scrapling/engines/_browsers/_controllers.py | 0 | a browser session manager with page pooling, it's using a persistent browser context by default with a temporary user pr | STRING |
| HIGH | scrapling/engines/_browsers/_stealth.py | 0 | a browser session manager with page pooling, it's using a persistent browser context by default with a temporary user pr | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …-skill/Scrapling-Skill/references/fetching/stealthy.md | 164 | This is for educational purposes only; this example was generated by AI, which also shows how easy it is to work with Sc | CODE |
| HIGH | tests/fetchers/test_utils.py | 17 | # A map generated by ChatGPT for most possible `content_type` values and the expected outcome | COMMENT |
| HIGH | docs/overview.md | 17 | Here's an HTML document generated by ChatGPT that we will be using as an example throughout this page: | CODE |
| HIGH | docs/development/adaptive_storage_system.md | 26 | Here's a more practical example generated by AI using Redis: | CODE |
| HIGH | docs/fetching/stealthy.md | 176 | This is for educational purposes only; this example was generated by AI, which also shows how easy it is to work with Sc | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapling/__init__.py | 24 | __all__ = ["Selector", "Fetcher", "AsyncFetcher", "StealthyFetcher", "DynamicFetcher"] | CODE |
| LOW | scrapling/core/storage.py | 97 | def _setup_database(self) -> None: | CODE |
| LOW | scrapling/fetchers/__init__.py | 23 | __all__ = [ | CODE |
| LOW | scrapling/fetchers/requests.py | 10 | __all__ = ["Fetcher", "AsyncFetcher", "FetcherSession"] | CODE |
| LOW | scrapling/spiders/__init__.py | 11 | __all__ = [ | CODE |
| LOW | scrapling/spiders/links.py | 27 | __all__ = ["LinkExtractor"] | CODE |
| LOW | scrapling/spiders/spider.py | 232 | def _setup_signal_handler(self) -> None: | CODE |
| LOW | scrapling/spiders/templates/shopify.py | 22 | __all__ = ["ShopifySpider"] | CODE |
| LOW | scrapling/spiders/templates/sitemap.py | 29 | __all__ = ["SitemapSpider"] | CODE |
| LOW | scrapling/spiders/templates/__init__.py | 5 | __all__ = [ | CODE |
| LOW | scrapling/spiders/templates/crawler.py | 23 | __all__ = ["CrawlRule", "CrawlSpider"] | CODE |
| LOW | scrapling/integrations/scrapy.py | 21 | __all__ = ["scrapling_response", "convert_response"] | CODE |
| LOW | scrapling/engines/toolbelt/__init__.py | 3 | __all__ = ["ProxyRotator", "is_proxy_error", "cyclic_rotation"] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/spiders/test_engine.py | 74 | CODE | |
| LOW | scrapling/parser.py | 80 | CODE | |
| LOW | scrapling/core/ai.py | 138 | CODE | |
| LOW | scrapling/core/ai.py | 281 | CODE | |
| LOW | scrapling/core/ai.py | 345 | CODE | |
| LOW | scrapling/core/ai.py | 419 | CODE | |
| LOW | scrapling/core/ai.py | 497 | CODE | |
| LOW | scrapling/core/ai.py | 582 | CODE | |
| LOW | scrapling/core/ai.py | 686 | CODE | |
| LOW | scrapling/core/ai.py | 786 | CODE | |
| LOW | scrapling/spiders/links.py | 194 | CODE | |
| LOW | scrapling/engines/static.py | 659 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/parser/test_attributes_handler.py | 279 | # Create an element with many attributes | COMMENT |
| MEDIUM | scrapling/core/shell.py | 55 | # Define the structure for parsed context - Simplified for Fetcher args | COMMENT |
| MEDIUM | scrapling/core/shell.py | 517 | # Create the namespace dictionary | COMMENT |
| MEDIUM | scrapling/engines/toolbelt/custom.py | 29 | """This class is returned by all engines as a way to unify the response type between different libraries. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/spiders/test_engine.py | 473 | CODE | |
| MEDIUM | scrapling/spiders/spider.py | 169 | CODE | |
| MEDIUM | scrapling/spiders/templates/sitemap.py | 81 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapling/core/shell.py | 239 | # Check if raw data looks like JSON, prefer 'json' param if so | COMMENT |
| LOW | scrapling/core/shell.py | 344 | # Check if this function has known parameters | COMMENT |
| LOW | scrapling/engines/_browsers/_base.py | 572 | # Check if turnstile captcha is embedded inside the page (Usually inside a closed Shadow iframe) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | agent-skill/README.md | 3 | The skill aligns with the [AgentSkill](https://agentskills.io/specification) specification, so it will be readable by [O | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …nt-skill/Scrapling-Skill/references/fetching/static.md | 350 | 'username': 'user@example.com', | CODE |
| LOW | docs/fetching/static.md | 357 | 'username': 'user@example.com', | CODE |