Scrapy, a fast high-level web crawling & scraping framework for Python.
This report summarizes scanner evidence for scrapy/scrapy, a Python project with 64,711 GitHub stars. SynthScan examined 106,238 lines of code across 567 source files, recording 2459 pattern matches distributed across 18 syntactic categories. The legacy adjusted pattern signal is 27.8 (Elevated pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 1694 finding(s).
Are imports declared and package/API names verified? 482 finding(s).
Are wrappers, guards, and repeated structures necessary? 1901 finding(s).
Which changes need focused human review? 2459 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 2459 distinct pattern matches across 18 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | conftest.py | 181 | def pytest_collection_modifyitems(items): | CODE |
| LOW | tests/test_request_dict.py | 114 | def test_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW | tests/test_request_dict.py | 123 | def test_reference_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW | tests/test_request_dict.py | 135 | def test_private_reference_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW⚡ | tests/test_request_dict.py | 147 | def test_private_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW⚡ | tests/test_request_dict.py | 156 | def test_mixin_private_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW⚡ | tests/test_request_dict.py | 165 | def test_delegated_callback_serialization(spider: MethodsSpider) -> None: | CODE |
| LOW | tests/test_request_dict.py | 204 | def test_callback_not_available() -> None: | CODE |
| LOW | tests/test_utils_signal.py | 57 | async def test_send_catch_log_dont_log( | CODE |
| LOW | tests/test_utils_signal.py | 162 | def test_error_logged_if_deferred_not_supported( | CODE |
| LOW | tests/test_http_response_text.py | 179 | def test_declared_encoding_invalid(self): | CODE |
| LOW | tests/test_http_response_text.py | 189 | def test_body_inferred_encoding_of_undeclared_legacy_page(self): | CODE |
| LOW | tests/test_http_response_text.py | 218 | def test_bom_is_removed_from_body(self): | CODE |
| LOW | tests/test_http_response_text.py | 245 | def test_replace_wrong_encoding(self): | CODE |
| LOW | tests/test_http_response_text.py | 288 | def test_selector_response_refcounting(self): | CODE |
| LOW | tests/test_http_response_text.py | 313 | def test_selector_shortcuts_kwargs(self): | CODE |
| LOW | tests/test_http_response_text.py | 334 | def test_urljoin_with_base_url(self): | CODE |
| LOW⚡ | tests/test_http_response_text.py | 393 | def test_follow_selector_list(self): | CODE |
| LOW⚡ | tests/test_http_response_text.py | 398 | def test_follow_selector_invalid(self): | CODE |
| LOW⚡ | tests/test_http_response_text.py | 403 | def test_follow_selector_attribute(self): | CODE |
| LOW⚡ | tests/test_http_response_text.py | 408 | def test_follow_selector_no_href(self): | CODE |
| LOW⚡ | tests/test_http_response_text.py | 416 | def test_follow_whitespace_selector(self): | CODE |
| LOW | tests/test_http_response_text.py | 466 | def test_follow_all_css_skip_invalid(self): | CODE |
| LOW | tests/test_http_response_text.py | 487 | def test_follow_all_xpath_skip_invalid(self): | CODE |
| LOW | tests/test_http_response_text.py | 504 | def test_follow_all_too_many_arguments(self): | CODE |
| LOW | tests/test_http_response_text.py | 534 | def test_json_response_wrong_charset(self): | CODE |
| LOW | tests/test_http_response_text.py | 647 | def test_selector_shortcuts_kwargs(self): | CODE |
| LOW⚡ | tests/test_contracts.py | 697 | def test_unregistered_at_line_is_not_a_tested_method(self): | CODE |
| LOW⚡ | tests/test_contracts.py | 705 | def test_unregistered_at_line_is_skipped(self): | CODE |
| LOW⚡ | tests/test_contracts.py | 712 | def test_unregistered_at_line_does_not_break_checks(self): | CODE |
| LOW | tests/test_contracts.py | 114 | def returns_request_cb_kwargs(self, response, url): | CODE |
| LOW | tests/test_contracts.py | 130 | def returns_item_cb_kwargs_error_unexpected_keyword(self, response): | CODE |
| LOW | tests/test_contracts.py | 138 | def returns_item_cb_kwargs_error_missing_argument(self, response, arg): | CODE |
| LOW | tests/test_contracts.py | 159 | def returns_request_range_fail(self, response): | CODE |
| LOW | tests/test_contracts.py | 242 | def scrapes_multiple_missing_fields(self, response): | CODE |
| LOW | tests/test_contracts.py | 266 | def invalid_regex_with_valid_contract(self, response): | CODE |
| LOW | tests/test_contracts.py | 290 | def returns_error_missing_meta(self, response): | CODE |
| LOW | tests/test_contracts.py | 566 | def test_returns_async_gen_sync(self): | CODE |
| LOW | tests/test_contracts.py | 583 | def test_returns_invalid_argument_count(self): | CODE |
| LOW | tests/test_contracts.py | 590 | def test_returns_default_bounds(self): | CODE |
| LOW | tests/test_contracts.py | 605 | def test_returns_and_scrapes_ignore_other_types(self): | CODE |
| LOW | tests/test_contracts.py | 615 | def test_multiple_urls_and_batches(self): | CODE |
| LOW | tests/test_contracts.py | 802 | def test_custom_tagged_request_contract(self): | CODE |
| LOW | tests/test_contracts.py | 874 | def test_pre_hook_keyboard_interrupt(self): | CODE |
| LOW | tests/test_contracts.py | 888 | def test_post_hook_keyboard_interrupt(self): | CODE |
| LOW | tests/test_contracts.py | 915 | def test_pre_hook_assertion_failure(self): | CODE |
| LOW | tests/test_contracts.py | 941 | async def test_pre_hook_async_callback(self): | CODE |
| LOW | tests/test_contracts.py | 955 | async def test_pre_hook_async_generator(self): | CODE |
| LOW | tests/test_contracts.py | 969 | async def test_post_hook_async_generator(self): | CODE |
| LOW | tests/test_contracts.py | 1002 | def test_hook_override_deprecation(): | CODE |
| LOW | tests/test_contracts.py | 1007 | def test_no_hook_override_deprecation(recwarn): | CODE |
| LOW | tests/test_command_parse.py | 116 | def parse_request_with_cb_kwargs(self, response, foo=None, key=None): | CODE |
| LOW | tests/test_command_parse.py | 122 | def parse_request_without_meta(self, response): | CODE |
| LOW | tests/test_command_parse.py | 518 | def test_crawlspider_matching_rule_default_callback( | CODE |
| LOW | tests/test_command_parse.py | 532 | def test_spider_with_no_rules_attribute( | CODE |
| LOW | tests/test_command_parse.py | 547 | def test_crawlspider_missing_callback( | CODE |
| LOW | tests/test_command_parse.py | 560 | def test_crawlspider_no_matching_rule( | CODE |
| LOW | tests/test_command_parse.py | 575 | def test_crawlspider_not_exists_with_not_matched_url( | CODE |
| LOW | tests/test_command_parse.py | 307 | def test_request_with_cb_kwargs( | STRING |
| LOW | tests/test_command_parse.py | 329 | def test_request_without_meta( | STRING |
| 1547 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | conftest.py | 1 | CODE | |
| LOW | tests/test_request_dict.py | 1 | CODE | |
| LOW | tests/test_utils_signal.py | 1 | CODE | |
| LOW | tests/test_http_response_text.py | 1 | CODE | |
| LOW | tests/test_contracts.py | 1 | CODE | |
| LOW | tests/test_command_parse.py | 1 | CODE | |
| LOW | tests/test_logstats.py | 1 | CODE | |
| LOW | tests/test_utils_template.py | 1 | CODE | |
| LOW | tests/test_command_genspider.py | 1 | CODE | |
| LOW | tests/test_extension_remote_control.py | 1 | CODE | |
| LOW | tests/test_utils_datatypes.py | 1 | CODE | |
| LOW | tests/test_engine.py | 1 | CODE | |
| LOW | tests/test_selector.py | 1 | CODE | |
| LOW | tests/test_zz_resources.py | 3 | CODE | |
| LOW | tests/test_spidermiddleware.py | 1 | CODE | |
| LOW | tests/test_utils_benchserver.py | 1 | CODE | |
| LOW | tests/test_utils_benchserver.py | 9 | CODE | |
| LOW | tests/test_scrapy__getattr__.py | 1 | CODE | |
| LOW | tests/test_extension_debug.py | 1 | CODE | |
| LOW | tests/test_downloader_handler_twisted_http11.py | 1 | CODE | |
| LOW | tests/test_downloader_handler_twisted_http11.py | 50 | CODE | |
| LOW | tests/test_stats.py | 1 | CODE | |
| LOW | tests/test_linkextractors.py | 1 | CODE | |
| LOW | tests/test_command_runspider.py | 1 | CODE | |
| LOW | tests/test_engine_close_spider.py | 1 | CODE | |
| LOW | tests/test_engine_close_spider.py | 18 | CODE | |
| LOW | tests/test_loader.py | 1 | CODE | |
| LOW | tests/test_middleware.py | 1 | CODE | |
| LOW | tests/test_http_request_json.py | 1 | CODE | |
| LOW | tests/test_http2_client_protocol.py | 1 | CODE | |
| LOW | tests/test_utils_iterators.py | 1 | CODE | |
| LOW | tests/test_dependencies.py | 1 | CODE | |
| LOW | tests/test_feedexport_async.py | 3 | CODE | |
| LOW | tests/test_poet.py | 3 | CODE | |
| LOW | tests/test_signals.py | 1 | CODE | |
| LOW | tests/test_utils_log.py | 1 | CODE | |
| LOW | tests/test_link.py | 1 | CODE | |
| LOW | tests/test_spider_sitemap.py | 1 | CODE | |
| LOW | tests/test_spidermiddleware_urllength.py | 1 | CODE | |
| LOW | tests/test_downloader_handler_twisted_http2.py | 3 | CODE | |
| LOW | tests/test_feedexport_postprocess.py | 1 | CODE | |
| LOW | tests/test_feedexport_uri_params.py | 1 | CODE | |
| LOW | tests/test_spidermiddleware_referer.py | 1 | CODE | |
| LOW | tests/test_utils_asyncgen.py | 1 | CODE | |
| LOW | tests/test_downloader_handler_aiohttp.py | 1 | CODE | |
| LOW | tests/spiders.py | 5 | CODE | |
| LOW | tests/test_downloadermiddleware_stats.py | 1 | CODE | |
| LOW | tests/test_utils_project.py | 1 | CODE | |
| LOW | tests/test_downloadermiddleware_robotstxt.py | 1 | CODE | |
| LOW | tests/test_crawler_subprocess.py | 1 | CODE | |
| LOW | tests/test_robotstxt_interface.py | 1 | CODE | |
| LOW | tests/test_command_genrequest.py | 1 | CODE | |
| LOW | tests/test_dupefilters.py | 1 | CODE | |
| LOW | tests/test_engine_stop_download_bytes.py | 1 | CODE | |
| LOW | tests/test_http_request_form.py | 1 | CODE | |
| LOW | tests/test_utils_trackref.py | 1 | CODE | |
| LOW | tests/test_pipelines.py | 1 | CODE | |
| LOW | tests/test_core_downloader.py | 1 | CODE | |
| LOW | tests/test_core_downloader.py | 59 | CODE | |
| LOW | tests/test_core_downloader.py | 61 | CODE | |
| 352 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_squeues_request.py | 54 | q.push(req) | CODE |
| HIGH | tests/test_squeues_request.py | 71 | q.push(req) | CODE |
| HIGH⚡ | tests/test_squeues_request.py | 83 | q.push(req1) | CODE |
| HIGH⚡ | tests/test_squeues_request.py | 84 | q.push(req2) | CODE |
| HIGH⚡ | tests/test_squeues_request.py | 85 | q.push(req3) | CODE |
| HIGH | tests/test_squeues_request.py | 242 | queue.push(Request("https://toscrape.com")) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 65 | queue.push(req1) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 66 | queue.push(req2) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 67 | queue.push(req3) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 114 | queue.push(req1) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 115 | queue.push(req2) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 116 | queue.push(req3) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 138 | queue.push(Request("https://example.org/1", meta={"is_start_request": True})) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 139 | queue.push(Request("https://example.org/2", priority=1)) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 175 | queue.push(req1) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 176 | queue.push(req2) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 177 | queue.push(req3) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 192 | queue.push(req1) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 193 | queue.push(req2) | CODE |
| HIGH⚡ | tests/test_pqueues.py | 194 | queue.push(req3) | CODE |
| HIGH | tests/test_pqueues.py | 48 | queue.push(req1) | CODE |
| HIGH | tests/test_pqueues.py | 90 | queue.push(req) | CODE |
| HIGH | tests/test_pqueues.py | 221 | queue.push(request) | CODE |
| HIGH | tests/test_pqueues.py | 247 | queue.push(request) | CODE |
| HIGH | tests/test_pqueues.py | 269 | queue.push(req) | CODE |
| HIGH | tests/test_pqueues.py | 281 | queue.push(req) | CODE |
| HIGH | tests/test_pqueues.py | 299 | queue.push(request) | CODE |
| HIGH | tests/test_pqueues.py | 305 | queue.push(request) | CODE |
| HIGH | tests/test_pqueues.py | 354 | queue.push(request) | CODE |
| HIGH | tests/test_scheduler.py | 432 | self._queue.push(request) | CODE |
| HIGH⚡ | tests/test_squeues.py | 77 | q.push(lambda x: x) | CODE |
| HIGH⚡ | tests/test_squeues.py | 83 | q.push(sel) | CODE |
| HIGH⚡ | tests/test_squeues.py | 93 | q.push("a") | CODE |
| HIGH⚡ | tests/test_squeues.py | 94 | q.push(123) | CODE |
| HIGH⚡ | tests/test_squeues.py | 95 | q.push({"a": "dict"}) | CODE |
| HIGH⚡ | tests/test_squeues.py | 136 | q.push(i) | CODE |
| HIGH⚡ | tests/test_squeues.py | 145 | q.push(loader) | CODE |
| HIGH⚡ | tests/test_squeues.py | 156 | q.push(r) | CODE |
| HIGH | tests/test_squeues.py | 169 | q.push(lambda x: x) | CODE |
| HIGH | tests/test_squeues.py | 178 | q.push(sel) | CODE |
| HIGH⚡ | tests/test_squeues.py | 205 | q.push("a") | CODE |
| HIGH⚡ | tests/test_squeues.py | 206 | q.push(123) | CODE |
| HIGH⚡ | tests/test_squeues.py | 207 | q.push({"a": "dict"}) | CODE |
| HIGH⚡ | tests/test_squeues.py | 228 | q.push(i) | CODE |
| HIGH⚡ | tests/test_squeues.py | 237 | q.push(loader) | CODE |
| HIGH⚡ | tests/test_squeues.py | 248 | q.push(r) | CODE |
| HIGH | tests/test_settings/__init__.py | 967 | {"FOO": '{"tests.test_settings.Component1": null}'}, | CODE |
| HIGH | scrapy/pqueues.py | 232 | q.push(request) # this may fail (eg. serialization error) | CODE |
| HIGH | scrapy/pqueues.py | 487 | queue.push(request) | CODE |
| HIGH | scrapy/core/scheduler.py | 450 | self.dqs.push(request) | STRING |
| HIGH | scrapy/core/scheduler.py | 473 | self.mqs.push(request) | STRING |
| HIGH | scrapy/extensions/remote_control.py | 68 | - ``start_time`` (number or null): the start time of the Scrapy process | STRING |
| HIGH | scrapy/extensions/remote_control.py | 87 | - ``traceback`` (string or null): the traceback if an exception was | STRING |
| HIGH | scrapy/spidermiddlewares/referer.py | 79 | If url is null, return no referrer. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_crawler_runners.py | 119 | except ValueError: | CODE |
| HIGH | tests/test_crawler_runners.py | 132 | except ValueError: | CODE |
| HIGH | tests/CrawlerProcess/asyncio_enabled_reactor.py | 17 | except RuntimeError: | CODE |
| HIGH | tests/CrawlerProcess/asyncio_enabled_reactor.py | 24 | except RuntimeError: | CODE |
| HIGH | tests/AsyncCrawlerProcess/asyncio_enabled_reactor.py | 16 | except RuntimeError: | CODE |
| HIGH | scrapy/pqueues.py | 240 | except KeyError: | CODE |
| HIGH | scrapy/cmdline.py | 179 | except KeyError: | CODE |
| HIGH | scrapy/exporters.py | 320 | except TypeError: # list in value may not contain strings | CODE |
| HIGH | scrapy/crawler.py | 1204 | except asyncio.CancelledError: | CODE |
| HIGH | scrapy/core/downloader/handlers/_httpx.py | 61 | except ImportError: # pragma: no cover | CODE |
| HIGH | scrapy/core/downloader/handlers/_httpx.py | 69 | except ImportError: # pragma: no cover | CODE |
| HIGH | scrapy/core/_http2/protocol.py | 413 | except KeyError: | CODE |
| HIGH | scrapy/core/_http2/protocol.py | 421 | except KeyError: | CODE |
| HIGH | scrapy/core/_http2/protocol.py | 440 | except KeyError: | CODE |
| HIGH | scrapy/core/_http2/protocol.py | 448 | except KeyError: | CODE |
| HIGH | scrapy/utils/reactor.py | 44 | except error.CannotListenError: | CODE |
| HIGH | scrapy/utils/_shell.py | 85 | except ImportError: | CODE |
| HIGH | scrapy/utils/_shell.py | 143 | except SystemExit: # raised when using exit() in python code.interact | CODE |
| HIGH | scrapy/utils/_shell.py | 43 | except RuntimeError: | CODE |
| HIGH | scrapy/utils/log.py | 215 | except ImportError: | CODE |
| HIGH | scrapy/utils/_datatypes.py | 35 | except KeyError: | CODE |
| HIGH | scrapy/extensions/debug.py | 36 | except AttributeError: | CODE |
| HIGH | scrapy/extensions/httpcache.py | 156 | except ValueError: | CODE |
| HIGH | scrapy/extensions/httpcache.py | 242 | except ValueError: | CODE |
| HIGH | scrapy/http/cookies.py | 122 | except ValueError: | CODE |
| HIGH | scrapy/http/request/form.py | 307 | except IndexError: | CODE |
| HIGH | scrapy/spidermiddlewares/referer.py | 418 | except ValueError: | CODE |
| HIGH | scrapy/commands/genspider.py | 207 | except KeyError: | CODE |
| HIGH | scrapy/commands/__init__.py | 192 | except ImportError: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_engine.py | 310 | CODE | |
| MEDIUM | tests/test_spidermiddleware.py | 81 | CODE | |
| MEDIUM | tests/spiders.py | 698 | CODE | |
| MEDIUM | tests/test_utils_python.py | 42 | CODE | |
| MEDIUM | tests/test_crawler.py | 247 | CODE | |
| MEDIUM | tests/test_crawler.py | 325 | CODE | |
| MEDIUM | tests/test_crawler.py | 403 | CODE | |
| MEDIUM | tests/test_crawler.py | 481 | CODE | |
| MEDIUM | tests/test_crawler.py | 595 | CODE | |
| MEDIUM | tests/test_engine_loop.py | 370 | CODE | |
| MEDIUM | tests/test_engine_loop.py | 412 | CODE | |
| MEDIUM | tests/test_engine_loop.py | 472 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/reactorless_simple.py | 15 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/custom_loop_same.py | 20 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/reactorless_custom_settings.py | 18 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/custom_loop_different.py | 20 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/multi_parallel.py | 15 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/reactorless_reactor.py | 14 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/no_reactor.py | 13 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/simple_default_reactor.py | 14 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/simple.py | 17 | CODE | |
| MEDIUM | tests/AsyncCrawlerRunner/multi_seq.py | 15 | CODE | |
| MEDIUM | tests/CrawlerProcess/asyncio_enabled_no_reactor.py | 17 | CODE | |
| MEDIUM | tests/CrawlerProcess/args_settings.py | 19 | CODE | |
| MEDIUM | tests/CrawlerProcess/reactor_select.py | 15 | CODE | |
| MEDIUM | tests/CrawlerProcess/multi.py | 10 | CODE | |
| MEDIUM | tests/CrawlerProcess/asyncio_enabled_reactor.py | 53 | CODE | |
| MEDIUM | …ts/CrawlerProcess/asyncio_enabled_reactor_same_loop.py | 22 | CODE | |
| MEDIUM | tests/CrawlerProcess/reactor_settings.py | 24 | CODE | |
| MEDIUM | …awlerProcess/reactor_default_twisted_reactor_select.py | 13 | CODE | |
| MEDIUM | …awlerProcess/asyncio_enabled_reactor_different_loop.py | 20 | CODE | |
| MEDIUM | …cess/reactor_select_subclass_twisted_reactor_select.py | 22 | CODE | |
| MEDIUM | tests/CrawlerProcess/dns_resolver_deprecated.py | 13 | CODE | |
| MEDIUM | tests/CrawlerProcess/reactor_default.py | 13 | CODE | |
| MEDIUM | tests/CrawlerProcess/simple.py | 12 | CODE | |
| MEDIUM | …rawlerProcess/reactor_select_twisted_reactor_select.py | 14 | CODE | |
| MEDIUM | tests/CrawlerProcess/asyncio_custom_loop.py | 10 | CODE | |
| MEDIUM | tests/CrawlerRunner/explicit_default_reactor.py | 17 | CODE | |
| MEDIUM | tests/CrawlerRunner/custom_loop_same.py | 19 | CODE | |
| MEDIUM | tests/CrawlerRunner/change_reactor.py | 15 | CODE | |
| MEDIUM | tests/CrawlerRunner/custom_loop_different.py | 19 | CODE | |
| MEDIUM | tests/CrawlerRunner/multi_parallel.py | 14 | CODE | |
| MEDIUM | tests/CrawlerRunner/no_reactor.py | 13 | CODE | |
| MEDIUM | tests/CrawlerRunner/simple.py | 16 | CODE | |
| MEDIUM | tests/CrawlerRunner/multi_seq.py | 15 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/asyncio_enabled_no_reactor.py | 17 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/reactorless_simple.py | 12 | CODE | |
| MEDIUM | …erProcess/asyncio_enabled_reactor_same_loop_default.py | 21 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/args_settings.py | 19 | CODE | |
| MEDIUM | …syncCrawlerProcess/reactorless_import_hook_multiple.py | 21 | CODE | |
| MEDIUM | …sts/AsyncCrawlerProcess/reactorless_custom_settings.py | 23 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/multi.py | 10 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/asyncio_enabled_reactor.py | 43 | CODE | |
| MEDIUM | …yncCrawlerProcess/asyncio_enabled_reactor_same_loop.py | 21 | CODE | |
| MEDIUM | tests/AsyncCrawlerProcess/reactor_settings.py | 24 | CODE | |
| MEDIUM | …ncCrawlerProcess/reactorless_telnetconsole_disabled.py | 10 | CODE | |
| MEDIUM | …ocess/asyncio_custom_loop_custom_settings_different.py | 13 | CODE | |
| MEDIUM | …lerProcess/asyncio_custom_loop_custom_settings_same.py | 13 | CODE | |
| MEDIUM | …awlerProcess/asyncio_enabled_reactor_different_loop.py | 19 | CODE | |
| MEDIUM | …yncCrawlerProcess/reactorless_telnetconsole_enabled.py | 10 | CODE | |
| 8 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_scheduler.py | 131 | def _setup(scheduler: Scheduler) -> None: | CODE |
| LOW | tests/test_scheduler.py | 144 | def _setup(scheduler: Scheduler) -> None: | CODE |
| LOW | tests/test_scheduler.py | 159 | def _setup(scheduler: Scheduler) -> None: | CODE |
| LOW | tests/test_scheduler.py | 300 | def _setup(scheduler: Scheduler) -> None: | CODE |
| LOW | tests/test_extension_periodic_log.py | 71 | def set_a(self) -> None: | CODE |
| LOW | tests/test_extension_periodic_log.py | 74 | def set_b(self) -> None: | CODE |
| LOW | tests/test_commands.py | 220 | logger = logging.getLogger(__name__) | CODE |
| LOW | tests/test_feedexport_mode.py | 54 | def _setup(self, mockserver: MockServer, tmp_path: Path) -> None: | CODE |
| LOW | tests/test_crawler.py | 859 | def update_settings(self, settings: Settings) -> None: | CODE |
| LOW | …syncCrawlerProcess/reactorless_import_hook_multiple.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | …yncCrawlerProcess/reactorless_import_hook_uninstall.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/mail.py | 38 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/robotstxt.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/addons.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/pqueues.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/pqueues.py | 267 | def _update_curprio(self) -> None: | CODE |
| LOW | scrapy/__init__.py | 15 | __all__ = [ | CODE |
| LOW | scrapy/statscollectors.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/statscollectors.py | 64 | def set_value(self, key: str, value: Any, spider: Spider | None = None) -> None: | CODE |
| LOW | scrapy/statscollectors.py | 68 | def set_stats(self, stats: StatsT, spider: Spider | None = None) -> None: | CODE |
| LOW | scrapy/statscollectors.py | 172 | def set_value(self, key: str, value: Any, spider: Spider | None = None) -> None: | CODE |
| LOW | scrapy/statscollectors.py | 175 | def set_stats(self, stats: StatsT, spider: Spider | None = None) -> None: | CODE |
| LOW | scrapy/exporters.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/exporters.py | 31 | __all__ = [ | CODE |
| LOW | scrapy/middleware.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/middleware.py | 67 | def _set_compat_spider(self, spider: Spider | None) -> None: | CODE |
| LOW | scrapy/crawler.py | 59 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/crawler.py | 172 | def _update_logging(self) -> None: | CODE |
| LOW | scrapy/crawler.py | 873 | def _setup_reactor(self, install_signal_handlers: bool) -> None: | CODE |
| LOW | scrapy/linkextractors/lxmlhtml.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/linkextractors/__init__.py | 141 | __all__ = [ | CODE |
| LOW | scrapy/downloadermiddlewares/cookies.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/downloadermiddlewares/robotstxt.py | 30 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/downloadermiddlewares/redirect.py | 24 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/downloadermiddlewares/offsite.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/downloadermiddlewares/offsite.py | 70 | def _update_host_regex(self, spider: Spider) -> None: | CODE |
| LOW | scrapy/downloadermiddlewares/httpcache.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/settings/default_settings.py | 21 | __all__ = [ | CODE |
| LOW | scrapy/selector/unified.py | 14 | __all__ = ["Selector", "SelectorList"] | CODE |
| LOW | scrapy/selector/__init__.py | 8 | __all__ = [ | CODE |
| LOW | scrapy/core/scraper.py | 54 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/spidermw.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/engine.py | 64 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/scheduler.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/downloader/tls.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/downloader/handlers/_base_streaming.py | 50 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/downloader/handlers/__init__.py | 30 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/downloader/handlers/http11.py | 77 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/downloader/handlers/http.py | 17 | __all__ = [ | CODE |
| LOW | scrapy/core/_http2/protocol.py | 52 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/core/_http2/stream.py | 35 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/spiders/sitemap.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/spiders/__init__.py | 81 | def _set_crawler(self, crawler: Crawler) -> None: | CODE |
| LOW | scrapy/spiders/__init__.py | 164 | def update_settings(cls, settings: BaseSettings) -> None: | CODE |
| LOW | scrapy/spiders/__init__.py | 190 | __all__ = [ | CODE |
| LOW | scrapy/pipelines/files.py | 67 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/pipelines/files.py | 513 | def _update_stores(cls, settings: BaseSettings) -> None: | CODE |
| LOW | scrapy/pipelines/media.py | 51 | logger = logging.getLogger(__name__) | CODE |
| LOW | scrapy/pipelines/images.py | 133 | def _update_stores(cls, settings: BaseSettings) -> None: | CODE |
| LOW | scrapy/utils/gz.py | 13 | __all__ = [ | CODE |
| 52 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_downloader_handler_httpx.py | 59 | except Exception: | CODE |
| LOW | tests/test_downloadermiddleware_httpcache.py | 396 | except Exception: | CODE |
| LOW | tests/test_pipeline_media.py | 143 | except Exception as exc: | CODE |
| LOW | …awlerProcess/twisted_reactor_custom_settings_select.py | 23 | except Exception: | CODE |
| LOW | …sts/AsyncCrawlerProcess/reactorless_custom_settings.py | 29 | except Exception: | CODE |
| LOW | scrapy/crawler.py | 310 | except Exception: | CODE |
| LOW | scrapy/crawler.py | 344 | except Exception: | CODE |
| LOW | scrapy/crawler.py | 614 | except Exception: | CODE |
| LOW | scrapy/crawler.py | 730 | except Exception: | CODE |
| LOW | scrapy/downloadermiddlewares/robotstxt.py | 105 | except Exception as e: | CODE |
| LOW | scrapy/downloadermiddlewares/httpcache.py | 85 | except Exception: | CODE |
| LOW | scrapy/core/scraper.py | 244 | except Exception: | CODE |
| LOW | scrapy/core/scraper.py | 275 | except Exception: | CODE |
| LOW | scrapy/core/scraper.py | 295 | except Exception as spider_exc: | CODE |
| LOW | scrapy/core/scraper.py | 318 | except Exception: | CODE |
| LOW | scrapy/core/scraper.py | 553 | except Exception as ex: | CODE |
| LOW | scrapy/core/spidermw.py | 97 | except Exception: | CODE |
| LOW | scrapy/core/spidermw.py | 111 | except Exception as ex: | CODE |
| LOW | scrapy/core/spidermw.py | 240 | except Exception as ex: | CODE |
| LOW | scrapy/core/engine.py | 239 | except Exception: | CODE |
| LOW | scrapy/core/engine.py | 422 | except Exception as exception: | CODE |
| LOW | scrapy/core/engine.py | 467 | except Exception: | CODE |
| LOW | scrapy/core/engine.py | 567 | except Exception: | CODE |
| LOW⚡ | scrapy/core/engine.py | 935 | except Exception: | CODE |
| LOW⚡ | scrapy/core/engine.py | 940 | except Exception: | CODE |
| LOW⚡ | scrapy/core/engine.py | 945 | except Exception: | CODE |
| LOW⚡ | scrapy/core/engine.py | 952 | except Exception: | CODE |
| LOW⚡ | scrapy/core/engine.py | 962 | except Exception: | CODE |
| LOW | scrapy/core/engine.py | 979 | except Exception: | CODE |
| LOW | scrapy/core/engine.py | 994 | except Exception: | CODE |
| LOW | scrapy/core/downloader/__init__.py | 344 | except Exception: | CODE |
| LOW | scrapy/core/downloader/middleware.py | 95 | except Exception as ex: | CODE |
| LOW | scrapy/core/downloader/middleware.py | 147 | except Exception as ex: | CODE |
| LOW | scrapy/core/downloader/handlers/_base_streaming.py | 226 | except Exception as e: | CODE |
| LOW | scrapy/core/downloader/handlers/__init__.py | 137 | except Exception as ex: | CODE |
| LOW | scrapy/contracts/__init__.py | 62 | except Exception: | CODE |
| LOW | scrapy/contracts/__init__.py | 241 | except Exception: | CODE |
| LOW | scrapy/contracts/__init__.py | 321 | except Exception: | CODE |
| LOW | scrapy/contracts/__init__.py | 331 | except Exception: | CODE |
| LOW | scrapy/pipelines/files.py | 419 | except Exception: | CODE |
| LOW | scrapy/pipelines/files.py | 676 | except Exception as exc: | CODE |
| LOW | scrapy/pipelines/media.py | 210 | except Exception: | CODE |
| LOW | scrapy/pipelines/media.py | 233 | except Exception: | CODE |
| LOW | scrapy/utils/_ssl.py | 37 | except Exception as ex: | CODE |
| LOW | scrapy/utils/signal.py | 64 | except Exception: | CODE |
| LOW | scrapy/utils/signal.py | 200 | except Exception as ex: | CODE |
| LOW | scrapy/utils/asyncio.py | 215 | except Exception: | CODE |
| LOW | scrapy/utils/engine.py | 36 | except Exception as e: | CODE |
| LOW | scrapy/utils/defer.py | 176 | except Exception: | CODE |
| LOW | scrapy/utils/defer.py | 429 | except Exception: | CODE |
| LOW | scrapy/utils/defer.py | 447 | except Exception: | CODE |
| LOW | scrapy/extensions/feedexport.py | 830 | except Exception: | CODE |
| LOW | scrapy/extensions/feedexport.py | 905 | except Exception as e: | CODE |
| LOW | scrapy/extensions/remote_control.py | 186 | except Exception: | CODE |
| LOW | scrapy/extensions/remote_control.py | 208 | except Exception: | CODE |
| LOW | scrapy/extensions/remote_control.py | 231 | except Exception: | CODE |
| LOW | scrapy/extensions/remote_control.py | 286 | except Exception: | CODE |
| MEDIUM | scrapy/extensions/remote_control.py | 152 | def _start(self) -> None: | CODE |
| LOW | scrapy/extensions/httpcache.py | 434 | except Exception: | CODE |
| MEDIUM | scrapy/commands/startproject.py | 56 | print(f"Error: Module {project_name!r} already exists") | CODE |
| 1 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | tests/test_engine_loop.py | 363 | spider.crawler.engine._slot.scheduler.enqueue_request(request) | CODE |
| CRITICAL | tests/test_engine_loop.py | 405 | spider.crawler.engine._slot.scheduler.enqueue_request(request) | CODE |
| CRITICAL | tests/test_engine_loop.py | 455 | spider.crawler.engine._slot.scheduler.enqueue_request(request) | CODE |
| CRITICAL | tests/test_engine_loop.py | 468 | spider.crawler.engine._slot.scheduler.enqueue_request(request) | CODE |
| CRITICAL | scrapy/extensions/throttle.py | 97 | return key, self.crawler.engine.downloader.slots.get(key) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 36 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 77 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 118 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 145 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 156 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 174 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 185 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 257 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 291 | # ================================================================================ | COMMENT |
| MEDIUM | tests/test_spidermiddleware_output_chain.py | 338 | # ================================================================================ | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | conftest.py | 121 | COMMENT | |
| LOW | pyproject.toml | 301 | "walltime: marks benchmarks that need CodSpeed's walltime mode to be meaningful", | COMMENT |
| LOW | pyproject.toml | 321 | # These are triggered by garbage collecting an unclosed AiohttpDownloadHandler. | COMMENT |
| LOW | tests/test_item.py | 341 | # This call to super() trigger the __classcell__ propagation | COMMENT |
| LOW | tests/utils/bases/http_request.py | 141 | url="http://www.scrapy.org/price/µ?currency=£", encoding="latin1" | COMMENT |
| LOW | tests/utils/bases/download_handlers_http.py | 81 | # of building its own (https://github.com/scrapy/scrapy/issues/4919) | COMMENT |
| LOW | tests/benchmarks/urls.txt | 1 | # Link targets for the URL benchmarks, as they would appear in the href | COMMENT |
| LOW | docs/requirements.txt | 1 | # This file was autogenerated by uv via the following command: | COMMENT |
| LOW | docs/requirements.txt | 21 | constantly==23.10.4 | COMMENT |
| LOW | docs/requirements.txt | 41 | # via tldextract | COMMENT |
| LOW | docs/requirements.txt | 61 | # itemloaders | COMMENT |
| LOW | docs/requirements.txt | 81 | # scrapy | COMMENT |
| LOW | docs/requirements.txt | 101 | # -r requirements.in | COMMENT |
| LOW | docs/requirements.txt | 121 | roman-numerals==4.1.0 | COMMENT |
| LOW | docs/requirements.txt | 141 | # sphinx-scrapy | COMMENT |
| LOW | docs/requirements.txt | 181 | # via sphinx-scrapy | COMMENT |
| LOW | docs/requirements.txt | 201 | # parsel | COMMENT |
| LOW | docs/conf.py | 121 | r"\w+Contract\.(adjust_request_args|(pre|post)_process)$", | COMMENT |
| LOW | scrapy/shell.py | 41 | # official ways to use it: | COMMENT |
| LOW | scrapy/shell.py | 61 | # 2. scrapy.commands.shell.Command.run() calls crawler_process.start() in a | COMMENT |
| LOW | scrapy/crawler.py | 1141 | ) -> None: | COMMENT |
| LOW | scrapy/crawler.py | 1161 | # 5. (after _run_loop() returns) _start_asyncio() calls _close_loop() | COMMENT |
| LOW | scrapy/crawler.py | 1181 | # 2. _signal_shutdown_reactorless() calls _shutdown_graceful_reactorless() | COMMENT |
| LOW | scrapy/core/_http2/protocol.py | 121 | COMMENT | |
| LOW | scrapy/extensions/throttle.py | 101 | COMMENT | |
| LOW | scrapy/http/request/__init__.py | 201 | #: | COMMENT |
| LOW | scrapy/http/request/__init__.py | 221 | #: :ref:`callback-output`. | COMMENT |
| LOW | scrapy/http/request/__init__.py | 241 | #: | COMMENT |
| LOW | scrapy/http/request/__init__.py | 261 | #: default), or those components should not filter out this request | COMMENT |
| LOW | scrapy/templates/project/module/spiders/__init__.py | 1 | # This package will contain the spiders of your Scrapy project | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_downloadermiddleware_cookies.py | 208 | CODE | |
| LOW | tests/test_downloadermiddleware_cookies.py | 261 | CODE | |
| LOW | tests/test_downloadermiddleware_httpcache.py | 582 | CODE | |
| LOW | tests/utils/bases/engine.py | 105 | CODE | |
| LOW | docs/utils/linkfix.py | 20 | CODE | |
| LOW | scrapy/pqueues.py | 236 | CODE | |
| LOW | scrapy/pqueues.py | 417 | CODE | |
| LOW | scrapy/downloadermiddlewares/httpproxy.py | 78 | CODE | |
| LOW | scrapy/selector/unified.py | 78 | CODE | |
| LOW | scrapy/core/downloader/middleware.py | 128 | CODE | |
| LOW | scrapy/core/_http2/protocol.py | 379 | CODE | |
| LOW | scrapy/core/_http2/stream.py | 410 | CODE | |
| LOW | scrapy/spiders/crawl.py | 292 | CODE | |
| LOW | scrapy/spiders/feed.py | 142 | CODE | |
| LOW | scrapy/pipelines/media.py | 144 | CODE | |
| LOW | scrapy/pipelines/media.py | 318 | CODE | |
| LOW | scrapy/pipelines/images.py | 211 | CODE | |
| LOW | scrapy/utils/_sitemap.py | 100 | CODE | |
| LOW | scrapy/utils/_sitemap.py | 55 | CODE | |
| LOW | scrapy/utils/request.py | 36 | CODE | |
| LOW | scrapy/utils/iterators.py | 21 | CODE | |
| LOW | scrapy/extensions/feedexport.py | 355 | CODE | |
| LOW | scrapy/extensions/feedexport.py | 885 | CODE | |
| LOW | scrapy/http/request/form.py | 171 | CODE | |
| LOW | scrapy/commands/check.py | 141 | CODE | |
| LOW | scrapy/commands/settings.py | 84 | CODE | |
| LOW | scrapy/commands/parse.py | 307 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scrapy/core/scraper.py | 1 | """This module implements the Scraper component which parses responses and | STRING |
| MEDIUM | scrapy/core/spidermw.py | 289 | # This method is only needed until _async compatibility methods are removed. | COMMENT |
| MEDIUM | scrapy/core/downloader/tls.py | 140 | """This method is needed to override the verify callback.""" | STRING |
| MEDIUM | scrapy/core/_http2/agent.py | 49 | # Create a deferred which will fire with the H2ClientProtocol | COMMENT |
| MEDIUM | scrapy/spiders/feed.py | 134 | """This method is called for the nodes matching the provided tag name | STRING |
| MEDIUM | scrapy/utils/trackref.py | 1 | """This module provides some functions and classes to record and report | STRING |
| MEDIUM | scrapy/commands/shell.py | 121 | # Create the engine in the main thread | COMMENT |
| MEDIUM | scrapy/commands/shell.py | 126 | # Create the engine in the event loop thread | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_downloadermiddleware_httpproxy.py | 285 | request.meta["proxy"] = "https://user1:password1@example.org" | CODE |
| LOW⚡ | tests/test_downloadermiddleware_httpproxy.py | 302 | request.meta["proxy"] = "https://user1:password1@example.org" | CODE |
| LOW⚡ | tests/test_downloadermiddleware_httpproxy.py | 325 | request.meta["proxy"] = "https://user2:password2@example.org" | CODE |
| LOW⚡ | tests/test_item.py | 68 | i["name"] = "John Doe" | CODE |
| LOW⚡ | tests/test_item.py | 72 | assert itemrepr == "{'name': 'John Doe', 'number': 123}" | CODE |
| LOW⚡ | tests/test_item.py | 75 | assert i2["name"] == "John Doe" | CODE |
| LOW⚡ | tests/test_settings/__init__.py | 692 | dummy_config_value = settings.pop("DUMMY_CONFIG", "dummy_value") | CODE |
| LOW⚡ | tests/test_settings/__init__.py | 693 | assert dummy_config_value == "dummy_value" | CODE |
| LOW⚡ | tests/test_settings/__init__.py | 697 | {"DUMMY_CONFIG": "dummy_value", "OTHER_DUMMY_CONFIG": "other_dummy_value"} | CODE |
| LOW⚡ | tests/test_settings/__init__.py | 700 | assert settings.pop("DUMMY_CONFIG") == "dummy_value" | CODE |
| LOW | scrapy/http/request/form.py | 68 | formdata={"name": "John Doe", "age": "27"}, | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_command_runspider.py | 0 | import scrapy class myspider(scrapy.spider): name = 'myspider' async def start(self): self.logger.debug('it works!') ret | STRING |
| HIGH | tests/test_command_crawl.py | 0 | import scrapy class myspider(scrapy.spider): name = 'myspider' async def start(self): self.logger.debug('it works!') ret | STRING |
| HIGH | tests/test_commands.py | 0 | import scrapy class myspider(scrapy.spider): name = 'myspider' async def start(self): self.logger.debug('it works!') ret | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scrapy/mail.py | 61 | CODE | |
| LOW | scrapy/linkextractors/lxmlhtml.py | 326 | CODE | |
| LOW | scrapy/spiders/crawl.py | 122 | CODE | |
| LOW | scrapy/extensions/feedexport.py | 282 | CODE | |
| LOW | scrapy/http/response/__init__.py | 68 | CODE | |
| LOW | scrapy/http/response/__init__.py | 255 | CODE | |
| LOW | scrapy/http/response/__init__.py | 309 | CODE | |
| LOW | scrapy/http/response/text.py | 190 | CODE | |
| LOW | scrapy/http/response/text.py | 246 | CODE | |
| LOW | scrapy/http/request/__init__.py | 168 | CODE | |
| LOW | scrapy/http/request/form.py | 114 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_http2_client_protocol.py | 401 | # Check if headers were sent successfully | COMMENT |
| LOW | scrapy/core/_http2/stream.py | 382 | # Check if we exceed the allowed max data size which can be received | COMMENT |
| LOW | scrapy/core/_http2/agent.py | 55 | # Check if we already have a connection to the remote | COMMENT |
| LOW | scrapy/pipelines/media.py | 192 | # Check if request is downloading right now to avoid doing it twice | COMMENT |
| LOW | scrapy/utils/asyncio.py | 95 | # Check if there is an installed asyncio reactor (it doesn't need to be | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/mockserver/http_resources.py | 470 | # just set this to trigger a test failure if no valid accept-encoding header was set | COMMENT |
| MEDIUM | scrapy/utils/signal.py | 42 | """Like ``pydispatch.robust.sendRobust()`` but it also logs errors and returns | STRING |
| LOW | scrapy/http/request/form.py | 294 | # If we don't have clickdata, we just use the first clickable element | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ils_misc/test_return_with_argument_inside_generator.py | 82 | def helper() -> int: | STRING |
| LOW | …ils_misc/test_return_with_argument_inside_generator.py | 136 | def helper() -> int: | STRING |
| LOW | …ils_misc/test_return_with_argument_inside_generator.py | 202 | def helper() -> int: | STRING |