小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫
This report summarizes scanner evidence for NanmiCoder/MediaCrawler, a Python project with 66,603 GitHub stars. SynthScan examined 56,313 lines of code across 290 source files, recording 1157 pattern matches distributed across 16 syntactic categories. The legacy adjusted pattern signal is 32.3 (High pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 786 finding(s).
Are imports declared and package/API names verified? 182 finding(s).
Are wrappers, guards, and repeated structures necessary? 554 finding(s).
Which changes need focused human review? 1157 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 1157 distinct pattern matches across 16 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | recv_sms.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | var.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | main.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | database/db.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | database/models.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | database/mongodb_store_base.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | database/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | database/db_session.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/crawler_util.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/async_file_writer.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/easing.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/utils.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/slider_util.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/words.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/browser_launcher.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/cdp_browser.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/app_runner.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/time_util.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/file_header_manager.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tools/file_header_manager.py | 81 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/types.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/proxy_mixin.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/proxy_ip_pool.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/base_proxy.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/providers/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/providers/kuaidl_proxy.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/providers/jishu_http_proxy.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | proxy/providers/wandou_http_proxy.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | cache/local_cache.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | cache/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | cache/abs_cache.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | cache/cache_factory.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | cache/redis_cache.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_utils.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_db_sync.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_expiring_local_cache.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_redis_cache.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_proxy_ip_pool.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | test/test_mongodb_integration.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/zhihu_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/base_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/ks_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/__init__.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/dy_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/tieba_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/xhs_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/db_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/weibo_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | config/bilibili_config.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/conftest.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_media_e2e.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_store_factory.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_douyin_argus_header.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_kuaishou_unavailable_video.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_media_ffmpeg.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_cmd_arg_media.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| MEDIUM | tests/test_excel_store.py | 4 | # This file is part of MediaCrawler project. | COMMENT |
| 120 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | recv_sms.py | 46 | def extract_verification_code(message: str) -> str: | CODE |
| LOW⚡ | main.py | 86 | async def _generate_wordcloud_if_needed() -> None: | CODE |
| LOW | database/db_session.py | 31 | async def create_database_if_not_exists(db_type: str): | CODE |
| LOW | tools/crawler_util.py | 66 | async def find_qrcode_img_from_canvas(page: Page, canvas_selector: str) -> str: | CODE |
| LOW | tools/crawler_util.py | 148 | async def convert_browser_context_cookies( | CODE |
| LOW | tools/crawler_util.py | 159 | def convert_str_cookie_to_dict(cookie_str: str) -> Dict: | CODE |
| LOW | tools/crawler_util.py | 177 | def match_interact_info_count(count_str: str) -> int: | CODE |
| LOW | tools/crawler_util.py | 226 | def extract_url_params_to_dict(url: str) -> Dict: | CODE |
| LOW | tools/async_file_writer.py | 62 | async def write_single_item_to_json(self, item: Dict, item_type: str): | CODE |
| LOW | tools/async_file_writer.py | 82 | async def generate_wordcloud_from_comments(self): | CODE |
| LOW | tools/words.py | 50 | async def generate_word_frequency_and_cloud(self, data, save_words_prefix): | CODE |
| LOW | tools/cdp_browser.py | 47 | def _register_cleanup_handlers(self): | CODE |
| LOW | tools/cdp_browser.py | 140 | async def _connect_existing_browser( | CODE |
| LOW | tools/cdp_browser.py | 288 | async def _get_browser_websocket_url(self, debug_port: int) -> str: | CODE |
| LOW | tools/time_util.py | 60 | def get_time_str_from_unix_time(unixtime): | CODE |
| LOW | tools/time_util.py | 71 | def get_date_str_from_unix_time(unixtime): | CODE |
| LOW | tools/time_util.py | 82 | def get_unix_time_from_time_str(time_str): | CODE |
| LOW | tools/file_header_manager.py | 67 | def generate_copyright_header(relative_path: str) -> str: | CODE |
| LOW | proxy/proxy_mixin.py | 57 | async def _refresh_proxy_if_expired(self) -> None: | CODE |
| LOW | test/test_utils.py | 38 | async def test_convert_browser_context_cookies_uses_url_filter(): | CODE |
| LOW | test/test_proxy_ip_pool.py | 94 | async def test_proxy_pool_auto_refresh(self): | CODE |
| LOW | test/test_proxy_ip_pool.py | 136 | async def test_ip_expiration_standalone(self): | CODE |
| LOW | test/test_proxy_ip_pool.py | 271 | async def test_create_ip_pool_unknown_provider_raises_value_error(self): | CODE |
| LOW | test/test_mongodb_integration.py | 222 | def test_xhs_store_implementation(self): | CODE |
| LOW | test/test_mongodb_integration.py | 274 | def test_douyin_store_implementation(self): | CODE |
| LOW | test/test_mongodb_integration.py | 319 | def test_concurrent_operations(self): | CODE |
| LOW | tests/test_weibo_no_user_info.py | 124 | def test_weibo_note_masks_user_info(): | CODE |
| LOW | tests/test_weibo_no_user_info.py | 164 | def test_weibo_comment_masks_user_info(): | CODE |
| LOW | tests/test_weibo_no_user_info.py | 203 | def test_weibo_store_end_to_end_sqlite(): | CODE |
| LOW | tests/test_cmd_arg_tieba.py | 10 | async def test_tieba_detail_cli_sets_specified_ids(): | CODE |
| LOW | tests/test_cmd_arg_tieba.py | 26 | async def test_tieba_creator_cli_sets_creator_urls(): | CODE |
| LOW | tests/test_cmd_arg_tieba.py | 45 | async def test_tieba_detail_reads_runtime_specified_ids(monkeypatch): | CODE |
| LOW | tests/test_media_e2e.py | 60 | async def test_bilibili_downloads_cover_and_video(tmp_path: Path): | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 40 | async def test_raw_response_rejects_access_http_status(monkeypatch, status_code): | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 75 | async def test_raw_response_rejects_known_business_block( | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 105 | async def test_raw_response_keeps_successful_html(monkeypatch): | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 126 | async def test_successful_json_keeps_raw_and_parsed_return_modes(monkeypatch): | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 157 | async def test_html_detail_does_not_multiply_transport_retries(monkeypatch): | CODE |
| LOW | tests/test_xhs_raw_response_errors.py | 179 | async def test_html_detail_still_retries_parse_failures(): | CODE |
| LOW | tests/test_store_factory.py | 67 | def test_create_mongodb_store(self): | CODE |
| LOW | tests/test_store_factory.py | 87 | def test_invalid_store_option(self): | CODE |
| LOW | tests/test_store_factory.py | 94 | def test_all_stores_registered(self): | CODE |
| LOW | tests/test_static_proxy_provider.py | 9 | def test_default_proxy_provider_remains_existing_provider(): | CODE |
| LOW | tests/test_static_proxy_provider.py | 16 | async def test_static_proxy_provider_parses_proxy_url(monkeypatch): | CODE |
| LOW | tests/test_static_proxy_provider.py | 32 | async def test_static_proxy_provider_rejects_invalid_url(monkeypatch): | CODE |
| LOW | tests/test_static_proxy_provider.py | 41 | async def test_static_proxy_pool_disables_validation(monkeypatch): | CODE |
| LOW | tests/test_api_limits.py | 12 | async def test_cmd_arg_crawler_max_notes_count(): | CODE |
| LOW | tests/test_api_limits.py | 29 | def test_crawler_manager_build_command(): | CODE |
| LOW | tests/test_api_limits.py | 66 | def test_crawler_manager_passes_media_switch(): | CODE |
| LOW | tests/test_api_limits.py | 91 | def test_api_schema_exposes_media_switch_default_off(): | CODE |
| LOW | tests/test_api_limits.py | 94 | def test_api_start_crawler_with_limits(): | CODE |
| LOW | tests/test_api_limits.py | 119 | def test_api_start_crawler_without_limits(): | CODE |
| LOW | tests/test_api_limits.py | 152 | def test_api_rejects_invalid_limits(field_name, value): | CODE |
| LOW⚡ | tests/test_douyin_argus_header.py | 47 | def test_argus_header_is_present_by_default(): | CODE |
| LOW⚡ | tests/test_douyin_argus_header.py | 54 | def test_uifid_header_comes_from_cookie(): | CODE |
| LOW⚡ | tests/test_douyin_argus_header.py | 61 | def test_uifid_header_falls_back_to_uifid_temp(): | CODE |
| LOW⚡ | tests/test_douyin_argus_header.py | 68 | def test_missing_uifid_omits_header(): | CODE |
| LOW⚡ | tests/test_douyin_argus_header.py | 76 | def test_caller_supplied_values_win(): | CODE |
| LOW | tests/test_kuaishou_unavailable_video.py | 83 | async def test_unavailable_video_is_skipped_instead_of_crashing(): | CODE |
| LOW | tests/test_kuaishou_unavailable_video.py | 95 | async def test_missing_photo_key_is_also_skipped(): | CODE |
| 300 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | media_platform/bilibili/login.py | 0 | check if the current login status is successful and return true otherwise return false retry decorator will retry 20 tim | STRING |
| HIGH | media_platform/weibo/login.py | 0 | check if the current login status is successful and return true otherwise return false retry decorator will retry 20 tim | STRING |
| HIGH | media_platform/kuaishou/login.py | 0 | check if the current login status is successful and return true otherwise return false retry decorator will retry 20 tim | STRING |
| HIGH | media_platform/xhs/client.py | 0 | update cookies method provided by api client, typically called after successful login args: browser_context: browser con | STRING |
| HIGH | media_platform/tieba/client.py | 0 | update cookies method provided by api client, typically called after successful login args: browser_context: browser con | STRING |
| HIGH | media_platform/zhihu/client.py | 0 | update cookies method provided by api client, typically called after successful login args: browser_context: browser con | STRING |
| HIGH | media_platform/xhs/core.py | 0 | search for notes and retrieve their comment information. returns: | STRING |
| HIGH | media_platform/zhihu/core.py | 0 | search for notes and retrieve their comment information. returns: | STRING |
| HIGH | media_platform/tieba/core.py | 0 | search for notes and retrieve their comment information. returns: | STRING |
| HIGH | media_platform/xhs/core.py | 0 | concurrently obtain the specified post list and save the data | STRING |
| HIGH | media_platform/douyin/core.py | 0 | concurrently obtain the specified post list and save the data | STRING |
| HIGH | media_platform/kuaishou/core.py | 0 | concurrently obtain the specified post list and save the data | STRING |
| HIGH | media_platform/weibo/core.py | 0 | 媒体请求头：平台 referer（防盗链）+ ua。 client.headers 里含 cookie，不能整体透传到 cdn 请求上，因此只取 ua。 | STRING |
| HIGH | media_platform/douyin/core.py | 0 | 媒体请求头：平台 referer（防盗链）+ ua。 client.headers 里含 cookie，不能整体透传到 cdn 请求上，因此只取 ua。 | STRING |
| HIGH | media_platform/kuaishou/core.py | 0 | 媒体请求头：平台 referer（防盗链）+ ua。 client.headers 里含 cookie，不能整体透传到 cdn 请求上，因此只取 ua。 | STRING |
| HIGH | media_platform/weibo/core.py | 0 | get creator's information and their notes and comments returns: | STRING |
| HIGH | media_platform/tieba/core.py | 0 | get creator's information and their notes and comments returns: | STRING |
| HIGH | media_platform/zhihu/core.py | 0 | get creator's information and their notes and comments returns: | STRING |
| HIGH | store/bilibili/_store_impl.py | 0 | content json storage implementation args: content_item: returns: | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | content json storage implementation args: content_item: returns: | STRING |
| HIGH | store/douyin/_store_impl.py | 0 | content json storage implementation args: content_item: returns: | STRING |
| HIGH | store/kuaishou/_store_impl.py | 0 | content json storage implementation args: content_item: returns: | STRING |
| HIGH | store/zhihu/_store_impl.py | 0 | content json storage implementation args: content_item: returns: | STRING |
| HIGH | store/bilibili/_store_impl.py | 0 | comment json storage implementation args: comment_item: returns: | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | comment json storage implementation args: comment_item: returns: | STRING |
| HIGH | store/douyin/_store_impl.py | 0 | comment json storage implementation args: comment_item: returns: | STRING |
| HIGH | store/kuaishou/_store_impl.py | 0 | comment json storage implementation args: comment_item: returns: | STRING |
| HIGH | store/zhihu/_store_impl.py | 0 | comment json storage implementation args: comment_item: returns: | STRING |
| HIGH | store/bilibili/_store_impl.py | 0 | creator json storage implementation 教学版：不采集/持久化创作者个人信息，空操作。 args: creator: returns: | STRING |
| HIGH | store/douyin/_store_impl.py | 0 | creator json storage implementation 教学版：不采集/持久化创作者个人信息，空操作。 args: creator: returns: | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | creator json storage implementation 教学版：不采集/持久化创作者个人信息，空操作。 args: creator: returns: | STRING |
| HIGH | store/bilibili/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/douyin/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/kuaishou/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/xhs/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/tieba/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/zhihu/_store_impl.py | 0 | store content to mongodb args: content_item: content data | STRING |
| HIGH | store/bilibili/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/xhs/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/tieba/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/douyin/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/kuaishou/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/zhihu/_store_impl.py | 0 | store comment to mongodb args: comment_item: comment data | STRING |
| HIGH | store/weibo/_store_impl.py | 0 | calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_s | STRING |
| HIGH | store/tieba/_store_impl.py | 0 | calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_s | STRING |
| HIGH | store/kuaishou/_store_impl.py | 0 | calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_s | STRING |
| HIGH | store/zhihu/_store_impl.py | 0 | calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_s | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | main.py | 32 | CODE | |
| LOW | database/db.py | 23 | CODE | |
| LOW | database/models.py | 27 | CODE | |
| LOW | database/models.py | 29 | CODE | |
| LOW | tools/crawler_util.py | 27 | CODE | |
| LOW | tools/crawler_util.py | 35 | CODE | |
| LOW | tools/async_file_writer.py | 24 | CODE | |
| LOW | tools/utils.py | 24 | CODE | |
| LOW | tools/utils.py | 25 | CODE | |
| LOW | tools/utils.py | 26 | CODE | |
| LOW | tools/browser_launcher.py | 28 | CODE | |
| LOW | tools/browser_launcher.py | 29 | CODE | |
| LOW | tools/app_runner.py | 21 | CODE | |
| LOW | proxy/__init__.py | 25 | CODE | |
| LOW | proxy/proxy_mixin.py | 31 | CODE | |
| LOW | proxy/proxy_ip_pool.py | 29 | CODE | |
| LOW | proxy/providers/__init__.py | 25 | CODE | |
| LOW | proxy/providers/__init__.py | 26 | CODE | |
| LOW | proxy/providers/__init__.py | 27 | CODE | |
| LOW | proxy/providers/kuaidl_proxy.py | 29 | CODE | |
| LOW | proxy/providers/jishu_http_proxy.py | 28 | CODE | |
| LOW | proxy/providers/wandou_http_proxy.py | 28 | CODE | |
| LOW | test/test_db_sync.py | 27 | CODE | |
| LOW | config/base_config.py | 146 | CODE | |
| LOW | config/base_config.py | 147 | CODE | |
| LOW | config/base_config.py | 148 | CODE | |
| LOW | config/base_config.py | 149 | CODE | |
| LOW | config/base_config.py | 150 | CODE | |
| LOW | config/base_config.py | 151 | CODE | |
| LOW | config/base_config.py | 152 | CODE | |
| LOW | config/__init__.py | 21 | CODE | |
| LOW | config/__init__.py | 22 | CODE | |
| LOW | tests/test_media_e2e.py | 21 | CODE | |
| LOW | tests/test_store_factory.py | 24 | CODE | |
| LOW | tests/test_store_factory.py | 27 | CODE | |
| LOW | tests/test_douyin_argus_header.py | 21 | CODE | |
| LOW | tests/test_douyin_argus_header.py | 23 | CODE | |
| LOW | tests/test_kuaishou_unavailable_video.py | 22 | CODE | |
| LOW | tests/test_media_ffmpeg.py | 16 | CODE | |
| LOW | tests/test_cmd_arg_media.py | 16 | CODE | |
| LOW | tests/test_excel_store.py | 25 | CODE | |
| LOW | tests/test_excel_store.py | 26 | CODE | |
| LOW | tests/test_kuaishou_url_parse.py | 21 | CODE | |
| LOW | tests/test_media_core_integration.py | 16 | CODE | |
| LOW | tests/media_server.py | 20 | CODE | |
| LOW | tests/test_douyin_aweme_detail_params.py | 21 | CODE | |
| LOW | tests/test_media_extractors.py | 16 | CODE | |
| LOW | tests/test_media_downloader.py | 16 | CODE | |
| LOW | tests/test_bilibili_login_state.py | 20 | CODE | |
| LOW | cmd_arg/__init__.py | 21 | CODE | |
| LOW | cmd_arg/arg.py | 21 | CODE | |
| LOW | media_platform/bilibili/media.py | 29 | CODE | |
| LOW | media_platform/bilibili/client.py | 30 | CODE | |
| LOW | media_platform/bilibili/client.py | 40 | CODE | |
| LOW | media_platform/bilibili/__init__.py | 26 | CODE | |
| LOW | media_platform/xhs/media.py | 26 | CODE | |
| LOW | media_platform/xhs/client.py | 23 | CODE | |
| LOW | media_platform/xhs/client.py | 25 | CODE | |
| LOW | media_platform/xhs/client.py | 42 | CODE | |
| LOW | media_platform/xhs/__init__.py | 21 | CODE | |
| 116 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | recv_sms.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | var.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | main.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | database/db.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | database/models.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | database/mongodb_store_base.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | database/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | database/db_session.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/crawler_util.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/async_file_writer.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/easing.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/utils.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/slider_util.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/words.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/browser_launcher.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/cdp_browser.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/app_runner.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/time_util.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tools/file_header_manager.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/__init__.py | 21 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/types.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/proxy_mixin.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/proxy_ip_pool.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/base_proxy.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/providers/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/providers/kuaidl_proxy.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/providers/jishu_http_proxy.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | proxy/providers/wandou_http_proxy.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | cache/local_cache.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | cache/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | cache/abs_cache.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | cache/cache_factory.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | cache/redis_cache.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_utils.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_db_sync.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_db_sync.py | 221 | # [*] Changed tables: | COMMENT |
| LOW | test/test_db_sync.py | 241 | # >>> Manual confirmation required: Synchronize ORM model to MySQL database? (y/N): y | COMMENT |
| LOW | test/test_expiring_local_cache.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_redis_cache.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_proxy_ip_pool.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | test/test_mongodb_integration.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/zhihu_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/base_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/base_config.py | 41 | COMMENT | |
| LOW | config/base_config.py | 61 | # CDP 调试端口，用于与浏览器通信 | COMMENT |
| LOW | config/ks_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/ks_config.py | 21 | COMMENT | |
| LOW | config/__init__.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/dy_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/dy_config.py | 21 | PUBLISH_TIME_TYPE = 0 | COMMENT |
| LOW | config/tieba_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/xhs_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/db_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/weibo_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | config/bilibili_config.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tests/conftest.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| LOW | tests/test_store_factory.py | 1 | # -*- coding: utf-8 -*- | COMMENT |
| 109 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | main.py | 82 | except Exception as e: | CODE |
| LOW⚡ | main.py | 96 | except Exception as e: | CODE |
| LOW | main.py | 129 | except Exception as e: | CODE |
| LOW | main.py | 137 | except Exception as e: | CODE |
| LOW | main.py | 158 | except Exception: | CODE |
| LOW | database/mongodb_store_base.py | 75 | except Exception as e: | CODE |
| LOW | database/mongodb_store_base.py | 111 | except Exception as e: | CODE |
| LOW | database/mongodb_store_base.py | 120 | except Exception as e: | CODE |
| LOW | database/mongodb_store_base.py | 132 | except Exception as e: | CODE |
| LOW | database/mongodb_store_base.py | 142 | except Exception as e: | CODE |
| LOW | database/db_session.py | 98 | except Exception as e: | CODE |
| LOW | tools/crawler_util.py | 61 | except Exception as e: | CODE |
| LOW | tools/async_file_writer.py | 146 | except Exception as e: | CODE |
| LOW | tools/browser_launcher.py | 187 | except Exception as e: | CODE |
| LOW | tools/browser_launcher.py | 206 | except Exception: | CODE |
| LOW | tools/browser_launcher.py | 238 | except Exception: | CODE |
| LOW | tools/browser_launcher.py | 288 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 135 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 246 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 309 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 330 | except Exception as direct_error: | CODE |
| LOW | tools/cdp_browser.py | 356 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 410 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 421 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 432 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 457 | except Exception as context_error: | CODE |
| LOW | tools/cdp_browser.py | 478 | except Exception as browser_error: | CODE |
| LOW | tools/cdp_browser.py | 507 | except Exception as e: | CODE |
| LOW | tools/cdp_browser.py | 533 | except Exception as e: | CODE |
| LOW | tools/app_runner.py | 82 | except Exception: | CODE |
| LOW | tools/app_runner.py | 102 | except Exception as e: | CODE |
| LOW | tools/time_util.py | 92 | except Exception as e: | CODE |
| LOW | tools/file_header_manager.py | 209 | except Exception as e: | CODE |
| LOW | proxy/proxy_ip_pool.py | 93 | except Exception as e: | CODE |
| LOW | proxy/proxy_ip_pool.py | 186 | except Exception as e: | CODE |
| LOW | proxy/base_proxy.py | 82 | except Exception as e: | CODE |
| LOW | test/test_db_sync.py | 196 | except Exception as e: | CODE |
| MEDIUM | test/test_db_sync.py | 197 | print(f"Error processing MySQL: {e}") | CODE |
| LOW | test/test_db_sync.py | 213 | except Exception as e: | CODE |
| MEDIUM | test/test_db_sync.py | 214 | print(f"Error processing SQLite: {e}") | CODE |
| LOW | test/test_mongodb_integration.py | 42 | except Exception as e: | CODE |
| LOW | test/test_mongodb_integration.py | 86 | except Exception as e: | CODE |
| MEDIUM | test/test_mongodb_integration.py | 36 | def setUpClass(cls): | CODE |
| LOW | media_platform/bilibili/client.py | 155 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 236 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 334 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 378 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 572 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 590 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 606 | except Exception as exc: | CODE |
| LOW | media_platform/bilibili/core.py | 741 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 785 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 808 | except Exception as e: | CODE |
| LOW | media_platform/bilibili/core.py | 831 | except Exception as e: | CODE |
| LOW | media_platform/xhs/client.py | 275 | except Exception as e: | CODE |
| LOW | media_platform/xhs/client.py | 546 | except Exception as e: | CODE |
| LOW | media_platform/xhs/client.py | 551 | except Exception as e: | CODE |
| LOW | media_platform/xhs/core.py | 470 | except Exception as e: | CODE |
| LOW | media_platform/xhs/core.py | 498 | except Exception as exc: | CODE |
| LOW | media_platform/xhs/login.py | 68 | except Exception: | CODE |
| 58 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | main.py | 158 | except Exception: | CODE |
| HIGH | tools/browser_launcher.py | 206 | except Exception: | CODE |
| HIGH | tools/app_runner.py | 58 | except asyncio.TimeoutError: | CODE |
| HIGH | tools/app_runner.py | 82 | except Exception: | CODE |
| HIGH | test/test_mongodb_integration.py | 78 | except: | CODE |
| HIGH | tests/media_server.py | 299 | except OSError: | CODE |
| HIGH | media_platform/xhs/client.py | 171 | except (TypeError, ValueError): | CODE |
| HIGH | media_platform/xhs/core.py | 313 | except RetryError: | CODE |
| HIGH | media_platform/xhs/login.py | 68 | except Exception: | CODE |
| HIGH | api/routers/data.py | 48 | except Exception: | CODE |
| HIGH | api/routers/websocket.py | 148 | except WebSocketDisconnect: | CODE |
| HIGH | api/routers/websocket.py | 150 | except Exception: | CODE |
| HIGH | api/services/crawler_manager.py | 281 | except asyncio.CancelledError: | CODE |
| HIGH | api/services/crawler_manager.py | 77 | except asyncio.QueueFull: | CODE |
| HIGH | api/services/crawler_manager.py | 110 | except asyncio.QueueEmpty: | CODE |
| HIGH | store/excel_store_base.py | 186 | except (TypeError, AttributeError): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | main.py | 123 | CODE | |
| LOW | tools/async_file_writer.py | 82 | CODE | |
| LOW | tools/slider_util.py | 84 | CODE | |
| LOW | tools/browser_launcher.py | 191 | CODE | |
| LOW | tools/browser_launcher.py | 214 | CODE | |
| LOW | tools/browser_launcher.py | 241 | CODE | |
| LOW | tools/cdp_browser.py | 437 | CODE | |
| LOW | cache/redis_cache.py | 77 | CODE | |
| LOW | test/test_db_sync.py | 101 | CODE | |
| LOW | test/test_db_sync.py | 133 | CODE | |
| LOW | test/test_proxy_ip_pool.py | 271 | CODE | |
| LOW | tests/media_server.py | 93 | CODE | |
| LOW | cmd_arg/arg.py | 154 | CODE | |
| LOW | cmd_arg/arg.py | 160 | CODE | |
| LOW | media_platform/bilibili/client.py | 257 | CODE | |
| LOW | media_platform/bilibili/core.py | 74 | CODE | |
| LOW | media_platform/bilibili/core.py | 196 | CODE | |
| LOW | media_platform/bilibili/core.py | 253 | CODE | |
| LOW | media_platform/xhs/client.py | 475 | CODE | |
| LOW | media_platform/xhs/core.py | 137 | CODE | |
| LOW | media_platform/weibo/core.py | 140 | CODE | |
| LOW | media_platform/weibo/core.py | 319 | CODE | |
| LOW | media_platform/weibo/help.py | 29 | CODE | |
| LOW | media_platform/tieba/client.py | 516 | CODE | |
| LOW | media_platform/tieba/core.py | 148 | CODE | |
| LOW | media_platform/tieba/core.py | 402 | CODE | |
| LOW | media_platform/tieba/help.py | 290 | CODE | |
| LOW | media_platform/douyin/client.py | 278 | CODE | |
| LOW | media_platform/douyin/core.py | 130 | CODE | |
| LOW | media_platform/douyin/core.py | 186 | CODE | |
| LOW | media_platform/kuaishou/core.py | 200 | CODE | |
| LOW | media_platform/zhihu/core.py | 150 | CODE | |
| LOW | media_platform/zhihu/help.py | 77 | CODE | |
| LOW | api/routers/data.py | 33 | CODE | |
| LOW | api/routers/data.py | 62 | CODE | |
| LOW | api/routers/data.py | 99 | CODE | |
| LOW | api/routers/data.py | 191 | CODE | |
| LOW | api/routers/websocket.py | 90 | CODE | |
| LOW | api/services/crawler_manager.py | 80 | CODE | |
| LOW | api/services/crawler_manager.py | 242 | CODE | |
| LOW | store/excel_store_base.py | 171 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_kuaishou_unavailable_video.py | 13 | ``visionVideoDetail: {photo: null, author: null}`` —— key 存在、值是 null。 | STRING |
| HIGH | media_platform/tieba/core.py | 502 | get: () => undefined, | CODE |
| HIGH⚡ | media_platform/douyin/help.py | 95 | "([params, post_data, ua]) => window.bdms.init._v[2].p[42].apply(null, [0, 1, 8, params, post_data, ua])", | CODE |
| HIGH | media_platform/kuaishou/core.py | 262 | # visionVideoDetail: {photo: null, author: null}——key 在、值是 null。 | COMMENT |
| HIGH | media_platform/kuaishou/help.py | 41 | if (!done && this && typeof this === "object" && this !== window && | CODE |
| HIGH | api/main.py | 79 | "note": "WebUI not found, please build it first: cd webui && npm run build" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | database/db_session.py | 46 | # Check if database exists | COMMENT |
| LOW | tools/browser_launcher.py | 97 | # Check if path exists and is executable | COMMENT |
| LOW | tools/cdp_browser.py | 448 | # Check if context is already closed | COMMENT |
| LOW | tools/cdp_browser.py | 472 | # Check if browser is still connected | COMMENT |
| LOW | tools/file_header_manager.py | 100 | # Check if contains Copyright keyword | COMMENT |
| LOW | tools/file_header_manager.py | 130 | # Check if first line is shebang | COMMENT |
| LOW | media_platform/bilibili/client.py | 81 | # Check if proxy has expired before each request | COMMENT |
| LOW | media_platform/bilibili/client.py | 324 | # Check if is_end and next exist | COMMENT |
| LOW | media_platform/bilibili/core.py | 190 | end_day = (start_day + timedelta(days=1) - timedelta(seconds=1)) # Set end_day to start_day + 1 day - 1 sec | CODE |
| LOW | media_platform/bilibili/core.py | 192 | end_day = (end_day + timedelta(days=1) - timedelta(seconds=1)) # Set end_day to end_day + 1 day - 1 second | CODE |
| LOW | media_platform/xhs/client.py | 145 | # Check if proxy is expired before each request | COMMENT |
| LOW | media_platform/weibo/client.py | 73 | # Check if proxy is expired before each request | COMMENT |
| LOW | media_platform/weibo/core.py | 455 | # Check if it's a long text | COMMENT |
| LOW | media_platform/tieba/client.py | 250 | # Check if proxy is expired before each request | COMMENT |
| LOW | media_platform/douyin/login.py | 150 | # Check if there is slider verification | COMMENT |
| LOW | media_platform/douyin/help.py | 121 | # Check if it's a short link (v.douyin.com) | COMMENT |
| LOW | media_platform/kuaishou/client.py | 69 | # Check if proxy is expired before each request | COMMENT |
| LOW | media_platform/zhihu/client.py | 97 | # Check if proxy is expired before each request | COMMENT |
| LOW | store/excel_store_base.py | 373 | # Check if there are any sheets left | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | media_platform/tieba/core.py | 415 | # Step 1: Visit Baidu homepage | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 419 | # Step 2: Wait for page loading, using delay setting from config file | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 423 | # Step 3: Find and click "Tieba" link | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 449 | # Step 4: Click Tieba link (check if it will open in a new tab) | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 479 | # Step 5: Wait for page to stabilize, using delay setting from config file | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | media_platform/tieba/core.py | 415 | # Step 1: Visit Baidu homepage | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 419 | # Step 2: Wait for page loading, using delay setting from config file | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 423 | # Step 3: Find and click "Tieba" link | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 449 | # Step 4: Click Tieba link (check if it will open in a new tab) | COMMENT |
| LOW⚡ | media_platform/tieba/core.py | 479 | # Step 5: Wait for page to stabilize, using delay setting from config file | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/media_server.py | 364 | def set_content(self, name: str, content: bytes) -> None: | CODE |
| LOW | tests/media_server.py | 367 | def set_content_type(self, name: str, content_type: str) -> None: | CODE |
| LOW | api/routers/__init__.py | 23 | __all__ = ["crawler_router", "data_router", "websocket_router"] | CODE |
| LOW | api/schemas/__init__.py | 29 | __all__ = [ | CODE |
| LOW | api/services/__init__.py | 21 | __all__ = ["CrawlerManager", "crawler_manager"] | CODE |
| LOW | media_downloader/__init__.py | 36 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | test/test_mongodb_integration.py | 360 | CODE | |
| LOW | cmd_arg/arg.py | 160 | CODE | |
| LOW | media_platform/bilibili/client.py | 416 | CODE | |
| LOW | media_platform/tieba/client.py | 216 | CODE | |
| LOW | media_downloader/downloader.py | 126 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | package-lock.json | 3425 | "resolved": "https://registry.npmmirror.com/robust-predicates/-/robust-predicates-3.0.2.tgz", | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/time_util.py | 130 | # Example usage | COMMENT |