Repository Analysis

NanmiCoder/MediaCrawler

小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 | 评论爬虫、微博帖子 | 评论爬虫、百度贴吧帖子 | 百度贴吧评论回复爬虫 | 知乎问答文章|评论爬虫

28.4 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of NanmiCoder/MediaCrawler, a Python project with 56,390 GitHub stars. SynthScan v2.0 examined 50,437 lines of code across 267 source files, recording 888 pattern matches distributed across 15 syntactic categories. The overall adjusted score of 28.4 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

28.4
Adjusted Score
28.4
Raw Score
100%
Time Factor
2026-07-10
Last Push
56.4K
Stars
Python
Language
50.4K
Lines of Code
267
Files
888
Pattern Hits
2026-07-14
Scan Date
0.18
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 49MEDIUM 166LOW 673

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 888 distinct pattern matches across 15 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Self-Referential Comments162 hits · 486 pts
SeverityFileLineSnippetContext
MEDIUMrecv_sms.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMvar.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMmain.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMdatabase/db.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMdatabase/models.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMdatabase/mongodb_store_base.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMdatabase/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMdatabase/db_session.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/crawler_util.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/async_file_writer.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/easing.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/utils.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/slider_util.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/words.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/browser_launcher.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/cdp_browser.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/app_runner.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/time_util.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/file_header_manager.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtools/file_header_manager.py81# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/types.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/proxy_mixin.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/proxy_ip_pool.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/base_proxy.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/providers/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/providers/kuaidl_proxy.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/providers/jishu_http_proxy.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMproxy/providers/wandou_http_proxy.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcache/local_cache.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcache/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcache/abs_cache.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcache/cache_factory.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcache/redis_cache.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_utils.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_db_sync.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_expiring_local_cache.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_redis_cache.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_proxy_ip_pool.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtest/test_mongodb_integration.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/zhihu_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/base_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/ks_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/dy_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/tieba_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/xhs_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/db_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/weibo_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMconfig/bilibili_config.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtests/conftest.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtests/test_store_factory.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtests/test_excel_store.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMtests/test_excel_store.py267 # Create an instanceCOMMENT
MEDIUMcmd_arg/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMcmd_arg/arg.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMmedia_platform/__init__.py4# This file is part of MediaCrawler project.COMMENT
MEDIUMmedia_platform/bilibili/field.py4# This file is part of MediaCrawler project.COMMENT
102 more matches not shown…
Cross-File Repetition46 hits · 230 pts
SeverityFileLineSnippetContext
HIGHmedia_platform/bilibili/login.py0check if the current login status is successful and return true otherwise return false retry decorator will retry 20 timSTRING
HIGHmedia_platform/weibo/login.py0check if the current login status is successful and return true otherwise return false retry decorator will retry 20 timSTRING
HIGHmedia_platform/kuaishou/login.py0check if the current login status is successful and return true otherwise return false retry decorator will retry 20 timSTRING
HIGHmedia_platform/xhs/client.py0update cookies method provided by api client, typically called after successful login args: browser_context: browser conSTRING
HIGHmedia_platform/tieba/client.py0update cookies method provided by api client, typically called after successful login args: browser_context: browser conSTRING
HIGHmedia_platform/zhihu/client.py0update cookies method provided by api client, typically called after successful login args: browser_context: browser conSTRING
HIGHmedia_platform/xhs/core.py0search for notes and retrieve their comment information. returns:STRING
HIGHmedia_platform/zhihu/core.py0search for notes and retrieve their comment information. returns:STRING
HIGHmedia_platform/tieba/core.py0search for notes and retrieve their comment information. returns:STRING
HIGHmedia_platform/xhs/core.py0concurrently obtain the specified post list and save the dataSTRING
HIGHmedia_platform/douyin/core.py0concurrently obtain the specified post list and save the dataSTRING
HIGHmedia_platform/kuaishou/core.py0concurrently obtain the specified post list and save the dataSTRING
HIGHmedia_platform/weibo/core.py0get creator's information and their notes and comments returns:STRING
HIGHmedia_platform/tieba/core.py0get creator's information and their notes and comments returns:STRING
HIGHmedia_platform/zhihu/core.py0get creator's information and their notes and comments returns:STRING
HIGHstore/bilibili/_store_impl.py0content json storage implementation args: content_item: returns:STRING
HIGHstore/weibo/_store_impl.py0content json storage implementation args: content_item: returns:STRING
HIGHstore/douyin/_store_impl.py0content json storage implementation args: content_item: returns:STRING
HIGHstore/kuaishou/_store_impl.py0content json storage implementation args: content_item: returns:STRING
HIGHstore/zhihu/_store_impl.py0content json storage implementation args: content_item: returns:STRING
HIGHstore/bilibili/_store_impl.py0comment json storage implementation args: comment_item: returns:STRING
HIGHstore/weibo/_store_impl.py0comment json storage implementation args: comment_item: returns:STRING
HIGHstore/douyin/_store_impl.py0comment json storage implementation args: comment_item: returns:STRING
HIGHstore/kuaishou/_store_impl.py0comment json storage implementation args: comment_item: returns:STRING
HIGHstore/zhihu/_store_impl.py0comment json storage implementation args: comment_item: returns:STRING
HIGHstore/bilibili/_store_impl.py0creator json storage implementation 教学版:不采集/持久化创作者个人信息,空操作。 args: creator: returns:STRING
HIGHstore/douyin/_store_impl.py0creator json storage implementation 教学版:不采集/持久化创作者个人信息,空操作。 args: creator: returns:STRING
HIGHstore/weibo/_store_impl.py0creator json storage implementation 教学版:不采集/持久化创作者个人信息,空操作。 args: creator: returns:STRING
HIGHstore/bilibili/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/douyin/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/kuaishou/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/xhs/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/weibo/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/tieba/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/zhihu/_store_impl.py0store content to mongodb args: content_item: content dataSTRING
HIGHstore/bilibili/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/xhs/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/weibo/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/tieba/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/douyin/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/kuaishou/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/zhihu/_store_impl.py0store comment to mongodb args: comment_item: comment dataSTRING
HIGHstore/weibo/_store_impl.py0calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_sSTRING
HIGHstore/tieba/_store_impl.py0calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_sSTRING
HIGHstore/kuaishou/_store_impl.py0calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_sSTRING
HIGHstore/zhihu/_store_impl.py0calculate the prefix sorting number for data save files, supporting writing to different files for each run args: file_sSTRING
Hyper-Verbose Identifiers183 hits · 182 pts
SeverityFileLineSnippetContext
LOWrecv_sms.py46def extract_verification_code(message: str) -> str:CODE
LOWmain.py86async def _generate_wordcloud_if_needed() -> None:CODE
LOWdatabase/db_session.py31async def create_database_if_not_exists(db_type: str):CODE
LOWtools/crawler_util.py66async def find_qrcode_img_from_canvas(page: Page, canvas_selector: str) -> str:CODE
LOWtools/crawler_util.py148async def convert_browser_context_cookies(CODE
LOWtools/crawler_util.py159def convert_str_cookie_to_dict(cookie_str: str) -> Dict:CODE
LOWtools/crawler_util.py177def match_interact_info_count(count_str: str) -> int:CODE
LOWtools/crawler_util.py226def extract_url_params_to_dict(url: str) -> Dict:CODE
LOWtools/async_file_writer.py62 async def write_single_item_to_json(self, item: Dict, item_type: str):CODE
LOWtools/async_file_writer.py82 async def generate_wordcloud_from_comments(self):CODE
LOWtools/words.py50 async def generate_word_frequency_and_cloud(self, data, save_words_prefix):CODE
LOWtools/cdp_browser.py47 def _register_cleanup_handlers(self):CODE
LOWtools/cdp_browser.py140 async def _connect_existing_browser(CODE
LOWtools/cdp_browser.py288 async def _get_browser_websocket_url(self, debug_port: int) -> str:CODE
LOWtools/time_util.py60def get_time_str_from_unix_time(unixtime):CODE
LOWtools/time_util.py71def get_date_str_from_unix_time(unixtime):CODE
LOWtools/time_util.py82def get_unix_time_from_time_str(time_str):CODE
LOWtools/file_header_manager.py67def generate_copyright_header(relative_path: str) -> str:CODE
LOWproxy/proxy_mixin.py57 async def _refresh_proxy_if_expired(self) -> None:CODE
LOWtest/test_utils.py38async def test_convert_browser_context_cookies_uses_url_filter():CODE
LOWtest/test_proxy_ip_pool.py94 async def test_proxy_pool_auto_refresh(self):CODE
LOWtest/test_proxy_ip_pool.py136 async def test_ip_expiration_standalone(self):CODE
LOWtest/test_proxy_ip_pool.py271 async def test_create_ip_pool_unknown_provider_raises_value_error(self):CODE
LOWtest/test_mongodb_integration.py222 def test_xhs_store_implementation(self):CODE
LOWtest/test_mongodb_integration.py274 def test_douyin_store_implementation(self):CODE
LOWtest/test_mongodb_integration.py319 def test_concurrent_operations(self):CODE
LOWtests/test_weibo_no_user_info.py124def test_weibo_note_masks_user_info():CODE
LOWtests/test_weibo_no_user_info.py164def test_weibo_comment_masks_user_info():CODE
LOWtests/test_weibo_no_user_info.py203def test_weibo_store_end_to_end_sqlite():CODE
LOWtests/test_cmd_arg_tieba.py10async def test_tieba_detail_cli_sets_specified_ids():CODE
LOWtests/test_cmd_arg_tieba.py26async def test_tieba_creator_cli_sets_creator_urls():CODE
LOWtests/test_cmd_arg_tieba.py45async def test_tieba_detail_reads_runtime_specified_ids(monkeypatch):CODE
LOWtests/test_store_factory.py67 def test_create_mongodb_store(self):CODE
LOWtests/test_store_factory.py87 def test_invalid_store_option(self):CODE
LOWtests/test_store_factory.py94 def test_all_stores_registered(self):CODE
LOWtests/test_static_proxy_provider.py9def test_default_proxy_provider_remains_existing_provider():CODE
LOWtests/test_static_proxy_provider.py16async def test_static_proxy_provider_parses_proxy_url(monkeypatch):CODE
LOWtests/test_static_proxy_provider.py32async def test_static_proxy_provider_rejects_invalid_url(monkeypatch):CODE
LOWtests/test_static_proxy_provider.py41async def test_static_proxy_pool_disables_validation(monkeypatch):CODE
LOWtests/test_api_limits.py12async def test_cmd_arg_crawler_max_notes_count():CODE
LOWtests/test_api_limits.py29def test_crawler_manager_build_command():CODE
LOWtests/test_api_limits.py65def test_api_start_crawler_with_limits():CODE
LOWtests/test_api_limits.py90def test_api_start_crawler_without_limits():CODE
LOWtests/test_api_limits.py123def test_api_rejects_invalid_limits(field_name, value):CODE
LOWtests/test_cdp_browser.py11async def test_existing_browser_connects_directly_to_devtools_browser(monkeypatch):CODE
LOWtests/test_cdp_browser.py37async def test_existing_browser_falls_back_to_discovered_websocket_url(monkeypatch):CODE
LOWtests/test_cdp_browser.py74async def test_launched_browser_uses_discovered_websocket_url(monkeypatch):CODE
LOWtests/test_no_user_info.py33def test_orm_has_no_forbidden_columns():CODE
LOWtests/test_no_user_info.py47def test_creator_tables_removed():CODE
LOWtests/test_no_user_info.py55def test_content_tables_have_creator_hash():CODE
LOWtests/test_no_user_info.py99def test_xhs_note_extraction_masks_user_info():CODE
LOWtests/test_no_user_info.py132def test_tieba_note_extraction_masks_user_info():CODE
LOWtests/test_no_user_info.py148def test_tieba_comment_extraction_masks_user_info():CODE
LOWtests/test_no_user_info.py164def test_zhihu_comment_extraction_masks_user_info():CODE
LOWtests/test_no_user_info.py182def test_bilibili_video_dict_masks_user_info():CODE
LOWtests/test_no_user_info.py216def test_store_no_forbidden_dict_keys():CODE
LOWtests/test_no_user_info.py226def test_store_no_creator_orm_imports():CODE
LOWtests/test_excel_store.py173 def test_empty_sheets_removed(self, excel_store):CODE
LOWtests/test_excel_store.py191def test_excel_import_availability():CODE
LOWtests/test_excel_store.py213 def test_get_instance_returns_same_instance(self):CODE
123 more matches not shown…
Over-Commented Block162 hits · 161 pts
SeverityFileLineSnippetContext
LOWrecv_sms.py1# -*- coding: utf-8 -*-COMMENT
LOWvar.py1# -*- coding: utf-8 -*-COMMENT
LOWmain.py1# -*- coding: utf-8 -*-COMMENT
LOWdatabase/db.py1# -*- coding: utf-8 -*-COMMENT
LOWdatabase/models.py1# -*- coding: utf-8 -*-COMMENT
LOWdatabase/mongodb_store_base.py1# -*- coding: utf-8 -*-COMMENT
LOWdatabase/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWdatabase/db_session.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/crawler_util.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/async_file_writer.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/easing.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/utils.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/slider_util.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/words.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/browser_launcher.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/cdp_browser.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/app_runner.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/time_util.py1# -*- coding: utf-8 -*-COMMENT
LOWtools/file_header_manager.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/__init__.py21# -*- coding: utf-8 -*-COMMENT
LOWproxy/types.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/proxy_mixin.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/proxy_ip_pool.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/base_proxy.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/providers/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/providers/kuaidl_proxy.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/providers/jishu_http_proxy.py1# -*- coding: utf-8 -*-COMMENT
LOWproxy/providers/wandou_http_proxy.py1# -*- coding: utf-8 -*-COMMENT
LOWcache/local_cache.py1# -*- coding: utf-8 -*-COMMENT
LOWcache/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWcache/abs_cache.py1# -*- coding: utf-8 -*-COMMENT
LOWcache/cache_factory.py1# -*- coding: utf-8 -*-COMMENT
LOWcache/redis_cache.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_utils.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_db_sync.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_db_sync.py221# [*] Changed tables:COMMENT
LOWtest/test_db_sync.py241# >>> Manual confirmation required: Synchronize ORM model to MySQL database? (y/N): yCOMMENT
LOWtest/test_expiring_local_cache.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_redis_cache.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_proxy_ip_pool.py1# -*- coding: utf-8 -*-COMMENT
LOWtest/test_mongodb_integration.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/zhihu_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/base_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/base_config.py41COMMENT
LOWconfig/base_config.py61# CDP 调试端口,用于与浏览器通信COMMENT
LOWconfig/ks_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/__init__.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/dy_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/dy_config.py21PUBLISH_TIME_TYPE = 0COMMENT
LOWconfig/tieba_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/xhs_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/db_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/weibo_config.py1# -*- coding: utf-8 -*-COMMENT
LOWconfig/bilibili_config.py1# -*- coding: utf-8 -*-COMMENT
LOWtests/conftest.py1# -*- coding: utf-8 -*-COMMENT
LOWtests/test_store_factory.py1# -*- coding: utf-8 -*-COMMENT
LOWtests/__init__.py1# -*- coding: utf-8 -*-COMMENT
102 more matches not shown…
Unused Imports144 hits · 144 pts
SeverityFileLineSnippetContext
LOWmain.py32CODE
LOWdatabase/db.py23CODE
LOWdatabase/models.py27CODE
LOWdatabase/models.py29CODE
LOWtools/crawler_util.py27CODE
LOWtools/crawler_util.py35CODE
LOWtools/async_file_writer.py24CODE
LOWtools/utils.py24CODE
LOWtools/utils.py25CODE
LOWtools/utils.py26CODE
LOWtools/browser_launcher.py28CODE
LOWtools/browser_launcher.py29CODE
LOWtools/app_runner.py21CODE
LOWproxy/__init__.py25CODE
LOWproxy/proxy_mixin.py31CODE
LOWproxy/proxy_ip_pool.py29CODE
LOWproxy/providers/__init__.py25CODE
LOWproxy/providers/__init__.py26CODE
LOWproxy/providers/__init__.py27CODE
LOWproxy/providers/kuaidl_proxy.py29CODE
LOWproxy/providers/jishu_http_proxy.py28CODE
LOWproxy/providers/wandou_http_proxy.py28CODE
LOWtest/test_db_sync.py27CODE
LOWconfig/base_config.py143CODE
LOWconfig/base_config.py144CODE
LOWconfig/base_config.py145CODE
LOWconfig/base_config.py146CODE
LOWconfig/base_config.py147CODE
LOWconfig/base_config.py148CODE
LOWconfig/base_config.py149CODE
LOWconfig/__init__.py21CODE
LOWconfig/__init__.py22CODE
LOWtests/test_store_factory.py24CODE
LOWtests/test_store_factory.py27CODE
LOWtests/test_excel_store.py25CODE
LOWtests/test_excel_store.py26CODE
LOWcmd_arg/__init__.py21CODE
LOWcmd_arg/arg.py21CODE
LOWmedia_platform/bilibili/client.py40CODE
LOWmedia_platform/bilibili/__init__.py26CODE
LOWmedia_platform/xhs/client.py23CODE
LOWmedia_platform/xhs/client.py36CODE
LOWmedia_platform/xhs/__init__.py21CODE
LOWmedia_platform/xhs/__init__.py22CODE
LOWmedia_platform/weibo/client.py43CODE
LOWmedia_platform/weibo/__init__.py25CODE
LOWmedia_platform/weibo/__init__.py26CODE
LOWmedia_platform/weibo/__init__.py27CODE
LOWmedia_platform/weibo/core.py29CODE
LOWmedia_platform/tieba/__init__.py22CODE
LOWmedia_platform/tieba/core.py24CODE
LOWmedia_platform/douyin/client.py38CODE
LOWmedia_platform/douyin/client.py39CODE
LOWmedia_platform/douyin/client.py40CODE
LOWmedia_platform/douyin/client.py36CODE
LOWmedia_platform/douyin/__init__.py21CODE
LOWmedia_platform/douyin/core.py24CODE
LOWmedia_platform/douyin/help.py29CODE
LOWmedia_platform/kuaishou/client.py27CODE
LOWmedia_platform/kuaishou/client.py37CODE
84 more matches not shown…
Excessive Try-Catch Wrapping110 hits · 114 pts
SeverityFileLineSnippetContext
LOWmain.py82 except Exception as e:CODE
LOWmain.py96 except Exception as e:CODE
LOWmain.py129 except Exception as e:CODE
LOWmain.py137 except Exception as e:CODE
LOWmain.py158 except Exception:CODE
LOWdatabase/mongodb_store_base.py75 except Exception as e:CODE
LOWdatabase/mongodb_store_base.py111 except Exception as e:CODE
LOWdatabase/mongodb_store_base.py120 except Exception as e:CODE
LOWdatabase/mongodb_store_base.py132 except Exception as e:CODE
LOWdatabase/mongodb_store_base.py142 except Exception as e:CODE
LOWdatabase/db_session.py98 except Exception as e:CODE
LOWtools/crawler_util.py61 except Exception as e:CODE
LOWtools/async_file_writer.py146 except Exception as e:CODE
LOWtools/browser_launcher.py187 except Exception as e:CODE
LOWtools/browser_launcher.py206 except Exception:CODE
LOWtools/browser_launcher.py238 except Exception:CODE
LOWtools/browser_launcher.py288 except Exception as e:CODE
LOWtools/cdp_browser.py135 except Exception as e:CODE
LOWtools/cdp_browser.py246 except Exception as e:CODE
LOWtools/cdp_browser.py309 except Exception as e:CODE
LOWtools/cdp_browser.py330 except Exception as direct_error:CODE
LOWtools/cdp_browser.py356 except Exception as e:CODE
LOWtools/cdp_browser.py410 except Exception as e:CODE
LOWtools/cdp_browser.py421 except Exception as e:CODE
LOWtools/cdp_browser.py432 except Exception as e:CODE
LOWtools/cdp_browser.py457 except Exception as context_error:CODE
LOWtools/cdp_browser.py478 except Exception as browser_error:CODE
LOWtools/cdp_browser.py507 except Exception as e:CODE
LOWtools/cdp_browser.py533 except Exception as e:CODE
LOWtools/app_runner.py82 except Exception:CODE
LOWtools/app_runner.py102 except Exception as e:CODE
LOWtools/time_util.py92 except Exception as e:CODE
LOWtools/file_header_manager.py209 except Exception as e:CODE
LOWproxy/proxy_ip_pool.py93 except Exception as e:CODE
LOWproxy/proxy_ip_pool.py186 except Exception as e:CODE
LOWproxy/base_proxy.py82 except Exception as e:CODE
LOWtest/test_db_sync.py196 except Exception as e:CODE
MEDIUMtest/test_db_sync.py197 print(f"Error processing MySQL: {e}")CODE
LOWtest/test_db_sync.py213 except Exception as e:CODE
MEDIUMtest/test_db_sync.py214 print(f"Error processing SQLite: {e}")CODE
LOWtest/test_mongodb_integration.py42 except Exception as e:CODE
LOWtest/test_mongodb_integration.py86 except Exception as e:CODE
MEDIUMtest/test_mongodb_integration.py36def setUpClass(cls):CODE
LOWmedia_platform/bilibili/client.py144 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py223 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py321 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py365 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py552 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py570 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py635 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py679 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py702 except Exception as e:CODE
LOWmedia_platform/bilibili/core.py725 except Exception as e:CODE
LOWmedia_platform/xhs/client.py256 except Exception as e:CODE
LOWmedia_platform/xhs/client.py527 except Exception as e:CODE
LOWmedia_platform/xhs/client.py532 except Exception as e:CODE
LOWmedia_platform/xhs/core.py447 except Exception as e:CODE
LOWmedia_platform/xhs/login.py68 except Exception:CODE
LOWmedia_platform/xhs/login.py117 except Exception as e:CODE
LOWmedia_platform/xhs/help.py368 except Exception as e:CODE
50 more matches not shown…
Deep Nesting39 hits · 38 pts
SeverityFileLineSnippetContext
LOWmain.py123CODE
LOWtools/async_file_writer.py82CODE
LOWtools/slider_util.py84CODE
LOWtools/browser_launcher.py191CODE
LOWtools/browser_launcher.py214CODE
LOWtools/browser_launcher.py241CODE
LOWtools/cdp_browser.py437CODE
LOWcache/redis_cache.py77CODE
LOWtest/test_db_sync.py101CODE
LOWtest/test_db_sync.py133CODE
LOWtest/test_proxy_ip_pool.py271CODE
LOWcmd_arg/arg.py154CODE
LOWcmd_arg/arg.py160CODE
LOWmedia_platform/bilibili/client.py260CODE
LOWmedia_platform/bilibili/core.py70CODE
LOWmedia_platform/bilibili/core.py183CODE
LOWmedia_platform/bilibili/core.py240CODE
LOWmedia_platform/xhs/playwright_sign.py72CODE
LOWmedia_platform/xhs/client.py456CODE
LOWmedia_platform/xhs/core.py129CODE
LOWmedia_platform/weibo/core.py137CODE
LOWmedia_platform/weibo/help.py29CODE
LOWmedia_platform/tieba/client.py516CODE
LOWmedia_platform/tieba/core.py148CODE
LOWmedia_platform/tieba/core.py402CODE
LOWmedia_platform/tieba/help.py290CODE
LOWmedia_platform/douyin/client.py253CODE
LOWmedia_platform/douyin/core.py127CODE
LOWmedia_platform/douyin/core.py183CODE
LOWmedia_platform/zhihu/core.py150CODE
LOWmedia_platform/zhihu/help.py77CODE
LOWapi/routers/data.py33CODE
LOWapi/routers/data.py62CODE
LOWapi/routers/data.py99CODE
LOWapi/routers/data.py191CODE
LOWapi/routers/websocket.py90CODE
LOWapi/services/crawler_manager.py80CODE
LOWapi/services/crawler_manager.py241CODE
LOWstore/excel_store_base.py171CODE
Redundant / Tautological Comments20 hits · 29 pts
SeverityFileLineSnippetContext
LOWdatabase/db_session.py46 # Check if database existsCOMMENT
LOWtools/browser_launcher.py97 # Check if path exists and is executableCOMMENT
LOWtools/cdp_browser.py448 # Check if context is already closedCOMMENT
LOWtools/cdp_browser.py472 # Check if browser is still connectedCOMMENT
LOWtools/file_header_manager.py100 # Check if contains Copyright keywordCOMMENT
LOWtools/file_header_manager.py130 # Check if first line is shebangCOMMENT
LOWmedia_platform/bilibili/client.py70 # Check if proxy has expired before each requestCOMMENT
LOWmedia_platform/bilibili/client.py307 # Check if is_end and next existCOMMENT
LOWmedia_platform/bilibili/core.py177 end_day = (start_day + timedelta(days=1) - timedelta(seconds=1)) # Set end_day to start_day + 1 day - 1 secCODE
LOWmedia_platform/bilibili/core.py179 end_day = (end_day + timedelta(days=1) - timedelta(seconds=1)) # Set end_day to end_day + 1 day - 1 secondCODE
LOWmedia_platform/xhs/client.py127 # Check if proxy is expired before each requestCOMMENT
LOWmedia_platform/xhs/client.py208 # Check if proxy is expired before requestCOMMENT
LOWmedia_platform/weibo/client.py74 # Check if proxy is expired before each requestCOMMENT
LOWmedia_platform/weibo/core.py435 # Check if it's a long textCOMMENT
LOWmedia_platform/tieba/client.py250 # Check if proxy is expired before each requestCOMMENT
LOWmedia_platform/douyin/login.py150 # Check if there is slider verificationCOMMENT
LOWmedia_platform/douyin/help.py121 # Check if it's a short link (v.douyin.com)COMMENT
LOWmedia_platform/kuaishou/client.py67 # Check if proxy is expired before each requestCOMMENT
LOWmedia_platform/zhihu/client.py97 # Check if proxy is expired before each requestCOMMENT
LOWstore/excel_store_base.py373 # Check if there are any sheets leftCOMMENT
Cross-Language Confusion3 hits · 18 pts
SeverityFileLineSnippetContext
HIGHmedia_platform/tieba/core.py502 get: () => undefined,CODE
HIGHmedia_platform/douyin/help.py95 "([params, post_data, ua]) => window.bdms.init._v[2].p[42].apply(null, [0, 1, 8, params, post_data, ua])",CODE
HIGHapi/main.py79 "note": "WebUI not found, please build it first: cd webui && npm run build"CODE
Verbosity Indicators5 hits · 11 pts
SeverityFileLineSnippetContext
LOWmedia_platform/tieba/core.py415 # Step 1: Visit Baidu homepageCOMMENT
LOWmedia_platform/tieba/core.py419 # Step 2: Wait for page loading, using delay setting from config fileCOMMENT
LOWmedia_platform/tieba/core.py423 # Step 3: Find and click "Tieba" linkCOMMENT
LOWmedia_platform/tieba/core.py449 # Step 4: Click Tieba link (check if it will open in a new tab)COMMENT
LOWmedia_platform/tieba/core.py479 # Step 5: Wait for page to stabilize, using delay setting from config fileCOMMENT
Structural Annotation Overuse5 hits · 11 pts
SeverityFileLineSnippetContext
LOWmedia_platform/tieba/core.py415 # Step 1: Visit Baidu homepageCOMMENT
LOWmedia_platform/tieba/core.py419 # Step 2: Wait for page loading, using delay setting from config fileCOMMENT
LOWmedia_platform/tieba/core.py423 # Step 3: Find and click "Tieba" linkCOMMENT
LOWmedia_platform/tieba/core.py449 # Step 4: Click Tieba link (check if it will open in a new tab)COMMENT
LOWmedia_platform/tieba/core.py479 # Step 5: Wait for page to stabilize, using delay setting from config fileCOMMENT
AI Structural Patterns4 hits · 4 pts
SeverityFileLineSnippetContext
LOWtest/test_mongodb_integration.py360CODE
LOWcmd_arg/arg.py160CODE
LOWmedia_platform/bilibili/client.py390CODE
LOWmedia_platform/tieba/client.py216CODE
Modern Structural Boilerplate3 hits · 3 pts
SeverityFileLineSnippetContext
LOWapi/routers/__init__.py23__all__ = ["crawler_router", "data_router", "websocket_router"]CODE
LOWapi/schemas/__init__.py29__all__ = [CODE
LOWapi/services/__init__.py21__all__ = ["CrawlerManager", "crawler_manager"]CODE
AI Slop Vocabulary1 hit · 2 pts
SeverityFileLineSnippetContext
MEDIUMpackage-lock.json3425 "resolved": "https://registry.npmmirror.com/robust-predicates/-/robust-predicates-3.0.2.tgz",CODE
Example Usage Blocks1 hit · 2 pts
SeverityFileLineSnippetContext
LOWtools/time_util.py130 # Example usageCOMMENT