AI Slop Code RIsk Monitor

WikiExtractor/wikiextractor

A tool for extracting plain text from Wikipedia dumps

61.7 High pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for WikiExtractor/wikiextractor, a Python project with 4,003 GitHub stars. SynthScan examined 21,028 lines of code across 83 source files, recording 993 pattern matches distributed across 11 syntactic categories. The legacy adjusted pattern signal is 61.7 (High pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

61.7
Adjusted Signal
61.7
Raw Pattern Score
100%
Time Factor
2026-10-04
Last Push
4.0K
Stars
Python
Language
21.0K
Lines of Code
83
Files
993
Pattern Hits
2026-10-12
Scan Date
0.05
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 93.1/100 · CRITICAL

Can maintainers explain intent, assumptions, and trade-offs? 933 finding(s).

Hallucinated Dependency & Supply Chain Risk 2.0/100 · LOW

Are imports declared and package/API names verified? 8 finding(s).

Agent Over-Engineering Ratio 94.2/100 · CRITICAL

Are wrappers, guards, and repeated structures necessary? 874 finding(s).

PR Review Velocity Impact 92.9/100 · CRITICAL

Which changes need focused human review? 993 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

Run additional scans to build a history of the legacy pattern signal. Score changes are not a measure of AI adoption or code quality.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 0HIGH 4MEDIUM 50LOW 939

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 993 distinct pattern matches across 11 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Hyper-Verbose Identifiers820 hits · 1027 pts
SeverityFileLineSnippetContext
LOWtools/confirm_redirect_keyword_causation.py137def get_source_wikitext_by_id(dump_path, wanted_ids):CODE
LOWtools/confirm_redirect_keyword_causation.py211def find_redirect_templates_invoked(ex, page_id, wikitext, non_english_redirect_titles, templates,CODE
LOWtools/confirm_redirect_keyword_causation.py245def find_non_english_redirect_titles(templates_path, redirect_keywords):CODE
LOWtools/extract_templates_by_id.py128def determine_template_prefix(templates_path):CODE
LOWtools/extract_templates_by_id.py222def find_titles_in_templates_file(templates_path, wanted_titles):CODE
LOWtools/extract_page_range.py268def extract_templates_for_pages(out, page_texts_by_id, templates_path, max_passes):CODE
LOWtools/report_page_timing.py133def report_last_job_per_worker(timings, in_progress):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py46 def test_space_before_comma_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py49 def test_space_before_colon_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py52 def test_space_before_period_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py55 def test_space_before_closing_paren_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py58 def test_space_before_closing_bracket_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py61 def test_space_before_guillemet_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py64 def test_space_after_opening_paren_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py67 def test_space_after_opening_bracket_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py70 def test_space_after_opening_guillemet_is_removed(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py73 def test_the_exact_real_article_regression(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py81 def test_multiple_spaces_before_punctuation_still_collapse_correctly(self):CODE
LOW⚡tests/test_punctuation_spacing_cleanup.py86 def test_punctuation_with_no_adjacent_space_is_unaffected(self):CODE
LOW⚡tests/test_br_close_tag.py47 def test_closing_br_between_words_becomes_a_space(self):CODE
LOW⚡tests/test_br_close_tag.py53 def test_closing_br_with_whitespace_variants(self):CODE
LOW⚡tests/test_br_close_tag.py59 def test_closing_hr_between_words_becomes_a_space(self):CODE
LOW⚡tests/test_br_close_tag.py63 def test_closing_br_at_start_of_line_does_not_add_leading_space(self):CODE
LOW⚡tests/test_br_close_tag.py70 def test_closing_br_at_end_of_line_does_not_add_trailing_space(self):CODE
LOW⚡tests/test_br_close_tag.py74 def test_paired_br_and_closing_br_both_handled(self):CODE
LOW⚡tests/test_br_close_tag.py81 def test_real_world_sindhi_article_shape(self):CODE
LOWtests/test_urlencode_urldecode.py45 def test_reserved_characters_are_encoded(self):CODE
LOWtests/test_urlencode_urldecode.py48 def test_non_ascii_text_is_encoded(self):CODE
LOWtests/test_urlencode_urldecode.py61 def test_reserved_characters_are_decoded(self):CODE
LOW⚡tests/test_urlencode_urldecode.py72 def test_round_trip_ascii_with_reserved_characters(self):CODE
LOW⚡tests/test_urlencode_urldecode.py76 def test_round_trip_non_ascii_text(self):CODE
LOW⚡tests/test_urlencode_urldecode.py80 def test_real_pipeline_round_trip_through_nested_template_calls(self):CODE
LOW⚡tests/test_sharp_time.py54 def test_year_four_and_two_digit(self):CODE
LOW⚡tests/test_sharp_time.py58 def test_month_padded_and_unpadded(self):CODE
LOW⚡tests/test_sharp_time.py62 def test_day_padded_and_unpadded(self):CODE
LOW⚡tests/test_sharp_time.py91 def test_day_of_year_is_zero_based(self):CODE
LOW⚡tests/test_sharp_time.py95 def test_combined_format_used_by_citation_templates(self):CODE
LOW⚡tests/test_sharp_time.py103 def test_non_alphabetic_characters_pass_through(self):CODE
LOW⚡tests/test_sharp_time.py106 def test_backslash_escapes_the_next_character(self):CODE
LOW⚡tests/test_sharp_time.py112 def test_double_quotes_mark_a_literal_run(self):CODE
LOW⚡tests/test_sharp_time.py115 def test_unclosed_quote_is_itself_a_literal(self):CODE
LOW⚡tests/test_sharp_time.py118 def test_trailing_backslash_is_dropped(self):CODE
LOW⚡tests/test_sharp_time.py128 def test_month_name_format_is_declined(self):CODE
LOW⚡tests/test_sharp_time.py131 def test_day_name_format_is_declined(self):CODE
LOW⚡tests/test_sharp_time.py134 def test_declining_is_not_the_error_span(self):CODE
LOW⚡tests/test_sharp_time.py137 def test_one_unsupported_character_declines_the_whole_format(self):CODE
LOW⚡tests/test_sharp_time.py149 def test_year_and_month_only_defaults_the_day(self):CODE
LOW⚡tests/test_sharp_time.py152 def test_iso_datetime_with_t_separator(self):CODE
LOW⚡tests/test_sharp_time.py155 def test_trailing_z_is_accepted(self):CODE
LOW⚡tests/test_sharp_time.py158 def test_mediawiki_fourteen_digit_timestamp(self):CODE
LOW⚡tests/test_sharp_time.py162 def test_empty_timestamp_means_now(self):CODE
LOW⚡tests/test_sharp_time.py166 def test_relative_offset_moves_from_now(self):CODE
LOW⚡tests/test_sharp_time.py171 def test_negative_relative_offset(self):CODE
LOW⚡tests/test_sharp_time.py176 def test_relative_units_without_a_fixed_length_are_rejected(self):CODE
LOW⚡tests/test_sharp_time.py184 def test_unparseable_text_gives_the_error_span(self):CODE
LOW⚡tests/test_sharp_time.py187 def test_well_formed_but_impossible_date_gives_the_error_span(self):CODE
LOW⚡tests/test_sharp_time.py191 def test_trailing_junk_after_a_date_gives_the_error_span(self):CODE
LOW⚡tests/test_sharp_time.py196 def test_the_error_span_is_what_iferror_detects(self):CODE
LOW⚡tests/test_sharp_time.py199 def test_a_good_date_does_not_trip_iferror(self):CODE
LOW⚡tests/test_sharp_time.py203 def test_the_error_span_leaves_no_visible_text(self):CODE
760 more matches not shown…
Decorative Section Separators43 hits · 146 pts
SeverityFileLineSnippetContext
MEDIUM⚡wikiextractor/cirrus-extract.py4# =============================================================================COMMENT
MEDIUM⚡wikiextractor/cirrus-extract.py8# =============================================================================COMMENT
MEDIUM⚡wikiextractor/cirrus-extract.py10# =============================================================================COMMENT
MEDIUMwikiextractor/cirrus-extract.py24# =============================================================================COMMENT
MEDIUMwikiextractor/cirrus-extract.py55# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/cirrus-extract.py124# ----------------------------------------------------------------------COMMENT
MEDIUM⚡wikiextractor/clean.py1# =============================================================================COMMENT
MEDIUM⚡wikiextractor/clean.py3# =============================================================================COMMENT
MEDIUMwikiextractor/clean.py17# =============================================================================COMMENT
MEDIUM⚡wikiextractor/WikiExtractor.py19# =============================================================================COMMENT
MEDIUM⚡wikiextractor/WikiExtractor.py21# =============================================================================COMMENT
MEDIUMwikiextractor/WikiExtractor.py4# =============================================================================COMMENT
MEDIUMwikiextractor/WikiExtractor.py35# =============================================================================COMMENT
MEDIUMwikiextractor/WikiExtractor.py81# ===========================================================================COMMENT
MEDIUMwikiextractor/WikiExtractor.py171# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/WikiExtractor.py183# ------------------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/WikiExtractor.py257# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/WikiExtractor.py873# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/WikiExtractor.py1164# ----------------------------------------------------------------------COMMENT
MEDIUM⚡wikiextractor/extractPage.py4# =============================================================================COMMENT
MEDIUM⚡wikiextractor/extractPage.py8# =============================================================================COMMENT
MEDIUM⚡wikiextractor/extractPage.py10# =============================================================================COMMENT
MEDIUMwikiextractor/extractPage.py24# =============================================================================COMMENT
MEDIUMwikiextractor/extractPage.py39# ----------------------------------------------------------------------COMMENT
MEDIUM⚡wikiextractor/extract.py3# =============================================================================COMMENT
MEDIUM⚡wikiextractor/extract.py5# =============================================================================COMMENT
MEDIUMwikiextractor/extract.py19# =============================================================================COMMENT
MEDIUMwikiextractor/extract.py59# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py244# ======================================================================COMMENT
MEDIUMwikiextractor/extract.py688# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py759# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py837# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py1157# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py1301# =========================================================================COMMENT
MEDIUMwikiextractor/extract.py1335# ------------------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py1631# ======================================================================COMMENT
MEDIUMwikiextractor/extract.py1758# ======================================================================COMMENT
MEDIUMwikiextractor/extract.py2011 # ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py2350# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py2737# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py2809# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py3281# ----------------------------------------------------------------------COMMENT
MEDIUMwikiextractor/extract.py3940# ----------------------------------------------------------------------COMMENT
Over-Commented Block68 hits · 40 pts
SeverityFileLineSnippetContext
LOWtests/test_sharp_switch.py41 self.assertEqual(ex.sharp_switch('zzz', 'a=A', 'b=B'), '')COMMENT
LOWtests/test_template_loop_guard.py161COMMENT
LOWtests/test_link_bracket_collapse.py201 self.assertEqual(result, "پاکستانبھارت")COMMENT
LOWtests/test_comment_removal.py101COMMENT
LOWtests/test_frame_cleanup.py81 # still have been correctly popped back to empty, not leftCOMMENT
LOWtests/test_sharp_expr_security.py81 def test_classic_sandbox_escape_chain_does_not_execute(self):COMMENT
LOWtests/test_sharp_expr_security.py101 # that one, and sharp_expr()'s own warnings.catch_warnings()COMMENT
LOWwikiextractor/cirrus-extract.py1#!/usr/bin/env pythonCOMMENT
LOWwikiextractor/clean.py1# =============================================================================COMMENT
LOWwikiextractor/WikiExtractor.py1#!/usr/bin/env pythonCOMMENT
LOWwikiextractor/WikiExtractor.py21# =============================================================================COMMENT
LOWwikiextractor/WikiExtractor.py81# ===========================================================================COMMENT
LOWwikiextractor/WikiExtractor.py161##COMMENT
LOWwikiextractor/WikiExtractor.py721 with template_blob_ctx as (_wrapper, template_blob_names), \COMMENT
LOWwikiextractor/WikiExtractor.py741 # but that particular issue is now fixed.COMMENT
LOWwikiextractor/WikiExtractor.py761 Process = mp_context.ProcessCOMMENT
LOWwikiextractor/WikiExtractor.py941 else contextlib.nullcontext(None))COMMENT
LOWwikiextractor/extractPage.py1#!/usr/bin/env pythonCOMMENT
LOWwikiextractor/template_blob.py121COMMENT
LOWwikiextractor/extract.py1# -*- coding: utf-8 -*-COMMENT
LOWwikiextractor/extract.py41# be enabled without the other.COMMENT
LOWwikiextractor/extract.py61# <nowiki>...</nowiki> marks its contents as literal text, exempt fromCOMMENT
LOWwikiextractor/extract.py81# containing no brace, bracket, or pipe character at all, so it can'tCOMMENT
LOWwikiextractor/extract.py161#COMMENT
LOWwikiextractor/extract.py201 # includeonly is the direct counterpart of noinclude above, and theCOMMENT
LOWwikiextractor/extract.py221]COMMENT
LOWwikiextractor/extract.py301 end = nextTag.end()COMMENT
LOWwikiextractor/extract.py321# base and reading run together into one string that is neither:COMMENT
LOWwikiextractor/extract.py421 text = text.replace("'''", '').replace("''", '"')COMMENT
LOWwikiextractor/extract.py441 # where there's actually something to merge with on both sides --COMMENT
LOWwikiextractor/extract.py481 # "geo/18aug2018-1"), so only a '/' immediately before theCOMMENT
LOWwikiextractor/extract.py501COMMENT
LOWwikiextractor/extract.py581 #COMMENT
LOWwikiextractor/extract.py921 i, j = m.start(), m.end()COMMENT
LOWwikiextractor/extract.py961 pos = curCOMMENT
LOWwikiextractor/extract.py981 # the far side of a whole separate "زبان|..." phrase, notCOMMENT
LOWwikiextractor/extract.py1301# =========================================================================COMMENT
LOWwikiextractor/extract.py1321COMMENT
LOWwikiextractor/extract.py1341# implicit line break around each of these), unlike the rest ofCOMMENT
LOWwikiextractor/extract.py1401 ns = normalizeNamespace(prefix)COMMENT
LOWwikiextractor/extract.py1461 return (left, right)COMMENT
LOWwikiextractor/extract.py1481 # ref/references/nowiki/templatestyles are NOT treated this way:COMMENT
LOWwikiextractor/extract.py1501# in clean() above): matched the same permissive way as nobr (trailingCOMMENT
LOWwikiextractor/extract.py1521# ends up referring to alternative 0's group (renumbered from thatCOMMENT
LOWwikiextractor/extract.py1541COMMENT
LOWwikiextractor/extract.py1641 # we must handle nesting, s.a.COMMENT
LOWwikiextractor/extract.py1661 e += 3COMMENT
LOWwikiextractor/extract.py2101 # will get default values (if available) during actual instantiation.COMMENT
LOWwikiextractor/extract.py2121 # Therefore, we don't check if the parameter has been assigned aCOMMENT
LOWwikiextractor/extract.py2261 title = redirectedCOMMENT
LOWwikiextractor/extract.py2281 # A tplarg may contain other parameters as well as templates, e.g.:COMMENT
LOWwikiextractor/extract.py2301COMMENT
LOWwikiextractor/extract.py2421COMMENT
LOWwikiextractor/extract.py2541 if i >= 0:COMMENT
LOWwikiextractor/extract.py2581 # brace outside it, taken as plain text. For any pattern of braces thisCOMMENT
LOWwikiextractor/extract.py2601 # We must resolve ambiguities like this:COMMENT
LOWwikiextractor/extract.py2781 m = _TEMPLATE_TITLE_COLON_RE.match(templateTitle)COMMENT
LOWwikiextractor/extract.py2941 for value_node in node.values:COMMENT
LOWwikiextractor/extract.py2961# of which lean on #expr comparisons heavily for their own internalCOMMENT
LOWwikiextractor/extract.py3021 # "trunc EXPR" is #expr's own prefix, unary truncate-toward-COMMENT
8 more matches not shown…
Deep Nesting36 hits · 28 pts
SeverityFileLineSnippetContext
LOWtools/confirm_redirect_keyword_causation.py111CODE
LOWtools/confirm_redirect_keyword_causation.py137CODE
LOWtools/confirm_redirect_keyword_causation.py245CODE
LOWtools/classify_missing_pages.py154CODE
LOWtools/classify_missing_pages.py191CODE
LOWtools/classify_missing_pages.py332CODE
LOWtools/extract_templates_by_id.py85CODE
LOWtools/check_magic_words_case.py93CODE
LOWtools/find_redirect_keywords.py152CODE
LOWtools/extract_page_range.py217CODE
LOWtools/extract_page_range.py337CODE
LOWtools/extract_page_range.py379CODE
LOWtests/test_dump_completeness.py93CODE
LOWtests/test_load_templates_output_file_whitespace.py84CODE
LOWtests/test_load_templates_output_file_whitespace.py128CODE
LOWtests/test_streaming_template_blob.py96CODE
LOWtests/test_template_loading_content.py125CODE
LOWtests/test_article_extraction_content.py144CODE
LOWwikiextractor/WikiExtractor.py264CODE
LOWwikiextractor/WikiExtractor.py415CODE
LOWwikiextractor/WikiExtractor.py505CODE
LOWwikiextractor/WikiExtractor.py640CODE
LOWwikiextractor/WikiExtractor.py877CODE
LOWwikiextractor/WikiExtractor.py1052CODE
LOWwikiextractor/WikiExtractor.py1170CODE
LOWwikiextractor/extractPage.py46CODE
LOWwikiextractor/extract.py270CODE
LOWwikiextractor/extract.py559CODE
LOWwikiextractor/extract.py690CODE
LOWwikiextractor/extract.py1099CODE
LOWwikiextractor/extract.py2354CODE
LOWwikiextractor/extract.py2573CODE
LOWwikiextractor/extract.py2847CODE
LOWwikiextractor/extract.py3013CODE
LOWwikiextractor/extract.py3204CODE
LOWwikiextractor/extract.py2065CODE
Self-Referential Comments6 hits · 26 pts
SeverityFileLineSnippetContext
MEDIUM⚡wikiextractor/cirrus-extract.py11# This file is part of Tanl.COMMENT
MEDIUM⚡wikiextractor/clean.py4# This file is part of Tanl.COMMENT
MEDIUM⚡wikiextractor/WikiExtractor.py22# This file is part of Tanl.COMMENT
MEDIUM⚡wikiextractor/extractPage.py11# This file is part of Tanl.COMMENT
MEDIUM⚡wikiextractor/extract.py6# This file is part of Tanl.COMMENT
MEDIUMwikiextractor/extract.py3851 # This function is used in some pages to construct linksCOMMENT
Empty Error Handlers3 hits · 15 pts
SeverityFileLineSnippetContext
HIGHtools/confirm_redirect_keyword_causation.py240except Exception:CODE
HIGHtools/extract_templates_by_id.py191except Exception:CODE
HIGHtests/test_frame_cleanup.py111except RuntimeError:CODE
Unused Imports7 hits · 6 pts
SeverityFileLineSnippetContext
LOWtests/test_sharp_time.py37CODE
LOWtests/test_streaming_template_blob.py31CODE
LOWtests/test_template_loading_content.py44CODE
LOWtests/test_extract_process_worker_summary.py40CODE
LOWwikiextractor/cirrus-extract.py37CODE
LOWwikiextractor/cirrus-extract.py41CODE
LOWwikiextractor/extract.py30CODE
Excessive Try-Catch Wrapping6 hits · 6 pts
SeverityFileLineSnippetContext
LOWtools/confirm_redirect_keyword_causation.py240 except Exception:CODE
LOWtools/classify_missing_pages.py306 except Exception as e:CODE
LOWtools/extract_templates_by_id.py191 except Exception:CODE
LOWtools/find_redirect_keywords.py138 except Exception as e:CODE
MEDIUMwikiextractor/extract.py3013def sharp_expr(expr, page_title=None, page_id=None, extractor=None):CODE
LOWwikiextractor/extract.py3048 except Exception:CODE
Cross-Language Confusion1 hit · 2 pts
SeverityFileLineSnippetContext
HIGHtests/test_template_argument_names.py108 Splitting there leaves {{{1}}} undefined, Template:BlockquoteSTRING
Overly Generic Function Names1 hit · 1 pts
SeverityFileLineSnippetContext
LOWwikiextractor/extractPage.py46def process_data(input_file, id, templates=False):CODE
AI Structural Patterns2 hits · 1 pts
SeverityFileLineSnippetContext
LOWwikiextractor/extract.py1827CODE
LOWwikiextractor/extract.py2063CODE