📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG
This report presents the forensic synthetic code analysis of VectifyAI/PageIndex, a Python project with 34,012 GitHub stars. SynthScan v2.0 examined 8,497 lines of code across 38 source files, recording 103 pattern matches distributed across 14 syntactic categories. The overall adjusted score of 16.7 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 103 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_issue_163.py | 20 | def test_toc_detector_empty_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 25 | def test_toc_detector_valid_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 30 | def test_toc_detector_malformed_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 35 | def test_extraction_complete_empty_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 40 | def test_extraction_complete_valid_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 45 | def test_transformation_complete_empty_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 50 | def test_transformation_complete_valid_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 55 | def test_detect_page_index_empty_response(self, mock_llm): | CODE |
| LOW⚡ | tests/test_issue_163.py | 63 | def test_completes_on_first_try(self, mock_llm, mock_check): | CODE |
| LOW⚡ | tests/test_issue_163.py | 72 | def test_continues_on_incomplete(self, mock_llm, mock_check): | CODE |
| LOW | tests/test_issue_163.py | 84 | def test_max_retries_raises_exception(self, mock_llm, mock_check): | CODE |
| LOW | tests/test_issue_163.py | 93 | def test_chat_history_grows_incrementally(self, mock_llm, mock_check): | CODE |
| LOW | tests/test_issue_163.py | 119 | def test_completes_on_first_try(self, mock_llm, mock_check): | CODE |
| LOW | tests/test_issue_163.py | 131 | def test_handles_missing_table_of_contents_key(self, mock_llm, mock_check): | CODE |
| LOW | pageindex/page_index.py | 48 | async def check_title_appearance_in_start(title, page_text, model=None, logger=None): | CODE |
| LOW | pageindex/page_index.py | 74 | async def check_title_appearance_in_start_concurrent(structure, page_list, model=None, logger=None): | CODE |
| LOW | pageindex/page_index.py | 124 | def check_if_toc_extraction_is_complete(content, toc, model=None): | CODE |
| LOW | pageindex/page_index.py | 142 | def check_if_toc_transformation_is_complete(content, toc, model=None): | CODE |
| LOW | pageindex/page_index.py | 368 | def extract_matching_page_pairs(toc_page, toc_physical_index, start_page_index): | CODE |
| LOW | pageindex/page_index.py | 405 | def add_page_offset_to_toc_json(data, offset): | CODE |
| LOW | pageindex/page_index.py | 483 | def remove_first_physical_index_section(text): | CODE |
| LOW | pageindex/page_index.py | 586 | def process_toc_no_page_numbers(toc_content, toc_page_list, page_list, start_index=1, model=None, logger=None): | CODE |
| LOW | pageindex/page_index.py | 611 | def process_toc_with_page_numbers(toc_content, toc_page_list, page_list, toc_check_page_num=None, model=None, logger=Non | CODE |
| LOW | pageindex/page_index.py | 645 | def process_none_page_numbers(toc_items, page_list, start_index=1, model=None): | CODE |
| LOW | pageindex/page_index.py | 729 | async def single_toc_item_index_fixer(section_title, content, model=None): | CODE |
| LOW | pageindex/page_index.py | 870 | async def fix_incorrect_toc_with_retries(toc_with_page_number, page_list, incorrect_results, start_index=1, max_attempts | CODE |
| LOW | pageindex/page_index.py | 992 | async def process_large_node_recursively(node, page_list, opt=None, logger=None): | CODE |
| LOW | pageindex/page_index.py | 1116 | def validate_and_truncate_physical_indices(toc_with_page_number, page_list_length, start_index=1, logger=None): | CODE |
| LOW | pageindex/client.py | 18 | def _normalize_retrieve_model(model: str) -> str: | CODE |
| LOW | pageindex/utils.py | 249 | def get_first_start_page_from_text(text): | CODE |
| LOW | pageindex/utils.py | 256 | def get_last_start_page_from_text(text): | CODE |
| LOW | pageindex/utils.py | 421 | def get_text_of_pdf_pages_with_labels(pdf_pages, start_page, end_page): | CODE |
| LOW | pageindex/utils.py | 519 | def convert_physical_index_to_int(data): | CODE |
| LOW | pageindex/utils.py | 566 | def add_node_text_with_labels(node, pdf_pages): | CODE |
| LOW | pageindex/utils.py | 590 | async def generate_summaries_for_structure(structure, model=None): | STRING |
| LOW | pageindex/utils.py | 600 | def create_clean_structure_for_description(structure): | STRING |
| LOW | pageindex/page_index_md.py | 19 | async def generate_summaries_for_structure_md(structure, summary_token_threshold, model=None): | CODE |
| LOW | pageindex/page_index_md.py | 32 | def extract_nodes_from_markdown(markdown_content): | CODE |
| LOW | pageindex/page_index_md.py | 62 | def extract_node_text_content(node_list, markdown_lines): | CODE |
| LOW | pageindex/page_index_md.py | 89 | def update_node_list_with_text_token_count(node_list, model=None): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | README.md | 14 | # PageIndex: Vectorless, Reasoning-based RAG | COMMENT |
| MEDIUM⚡ | README.md | 33 | - 🔥 [**Agentic Vectorless RAG**](https://github.com/VectifyAI/PageIndex/blob/main/examples/agentic_vectorless_rag_demo.p | CODE |
| MEDIUM⚡ | README.md | 33 | - 🔥 [**Agentic Vectorless RAG**](https://github.com/VectifyAI/PageIndex/blob/main/examples/agentic_vectorless_rag_demo.p | CODE |
| MEDIUM⚡ | README.md | 36 | - [PageIndex Framework](https://pageindex.ai/blog/pageindex-intro) — Deep dive into PageIndex: an *agentic, in-context t | CODE |
| MEDIUM | README.md | 50 | Inspired by AlphaGo, we propose **[PageIndex](https://vectify.ai/pageindex)** — a **vectorless**, **reasoning-based RAG* | CODE |
| MEDIUM | README.md | 88 | - 🔥 [**Agentic Vectorless RAG**](examples/agentic_vectorless_rag_demo.py) *(latest)* — a simple but complete **agentic v | CODE |
| MEDIUM | README.md | 200 | ## 🚀 Agentic Vectorless RAG: An Example | COMMENT |
| MEDIUM | …es/workspace/12345678-abcd-4321-abcd-123456789abc.json | 223 | "content": "Attention ResidualsTECHNICALREPORT\nTable 2: Baseline vs Block AttnRes ( N= 8 ) vs Full AttnRes vs mHC | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | run_pageindex.py | 4 | CODE | |
| LOW | tests/test_issue_163.py | 4 | CODE | |
| LOW | examples/agentic_vectorless_rag_demo.py | 31 | CODE | |
| LOW | pageindex/page_index.py | 7 | CODE | |
| LOW | pageindex/page_index.py | 9 | CODE | |
| LOW | pageindex/page_index.py | 9 | CODE | |
| LOW | pageindex/__init__.py | 1 | CODE | |
| LOW | pageindex/__init__.py | 2 | CODE | |
| LOW | pageindex/__init__.py | 3 | CODE | |
| LOW | pageindex/__init__.py | 3 | CODE | |
| LOW | pageindex/__init__.py | 3 | CODE | |
| LOW | pageindex/__init__.py | 4 | CODE | |
| LOW | pageindex/page_index_md.py | 6 | CODE | |
| LOW | pageindex/page_index_md.py | 8 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/agentic_vectorless_rag_demo.py | 55 | CODE | |
| LOW | examples/agentic_vectorless_rag_demo.py | 89 | CODE | |
| LOW | pageindex/page_index.py | 330 | CODE | |
| LOW | pageindex/page_index.py | 368 | CODE | |
| LOW | pageindex/page_index.py | 645 | CODE | |
| LOW | pageindex/page_index.py | 685 | CODE | |
| LOW | pageindex/page_index.py | 1116 | CODE | |
| LOW | pageindex/client.py | 55 | CODE | |
| LOW | pageindex/utils.py | 33 | CODE | |
| LOW | pageindex/utils.py | 174 | CODE | |
| LOW | pageindex/utils.py | 192 | CODE | |
| LOW | pageindex/utils.py | 388 | CODE | |
| LOW | pageindex/utils.py | 519 | CODE | |
| LOW | pageindex/utils.py | 194 | CODE | |
| LOW | pageindex/page_index_md.py | 135 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | pageindex/utils.py | 268 | # In Linux, only '/' and '\0' (null) are invalid in filenames. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/agentic_vectorless_rag_demo.py | 158 | # Step 1: Index PDF and view tree structure | COMMENT |
| LOW⚡ | examples/agentic_vectorless_rag_demo.py | 175 | # Step 2: View document metadata | COMMENT |
| LOW⚡ | examples/agentic_vectorless_rag_demo.py | 182 | # Step 3: Agent Query | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/agentic_vectorless_rag_demo.py | 158 | # Step 1: Index PDF and view tree structure | COMMENT |
| LOW⚡ | examples/agentic_vectorless_rag_demo.py | 175 | # Step 2: View document metadata | COMMENT |
| LOW⚡ | examples/agentic_vectorless_rag_demo.py | 182 | # Step 3: Agent Query | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pageindex/page_index.py | 1087 | # Create a clean structure without unnecessary fields for description generation | COMMENT |
| MEDIUM | pageindex/page_index_md.py | 280 | # Create a clean structure without unnecessary fields for description generation | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pageindex/page_index.py | 763 | # Check if list_index is valid | COMMENT |
| LOW | pageindex/page_index.py | 817 | # Check if the result is correct | COMMENT |
| LOW | pageindex/utils.py | 213 | # Check if the node is a leaf node | COMMENT |
| LOW | pageindex/utils.py | 522 | # Check if item is a dictionary and has 'physical_index' key | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pageindex/utils.py | 100 | def extract_json(content): | CODE |
| LOW | pageindex/utils.py | 50 | except Exception as e: | CODE |
| LOW | pageindex/utils.py | 76 | except Exception as e: | CODE |
| LOW | pageindex/utils.py | 129 | except Exception as e: | CODE |
| LOW | pageindex/retrieve.py | 134 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pageindex/retrieve.py | 10 | # ── Helpers ────────────────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | pageindex/retrieve.py | 79 | # ── Tool functions ──────────────────────────────────────────────────────────── | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pageindex/page_index.py | 1105 | CODE | |
| LOW | pageindex/utils.py | 587 | CODE | |
| LOW | pageindex/utils.py | 632 | CODE | |
| LOW | pageindex/page_index_md.py | 243 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …es/workspace/12345678-abcd-4321-abcd-123456789abc.json | 263 | "content": "Attention ResidualsTECHNICALREPORT\n[52] Ashish Vaswani et al. “Attention is All you Need”. In:Advance | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .github/scripts/comment-on-duplicates.sh | 5 | # Usage: | COMMENT |