Repository Analysis

quickwit-oss/tantivy

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust

3.6 Likely human-written View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of quickwit-oss/tantivy, a Rust project with 15,549 GitHub stars. SynthScan v2.0 examined 153,407 lines of code across 490 source files, recording 493 pattern matches distributed across 7 syntactic categories. The overall adjusted score of 3.6 places this repository in the Likely human-written band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

3.6
Adjusted Score
3.6
Raw Score
100%
Time Factor
2026-07-13
Last Push
15.5K
Stars
Rust
Language
153.4K
Lines of Code
490
Files
493
Pattern Hits
2026-07-14
Scan Date
0.00
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 0MEDIUM 16LOW 477

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 493 distinct pattern matches across 7 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Over-Commented Block449 hits · 449 pts
SeverityFileLineSnippetContext
LOWcliff.toml61COMMENT
LOWcolumnar/src/dynamic_column.rs241 pub fn open(&self) -> io::Result<DynamicColumn> {COMMENT
LOWcolumnar/src/lib.rs1//! # Tantivy-ColumnarCOMMENT
LOWcolumnar/src/value.rs101 1 => Ok(NumericalType::U64),COMMENT
LOWcolumnar/src/dictionary.rs21COMMENT
LOWcolumnar/src/columnar/merge/mod.rs61/// If several columns with the same name are conflicting with the numerical types in theCOMMENT
LOWcolumnar/src/columnar/merge/tests.rs41 let columnar2 = make_columnar("numbers", &[2u64]);COMMENT
LOWcolumnar/src/column_index/multivalued_index.rs101 }COMMENT
LOWcolumnar/src/column_index/multivalued_index.rs241 }COMMENT
LOWcolumnar/src/column_index/multivalued_index.rs281 pub fn num_docs(&self) -> u32 {COMMENT
LOWcolumnar/src/column_index/mod.rs141 row_start..row_endCOMMENT
LOWcolumnar/src/column_index/optional_index/mod.rs41 pub fn num_bytes_in_block(&self) -> u32 {COMMENT
LOWcolumnar/src/column_index/optional_index/mod.rs61/// block]COMMENT
LOWcolumnar/src/column_values/monotonic_mapping.rs21 fn from_u64(val: u64) -> Self;COMMENT
LOWcolumnar/src/column_values/mod.rs41COMMENT
LOWcolumnar/src/column_values/mod.rs141 if value_range.contains(&val) {COMMENT
LOW…values/u128_based/compact_space/build_compact_space.rs101 let saved_bits = (amplitude_bits - amplitude_new_bits) as usize * total_num_values as usize;COMMENT
LOW…mnar/src/column_values/u128_based/compact_space/mod.rs1/// This codec takes a large number space (u128) and reduces it to a compact number space.COMMENT
LOWcolumnar/src/column_values/u64_based/line.rs41 // This is outside of realm we handle.COMMENT
LOWcolumnar/src/column_values/u64_based/line.rs81 return Line::default();COMMENT
LOWcolumnar/src/column_values/u64_based/line.rs101 // Without sorting our values, this is a difficult problem.COMMENT
LOWcolumnar/src/column_values/u64_based/mod.rs21use crate::iterable::Iterable;COMMENT
LOWquery-grammar/src/query_grammar.rs1061fn ast(inp: &str) -> IResult<&str, UserInputAst> {COMMENT
LOWsstable/src/block_match_automaton.rs41 for kb in &start_key[0..common_prefix_len] {COMMENT
LOWsstable/src/block_match_automaton.rs61 // e.* |COMMENT
LOWsstable/src/block_match_automaton.rs81 // pCOMMENT
LOWsstable/src/lib.rs1//! `tantivy_sstable` is a crate that provides a sorted string table data structure.COMMENT
LOWsstable/src/lib.rs21//! builder.insert(b"banana", &2).unwrap();COMMENT
LOWsstable/src/streamer.rs261 /// If the end of the stream as been reached, and `.next()`COMMENT
LOWsstable/src/dictionary.rs21/// to any kind of typed values.COMMENT
LOWsstable/src/dictionary.rs161 block_addr: BlockAddr,COMMENT
LOWsstable/src/dictionary.rs421 /// lower_bound: Bound::Included(aaa) => Included(0) // "Next" term idCOMMENT
LOWsstable/src/index/v3.rs561 max_slope_idx = index;COMMENT
LOWstacker/src/fastcmp.rs1/// fastcmp employs a trick to speed up the comparison of two slices of bytes.COMMENT
LOWstacker/src/arena_hashmap.rs1use super::{Addr, MemoryArena};COMMENT
LOWstacker/src/arena_hashmap.rs61 }COMMENT
LOWstacker/src/expull.rs1use std::mem;COMMENT
LOWstacker/src/expull.rs21/// It combines the idea of the unrolled linked list and tries to address theCOMMENT
LOWstacker/src/shared_arena_hashmap.rs41 fn is_empty(&self) -> bool {COMMENT
LOWstacker/src/shared_arena_hashmap.rs281 let v = memory_arena.read(val_addr);COMMENT
LOWstacker/src/memory_arena.rs1//! 32-bits Memory arena for types implementing `Copy`.COMMENT
LOWstacker/src/memory_arena.rs21//! access them as references.COMMENT
LOWtokenizer-api/src/lib.rs1//! Tokenizer are in charge of chopping text into a stream of tokensCOMMENT
LOWcommon/src/group_by.rs1use std::cell::RefCell;COMMENT
LOWcommon/src/lib.rs41 fn len(&self) -> usize {COMMENT
LOWcommon/src/lib.rs61///COMMENT
LOWcommon/src/lib.rs81/// For simplicity, tantivy internally handles `f64` as `u64`.COMMENT
LOWcommon/src/file_slice.rs281COMMENT
LOWcommon/src/datetime.rs21 Milliseconds,COMMENT
LOWcommon/src/bitset.rs161 // `trailing_zeros` and the bit-clear in parallel instead ofCOMMENT
LOWexamples/index_from_multiple_threads.rs1// # Indexing from different threads.COMMENT
LOWexamples/iterating_docs_and_positions.rs1// # Iterating docs and positions.COMMENT
LOWexamples/iterating_docs_and_positions.rs41 // (Because we indexed a very small number of documents over one threadCOMMENT
LOWexamples/iterating_docs_and_positions.rs101 // and the [`Postings`](https://docs.rs/tantivy/~0/tantivy/trait.Postings.html) traitCOMMENT
LOWexamples/deleting_updating_documents.rs1// # Deleting and Updating (?) documentsCOMMENT
LOWexamples/deleting_updating_documents.rs41COMMENT
LOWexamples/deleting_updating_documents.rs101 // # Update = Delete + InsertCOMMENT
LOWexamples/faceted_search.rs1// # Faceted SearchCOMMENT
LOWexamples/aggregation.rs21 // category, stock and price will be fast fields as that's the requirementCOMMENT
LOWexamples/aggregation.rs181 // In this Aggregation we want to get the average price for different groups, depending on howCOMMENT
389 more matches not shown…
Structural Annotation Overuse21 hits · 44 pts
SeverityFileLineSnippetContext
LOW.claude/skills/simple-pr/SKILL.md11## Step 1: Check workspace stateCOMMENT
LOW.claude/skills/simple-pr/SKILL.md19## Step 2: Ensure main is up to dateCOMMENT
LOW.claude/skills/simple-pr/SKILL.md25## Step 3: Review staged changesCOMMENT
LOW.claude/skills/simple-pr/SKILL.md31## Step 4: Generate commit messageCOMMENT
LOW.claude/skills/simple-pr/SKILL.md37## Step 5: Create a new branchCOMMENT
LOW.claude/skills/simple-pr/SKILL.md45## Step 6: Commit changesCOMMENT
LOW.claude/skills/simple-pr/SKILL.md52## Step 7: Push and open a PRCOMMENT
LOW.claude/skills/rationalize-deps/SKILL.md18## Step 1: Identify the targetCOMMENT
LOW.claude/skills/rationalize-deps/SKILL.md25## Step 2: Analyze current dependenciesCOMMENT
LOW.claude/skills/rationalize-deps/SKILL.md33## Step 3: For each candidate dependencyCOMMENT
LOW.claude/skills/rationalize-deps/SKILL.md76## Step 4: Document findingsCOMMENT
LOW.claude/skills/rationalize-deps/SKILL.md84## Step 5: Verify full buildCOMMENT
LOW.claude/skills/update-changelog/SKILL.md10## Step 1: Determine the changelog scopeCOMMENT
LOW.claude/skills/update-changelog/SKILL.md16## Step 2: Find merged PRs not yet in the changelogCOMMENT
LOW.claude/skills/update-changelog/SKILL.md26## Step 3: Consolidate related PRsCOMMENT
LOW.claude/skills/update-changelog/SKILL.md38## Step 4: Review the actual code diffCOMMENT
LOW.claude/skills/update-changelog/SKILL.md49## Step 5: Categorize each PR groupCOMMENT
LOW.claude/skills/update-changelog/SKILL.md61## Step 6: Format entriesCOMMENT
LOW.claude/skills/update-changelog/SKILL.md75## Step 7: Present changes to the userCOMMENT
LOW.claude/skills/update-changelog/SKILL.md79## Step 8: Update CHANGELOG.mdCOMMENT
LOW.claude/skills/update-changelog/SKILL.md85## Step 9: VerifyCOMMENT
Self-Referential Comments12 hits · 36 pts
SeverityFileLineSnippetContext
MEDIUMexamples/snippet.rs22 // # Defining the schemaCOMMENT
MEDIUMexamples/json_field.rs13 // # Defining the schemaCOMMENT
MEDIUMexamples/index_from_multiple_threads.rs36 // # Defining the schemaCOMMENT
MEDIUMexamples/deleting_updating_documents.rs43 // # Defining the schemaCOMMENT
MEDIUMexamples/basic_search.rs26 // # Defining the schemaCOMMENT
MEDIUMexamples/ip_field.rs12 // # Defining the schemaCOMMENT
MEDIUMexamples/custom_collector.rs123 // # Defining the schemaCOMMENT
MEDIUMexamples/custom_tokenizer.rs1// # Defining a tokenizer pipelineCOMMENT
MEDIUMexamples/custom_tokenizer.rs12 // # Defining the schemaCOMMENT
MEDIUMexamples/fuzzy_search.rs25 // # Defining the schemaCOMMENT
MEDIUMexamples/date_time_field.rs11 // # Defining the schemaCOMMENT
MEDIUMsrc/query/fuzzy_query.rs194 // # Defining the schemaCOMMENT
AI Slop Vocabulary3 hits · 9 pts
SeverityFileLineSnippetContext
MEDIUMsrc/aggregation/bucket/term_agg/term_histogram.rs233 // value count. (Essentially always true here: the column is full, so its value countCOMMENT
MEDIUMsrc/schema/mod.rs101//! Some queries may leverage Fast fields when run on a field that is not indexed. This can beCOMMENT
MEDIUMsrc/schema/text_options.rs192/// Essentially, should we store the term frequency and/or the positions (SeeCOMMENT
Fake / Example Data6 hits · 6 pts
SeverityFileLineSnippetContext
LOWsrc/functional_test.rs132const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod \CODE
LOWsrc/functional_test.rs132const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod \CODE
LOWsrc/indexer/index_writer.rs844 const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do \CODE
LOWsrc/indexer/index_writer.rs844 const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do \CODE
LOWsrc/store/mod.rs68 const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do \CODE
LOWsrc/store/mod.rs68 const LOREM: &str = "Doc Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do \CODE
Decorative Section Separators1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUMsrc/postings/recorder.rs408 // ── TermFrequencyRecorder ─────────────────────────────────────────────────COMMENT
Verbosity Indicators1 hit · 2 pts
SeverityFileLineSnippetContext
LOWsrc/query/phrase_query/phrase_scorer.rs570 // So the cost estimation would be the number of times we need to check if a doc is a hit *COMMENT