Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.
This report presents the forensic synthetic code analysis of castorini/pyserini, a Python project with 2,146 GitHub stars. SynthScan v2.0 examined 130,151 lines of code across 682 source files, recording 974 pattern matches distributed across 17 syntactic categories. The overall adjusted score of 9.7 places this repository in the Low AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 974 distinct pattern matches across 17 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/_jvm.py | 55 | def is_jvm_already_running_error(exception): | CODE |
| LOW | pyserini/_jvm.py | 65 | def suppress_jvm_startup_stderr(): | CODE |
| LOW | pyserini/util.py | 66 | def _archive_name_to_dir_name(archive_name): | CODE |
| LOW | pyserini/util.py | 165 | def compare_trec_strings_with_tolerance(trec_strings1, trec_strings2, tolerance=1e-4): | CODE |
| LOW | pyserini/util.py | 203 | def compare_trec_files_with_tolerance(file1_path, file2_path, tolerance=1e-4): | CODE |
| LOW | pyserini/util.py | 303 | def download_and_unpack_archive(url, output_dir='indexes', local_filename=False, md5=None, | CODE |
| LOW | pyserini/util.py | 465 | def download_evaluation_script(evaluation_name, force=False, verbose=True, mirror=None): | CODE |
| LOW | pyserini/prebuilt_index_info.py | 29 | def _import_from_lucene_prebuilt_inverted_index_json(index_metadata): | CODE |
| LOW | pyserini/2cr/m_beir.py | 67 | def print_results_by_metric_position(table, position, metric_name): | CODE |
| LOW | pyserini/2cr/_base.py | 34 | def run_eval_and_return_metric(metric, eval_key, defs, runfile, display_command=False): | CODE |
| LOW | pyserini/2cr/_base.py | 50 | def run_dpr_retrieval_eval_and_return_metric(defs, json_file): | CODE |
| LOW | pyserini/2cr/_base.py | 70 | def convert_trec_run_to_dpr_retrieval_json(topics,index,runfile,output): | CODE |
| LOW | pyserini/2cr/miracl.py | 221 | def extract_topic_fn_from_cmd(cmd): | CODE |
| LOW | pyserini/2cr/ciral.py | 159 | def extract_topic_fn_from_cmd(cmd): | CODE |
| LOW | pyserini/2cr/mmeb.py | 81 | def print_results_by_metric_position(table, position, metric_name): | CODE |
| LOW | pyserini/2cr/odqa.py | 93 | def format_hybrid_search_command(raw): | CODE |
| LOW | pyserini/encode/_dpr.py | 90 | def _normalize_legacy_wordpiece(text): | CODE |
| LOW | pyserini/encode/_dse.py | 57 | def patch_processor_for_batching(processor): | CODE |
| LOW | pyserini/encode/_dse.py | 64 | def patched_convert_images_texts_to_inputs(self, images, texts, padding=False, truncation=None, max_length=None, ret | CODE |
| LOW | pyserini/encode/_unicoil.py | 196 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/_slim.py | 89 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/_splade.py | 28 | def _from_pretrained_without_safetensors_conversion(model_name_or_path): | CODE |
| LOW | pyserini/encode/_splade.py | 53 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 24 | def _ensure_transformers_additional_special_token_ids(): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 40 | def _ensure_transformers_tied_weights_keys(): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 80 | def _ensure_transformers_compatibility(): | CODE |
| LOW | pyserini/server/document_format.py | 75 | def truncate_document_payload( | CODE |
| LOW | pyserini/server/backend.py | 254 | def _params_require_searcher_rebuild( | CODE |
| LOW | pyserini/server/backend.py | 412 | def _doc_store_lucene_searcher(self, start_index_name: str, *, allow_local_index: bool) -> LuceneSearcher: | CODE |
| LOW | pyserini/server/backend.py | 427 | def _bulk_fetch_and_format_documents( | CODE |
| LOW | pyserini/server/backend.py | 467 | def _resolve_mbeir_instruction_config(self, index_name: str) -> str | None: | CODE |
| LOW | pyserini/server/rest/app.py | 133 | def _sync_counts_from_hits_if_needed(self) -> None: | CODE |
| LOW | pyserini/server/rest/app.py | 223 | def _compute_token_fingerprint(token: str | None) -> str: | CODE |
| LOW | pyserini/server/rest/app.py | 244 | def _build_uvicorn_log_config( | CODE |
| LOW | pyserini/server/rest/app.py | 346 | async def rest_api_key_and_access_log(request: Request, call_next): | CODE |
| LOW⚡ | pyserini/server/rest/app.py | 422 | async def unhandled_exception_handler(request: Request, exc: Exception): | CODE |
| LOW⚡ | pyserini/server/rest/app.py | 427 | async def starlette_http_exception_handler(request: Request, exc: StarletteHTTPException): | CODE |
| LOW⚡ | pyserini/server/rest/app.py | 437 | async def validation_exception_handler(request: Request, exc: RequestValidationError): | CODE |
| LOW | pyserini/server/rest/routes/v1.py | 63 | def _parse_optional_positive_int(raw: str | None, name: str) -> tuple[int | None, JSONResponse | None]: | CODE |
| LOW | pyserini/search/_base.py | 68 | def _get_topics_and_qrels_cache_path(): | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 49 | def _reader_has_stored_raw_field(reader) -> bool: | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 373 | def _init_query_encoder_from_str(query_encoder): | CODE |
| LOW | pyserini/search/lucene/querybuilder.py | 47 | def get_boolean_query_builder(): | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 141 | def _normalize_to_unit_interval(self, distances: np.ndarray) -> np.ndarray: | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 509 | def _read_legacy_binary_idmap_index(index_path: str): | CODE |
| LOW | pyserini/index/lucene/_base.py | 509 | def compute_query_document_score(self, docid: str, query: str, similarity=None): | CODE |
| LOW | pyserini/index/lucene/_base.py | 515 | def convert_internal_docid_to_collection_docid(self, docid: int) -> str: | CODE |
| LOW | pyserini/index/lucene/_base.py | 530 | def convert_collection_docid_to_internal_docid(self, docid: str) -> int: | CODE |
| LOW | tests/core/test_hybrid_search.py | 44 | def test_hybrid_searcher_batch(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 207 | def test_reciprocal_rank_fusion_simple(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 216 | def test_interpolation_fusion_simple(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 225 | def test_average_fusion_simple(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 234 | def test_normalize_fusion_simple(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 243 | def test_reciprocal_rank_fusion_complex(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 259 | def test_lucene_flat_dense_rrf_fusion(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 262 | def test_lucene_flat_dense_interpolation_fusion(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 265 | def test_lucene_flat_dense_average_fusion(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 268 | def test_lucene_flat_dense_normalize_fusion(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 271 | def test_faiss_flat_dense_rrf_fusion_normalize_distances(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 274 | def test_faiss_flat_dense_interpolation_fusion_normalize_distances(self): | CODE |
| 261 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/_jvm.py | 1 | # | COMMENT |
| LOW | pyserini/encoded_corpus_info.py | 1 | # | COMMENT |
| LOW | pyserini/util.py | 1 | # | COMMENT |
| LOW | pyserini/output_writer.py | 1 | # | COMMENT |
| LOW | pyserini/external_query_info.py | 1 | # | COMMENT |
| LOW | pyserini/prebuilt_index_info.py | 1 | # | COMMENT |
| LOW | pyserini/query_iterator.py | 1 | # | COMMENT |
| LOW | pyserini/tokenize_json_collection.py | 1 | # | COMMENT |
| LOW | pyserini/encoded_query_info.py | 1 | # | COMMENT |
| LOW | pyserini/query_iterator_order_info.py | 1 | # | COMMENT |
| LOW | pyserini/evaluate_script_info.py | 1 | # | COMMENT |
| LOW | pyserini/pyclass.py | 1 | # | COMMENT |
| LOW | pyserini/multithreading.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/_base.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/__main__.py | 1 | # | COMMENT |
| LOW | pyserini/demo/miracl.py | 1 | # | COMMENT |
| LOW | pyserini/demo/acl.py | 1 | # | COMMENT |
| LOW | pyserini/demo/dpr.py | 1 | # | COMMENT |
| LOW | pyserini/demo/atomic.py | 1 | # | COMMENT |
| LOW | pyserini/demo/msmarco.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/m_beir.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/_base.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/miracl.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/ciral.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/mmeb.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/dse.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/atomic.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/bright.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/msmarco.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/beir.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/mrtydi.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/odqa.py | 1 | # | COMMENT |
| LOW | pyserini/collection/_base.py | 1 | # | COMMENT |
| LOW | pyserini/collection/_collection_support.py | 1 | # | COMMENT |
| LOW | pyserini/collection/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/analysis/_base.py | 1 | # | COMMENT |
| LOW | pyserini/analysis/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/vectorizer/_base.py | 1 | # | COMMENT |
| LOW | pyserini/vectorizer/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/trectools/_base.py | 1 | # | COMMENT |
| LOW | pyserini/trectools/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dkrr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_base.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_auto.py | 1 | # | COMMENT |
| LOW | pyserini/encode/query.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_clip.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_aggretriever.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_bpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_openai.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_tct_colbert.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_cosdpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dse.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_unicoil.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_cached_data.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_ance.py | 1 | # | COMMENT |
| LOW | pyserini/encode/merge_faiss_index.py | 1 | # | COMMENT |
| 223 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | pyserini/demo/miracl.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/demo/acl.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/demo/atomic.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/search/lucene/_impact_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/lucene/_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/faiss/_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/index/lucene/_base.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/lucene/_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/search/lucene/_hnsw_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/search/lucene/_geo_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/index/lucene/_base.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | scripts/tokenize_corpus.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_queries.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_corpus.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_queries.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| HIGH | scripts/beir/tokenize_corpus.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| HIGH | scripts/ltr_msmarco/convert_common.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 18 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/vectorizer/__init__.py | 17 | CODE | |
| LOW | pyserini/vectorizer/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/_base.py | 17 | CODE | |
| LOW | pyserini/trectools/_base.py | 18 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 24 | CODE | |
| LOW | pyserini/encode/__init__.py | 28 | CODE | |
| LOW | pyserini/encode/__init__.py | 29 | CODE | |
| LOW | pyserini/encode/__init__.py | 29 | CODE | |
| LOW | pyserini/encode/__init__.py | 30 | CODE | |
| LOW | pyserini/encode/__init__.py | 35 | CODE | |
| LOW | pyserini/encode/__init__.py | 36 | CODE | |
| LOW | pyserini/encode/__init__.py | 37 | CODE | |
| LOW | pyserini/encode/__init__.py | 39 | CODE | |
| LOW | pyserini/encode/__init__.py | 40 | CODE | |
| LOW | pyserini/encode/__init__.py | 40 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 23 | CODE | |
| LOW | pyserini/encode/optional/__init__.py | 17 | CODE | |
| LOW | pyserini/encode/optional/__init__.py | 18 | CODE | |
| LOW | pyserini/server/document_format.py | 23 | CODE | |
| LOW | pyserini/server/backend.py | 17 | CODE | |
| LOW | pyserini/server/config.py | 19 | CODE | |
| LOW | pyserini/server/mcp/extension.py | 17 | CODE | |
| LOW | pyserini/server/rest/app.py | 32 | CODE | |
| LOW | pyserini/server/rest/routes/v1.py | 24 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 28 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 28 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 30 | CODE | |
| 66 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 165 | CODE | |
| LOW | pyserini/util.py | 224 | CODE | |
| LOW | pyserini/util.py | 359 | CODE | |
| LOW | pyserini/util.py | 410 | CODE | |
| LOW | pyserini/prebuilt_index_info.py | 54 | CODE | |
| LOW | pyserini/query_iterator.py | 97 | CODE | |
| LOW | pyserini/query_iterator.py | 233 | CODE | |
| LOW | pyserini/query_iterator.py | 321 | CODE | |
| LOW | pyserini/demo/msmarco.py | 56 | CODE | |
| LOW | pyserini/2cr/m_beir.py | 90 | CODE | |
| LOW | pyserini/2cr/m_beir.py | 200 | CODE | |
| LOW | pyserini/2cr/miracl.py | 227 | CODE | |
| LOW | pyserini/2cr/miracl.py | 299 | CODE | |
| LOW | pyserini/2cr/ciral.py | 165 | CODE | |
| LOW | pyserini/2cr/ciral.py | 230 | CODE | |
| LOW | pyserini/2cr/mmeb.py | 106 | CODE | |
| LOW | pyserini/2cr/mmeb.py | 180 | CODE | |
| LOW | pyserini/2cr/dse.py | 55 | CODE | |
| LOW | pyserini/2cr/dse.py | 158 | CODE | |
| LOW | pyserini/2cr/atomic.py | 106 | CODE | |
| LOW | pyserini/2cr/atomic.py | 183 | CODE | |
| LOW | pyserini/2cr/bright.py | 98 | CODE | |
| LOW | pyserini/2cr/bright.py | 166 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 275 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 359 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 523 | CODE | |
| LOW | pyserini/2cr/beir.py | 121 | CODE | |
| LOW | pyserini/2cr/beir.py | 237 | CODE | |
| LOW | pyserini/2cr/mrtydi.py | 180 | CODE | |
| LOW | pyserini/2cr/mrtydi.py | 229 | CODE | |
| LOW | pyserini/2cr/odqa.py | 196 | CODE | |
| LOW | pyserini/2cr/odqa.py | 338 | CODE | |
| LOW | pyserini/analysis/_base.py | 54 | CODE | |
| LOW | pyserini/trectools/_base.py | 141 | CODE | |
| LOW | pyserini/encode/_base.py | 206 | CODE | |
| LOW | pyserini/encode/query.py | 45 | CODE | |
| LOW | pyserini/encode/_tct_colbert.py | 76 | CODE | |
| LOW | pyserini/encode/_unicoil.py | 118 | CODE | |
| LOW | pyserini/encode/_slim.py | 62 | CODE | |
| LOW | pyserini/server/backend.py | 196 | CODE | |
| LOW | pyserini/server/backend.py | 274 | CODE | |
| LOW | pyserini/server/backend.py | 303 | CODE | |
| LOW | pyserini/server/backend.py | 485 | CODE | |
| LOW | pyserini/server/backend.py | 576 | CODE | |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 66 | CODE | |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 237 | CODE | |
| LOW | pyserini/server/mcp/extension.py | 72 | CODE | |
| LOW | pyserini/server/rest/app.py | 292 | CODE | |
| LOW | pyserini/server/rest/app.py | 346 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 168 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 259 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 373 | CODE | |
| LOW | pyserini/search/lucene/_searcher.py | 464 | CODE | |
| LOW | pyserini/search/lucene/__main__.py | 34 | CODE | |
| LOW | pyserini/search/faiss/_searcher.py | 165 | CODE | |
| LOW | pyserini/search/faiss/_searcher.py | 353 | CODE | |
| LOW | pyserini/search/faiss/__main__.py | 249 | CODE | |
| LOW | pyserini/eval/evaluate_kilt_retrieval.py | 74 | CODE | |
| LOW | pyserini/eval/evaluate_kilt_retrieval.py | 104 | CODE | |
| LOW | pyserini/eval/evaluate_qa_overlap_retrieval.py | 251 | CODE | |
| 34 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/bright/dataset_analysis.py | 33 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 35 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 60 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 62 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 162 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 164 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 193 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 195 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 234 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 236 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 270 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 272 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 14 | # ------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 16 | # ------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 20 | # ------------------------------------------------------------ | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 114 | except Exception as e: | CODE |
| LOW | pyserini/query_iterator.py | 286 | except Exception as e: | CODE |
| LOW | pyserini/query_iterator.py | 305 | except Exception as e: | CODE |
| LOW | pyserini/pyclass.py | 36 | except Exception as e: | CODE |
| LOW | pyserini/encode/_base.py | 56 | except Exception: | CODE |
| LOW | pyserini/encode/_clip.py | 81 | except Exception as e: | CODE |
| MEDIUM | pyserini/encode/_clip.py | 82 | print(f"Error loading image {image}: {e}") | CODE |
| LOW | pyserini/encode/__init__.py | 49 | except Exception as e: | CODE |
| LOW | pyserini/encode/_openai.py | 41 | except Exception as e: | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 151 | except Exception as e: | CODE |
| LOW | pyserini/server/backend.py | 88 | except Exception: | CODE |
| LOW | pyserini/server/backend.py | 179 | except Exception: | CODE |
| LOW | pyserini/server/backend.py | 347 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 60 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 138 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 213 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 271 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/mcpyserini.py | 66 | except Exception as e: | CODE |
| LOW | pyserini/server/rest/app.py | 409 | except Exception: | CODE |
| LOW | pyserini/search/lucene/rerank/reranker_base.py | 34 | except Exception: | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 324 | except Exception as e: | CODE |
| LOW | pyserini/index/lucene/_base.py | 369 | except Exception as e: | CODE |
| MEDIUM | pyserini/index/lucene/_base.py | 118 | def _get_generator(self): | CODE |
| MEDIUM | scripts/run_jobs_with_load.py | 38 | print(f"Error executing the command: {e}") | CODE |
| LOW | scripts/kilt/anserini_retriever.py | 109 | except Exception as e: | CODE |
| LOW | …ripts/verify_encoded_queries/verify_encoded_queries.py | 537 | except Exception as e: | CODE |
| LOW | scripts/dse/download_slidevqa_corpus.py | 46 | except Exception: | CODE |
| LOW | scripts/dse/download_slidevqa_corpus.py | 130 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 43 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/pyclass.py | 55 | __all__ = [ | CODE |
| LOW | pyserini/collection/_base.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/trectools/__init__.py | 19 | __all__ = ['AggregationMethod', 'RescoreMethod', 'TrecRun', 'Qrels'] | CODE |
| LOW | pyserini/encode/_dse.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/backend.py | 41 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/mcp/tools.py | 31 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/mcp/extension.py | 36 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/rest/app.py | 57 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/rest/app.py | 552 | __all__ = ['RestBackpressure', 'app', 'create_app', 'main', 'VERSION', 'API_VERSION'] | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_searcher.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_hnsw_searcher.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_geo_searcher.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/querybuilder.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 50 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/index/lucene/_base.py | 36 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/index/lucene/_indexer.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/eval/evaluate_qa_overlap_retrieval.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/eval/evaluate_dpr_retrieval.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/dse/evaluate_wiki_ss_run.py | 12 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 238 | # Check to see if file already exists, if so, simply return (quietly) unless force=True, in which case we remove | COMMENT |
| LOW | pyserini/2cr/miracl.py | 394 | # If results are within 0.0005, just call it "OKish". | COMMENT |
| LOW | pyserini/2cr/atomic.py | 233 | # If results are within 0.0005, just call it "OKish". | COMMENT |
| LOW | pyserini/2cr/bright.py | 215 | # If results are within 0.005, just call it "OKish". | COMMENT |
| LOW | pyserini/2cr/msmarco.py | 579 | # If results are within 0.0005, just call it "OKish". | COMMENT |
| LOW | pyserini/2cr/beir.py | 290 | # If results are within 0.0005, just call it "OKish". | COMMENT |
| LOW | pyserini/2cr/mrtydi.py | 279 | # If results are within 0.0005, just call it "OKish". | COMMENT |
| LOW | pyserini/eval/trec_eval.py | 137 | # Obvious question here: why we *not* just call the trec_eval main (Java) class, which already wraps the executable? | COMMENT |
| MEDIUM | pyserini/eval/evaluate_kilt_retrieval.py | 135 | # 2. check what's the minimum number of predicted pages needed to get a robust P/R@k | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | pyserini/trectools/_base.py | 253 | # Step 1: Collect all dataframes from all runs | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 263 | # Step 2: Concatenate all dataframes at once (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 266 | # Step 3: Group by topic and docid, sum scores (vectorized aggregation) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 269 | # Step 4: Sort by topic (ascending), score (descending), docid (ascending for tie-breaking) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 274 | # Step 5: Apply k limit per topic if specified (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 278 | # Step 6: Add required columns and assign ranks (vectorized) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | pyserini/trectools/_base.py | 253 | # Step 1: Collect all dataframes from all runs | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 263 | # Step 2: Concatenate all dataframes at once (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 266 | # Step 3: Group by topic and docid, sum scores (vectorized aggregation) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 269 | # Step 4: Sort by topic (ascending), score (descending), docid (ascending for tie-breaking) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 274 | # Step 5: Apply k limit per topic if specified (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 278 | # Step 6: Add required columns and assign ranks (vectorized) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/es/create_es_ann.py | 19 | # Create the client instance | COMMENT |
| MEDIUM | scripts/es/create_es.py | 18 | # Create the client instance | COMMENT |
| MEDIUM | scripts/es/check_status.py | 18 | # Create the client instance | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/server/backend.py | 664 | CODE | |
| LOW | pyserini/server/mcp/tools.py | 39 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 299 | CODE | |
| LOW | pyserini/search/lucene/_searcher.py | 511 | CODE | |
| LOW | scripts/beir/compare_domains.py | 52 | CODE | |
| LOW | scripts/beir/compare_domains.py | 66 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pyserini/server/mcp/mcp-client/README.md | 2 | # MCP Client for RAG | COMMENT |
| MEDIUM | docs/experiments-msmarco-v2.1.md | 14 | ## Batch Runs on TREC 2024 RAG Topics | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/eval/trec_eval.py | 17 | # Example usage: | COMMENT |
| LOW | scripts/msmarco_v2/generate_train_triplet.py | 19 | # Usage: | COMMENT |
| LOW | scripts/msmarco_v2/convert_passage_run_to_doc_run.py | 23 | # Usage: | COMMENT |
| LOW | scripts/msmarco_v2/build_passage_to_doc_id_map.py | 19 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/dse/download_slidevqa_corpus.py | 70 | # Check if page exists and is not None | COMMENT |
| LOW | scripts/dse/download_slidevqa_corpus.py | 82 | # Check if we've already processed this page | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/base/encoder/test_encoder_model_arctic.py | 45 | #TODO: Implement test_arctic_query_encoder after beir regression incorporated | COMMENT |