Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.
This report summarizes scanner evidence for castorini/pyserini, a Python project with 2,170 GitHub stars. SynthScan examined 133,646 lines of code across 687 source files, recording 1049 pattern matches distributed across 18 syntactic categories. The legacy adjusted pattern signal is 10.1 (Moderate pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 803 finding(s).
Are imports declared and package/API names verified? 128 finding(s).
Are wrappers, guards, and repeated structures necessary? 510 finding(s).
Which changes need focused human review? 1049 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 1049 distinct pattern matches across 18 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/_jvm.py | 55 | def is_jvm_already_running_error(exception): | CODE |
| LOW | pyserini/_jvm.py | 65 | def suppress_jvm_startup_stderr(): | CODE |
| LOW | pyserini/util.py | 109 | def _archive_name_to_dir_name(archive_name): | CODE |
| LOW | pyserini/util.py | 208 | def compare_trec_strings_with_tolerance(trec_strings1, trec_strings2, tolerance=1e-4): | CODE |
| LOW | pyserini/util.py | 246 | def compare_trec_files_with_tolerance(file1_path, file2_path, tolerance=1e-4): | CODE |
| LOW | pyserini/util.py | 346 | def _rewrite_mbeir_instruction_config(instruction_config_path, instructions_dir): | CODE |
| LOW | pyserini/util.py | 373 | def get_mbeir_instruction_config(instr_file=None): | CODE |
| LOW | pyserini/util.py | 405 | def download_and_unpack_archive(url, output_dir='indexes', local_filename=False, md5=None, | CODE |
| LOW | pyserini/prebuilt_index_info.py | 29 | def _import_from_lucene_prebuilt_inverted_index_json(index_metadata): | CODE |
| LOW | pyserini/2cr/m_beir.py | 55 | def print_results_by_metric_position(table, position, metric_name): | CODE |
| LOW | pyserini/2cr/_base.py | 39 | def compare_reproduction_score(observed: float, expected: float) -> ReproductionStatus: | CODE |
| LOW | pyserini/2cr/_base.py | 60 | def format_reproduction_status(status: ReproductionStatus, expected: float, *, precision: int = 4) -> str: | CODE |
| LOW | pyserini/2cr/_base.py | 103 | def run_eval_and_return_metric(metric, eval_key, defs, runfile, display_command=False): | CODE |
| LOW | pyserini/2cr/_base.py | 120 | def run_dpr_retrieval_eval_and_return_metric(defs, json_file): | CODE |
| LOW | pyserini/2cr/_base.py | 141 | def convert_trec_run_to_dpr_retrieval_json(topics,index,runfile,output): | CODE |
| LOW | pyserini/2cr/miracl.py | 206 | def extract_topic_fn_from_cmd(cmd): | CODE |
| LOW | pyserini/2cr/ciral.py | 144 | def extract_topic_fn_from_cmd(cmd): | CODE |
| LOW | pyserini/2cr/mmeb.py | 67 | def print_results_by_metric_position(table, position, metric_name): | CODE |
| LOW | pyserini/2cr/odqa.py | 84 | def format_hybrid_search_command(raw): | CODE |
| LOW | pyserini/encode/_dpr.py | 90 | def _normalize_legacy_wordpiece(text): | CODE |
| LOW | pyserini/encode/_dse.py | 57 | def patch_processor_for_batching(processor): | CODE |
| LOW | pyserini/encode/_dse.py | 64 | def patched_convert_images_texts_to_inputs(self, images, texts, padding=False, truncation=None, max_length=None, ret | CODE |
| LOW | pyserini/encode/_unicoil.py | 196 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/_slim.py | 89 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/_splade.py | 28 | def _from_pretrained_without_safetensors_conversion(model_name_or_path): | CODE |
| LOW | pyserini/encode/_splade.py | 53 | def _get_encoded_query_token_wight_dicts(self, tok_weights): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 24 | def _ensure_transformers_additional_special_token_ids(): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 40 | def _ensure_transformers_tied_weights_keys(): | CODE |
| LOW | pyserini/encode/optional/_uniir.py | 80 | def _ensure_transformers_compatibility(): | CODE |
| LOW | pyserini/server/document_format.py | 75 | def truncate_document_payload( | CODE |
| LOW | pyserini/server/backend.py | 254 | def _params_require_searcher_rebuild( | CODE |
| LOW | pyserini/server/backend.py | 412 | def _doc_store_lucene_searcher(self, start_index_name: str, *, allow_local_index: bool) -> LuceneSearcher: | CODE |
| LOW | pyserini/server/backend.py | 427 | def _bulk_fetch_and_format_documents( | CODE |
| LOW | pyserini/server/backend.py | 467 | def _resolve_mbeir_instruction_config(self, index_name: str) -> str | None: | CODE |
| LOW | pyserini/server/config.py | 237 | def load_token_issuance_config(config_path: str | None) -> TokenIssuanceConfig | None: | CODE |
| LOW | pyserini/server/rest/app.py | 187 | def _sync_counts_from_hits_if_needed(self) -> None: | CODE |
| LOW | pyserini/server/rest/app.py | 329 | def _compute_token_fingerprint(token: str | None) -> str: | CODE |
| LOW | pyserini/server/rest/app.py | 373 | def _build_uvicorn_log_config( | CODE |
| LOW | pyserini/server/rest/app.py | 498 | async def rest_api_key_and_access_log(request: Request, call_next): | CODE |
| LOW | pyserini/server/rest/app.py | 578 | async def unhandled_exception_handler(request: Request, exc: Exception): | CODE |
| LOW | pyserini/server/rest/app.py | 583 | async def starlette_http_exception_handler(request: Request, exc: StarletteHTTPException): | CODE |
| LOW | pyserini/server/rest/app.py | 595 | async def validation_exception_handler(request: Request, exc: RequestValidationError): | CODE |
| LOW | pyserini/server/rest/routes/v1.py | 67 | def _parse_optional_positive_int(raw: str | None, name: str) -> tuple[int | None, JSONResponse | None]: | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 49 | def _reader_has_stored_raw_field(reader) -> bool: | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 373 | def _init_query_encoder_from_str(query_encoder): | CODE |
| LOW | pyserini/search/lucene/querybuilder.py | 47 | def get_boolean_query_builder(): | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 141 | def _normalize_to_unit_interval(self, distances: np.ndarray) -> np.ndarray: | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 509 | def _read_legacy_binary_idmap_index(index_path: str): | CODE |
| LOW | pyserini/index/lucene/_base.py | 509 | def compute_query_document_score(self, docid: str, query: str, similarity=None): | CODE |
| LOW | pyserini/index/lucene/_base.py | 515 | def convert_internal_docid_to_collection_docid(self, docid: int) -> str: | CODE |
| LOW | pyserini/index/lucene/_base.py | 530 | def convert_collection_docid_to_internal_docid(self, docid: str) -> int: | CODE |
| LOW | pyserini/eval/msmarco_doc_eval_raw.py | 31 | def load_reference_from_stream(f): | CODE |
| LOW | pyserini/eval/msmarco_doc_eval_raw.py | 61 | def validate_candidate_has_enough_ranking(qid_to_ranked_candidate_documents): | CODE |
| LOW | pyserini/eval/msmarco_doc_eval_raw.py | 67 | def load_candidate_from_stream(f): | CODE |
| LOW | pyserini/eval/msmarco_doc_eval_raw.py | 167 | def compute_metrics_from_files(path_to_reference, path_to_candidate, exclude_qids, perform_checks=True): | CODE |
| LOW | pyserini/eval/msmarco_passage_eval_raw.py | 18 | def load_reference_from_stream(f): | CODE |
| LOW | pyserini/eval/msmarco_passage_eval_raw.py | 46 | def load_candidate_from_stream(f): | CODE |
| LOW | pyserini/eval/msmarco_passage_eval_raw.py | 140 | def compute_metrics_from_files(path_to_reference, path_to_candidate, perform_checks=True): | CODE |
| LOW | tests/core/test_hybrid_search.py | 44 | def test_hybrid_searcher_batch(self): | CODE |
| LOW⚡ | tests/core/test_fusion.py | 240 | def test_reciprocal_rank_fusion_simple(self): | CODE |
| 323 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/_jvm.py | 1 | # | COMMENT |
| LOW | pyserini/encoded_corpus_info.py | 1 | # | COMMENT |
| LOW | pyserini/util.py | 1 | # | COMMENT |
| LOW | pyserini/output_writer.py | 1 | # | COMMENT |
| LOW | pyserini/external_query_info.py | 1 | # | COMMENT |
| LOW | pyserini/prebuilt_index_info.py | 1 | # | COMMENT |
| LOW | pyserini/query_iterator.py | 1 | # | COMMENT |
| LOW | pyserini/tokenize_json_collection.py | 1 | # | COMMENT |
| LOW | pyserini/encoded_query_info.py | 1 | # | COMMENT |
| LOW | pyserini/query_iterator_order_info.py | 1 | # | COMMENT |
| LOW | pyserini/pyclass.py | 1 | # | COMMENT |
| LOW | pyserini/multithreading.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/_base.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/fusion/__main__.py | 1 | # | COMMENT |
| LOW | pyserini/demo/miracl.py | 1 | # | COMMENT |
| LOW | pyserini/demo/acl.py | 1 | # | COMMENT |
| LOW | pyserini/demo/dpr.py | 1 | # | COMMENT |
| LOW | pyserini/demo/atomic.py | 1 | # | COMMENT |
| LOW | pyserini/demo/msmarco.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/m_beir.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/_base.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/miracl.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/ciral.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/mmeb.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/dse.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/atomic.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/bright.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/msmarco.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/beir.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/mrtydi.py | 1 | # | COMMENT |
| LOW | pyserini/2cr/odqa.py | 1 | # | COMMENT |
| LOW | pyserini/collection/_base.py | 1 | # | COMMENT |
| LOW | pyserini/collection/_collection_support.py | 1 | # | COMMENT |
| LOW | pyserini/collection/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/analysis/_base.py | 1 | # | COMMENT |
| LOW | pyserini/analysis/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/vectorizer/_base.py | 1 | # | COMMENT |
| LOW | pyserini/vectorizer/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/trectools/_base.py | 1 | # | COMMENT |
| LOW | pyserini/trectools/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dkrr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_base.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_auto.py | 1 | # | COMMENT |
| LOW | pyserini/encode/query.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_clip.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_aggretriever.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_bpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/__init__.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_openai.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_tct_colbert.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_cosdpr.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_dse.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_unicoil.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_cached_data.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_ance.py | 1 | # | COMMENT |
| LOW | pyserini/encode/merge_faiss_index.py | 1 | # | COMMENT |
| LOW | pyserini/encode/_slim.py | 1 | # | COMMENT |
| 223 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | pyserini/demo/miracl.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/demo/acl.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/demo/atomic.py | 0 | this script provides an interactive web interface demo for retrieval on the atomic dataset. it requires `flask` (`pip in | STRING |
| HIGH | pyserini/search/lucene/_impact_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/lucene/_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/faiss/_searcher.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/index/lucene/_base.py | 0 | display information about available prebuilt indexes. | STRING |
| HIGH | pyserini/search/lucene/_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/search/lucene/_hnsw_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/search/lucene/_geo_searcher.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/index/lucene/_base.py | 0 | wrapper class for ``indexreaderutils`` in anserini. parameters ---------- index_dir : str path to lucene index directory | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns the original text (with whitespace reinserted). | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of the text of each token args: uncased: lower cases text | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of [start, end) character offsets of each token. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of named-entity-recognition tags of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of the lemmatized text of each token. returns none if this annotation was not included. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | returns a list of all ngrams from length 1 to n. args: n: upper limit of ngram length uncased: lower cases text filter_f | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | group consecutive entity tokens with the same ner tag. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | base tokenizer class. tokenizers implement tokenize, which should return a tokens class. | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | args: annotators: none or empty set (only tokenizes). | STRING |
| HIGH | pyserini/eval/evaluate_qa_overlap_retrieval.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | pyserini/eval/evaluate_dpr_retrieval.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | scripts/dse/evaluate_wiki_ss_run.py | 0 | test if a regex pattern is contained within a text. | STRING |
| HIGH | scripts/tokenize_corpus.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_queries.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_corpus.py | 0 | replace original contents fields with bert tokenization | STRING |
| HIGH | scripts/beir/tokenize_queries.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| HIGH | scripts/beir/tokenize_corpus.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| HIGH | scripts/ltr_msmarco/convert_common.py | 0 | obtain a space separated re-tokenized text. :param tokenizer: a tokenizer that has the function tokenize that returns an | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/fusion/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 17 | CODE | |
| LOW | pyserini/collection/__init__.py | 18 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/analysis/__init__.py | 17 | CODE | |
| LOW | pyserini/vectorizer/__init__.py | 17 | CODE | |
| LOW | pyserini/vectorizer/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/_base.py | 17 | CODE | |
| LOW | pyserini/trectools/_base.py | 18 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/trectools/__init__.py | 17 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 20 | CODE | |
| LOW | pyserini/encode/__init__.py | 24 | CODE | |
| LOW | pyserini/encode/__init__.py | 28 | CODE | |
| LOW | pyserini/encode/__init__.py | 29 | CODE | |
| LOW | pyserini/encode/__init__.py | 29 | CODE | |
| LOW | pyserini/encode/__init__.py | 30 | CODE | |
| LOW | pyserini/encode/__init__.py | 35 | CODE | |
| LOW | pyserini/encode/__init__.py | 36 | CODE | |
| LOW | pyserini/encode/__init__.py | 37 | CODE | |
| LOW | pyserini/encode/__init__.py | 39 | CODE | |
| LOW | pyserini/encode/__init__.py | 40 | CODE | |
| LOW | pyserini/encode/__init__.py | 40 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 21 | CODE | |
| LOW | pyserini/encode/_dse.py | 23 | CODE | |
| LOW | pyserini/encode/optional/__init__.py | 17 | CODE | |
| LOW | pyserini/encode/optional/__init__.py | 18 | CODE | |
| LOW | pyserini/server/document_format.py | 23 | CODE | |
| LOW | pyserini/server/backend.py | 17 | CODE | |
| LOW | pyserini/server/config.py | 19 | CODE | |
| LOW | pyserini/server/token_delivery.py | 19 | CODE | |
| LOW | pyserini/server/mcp/extension.py | 17 | CODE | |
| LOW | pyserini/server/rest/app.py | 34 | CODE | |
| LOW | pyserini/server/rest/routes/v1.py | 24 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/__init__.py | 17 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 28 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 28 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| LOW | pyserini/search/lucene/__init__.py | 29 | CODE | |
| 68 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 208 | CODE | |
| LOW | pyserini/util.py | 267 | CODE | |
| LOW | pyserini/util.py | 461 | CODE | |
| LOW | pyserini/util.py | 512 | CODE | |
| LOW | pyserini/prebuilt_index_info.py | 54 | CODE | |
| LOW | pyserini/query_iterator.py | 97 | CODE | |
| LOW | pyserini/query_iterator.py | 233 | CODE | |
| LOW | pyserini/query_iterator.py | 321 | CODE | |
| LOW | pyserini/demo/msmarco.py | 56 | CODE | |
| LOW | pyserini/2cr/m_beir.py | 78 | CODE | |
| LOW | pyserini/2cr/m_beir.py | 182 | CODE | |
| LOW | pyserini/2cr/miracl.py | 212 | CODE | |
| LOW | pyserini/2cr/miracl.py | 283 | CODE | |
| LOW | pyserini/2cr/ciral.py | 150 | CODE | |
| LOW | pyserini/2cr/ciral.py | 219 | CODE | |
| LOW | pyserini/2cr/mmeb.py | 92 | CODE | |
| LOW | pyserini/2cr/mmeb.py | 163 | CODE | |
| LOW | pyserini/2cr/dse.py | 51 | CODE | |
| LOW | pyserini/2cr/dse.py | 143 | CODE | |
| LOW | pyserini/2cr/atomic.py | 96 | CODE | |
| LOW | pyserini/2cr/atomic.py | 172 | CODE | |
| LOW | pyserini/2cr/bright.py | 80 | CODE | |
| LOW | pyserini/2cr/bright.py | 147 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 281 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 357 | CODE | |
| LOW | pyserini/2cr/msmarco.py | 510 | CODE | |
| LOW | pyserini/2cr/beir.py | 103 | CODE | |
| LOW | pyserini/2cr/beir.py | 218 | CODE | |
| LOW | pyserini/2cr/mrtydi.py | 168 | CODE | |
| LOW | pyserini/2cr/mrtydi.py | 216 | CODE | |
| LOW | pyserini/2cr/odqa.py | 179 | CODE | |
| LOW | pyserini/2cr/odqa.py | 321 | CODE | |
| LOW | pyserini/analysis/_base.py | 54 | CODE | |
| LOW | pyserini/trectools/_base.py | 141 | CODE | |
| LOW | pyserini/encode/_base.py | 206 | CODE | |
| LOW | pyserini/encode/query.py | 45 | CODE | |
| LOW | pyserini/encode/_tct_colbert.py | 76 | CODE | |
| LOW | pyserini/encode/_unicoil.py | 118 | CODE | |
| LOW | pyserini/encode/_slim.py | 62 | CODE | |
| LOW | pyserini/server/backend.py | 196 | CODE | |
| LOW | pyserini/server/backend.py | 274 | CODE | |
| LOW | pyserini/server/backend.py | 303 | CODE | |
| LOW | pyserini/server/backend.py | 471 | CODE | |
| LOW | pyserini/server/backend.py | 562 | CODE | |
| LOW | pyserini/server/config.py | 379 | CODE | |
| LOW | pyserini/server/config.py | 431 | CODE | |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 66 | CODE | |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 237 | CODE | |
| LOW | pyserini/server/mcp/extension.py | 72 | CODE | |
| LOW | pyserini/server/rest/app.py | 421 | CODE | |
| LOW | pyserini/server/rest/app.py | 498 | CODE | |
| LOW | pyserini/server/rest/app.py | 617 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 168 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 259 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 373 | CODE | |
| LOW | pyserini/search/lucene/_searcher.py | 464 | CODE | |
| LOW | pyserini/search/lucene/__main__.py | 34 | CODE | |
| LOW | pyserini/search/faiss/_searcher.py | 165 | CODE | |
| LOW | pyserini/search/faiss/_searcher.py | 353 | CODE | |
| LOW | pyserini/search/faiss/__main__.py | 249 | CODE | |
| 41 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/bright/dataset_analysis.py | 33 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 35 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 60 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 62 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 162 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 164 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 193 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 195 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 234 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 236 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 270 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM | scripts/bright/dataset_analysis.py | 272 | # ------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 14 | # ------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 16 | # ------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | scripts/bright/dataset_analysis.sh | 20 | # ------------------------------------------------------------ | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 157 | except Exception as e: | CODE |
| LOW | pyserini/util.py | 395 | except Exception as e: | CODE |
| LOW | pyserini/query_iterator.py | 286 | except Exception as e: | CODE |
| LOW | pyserini/query_iterator.py | 305 | except Exception as e: | CODE |
| LOW | pyserini/pyclass.py | 36 | except Exception as e: | CODE |
| LOW | pyserini/encode/_base.py | 56 | except Exception: | CODE |
| LOW | pyserini/encode/_clip.py | 81 | except Exception as e: | CODE |
| MEDIUM | pyserini/encode/_clip.py | 82 | print(f"Error loading image {image}: {e}") | CODE |
| LOW | pyserini/encode/__init__.py | 49 | except Exception as e: | CODE |
| LOW | pyserini/encode/_openai.py | 41 | except Exception as e: | CODE |
| LOW | pyserini/server/backend.py | 88 | except Exception: | CODE |
| LOW | pyserini/server/backend.py | 179 | except Exception: | CODE |
| LOW | pyserini/server/backend.py | 347 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 60 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 138 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 213 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/pyserini_bridge.py | 271 | except Exception as e: | CODE |
| LOW | pyserini/server/mcp/mcpyserini.py | 66 | except Exception as e: | CODE |
| LOW | pyserini/server/rest/app.py | 565 | except Exception: | CODE |
| LOW | pyserini/server/rest/app.py | 668 | except Exception: | CODE |
| LOW | pyserini/search/lucene/rerank/reranker_base.py | 34 | except Exception: | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 324 | except Exception as e: | CODE |
| LOW | pyserini/index/lucene/_base.py | 369 | except Exception as e: | CODE |
| MEDIUM | pyserini/index/lucene/_base.py | 118 | def _get_generator(self): | CODE |
| MEDIUM | scripts/run_jobs_with_load.py | 38 | print(f"Error executing the command: {e}") | CODE |
| LOW | scripts/kilt/anserini_retriever.py | 109 | except Exception as e: | CODE |
| LOW | …ripts/verify_encoded_queries/verify_encoded_queries.py | 537 | except Exception as e: | CODE |
| LOW | scripts/dse/download_slidevqa_corpus.py | 46 | except Exception: | CODE |
| LOW | scripts/dse/download_slidevqa_corpus.py | 130 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | pyserini/server/config.py | 75 | except OSError: | CODE |
| HIGH | pyserini/server/config.py | 85 | except FileNotFoundError: | CODE |
| HIGH | pyserini/server/config.py | 70 | except PermissionError: | CODE |
| HIGH | pyserini/eval/trec_eval.py | 50 | except: | CODE |
| HIGH | scripts/dse/download_wiki_ss_corpus.py | 66 | except: | CODE |
| HIGH | scripts/dse/download_slidevqa_corpus.py | 46 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 44 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/pyclass.py | 55 | __all__ = [ | CODE |
| LOW | pyserini/collection/_base.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/trectools/__init__.py | 19 | __all__ = ['AggregationMethod', 'RescoreMethod', 'TrecRun', 'Qrels'] | CODE |
| LOW | pyserini/encode/_dse.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/backend.py | 41 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/mcp/tools.py | 31 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/mcp/extension.py | 36 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/rest/app.py | 72 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/server/rest/app.py | 813 | __all__ = ['RestBackpressure', 'app', 'create_app', 'main', 'VERSION', 'API_VERSION'] | CODE |
| LOW | pyserini/search/lucene/_impact_searcher.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_searcher.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_hnsw_searcher.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/_geo_searcher.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/lucene/querybuilder.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/search/faiss/_searcher.py | 50 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/index/lucene/_base.py | 36 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/index/lucene/_indexer.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/eval/evaluate_qa_overlap_retrieval.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | pyserini/eval/evaluate_dpr_retrieval.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | scripts/dse/evaluate_wiki_ss_run.py | 12 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | pyserini/trectools/_base.py | 253 | # Step 1: Collect all dataframes from all runs | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 263 | # Step 2: Concatenate all dataframes at once (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 266 | # Step 3: Group by topic and docid, sum scores (vectorized aggregation) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 269 | # Step 4: Sort by topic (ascending), score (descending), docid (ascending for tie-breaking) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 274 | # Step 5: Apply k limit per topic if specified (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 278 | # Step 6: Add required columns and assign ranks (vectorized) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | pyserini/trectools/_base.py | 253 | # Step 1: Collect all dataframes from all runs | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 263 | # Step 2: Concatenate all dataframes at once (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 266 | # Step 3: Group by topic and docid, sum scores (vectorized aggregation) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 269 | # Step 4: Sort by topic (ascending), score (descending), docid (ascending for tie-breaking) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 274 | # Step 5: Apply k limit per topic if specified (vectorized) | COMMENT |
| LOW⚡ | pyserini/trectools/_base.py | 278 | # Step 6: Add required columns and assign ranks (vectorized) | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/server/backend.py | 650 | CODE | |
| LOW | pyserini/server/mcp/tools.py | 39 | CODE | |
| LOW | pyserini/server/rest/app.py | 421 | CODE | |
| LOW | pyserini/server/rest/routes/v1.py | 97 | CODE | |
| LOW | pyserini/server/rest/routes/v1.py | 181 | CODE | |
| LOW | pyserini/search/lucene/_impact_searcher.py | 299 | CODE | |
| LOW | pyserini/search/lucene/_searcher.py | 511 | CODE | |
| LOW | scripts/beir/compare_domains.py | 52 | CODE | |
| LOW | scripts/beir/compare_domains.py | 66 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | scripts/es/create_es_ann.py | 19 | # Create the client instance | COMMENT |
| MEDIUM | scripts/es/create_es.py | 18 | # Create the client instance | COMMENT |
| MEDIUM | scripts/es/check_status.py | 18 | # Create the client instance | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/util.py | 281 | # Check to see if file already exists, if so, simply return (quietly) unless force=True, in which case we remove | COMMENT |
| LOW | pyserini/eval/trec_eval.py | 137 | # Obvious question here: why we *not* just call the trec_eval main (Java) class, which already wraps the executable? | COMMENT |
| MEDIUM | pyserini/eval/evaluate_kilt_retrieval.py | 135 | # 2. check what's the minimum number of predicted pages needed to get a robust P/R@k | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pyserini/server/mcp/mcp-client/README.md | 2 | # MCP Client for RAG | COMMENT |
| MEDIUM | docs/experiments-msmarco-v2.1.md | 14 | ## Batch Runs on TREC 2024 RAG Topics | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyserini/eval/trec_eval.py | 17 | # Example usage: | COMMENT |
| LOW | scripts/msmarco_v2/generate_train_triplet.py | 19 | # Usage: | COMMENT |
| LOW | scripts/msmarco_v2/convert_passage_run_to_doc_run.py | 23 | # Usage: | COMMENT |
| LOW | scripts/msmarco_v2/build_passage_to_doc_id_map.py | 19 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | scripts/dse/download_slidevqa_corpus.py | 70 | # Check if page exists and is not None | COMMENT |
| LOW | scripts/dse/download_slidevqa_corpus.py | 82 | # Check if we've already processed this page | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/base/encoder/test_encoder_model_arctic.py | 45 | #TODO: Implement test_arctic_query_encoder after beir regression incorporated | COMMENT |