Repository Analysis

NVIDIA/cosmos

NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.

25.3 Moderate AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of NVIDIA/cosmos, a Jupyter Notebook project with 11,040 GitHub stars. SynthScan v2.0 examined 204,181 lines of code across 943 source files, recording 3794 pattern matches distributed across 24 syntactic categories. The overall adjusted score of 25.3 places this repository in the Moderate AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

25.3
Adjusted Score
25.3
Raw Score
100%
Time Factor
2026-07-13
Last Push
11.0K
Stars
Jupyter Notebook
Language
204.2K
Lines of Code
943
Files
3.8K
Pattern Hits
2026-07-14
Scan Date
0.24
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 1HIGH 224MEDIUM 332LOW 3237

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 3794 distinct pattern matches across 24 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Excessive Try-Catch Wrapping889 hits · 1014 pts
SeverityFileLineSnippetContext
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py24 except Exception:CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py540 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py350 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py377 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py391 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py318 except Exception:STRING
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/inference_mt.py38 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py46 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py67 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py205 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py222 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py240 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py257 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py402 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py452 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py489 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py574 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py614 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py623 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py644 except Exception as e:CODE
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py609def run_eval():CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py68 except Exception as err:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py82 except Exception as e:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py126 except Exception as e:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py178 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py172 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py180 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py187 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py195 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py253 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py263 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py92 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py136 except Exception as err:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py71 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py85 except Exception as e:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py102 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py114 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py121 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py158 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py165 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py321 except Exception as e:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py374 except Exception as e:CODE
MEDIUM…valkit/vlmeval/dataset/medqbench_paired_description.py375 print(f"Error during judge evaluation: {e}")CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py395 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py398 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py449 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py42 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py125 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py254 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py283 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py319 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py329 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py351 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py416 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py428 except Exception:CODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py410def str_to_dict(s):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py170 except Exception:STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py134 except Exception:STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py149 except Exception:STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py212 except Exception as e:STRING
829 more matches not shown…
Cross-File Repetition194 hits · 970 pts
SeverityFileLineSnippetContext
HIGH…lmevalkit/vlmeval/dataset/avspecial_collision_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py0stub: dataset implementation not included in this public release.STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py0stub: dataset implementation not included in this public release.STRING
HIGH…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py0stub: dataset implementation not included in this public release.STRING
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py0stub: dataset implementation not included in this public release.STRING
HIGH…lmevalkit/vlmeval/dataset/sop_temporal_localization.py0stub: dataset implementation not included in this public release.STRING
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…evalkit/vlmeval/dataset/temporal_localization_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…oner/vlmevalkit/vlmeval/dataset/warehouse_near_miss.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_reason.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/intphys2.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/cv_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…r/vlmevalkit/vlmeval/dataset/sop_action_recognition.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/robospatial_home.py0stub: dataset implementation not included in this public release.STRING
HIGH…evalkit/vlmeval/dataset/avspecial_environment_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/blink_depth.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/its_collision.py0stub: dataset implementation not included in this public release.STRING
HIGH…ner/vlmevalkit/vlmeval/dataset/warehouse_spatial_ai.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/camera_intrinsic.py0stub: dataset implementation not included in this public release.STRING
HIGH…soner/vlmevalkit/vlmeval/dataset/health_surgi_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/locate_anything_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_video.py0stub: dataset implementation not included in this public release.STRING
HIGH…ner/vlmevalkit/vlmeval/dataset/lvs_ai_hallucination.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_erqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/lv_event_verification.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/where2place.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_image.py0stub: dataset implementation not included in this public release.STRING
HIGH…alkit/vlmeval/dataset/avspecial_ood_reasoning_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…oner/vlmevalkit/vlmeval/dataset/av_prompt_following.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/tailgating.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/Omni3D/omni3d_dataset.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/IFBench/ifbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
HIGH…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
134 more matches not shown…
Deep Nesting885 hits · 833 pts
SeverityFileLineSnippetContext
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py227CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py31CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py196CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py8CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py25CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py58CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py66CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py226CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py385CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py443CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py526CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py575CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py595CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py103CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py509CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py200CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py73CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/image_base.py129CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py54CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py85CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py216CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py293CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py327CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py329CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py91CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py143CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py272CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py325CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py382CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py182CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py405CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py242CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py354CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py64CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py208CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py243CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py85CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/image_shortqa.py92CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py124CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py276CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py309CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py472CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py59CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py74CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py118CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py149CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py214CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py265CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py308CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py341CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py581CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py132CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py162CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py225CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py252CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py300CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py354CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/macbench.py150CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py21CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py64CODE
825 more matches not shown…
Unused Imports674 hits · 587 pts
SeverityFileLineSnippetContext
LOW…uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py38CODE
LOW…uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py30CODE
LOW…lmevalkit/vlmeval/dataset/avspecial_collision_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py6CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py6CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py6CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py6CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py6CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py575CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py576CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py577CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py578CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py578CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py6CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py6CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py7CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py11CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py15CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py17CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py19CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py19CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py6CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py6CODE
LOW…ner/vlmevalkit/vlmeval/dataset/video_dataset_config.py3CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py2CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py7CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py10CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py13CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py15CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py17CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py19CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py19CODE
LOW…alkit/vlmeval/dataset/metropolis_event_verification.py7CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py34CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/foxbench.py1CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py3CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py4CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py5CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py7CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py9CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py6CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py6CODE
LOW…lmevalkit/vlmeval/dataset/sop_temporal_localization.py6CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py6CODE
LOW…evalkit/vlmeval/dataset/temporal_localization_bench.py6CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py68CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py6CODE
614 more matches not shown…
Hyper-Verbose Identifiers427 hits · 422 pts
SeverityFileLineSnippetContext
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py84def build_dataset_from_config(cfg, dataset_name):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py605 async def _run_eval_in_main_process(self, dataset_name: str):CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py50 def trans_greater_sign_to_interval(self, expr: str):CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py91 def parse_score_from_response(resp):CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/medqbench_mcq.py84 def _convert_answer_to_letter(self, answer):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py518def download_vladbench_from_hf():CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py168def _convert_single_page_pdf_to_png(pdf_path, output_path, dpi=350):CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py426 def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py13def format_model_answer_tolist(model_answer, task_gt):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ost_bench.py66 def longest_common_subsequence(str1, str2):STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py373 def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name):CODE
LOW…alkit/vlmeval/dataset/metropolis_event_verification.py319def test_metropolis_event_verification_dataset():CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py120def extract_score_from_direct_gpt_resp(raw_score):STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py158def extract_score_from_p_level_gpt_resp(raw_score):STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py178def extract_score_from_cmp_gpt_resp(response_text):STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py233def _regex_extract_frame_bboxes(CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/omtgbench.py122def compute_one_to_many_metrics(CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/spatial457.py149 def build_subtask_instruction(self, level):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py202def parse_multi_choice_response(response, all_choices, index2ans):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py701 def doc_to_text_perception_comprehension(self, doc):CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/av_speakerbench.py17def _parse_multi_choice_response(response, all_choices):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py20def evaluate_hipho_single_problem(judge_model, row, index, judge_kwargs, dataset_instance):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py594 def _evaluate_single_criterion(self, prediction, criterion, judge_model,CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py701 def _has_multiple_marking_sets(self, marking):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py715 def _evaluate_multiple_marking_sets(self, prediction, marking_sets, points, judge_model, question):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py736 def _evaluate_single_marking_set(self, prediction, marking, points, judge_model, question):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py796 def _extract_score_from_response(self, response):CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py111 def proportional_sample_from_lists(num_list, k):STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py123 def split_txt_based_on_images(text, image_paths):STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py166 def _resize_frames_to_fit_pixels(self, images, total_pixels):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py16def _find_first_bracketed_list(text: str) -> Optional[str]:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py69def _parse_python_list_from_text(text) -> Optional[List[int]]:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py89def _parse_all_python_int_lists_from_text(text) -> List[List[int]]:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py129def _parse_subdataset_from_path(rel_path):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py301 def _extract_yesno_and_explanation(text):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py325 def _extract_letter_and_explanation(text):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py371 def _assert_nonempty_references(references, task_name):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py430 def _parse_timestamp_to_seconds(ts):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py470 def _eval_temporal_localization(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py518 def _eval_temporal_description(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py521 def _eval_video_summarization(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/visfactor.py61 def extract_last_uppercase_letter(self, s):CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py114 def parse_score_from_response(resp):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py100 def _save_images_and_update_df(self, df, base_image_dir):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py69 def _download_from_huggingface(self, data_path, image_dir):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py209 def parse_answer_from_response(response, category):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py495 def _process_graph_isomorphism_pairs(pairs_dict, results_dict, data_judged, results_by_category):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/longvideobench.py37def insert_subtitles_into_frames(CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py325 def load_into_video_and_process(self, line):STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py156 def _download_dataset_from_dss(self, dataset_name):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py212 def _generate_tsv_from_events(self, local_dir, tsv_path):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py573 def compute_bert_scores_remote(candidates: List[str], references: List[str]) -> List[float]:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2499 def extract_content_in_braces(self, input_str):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2508 def compare_string_with_values(self, input_str, target_values):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2594 def convert_to_vlmeval_format(example):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py15def get_multi_choice_prediction(response, all_choices, index2ans):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py113def get_fill_blank_prediction(response, answer):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mvbench.py295 def load_into_video_and_process(self, line):STRING
LOW…lmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py19def extract_answer_from_response(response):CODE
LOW…alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py223def calculate_semantic_repetition(text: str) -> float:CODE
367 more matches not shown…
Decorative Section Separators139 hits · 420 pts
SeverityFileLineSnippetContext
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py199 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py201 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py215 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py217 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py250 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py252 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py281 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py283 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py354 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py356 # ==============================================STRING
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py74# ==========================================COMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py76# ==========================================COMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py138# ==========================================COMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py140# ==========================================COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py38# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py40# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py55# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py57# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py168# ---------------------------------------------------------------------------STRING
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py170# ---------------------------------------------------------------------------STRING
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py222# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py224# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py343# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py345# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py511# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py513# ---------------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py558 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py560 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py613 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py615 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py732 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py734 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py763 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py765 # ------------------------------------------------------------------COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py20# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py22# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py58# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py60# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py134# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py136# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py185# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py187# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py18# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py20# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py29# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py62# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py64# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py422# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py424# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py498 # ========================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py500 # ========================================================================COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py25# ---------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py56# ---------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py279#--------------------------------------------------COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py835# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py837# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py851# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py853# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py906# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py908# # ===================COMMENT
79 more matches not shown…
Redundant / Tautological Comments135 hits · 196 pts
SeverityFileLineSnippetContext
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py201 # Check if two numerical values are equal within an allowed error rangeCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py267 # Check if two equations are mathematically equivalentCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py101 # Check if two < or > in expressionCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py121 # Check if elements in both lists can be paired and are equalCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py158 # Set timeout to 30 seconds for is_equalCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py217 # Check if two expressions are mathematically equivalentCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py294 # Check if two intervals are mathematically equivalentCOMMENT
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py333 # Check if prediction column exists, if not, evaluation cannot proceedCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py102 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py120 # Check if the member is a file (not a directory)STRING
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py100 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py482 # Check if py_root/v1 existsCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py219 # Check if video directory exists and has videosCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py504 # Check if any category name (lowercased) appears in the video nameCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py893 # Check if verbose is enabled in judge_kwargsCOMMENT
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py173 # Check if video directory exists and has videosCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py27 # Check if the options are already appended with lettersCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py503 # Check if the member is a file (not a directory)COMMENT
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py108 # Check if the member is a file (not a directory)STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py126 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py113 # Check if TSV already existsCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py485 # Check if point is in mask (non-zero)COMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py199 # Check if this is HiPhO_ALL modeCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py219 # Check if this subset is already completedCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py510 # Check if API failedCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py622 # Check if it's an API failure messageCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py626 # Check if response is empty or invalidCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py125 # Check if all frames already existCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py68 # Check if already extractedCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py264 # Check if we need LLM judgeCOMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py336 # Check if prediction column exists, if not, evaluation cannot proceedCOMMENT
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py109 # Check if image_list is emptyCOMMENT
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py292 # Check if image name is in reverse_xxx formatCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py54 # Check if TSV exists locallyCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py110 # Check if question has <image> placeholdersCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py113 # Check if the data file existsSTRING
LOW…s3/reasoner/vlmevalkit/vlmeval/dataset/video_holmes.py98 # Check if the member is a file (not a directory)STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py683 # Print results tableCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/asclepius.py126 # Check if evaluation results already existCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py949 # Check if dataset is cached and has integrityCOMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py184 # Check if this line has pipe characters (a table row indicator)COMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py287 # Check if this looks like a separator row (contains mostly dashes)COMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1003 # Check if a 2x2 block is formed by the same valueSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1013 # Check if the response is a 6x6 matrixSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1017 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1025 # Check if 0s and 1s form connected componentsSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1030 # Check if there are any 2x2 blocks of 0s or 1sSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py820 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py910 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py954 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py992 # Check if all cells with the given value are visitedSTRING
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/tempcompass.py151 # Check if the chatgpt answer is the ground-truth answerSTRING
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py54 # Check if in main threadCOMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1027 # Check if judge model is availableCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py641 # Check if the expression is a power expressionCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py646 # Check if the base and the exponent are numbersCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py651 # Check if the exponent is greater than the thresholdCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/judge_util.py122 # Check if using NVIDIA APICOMMENT
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py329 # Check if there is an overlapCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vgrpbench/score.py110 # Check if cell is a number (not 0) or not a string/characterCOMMENT
75 more matches not shown…
AI Structural Patterns197 hits · 192 pts
SeverityFileLineSnippetContext
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py100CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py432CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py122CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py91CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py552CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py219CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py228CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py713CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py132CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py527CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vcr.py182CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py3586CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py683CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py958CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py1089CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/slidevqa.py62CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py114CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2029CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2091CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2282CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2422CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py325CODE
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dude.py48CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lens.py219CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/utils/mathvista.py167CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py47CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py93CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py145CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py187CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py210CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py232CODE
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/logicvista.py151CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/omni3dbench.py100CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/videoholmes.py38CODE
LOW…oner/vlmevalkit/vlmeval/dataset/utils/ocr_reasoning.py173CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py107CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py142CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py165CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py513CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py539CODE
LOW…vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py329CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py35CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py99CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py187CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py49CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py115CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py205CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py65CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py132CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py232CODE
LOW…val/dataset/utils/SArena/video/viclip/viclip_vision.py335CODE
LOW…meval/dataset/utils/SArena/video/viclip/viclip_text.py216CODE
LOW…dataset/utils/megabench/scoring/latex_expr_equality.py75CODE
LOW…eval/dataset/utils/megabench/scoring/sacrebleu_bleu.py23CODE
LOW…eval/dataset/utils/megabench/parsing/common/parsers.py146CODE
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py338CODE
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py876CODE
LOW…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py426CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/vlm/cosmos_reason_hf.py47CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py25CODE
137 more matches not shown…
Self-Referential Comments38 hits · 111 pts
SeverityFileLineSnippetContext
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run.py529 # Create the symbolic links for the prediction filesSTRING
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py592 # Create a simple test PDF using matplotlibCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py150 # Create a copy of line to avoid SettingWithCopyWarningCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py24 # Define the option letters based on the number of optionsCOMMENT
MEDIUM…er/vlmevalkit/vlmeval/dataset/olmOCRBench/evaluator.py98 # Define an inner function to evaluate a single testCOMMENT
MEDIUM…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py216 # Create a new page.COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py430 # Create a base temporary directory for this evaluation runCOMMENT
MEDIUM…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py430 # Create a dummy video on failure to prevent crash in metricsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py43 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py122 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py77 # Define the keys to locate in the textSTRING
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py111 # Define the keys to locate in the textSTRING
MEDIUM…meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py40 # Create a list of non-zero values in the rowCOMMENT
MEDIUM…meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py55 # Create a list of non-zero values in the columnCOMMENT
MEDIUM…taset/utils/vgrpbench/puzzles/common_puzzle_factory.py30 # Create a new board filled with zerosCOMMENT
MEDIUM…lmeval/dataset/utils/vgrpbench/puzzles/thermometers.py23 # Create a set of all thermometer positions for efficient lookupCOMMENT
MEDIUM…eval/dataset/utils/chartmimic/evaluator/color_utils.py24 # Create an sRGBColor object from RGB valuesCOMMENT
MEDIUM…ataset/utils/mmhelix/evaluators/graph_problems_eval.py526 # Create a new cycle with the first node repeated at the end for validationCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py132 # Create a new simulator to test the ground truth solutionCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py142 # Define a Sokoban simulator class within the evaluatorCOMMENT
MEDIUM…vlmeval/dataset/utils/design2code/screenshot_single.py66 # Define the argumentsSTRING
MEDIUM…lmeval/dataset/utils/megabench/tools/analysis_utils.py178 meta = all_meta[task_name].copy() # Create a copy to avoid modifying originalCODE
MEDIUM…set/utils/megabench/scoring/general_numerical_match.py35 # Create a multiprocessing.Queue to receive the outputCOMMENT
MEDIUM…meval/dataset/utils/megabench/scoring/program_judge.py45 # Create a CodeTester instance with the response and the found test casesCOMMENT
MEDIUM…/dataset/utils/megabench/scoring/common/conversions.py221 # Create a new image with the calculated sizeCOMMENT
MEDIUM…vlmevalkit/vlmeval/dataset/utils/ssi_bench/__init__.py1# This file is intentionally left empty to make this directory a Python package.COMMENT
MEDIUM…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py376 # Create a dictionary for each combination of values.COMMENT
MEDIUM…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py388 # Create a dictionary for each combination of values.COMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py78 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py157 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py1775 # Define a regex pattern to find Markdown tables with newlinesCOMMENT
MEDIUM…soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py1905 # Define a regex pattern to find Markdown tables with newlinesCOMMENT
MEDIUM…uation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vila.py173 # Create a unique temporary directory for this inference callCOMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/long_vita.py129 # Create a new image with the target size and paste the resized image onto itCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py64# Define the temporary folder for processed imagesCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py211# Define a minimum dimension for the cropCOMMENT
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/vlm/qwen2_vl/model.py115 # Create a temporary file for the frameCOMMENT
MEDIUM…s3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/mm_utils.py69 # Create a new image with the target size and paste the resized image onto itCOMMENT
Cross-Language Confusion18 hits · 102 pts
SeverityFileLineSnippetContext
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/groundingme.py13Ensure that all details mentioned about the object are accurate. Provide at most one bounding box. If a matching object CODE
HIGH…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py264 return null;CODE
HIGH…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py310 list.push({CODE
HIGH…vlmevalkit/vlmeval/dataset/utils/physics_eval_utils.py83 sympy_result = sympy_result or expr1_sympy.equals(expr2_sympy)CODE
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py406 if (root && typeof root.pauseAnimations === 'function') root.pauseAnimations();CODE
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py407 if (root && typeof root.setCurrentTime === 'function') root.setCurrentTime({t});CODE
HIGH…oner/vlmevalkit/vlmeval/dataset/utils/tablevqabench.py21 'If you have multiple answers, please separate them with || marks. Example: Apple||Banana||Tomato\n\n'CODE
HIGH…kit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py81 # 2. We don't have enough potential lights (current + undefined) to reach the required numberCOMMENT
HIGH…lmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py37 # Check if current mines <= number <= potential mines (current + undefined)COMMENT
HIGH…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py14 # Convert line to runs of filled cells ('s' = filled, 'e' = empty, 0 = undefined)COMMENT
HIGH…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py33 else: # cell is 0 (undefined)CODE
HIGH…dataset/utils/megabench/scoring/latex_expr_equality.py36 result = 1 if responses_expr.equals(targets_expr) else 0CODE
HIGH…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py69 " cd VLM-3R && pip install -e .\n"CODE
HIGH…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py70 " cd CUT3R && pip install -r requirements.txt\n"CODE
HIGH…/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/cambrian_s.py66 " cd cambrian-s && pip install -e .\n"CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py154 " cd Bagel && uv pip install -e .\n"CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py155 " OR: cd Bagel && pip install -e .\n"CODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/vlm/spatial_mllm.py114 "2. Install in development mode: cd Spatial-MLLM && uv pip install -e .\n"CODE
Over-Commented Block77 hits · 66 pts
SeverityFileLineSnippetContext
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py81 ans = judge_model.generate(messages)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py221 # 'mcq_openended_bleu': 1.0,COMMENT
LOW…/reasoner/vlmevalkit/vlmeval/dataset/reasonmap_plus.py1# ReasonMap Plus Dataset is an extension of the original ReasonMap dataset,COMMENT
LOW…valkit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py21 # 3: parameter list (e.g.,COMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py1# flake8: noqaCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py21# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py41# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py61# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py801 grid = response_listCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/utils/mmsci.py261COMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1# Copyright 2025 Garena Online Private LimitedCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py121# def _get_prefix(self):COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py141# # name = func.__name__COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py161# Axes.hist = log_function(Axes.hist)COMMENT
LOW…/dataset/utils/chartmimic/evaluator/color_evaluator.py21COMMENT
LOW…/dataset/utils/chartmimic/evaluator/color_evaluator.py201 # similarities = [item[1] for item in similarities]COMMENT
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py361# Axes.errorbar = log_function(Axes.errorbar)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py521Axes3D.plot_surface = log_function_for_3d(Axes3D.plot_surface)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py541# ax.axvline(0.5)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py561# loglog testCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py581# ax.fill_betweenx(x, y1, y2, cmap='viridis')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py601# plt.show()COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py621# ax.set_xlabel('X-axis')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py641# fig, ax = plt.subplots()COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py661# v = np.ones(10)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py681# ax.plot([0, 1], [0, 1], [1, 0], color='b')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py701# dz = np.random.rand(10)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py721# plot in testCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py741# draw the pcolor plot, with a beautiful colormapCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py761# X, Y = np.meshgrid(x, y)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py781# x = np.linspace(0, 10, 100)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py821# fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py841# num_vars = len(values_pc1)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py861# ax.scatter(angles[:-1], values_pc2[:-1], color="red", s=50)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py881# Line2D(COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py901# # Add legend and titleCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py921# # ===================COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py941# ]COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py961# ax.set_yticks([1, 2, 3])COMMENT
LOW…kit/vlmeval/dataset/utils/mmif/function_and_compare.py401COMMENT
LOW…kit/vlmeval/dataset/utils/mmif/function_and_compare.py421# else:COMMENT
LOW…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py561 # initial_state: JSON string representing the grid structureCOMMENT
LOW…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py581 # converted_answer = {}COMMENT
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py1# flake8: noqaCOMMENT
LOW…eval/dataset/utils/Ocrbench_v2/spotting_eval/script.py1# flake8: noqaCOMMENT
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/GUI/osworld_g.py341 click_point = self.parse_response_func(prediction)COMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/Omni3D/install_patch.sh61# echo "[INFO] Updating ~/.bashrc (tclsh + PYTHONPATH)..."COMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py41 if ord(char) > 127: # Non-ASCII charactersCOMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py341# for match in inline_matches:COMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py361# # 'category_type': 'equation_inline',COMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py161 # Second pass: Inline images (re-using the same images? Need to check if processorCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py181 # the content structure.COMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py201 passCOMMENT
LOW…vlmevalkit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py1# Adapted from LLaVA code.COMMENT
LOW…awk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py1# --------------------------------------------------------COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/utils.py1# Adopted from https://github.com/haotian-liu/LLaVA. Below is the original copyright:COMMENT
LOW…meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py461COMMENT
LOW…r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py1# Copyright (c) 2023-2024 DeepSeek.COMMENT
LOW…it/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py1# Copyright (c) 2023-2024 DeepSeek.COMMENT
LOW…lmevalkit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py241 # if vision_feature_select_strategy == "default":COMMENT
17 more matches not shown…
Docstring Block Structure9 hits · 45 pts
SeverityFileLineSnippetContext
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py74 Calculate scores by category from evaluation results. Args: result_file: Path to the file STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py326 Loads a video or image sequence, processes it, and returns the path to the processed video. Args: STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py399 Builds a prompt for a language model based on the provided data and settings. Args: line (STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py447 Evaluates the given evaluation file and generates ratings based on different dimensions. Args: STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py58 Map question IDs to their categories based on the evaluation mode. Args: mode: Either "descriptive" orSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py139 Calculate scores by category from evaluation results. Args: result_file: Path to the file STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py1077 Load a single test from a JSON line string or JSON object. Args: data: Either a JSON string to parse oSTRING
HIGH…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py48 Load bbox_3d and label arrays from a JSON annotation file. Args: json_path: Path to JSON file containiSTRING
HIGH…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py71 Normalize label by extracting base name and removing suffixes. Removes numeric suffixes and underscores (e.g.,STRING
AI Slop Vocabulary17 hits · 41 pts
SeverityFileLineSnippetContext
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py53 "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference imageCODE
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py234 # Save comprehensive resultsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py256 """image is [good/usable/reject]". Please provide a comprehensive but concise assessment in """STRING
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/sitebench.py375 # Use os.path.relpath for robust relative path extractionCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/bridges_eval.py94 # Various bridge patterns - more comprehensiveCOMMENT
MEDIUM…val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py314 # Extract the predicted grid using robust parsingCOMMENT
MEDIUM…val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py476 # Extract predicted answer using the robust extract_answer methodCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/binario_eval.py518 # More comprehensive patterns for matrix keywordsCOMMENT
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py32 """Enhanced robust extraction of answer from model output with comprehensive patterns."""STRING
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py101 """Extract coordinate-value patterns with comprehensive regex."""STRING
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py569 # # If it's a string, try to extract the answer using robust parsingCOMMENT
LOW…set/utils/megabench/scoring/xml_norm_point_distance.py36 # Instead of normalizing by 1/sqrt(2), we just set it to 0 if the distance is above 1.COMMENT
LOW…lmeval/dataset/utils/megabench/parsing/common/utils.py137 # the result overflows, simply return an empty stringCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py334 # ThreeDAVGroundingBench: the harness writes a transposed 1-row DF as a flatCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py355 # AETCBench: harness writes a transposed 1-row DF as a flat dict — metricCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py632# --- RynnScale harness adapter (layer-1: native <suite>.json -> standardized scores) ---COMMENT
MEDIUM…luation/cosmos3/generator/paibench_c/run_paibench_c.sh707# Use a robust python for printing results - prefer cosmos3 venv if functional,COMMENT
Structural Annotation Overuse21 hits · 37 pts
SeverityFileLineSnippetContext
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py179 # Step 1: Find the last occurrence of 'summary:'STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py184 # Step 2: Slice the string after 'summary:' and extract valueSTRING
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py210 # Step 1: Clean up punctuation from the responseCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py235 # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py243 # Step 4: If no candidates, look for choices with a space after (A B C D)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py250 # Step 5: If no candidates and response has more than 5 tokens, try parsing based on contentCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py258 # Step 6: If still no candidates, randomly choose oneCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py262 # Step 7: If multiple candidates found, use the one appearing lastCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py222 # Step 2: If no candidates, look for choices with a period after (A. B. C. D.)COMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py211 # Step 1: Try template matchingCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py216 # Step 2: Fall back to LLM matchingCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py74Predicted answer: ### Step 1: Calculate ( l_1 )CODE
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py55# Step 1: Reformatting the SolutionCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py83# Step 2: Evaluating CorrectnessCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py355 # Step 3: Validate segments formatCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py359 # Step 4: Validate using initial state and game rulesCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py313 # Step 1: Extract and validate predicted_answerCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py331 # Step 2: Validate initial_stateCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md5## Step 0: Install LMDeployCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md12## Step 1: Start the Inference ServiceCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md23## Step 2: EvaluationCOMMENT
Verbosity Indicators20 hits · 36 pts
SeverityFileLineSnippetContext
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py179 # Step 1: Find the last occurrence of 'summary:'STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py184 # Step 2: Slice the string after 'summary:' and extract valueSTRING
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py210 # Step 1: Clean up punctuation from the responseCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py235 # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py243 # Step 4: If no candidates, look for choices with a space after (A B C D)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py250 # Step 5: If no candidates and response has more than 5 tokens, try parsing based on contentCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py258 # Step 6: If still no candidates, randomly choose oneCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py262 # Step 7: If multiple candidates found, use the one appearing lastCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py222 # Step 2: If no candidates, look for choices with a period after (A. B. C. D.)COMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py211 # Step 1: Try template matchingCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py216 # Step 2: Fall back to LLM matchingCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py74Predicted answer: ### Step 1: Calculate ( l_1 )CODE
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py55# Step 1: Reformatting the SolutionCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py83# Step 2: Evaluating CorrectnessCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py355 # Step 3: Validate segments formatCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py359 # Step 4: Validate using initial state and game rulesCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py313 # Step 1: Extract and validate predicted_answerCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py331 # Step 2: Validate initial_stateCOMMENT
LOW…ataset/utils/mmhelix/evaluators/graph_problems_eval.py283 # we need to check if the last node connects back to the first nodeCOMMENT
LOW…ataset/utils/mmhelix/evaluators/graph_problems_eval.py508 # If not, we need to check if the last node connects back to the first nodeCOMMENT
Modern Structural Boilerplate32 hits · 34 pts
SeverityFileLineSnippetContext
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py486__all__ = [CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/utils/__init__.py7__all__ = [CODE
LOW…val/dataset/utils/SArena/video/viclip/viclip_vision.py18logger = logging.getLogger(__name__)CODE
LOW…meval/dataset/utils/SArena/video/viclip/viclip_text.py18logger = logging.getLogger(__name__)CODE
LOW…it/vlmeval/dataset/utils/SArena/video/viclip/viclip.py16logger = logging.getLogger(__name__)CODE
LOW…vlmevalkit/vlmeval/dataset/utils/megabench/__init__.py5__all__ = [AggregationType, MetricType, ResponseParseType]CODE
LOW…t/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py16logger = logging.getLogger(__name__)CODE
LOW…valkit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py5logger = logging.getLogger(__name__)CODE
LOW…er/vlmevalkit/vlmeval/dataset/metropolis2d/__init__.py5__all__ = ['Metropolis2DDetectionDataset', 'Metropolis2DGroundingDataset', 'Astro2DDetectionDataset']CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/utils/__init__.py5__all__ = [CODE
LOW…tion/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/points.py226 def set_image_processor(self, dataset: str) -> None:CODE
LOW…/reasoner/vlmevalkit/vlmeval/vlm/video_llm/__init__.py8__all__ = ['VideoLLaVA', 'VideoLLaVA_HF', 'Chatunivi', 'VideoChatGPT', 'LLaMAVID', 'VideoChat2_HD', 'PLLaVA']CODE
LOW…/reasoner/vlmevalkit/vlmeval/vlm/xcomposer/__init__.py7__all__ = ['ShareCaptioner', 'XComposer', 'XComposer2', 'XComposer2_4KHD', 'XComposer2d5']CODE
LOW…oner/vlmevalkit/vlmeval/vlm/granite_vision/__init__.py3__all__ = ['GraniteVision3']CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/vlm/qtunevl/__init__.py4__all__ = ['QTuneVL', 'QTuneVLChat']CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/interns1/__init__.py3__all__ = ['InternS1Chat']CODE
LOW…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py105 def set_dump_image(self, flag: bool) -> None:CODE
LOW…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py108 def set_batch_size(self, bs: int) -> None:CODE
LOW…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py111 def set_output_folder(self, out_dir: str) -> None:CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/internvl/__init__.py3__all__ = ['InternVLChat']CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/vlm/llava/__init__.py5__all__ = [CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/vlm/ovis/__init__.py3__all__ = ['Ovis', 'Ovis1_6', 'Ovis1_6_Plus', 'Ovis2', 'OvisU1', 'Ovis2_5']CODE
LOW…meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py504 def set_grad_checkpointing(self, enable: bool = True) -> None:CODE
LOW…lmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py26logger = logging.getLogger(__name__)CODE
LOW…lmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py8__all__ = [CODE
LOW…l/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py27logger = logging.getLogger(__name__)CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/qwen3_vl/__init__.py3__all__ = [CODE
LOW…r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py436 def set_grad_checkpointing(self, enable: bool = True) -> None:CODE
LOW…ner/vlmevalkit/vlmeval/vlm/ursa/ursa_model/__init__.py20__all__ = [CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/api/sensechat_vision.py193 def set_max_num(self, dataset: Optional[str] = None) -> None:CODE
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/api/__init__.py41__all__ = [CODE
LOW…3/reasoner/vlmevalkit/vlmeval/api/adapters/__init__.py8__all__ = [CODE
Synthetic Comment Markers3 hits · 20 pts
SeverityFileLineSnippetContext
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py53 "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference imageCODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py43 # I will dump all original keys as requested.COMMENT
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py158 # Format as requested: dictionary composed of lists, organized into a pandas.DataFrameCOMMENT
Slop Phrases8 hits · 16 pts
SeverityFileLineSnippetContext
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py2502 # Once you get the permission, you can use the helper code here to download and extract necessary images:COMMENT
MEDIUM…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py29Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py11Your task is to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's hoCODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py32Here's how you can accomplish the task:CODE
MEDIUM…easoner/vlmevalkit/vlmeval/dataset/utils/video_mmlu.py44Here's how you can accomplish the task:CODE
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py39Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
MEDIUM…reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py33Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
LOW…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py36 # Don't forget to add the last run if it existsCOMMENT
Dead Code5 hits · 10 pts
SeverityFileLineSnippetContext
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py321CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py331CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py386CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py397CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py407CODE
Hallucination Indicators1 hit · 10 pts
SeverityFileLineSnippetContext
CRITICAL…easoner/vlmevalkit/vlmeval/vlm/video_llm/videochat2.py263 seg_embs = [model.mistral_model.base_model.model.model.embed_tokens(seg_t) for seg_t in seg_tokens]CODE
Modern AI Meta-Vocabulary2 hits · 5 pts
SeverityFileLineSnippetContext
MEDIUMcookbooks/cosmos3/README.md102### System packages (required for Framework guardrails)COMMENT
MEDIUMcookbooks/cosmos3/README.md138license and set `HF_TOKEN` as in [Prerequisites](#prerequisites)). To disable guardrailsCODE
Fake / Example Data1 hit · 1 pts
SeverityFileLineSnippetContext
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py538 self.data['question'] = ['placeholder'] * len(self.data)CODE
Overly Generic Function Names1 hit · 1 pts
SeverityFileLineSnippetContext
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py328 def process_data(obj):CODE
Example Usage Blocks1 hit · 0 pts
SeverityFileLineSnippetContext
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1343 # Example usageSTRING