NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.
This report presents the forensic synthetic code analysis of NVIDIA/cosmos, a Jupyter Notebook project with 11,040 GitHub stars. SynthScan v2.0 examined 204,181 lines of code across 943 source files, recording 3794 pattern matches distributed across 24 syntactic categories. The overall adjusted score of 25.3 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 3794 distinct pattern matches across 24 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run.py | 24 | except Exception: | CODE |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run.py | 540 | except Exception as e: | STRING |
| LOW⚡ | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 350 | except Exception as e: | STRING |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 377 | except Exception as e: | STRING |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 391 | except Exception as e: | STRING |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 318 | except Exception: | STRING |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/inference_mt.py | 38 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 46 | except Exception as e: | CODE |
| LOW⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 67 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 205 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 222 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 240 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 257 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 402 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 452 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 489 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 574 | except Exception as e: | CODE |
| LOW⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 614 | except Exception as e: | CODE |
| LOW⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 623 | except Exception as e: | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 644 | except Exception as e: | CODE |
| MEDIUM | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 609 | def run_eval(): | CODE |
| LOW | …ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py | 68 | except Exception as err: | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py | 82 | except Exception as e: | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py | 126 | except Exception as e: | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py | 178 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 172 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 180 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 187 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 195 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 253 | except Exception: | CODE |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 263 | except Exception: | CODE |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 92 | except Exception: | CODE |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 136 | except Exception as err: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 71 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 85 | except Exception as e: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 102 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 114 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 121 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 158 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 165 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 321 | except Exception as e: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 374 | except Exception as e: | CODE |
| MEDIUM | …valkit/vlmeval/dataset/medqbench_paired_description.py | 375 | print(f"Error during judge evaluation: {e}") | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 395 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 398 | except Exception: | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 449 | except Exception as e: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 42 | except Exception: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 125 | except Exception: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 254 | except Exception: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 283 | except Exception as e: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 319 | except Exception as e: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 329 | except Exception as e: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 351 | except Exception: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 416 | except Exception: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 428 | except Exception: | CODE |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 410 | def str_to_dict(s): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py | 170 | except Exception: | STRING |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 134 | except Exception: | STRING |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 149 | except Exception: | STRING |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 212 | except Exception as e: | STRING |
| 829 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …lmevalkit/vlmeval/dataset/avspecial_collision_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …lmevalkit/vlmeval/dataset/sop_temporal_localization.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …evalkit/vlmeval/dataset/temporal_localization_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldbench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …oner/vlmevalkit/vlmeval/dataset/warehouse_near_miss.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_reason.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/intphys2.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/cv_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …r/vlmevalkit/vlmeval/dataset/sop_action_recognition.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …easoner/vlmevalkit/vlmeval/dataset/robospatial_home.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …evalkit/vlmeval/dataset/avspecial_environment_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/blink_depth.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …3/reasoner/vlmevalkit/vlmeval/dataset/its_collision.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …ner/vlmevalkit/vlmeval/dataset/warehouse_spatial_ai.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …easoner/vlmevalkit/vlmeval/dataset/camera_intrinsic.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …soner/vlmevalkit/vlmeval/dataset/health_surgi_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …er/vlmevalkit/vlmeval/dataset/locate_anything_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_video.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …ner/vlmevalkit/vlmeval/dataset/lvs_ai_hallucination.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_erqa.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …er/vlmevalkit/vlmeval/dataset/lv_event_verification.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/where2place.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_image.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …alkit/vlmeval/dataset/avspecial_ood_reasoning_bench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …oner/vlmevalkit/vlmeval/dataset/av_prompt_following.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/tailgating.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …er/vlmevalkit/vlmeval/dataset/Omni3D/omni3d_dataset.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/dataset/IFBench/ifbench.py | 0 | stub: dataset implementation not included in this public release. | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py | 0 | evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judg | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 0 | evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judg | STRING |
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py | 0 | evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judg | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py | 0 | evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judg | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py | 0 | evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judg | STRING |
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 0 | these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice q | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py | 0 | these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice q | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py | 0 | these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice q | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py | 0 | these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice q | STRING |
| HIGH | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 0 | build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py | 0 | build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py | 0 | build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: | STRING |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py | 0 | 1. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted a | STRING |
| HIGH | …oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py | 0 | 1. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted a | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py | 0 | 1. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted a | STRING |
| 134 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run.py | 227 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 31 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 196 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/setup.py | 8 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/setup.py | 25 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/setup.py | 58 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/setup.py | 66 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 226 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 385 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 443 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 526 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 575 | CODE | |
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py | 595 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py | 103 | CODE | |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 509 | CODE | |
| LOW | …ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py | 200 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py | 73 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/image_base.py | 129 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 54 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 85 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 216 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 293 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 327 | CODE | |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 329 | CODE | |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 91 | CODE | |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 143 | CODE | |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 272 | CODE | |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 325 | CODE | |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 382 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 182 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 405 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 242 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py | 354 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 64 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 208 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 243 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 85 | CODE | |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/image_shortqa.py | 92 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 124 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 276 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 309 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 472 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py | 59 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py | 74 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 118 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 149 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 214 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 265 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 308 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 341 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 581 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 132 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 162 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 225 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 252 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 300 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py | 354 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/macbench.py | 150 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py | 21 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py | 64 | CODE | |
| 825 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py | 38 | CODE | |
| LOW | …uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py | 30 | CODE | |
| LOW | …lmevalkit/vlmeval/dataset/avspecial_collision_bench.py | 6 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py | 6 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py | 6 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py | 6 | CODE | |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py | 6 | CODE | |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py | 6 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 575 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 576 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 577 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 578 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 578 | CODE | |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py | 6 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py | 6 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py | 6 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 7 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 11 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 15 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 17 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 19 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 19 | CODE | |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py | 6 | CODE | |
| LOW | …reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py | 6 | CODE | |
| LOW | …ner/vlmevalkit/vlmeval/dataset/video_dataset_config.py | 3 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 2 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 7 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 10 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 11 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 11 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 11 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 11 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 13 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 15 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 17 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 19 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 19 | CODE | |
| LOW | …alkit/vlmeval/dataset/metropolis_event_verification.py | 7 | CODE | |
| LOW | …reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py | 6 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 34 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/foxbench.py | 1 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 3 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 4 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 5 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 6 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 7 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 9 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py | 6 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py | 6 | CODE | |
| LOW | …lmevalkit/vlmeval/dataset/sop_temporal_localization.py | 6 | CODE | |
| LOW | …s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py | 6 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py | 6 | CODE | |
| LOW | …evalkit/vlmeval/dataset/temporal_localization_bench.py | 6 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 68 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 179 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 179 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 179 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 179 | CODE | |
| LOW | …alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py | 6 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py | 6 | CODE | |
| 614 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | evaluation/cosmos3/reasoner/vlmevalkit/run.py | 84 | def build_dataset_from_config(cfg, dataset_name): | CODE |
| LOW⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 605 | async def _run_eval_in_main_process(self, dataset_name: str): | CODE |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 50 | def trans_greater_sign_to_interval(self, expr: str): | CODE |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 91 | def parse_score_from_response(resp): | CODE |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/medqbench_mcq.py | 84 | def _convert_answer_to_letter(self, answer): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py | 518 | def download_vladbench_from_hf(): | CODE |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 168 | def _convert_single_page_pdf_to_png(pdf_path, output_path, dpi=350): | CODE |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 426 | def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py | 13 | def format_model_answer_tolist(model_answer, task_gt): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/ost_bench.py | 66 | def longest_common_subsequence(str1, str2): | STRING |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 373 | def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name): | CODE |
| LOW | …alkit/vlmeval/dataset/metropolis_event_verification.py | 319 | def test_metropolis_event_verification_dataset(): | CODE |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 120 | def extract_score_from_direct_gpt_resp(raw_score): | STRING |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 158 | def extract_score_from_p_level_gpt_resp(raw_score): | STRING |
| LOW⚡ | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 178 | def extract_score_from_cmp_gpt_resp(response_text): | STRING |
| LOW⚡ | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 233 | def _regex_extract_frame_bboxes( | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/omtgbench.py | 122 | def compute_one_to_many_metrics( | CODE |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/spatial457.py | 149 | def build_subtask_instruction(self, level): | CODE |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 202 | def parse_multi_choice_response(response, all_choices, index2ans): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 701 | def doc_to_text_perception_comprehension(self, doc): | CODE |
| LOW | …reasoner/vlmevalkit/vlmeval/dataset/av_speakerbench.py | 17 | def _parse_multi_choice_response(response, all_choices): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 20 | def evaluate_hipho_single_problem(judge_model, row, index, judge_kwargs, dataset_instance): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 594 | def _evaluate_single_criterion(self, prediction, criterion, judge_model, | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 701 | def _has_multiple_marking_sets(self, marking): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 715 | def _evaluate_multiple_marking_sets(self, prediction, marking_sets, points, judge_model, question): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 736 | def _evaluate_single_marking_set(self, prediction, marking, points, judge_model, question): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 796 | def _extract_score_from_response(self, response): | CODE |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py | 111 | def proportional_sample_from_lists(num_list, k): | STRING |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py | 123 | def split_txt_based_on_images(text, image_paths): | STRING |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py | 166 | def _resize_frames_to_fit_pixels(self, images, total_pixels): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py | 16 | def _find_first_bracketed_list(text: str) -> Optional[str]: | CODE |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py | 69 | def _parse_python_list_from_text(text) -> Optional[List[int]]: | CODE |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py | 89 | def _parse_all_python_int_lists_from_text(text) -> List[List[int]]: | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 129 | def _parse_subdataset_from_path(rel_path): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 301 | def _extract_yesno_and_explanation(text): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 325 | def _extract_letter_and_explanation(text): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 371 | def _assert_nonempty_references(references, task_name): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 430 | def _parse_timestamp_to_seconds(ts): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 470 | def _eval_temporal_localization(self, data, **judge_kwargs): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 518 | def _eval_temporal_description(self, data, **judge_kwargs): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 521 | def _eval_video_summarization(self, data, **judge_kwargs): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/visfactor.py | 61 | def extract_last_uppercase_letter(self, s): | CODE |
| LOW⚡ | …asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py | 114 | def parse_score_from_response(resp): | CODE |
| LOW⚡ | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 100 | def _save_images_and_update_df(self, df, base_image_dir): | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 69 | def _download_from_huggingface(self, data_path, image_dir): | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 209 | def parse_answer_from_response(response, category): | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 495 | def _process_graph_isomorphism_pairs(pairs_dict, results_dict, data_judged, results_by_category): | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/longvideobench.py | 37 | def insert_subtitles_into_frames( | CODE |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py | 325 | def load_into_video_and_process(self, line): | STRING |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py | 156 | def _download_dataset_from_dss(self, dataset_name): | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py | 212 | def _generate_tsv_from_events(self, local_dir, tsv_path): | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py | 573 | def compute_bert_scores_remote(candidates: List[str], references: List[str]) -> List[float]: | CODE |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2499 | def extract_content_in_braces(self, input_str): | CODE |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2508 | def compare_string_with_values(self, input_str, target_values): | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2594 | def convert_to_vlmeval_format(example): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py | 15 | def get_multi_choice_prediction(response, all_choices, index2ans): | CODE |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py | 113 | def get_fill_blank_prediction(response, answer): | CODE |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mvbench.py | 295 | def load_into_video_and_process(self, line): | STRING |
| LOW | …lmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py | 19 | def extract_answer_from_response(response): | CODE |
| LOW | …alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py | 223 | def calculate_semantic_repetition(text: str) -> float: | CODE |
| 367 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 199 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 201 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 215 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 217 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 250 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 252 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 281 | # ============================================== | STRING |
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 283 | # ============================================== | STRING |
| MEDIUM⚡ | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 354 | # ============================================== | STRING |
| MEDIUM⚡ | evaluation/cosmos3/reasoner/vlmevalkit/run_api.py | 356 | # ============================================== | STRING |
| MEDIUM⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 74 | # ========================================== | COMMENT |
| MEDIUM⚡ | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 76 | # ========================================== | COMMENT |
| MEDIUM | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 138 | # ========================================== | COMMENT |
| MEDIUM | …n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py | 140 | # ========================================== | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 38 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 40 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 55 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 57 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 168 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 170 | # --------------------------------------------------------------------------- | STRING |
| MEDIUM⚡ | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 222 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 224 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 343 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 345 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 511 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 513 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 558 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 560 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 613 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 615 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 732 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 734 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 763 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 765 | # ------------------------------------------------------------------ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 20 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 22 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 58 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 60 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 134 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 136 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 185 | # ============================================================================ | COMMENT |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 187 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 18 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 20 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 29 | # ============================================================================ | COMMENT |
| MEDIUM | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 62 | # ============================================================================ | COMMENT |
| MEDIUM | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 64 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 422 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 424 | # ============================================================================ | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 498 | # ======================================================================== | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 500 | # ======================================================================== | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py | 25 | # --------------------------------- | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py | 56 | # --------------------------------- | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py | 279 | #-------------------------------------------------- | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 835 | # # =================== | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 837 | # # =================== | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 851 | # # =================== | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 853 | # # =================== | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 906 | # # =================== | COMMENT |
| MEDIUM | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 908 | # # =================== | COMMENT |
| 79 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 201 | # Check if two numerical values are equal within an allowed error range | COMMENT |
| LOW⚡ | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 267 | # Check if two equations are mathematically equivalent | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 101 | # Check if two < or > in expression | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 121 | # Check if elements in both lists can be paired and are equal | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 158 | # Set timeout to 30 seconds for is_equal | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 217 | # Check if two expressions are mathematically equivalent | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py | 294 | # Check if two intervals are mathematically equivalent | COMMENT |
| LOW | …valkit/vlmeval/dataset/medqbench_paired_description.py | 333 | # Check if prediction column exists, if not, evaluation cannot proceed | COMMENT |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 102 | # Check if the member is a file (not a directory) | STRING |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py | 120 | # Check if the member is a file (not a directory) | STRING |
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py | 100 | # Check if the member is a file (not a directory) | STRING |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 482 | # Check if py_root/v1 exists | COMMENT |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 219 | # Check if video directory exists and has videos | COMMENT |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 504 | # Check if any category name (lowercased) appears in the video name | COMMENT |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 893 | # Check if verbose is enabled in judge_kwargs | COMMENT |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 173 | # Check if video directory exists and has videos | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 27 | # Check if the options are already appended with letters | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 503 | # Check if the member is a file (not a directory) | COMMENT |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py | 108 | # Check if the member is a file (not a directory) | STRING |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py | 126 | # Check if the member is a file (not a directory) | STRING |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py | 113 | # Check if TSV already exists | COMMENT |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py | 485 | # Check if point is in mask (non-zero) | COMMENT |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 199 | # Check if this is HiPhO_ALL mode | COMMENT |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 219 | # Check if this subset is already completed | COMMENT |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 510 | # Check if API failed | COMMENT |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 622 | # Check if it's an API failure message | COMMENT |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 626 | # Check if response is empty or invalid | COMMENT |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py | 125 | # Check if all frames already exist | COMMENT |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py | 68 | # Check if already extracted | COMMENT |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py | 264 | # Check if we need LLM judge | COMMENT |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py | 336 | # Check if prediction column exists, if not, evaluation cannot proceed | COMMENT |
| LOW⚡ | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 109 | # Check if image_list is empty | COMMENT |
| LOW⚡ | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py | 292 | # Check if image name is in reverse_xxx format | COMMENT |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py | 54 | # Check if TSV exists locally | COMMENT |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py | 110 | # Check if question has <image> placeholders | COMMENT |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py | 113 | # Check if the data file exists | STRING |
| LOW | …s3/reasoner/vlmevalkit/vlmeval/dataset/video_holmes.py | 98 | # Check if the member is a file (not a directory) | STRING |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py | 683 | # Print results table | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/asclepius.py | 126 | # Check if evaluation results already exist | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 949 | # Check if dataset is cached and has integrity | COMMENT |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py | 184 | # Check if this line has pipe characters (a table row indicator) | COMMENT |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py | 287 | # Check if this looks like a separator row (contains mostly dashes) | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 1003 | # Check if a 2x2 block is formed by the same value | STRING |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 1013 | # Check if the response is a 6x6 matrix | STRING |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 1017 | # Verify if the base matrix has been changed | STRING |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 1025 | # Check if 0s and 1s form connected components | STRING |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 1030 | # Check if there are any 2x2 blocks of 0s or 1s | STRING |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 820 | # Verify if the base matrix has been changed | STRING |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 910 | # Verify if the base matrix has been changed | STRING |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 954 | # Verify if the base matrix has been changed | STRING |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 992 | # Check if all cells with the given value are visited | STRING |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/utils/tempcompass.py | 151 | # Check if the chatgpt answer is the ground-truth answer | STRING |
| LOW | …ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py | 54 | # Check if in main thread | COMMENT |
| LOW | …ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py | 1027 | # Check if judge model is available | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py | 641 | # Check if the expression is a power expression | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py | 646 | # Check if the base and the exponent are numbers | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py | 651 | # Check if the exponent is greater than the threshold | COMMENT |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/judge_util.py | 122 | # Check if using NVIDIA API | COMMENT |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py | 329 | # Check if there is an overlap | COMMENT |
| LOW | …er/vlmevalkit/vlmeval/dataset/utils/vgrpbench/score.py | 110 | # Check if cell is a number (not 0) or not a string/character | COMMENT |
| 75 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py | 100 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py | 432 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py | 122 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py | 91 | CODE | |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py | 552 | CODE | |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py | 219 | CODE | |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py | 228 | CODE | |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py | 713 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py | 132 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 527 | CODE | |
| LOW | …ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vcr.py | 182 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py | 3586 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py | 683 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py | 958 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py | 1089 | CODE | |
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/slidevqa.py | 62 | CODE | |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py | 114 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2029 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2091 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2282 | CODE | |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 2422 | CODE | |
| LOW | …n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py | 325 | CODE | |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dude.py | 48 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lens.py | 219 | CODE | |
| LOW | …reasoner/vlmevalkit/vlmeval/dataset/utils/mathvista.py | 167 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 47 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 93 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 145 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 187 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 210 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py | 232 | CODE | |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/logicvista.py | 151 | CODE | |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/utils/omni3dbench.py | 100 | CODE | |
| LOW | …asoner/vlmevalkit/vlmeval/dataset/utils/videoholmes.py | 38 | CODE | |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/ocr_reasoning.py | 173 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py | 107 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py | 142 | CODE | |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py | 165 | CODE | |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py | 513 | CODE | |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py | 539 | CODE | |
| LOW | …vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py | 329 | CODE | |
| LOW | …ls/chartmimic/evaluator/chart_type_evaluator_prefix.py | 35 | CODE | |
| LOW | …ls/chartmimic/evaluator/chart_type_evaluator_prefix.py | 99 | CODE | |
| LOW | …ls/chartmimic/evaluator/chart_type_evaluator_prefix.py | 187 | CODE | |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 49 | CODE | |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 115 | CODE | |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 205 | CODE | |
| LOW | …t/utils/chartmimic/evaluator/color_evaluator_prefix.py | 65 | CODE | |
| LOW | …t/utils/chartmimic/evaluator/color_evaluator_prefix.py | 132 | CODE | |
| LOW | …t/utils/chartmimic/evaluator/color_evaluator_prefix.py | 232 | CODE | |
| LOW | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 335 | CODE | |
| LOW | …meval/dataset/utils/SArena/video/viclip/viclip_text.py | 216 | CODE | |
| LOW | …dataset/utils/megabench/scoring/latex_expr_equality.py | 75 | CODE | |
| LOW | …eval/dataset/utils/megabench/scoring/sacrebleu_bleu.py | 23 | CODE | |
| LOW | …eval/dataset/utils/megabench/parsing/common/parsers.py | 146 | CODE | |
| LOW | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 338 | CODE | |
| LOW | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 876 | CODE | |
| LOW | …reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py | 426 | CODE | |
| LOW | …s3/reasoner/vlmevalkit/vlmeval/vlm/cosmos_reason_hf.py | 47 | CODE | |
| LOW | …ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py | 25 | CODE | |
| 137 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | evaluation/cosmos3/reasoner/vlmevalkit/run.py | 529 | # Create the symbolic links for the prediction files | STRING |
| MEDIUM | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 592 | # Create a simple test PDF using matplotlib | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py | 150 | # Create a copy of line to avoid SettingWithCopyWarning | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 24 | # Define the option letters based on the number of options | COMMENT |
| MEDIUM | …er/vlmevalkit/vlmeval/dataset/olmOCRBench/evaluator.py | 98 | # Define an inner function to evaluate a single test | COMMENT |
| MEDIUM | …vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py | 216 | # Create a new page. | COMMENT |
| MEDIUM⚡ | …3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py | 430 | # Create a base temporary directory for this evaluation run | COMMENT |
| MEDIUM⚡ | …s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py | 430 | # Create a dummy video on failure to prevent crash in metrics | COMMENT |
| MEDIUM | …asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py | 43 | # Create the prediction column based on the Score and Answer columns | COMMENT |
| MEDIUM | …asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py | 122 | # Create the prediction column based on the Score and Answer columns | COMMENT |
| MEDIUM | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py | 77 | # Define the keys to locate in the text | STRING |
| MEDIUM | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py | 111 | # Define the keys to locate in the text | STRING |
| MEDIUM | …meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py | 40 | # Create a list of non-zero values in the row | COMMENT |
| MEDIUM | …meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py | 55 | # Create a list of non-zero values in the column | COMMENT |
| MEDIUM | …taset/utils/vgrpbench/puzzles/common_puzzle_factory.py | 30 | # Create a new board filled with zeros | COMMENT |
| MEDIUM | …lmeval/dataset/utils/vgrpbench/puzzles/thermometers.py | 23 | # Create a set of all thermometer positions for efficient lookup | COMMENT |
| MEDIUM | …eval/dataset/utils/chartmimic/evaluator/color_utils.py | 24 | # Create an sRGBColor object from RGB values | COMMENT |
| MEDIUM | …ataset/utils/mmhelix/evaluators/graph_problems_eval.py | 526 | # Create a new cycle with the first node repeated at the end for validation | COMMENT |
| MEDIUM⚡ | …meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py | 132 | # Create a new simulator to test the ground truth solution | COMMENT |
| MEDIUM⚡ | …meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py | 142 | # Define a Sokoban simulator class within the evaluator | COMMENT |
| MEDIUM | …vlmeval/dataset/utils/design2code/screenshot_single.py | 66 | # Define the arguments | STRING |
| MEDIUM | …lmeval/dataset/utils/megabench/tools/analysis_utils.py | 178 | meta = all_meta[task_name].copy() # Create a copy to avoid modifying original | CODE |
| MEDIUM | …set/utils/megabench/scoring/general_numerical_match.py | 35 | # Create a multiprocessing.Queue to receive the output | COMMENT |
| MEDIUM | …meval/dataset/utils/megabench/scoring/program_judge.py | 45 | # Create a CodeTester instance with the response and the found test cases | COMMENT |
| MEDIUM | …/dataset/utils/megabench/scoring/common/conversions.py | 221 | # Create a new image with the calculated size | COMMENT |
| MEDIUM | …vlmevalkit/vlmeval/dataset/utils/ssi_bench/__init__.py | 1 | # This file is intentionally left empty to make this directory a Python package. | COMMENT |
| MEDIUM | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 376 | # Create a dictionary for each combination of values. | COMMENT |
| MEDIUM | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 388 | # Create a dictionary for each combination of values. | COMMENT |
| MEDIUM | …asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py | 78 | # Create the prediction column based on the Score and Answer columns | COMMENT |
| MEDIUM | …asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py | 157 | # Create the prediction column based on the Score and Answer columns | COMMENT |
| MEDIUM | …soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py | 1775 | # Define a regex pattern to find Markdown tables with newlines | COMMENT |
| MEDIUM | …soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py | 1905 | # Define a regex pattern to find Markdown tables with newlines | COMMENT |
| MEDIUM | …uation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vila.py | 173 | # Create a unique temporary directory for this inference call | COMMENT |
| MEDIUM | …n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/long_vita.py | 129 | # Create a new image with the target size and paste the resized image onto it | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py | 64 | # Define the temporary folder for processed images | COMMENT |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py | 211 | # Define a minimum dimension for the crop | COMMENT |
| MEDIUM | …mos3/reasoner/vlmevalkit/vlmeval/vlm/qwen2_vl/model.py | 115 | # Create a temporary file for the frame | COMMENT |
| MEDIUM | …s3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/mm_utils.py | 69 | # Create a new image with the target size and paste the resized image onto it | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/groundingme.py | 13 | Ensure that all details mentioned about the object are accurate. Provide at most one bounding box. If a matching object | CODE |
| HIGH | …vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py | 264 | return null; | CODE |
| HIGH | …vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py | 310 | list.push({ | CODE |
| HIGH | …vlmevalkit/vlmeval/dataset/utils/physics_eval_utils.py | 83 | sympy_result = sympy_result or expr1_sympy.equals(expr2_sympy) | CODE |
| HIGH | …s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py | 406 | if (root && typeof root.pauseAnimations === 'function') root.pauseAnimations(); | CODE |
| HIGH | …s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py | 407 | if (root && typeof root.setCurrentTime === 'function') root.setCurrentTime({t}); | CODE |
| HIGH | …oner/vlmevalkit/vlmeval/dataset/utils/tablevqabench.py | 21 | 'If you have multiple answers, please separate them with || marks. Example: Apple||Banana||Tomato\n\n' | CODE |
| HIGH | …kit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py | 81 | # 2. We don't have enough potential lights (current + undefined) to reach the required number | COMMENT |
| HIGH | …lmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py | 37 | # Check if current mines <= number <= potential mines (current + undefined) | COMMENT |
| HIGH | …it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py | 14 | # Convert line to runs of filled cells ('s' = filled, 'e' = empty, 0 = undefined) | COMMENT |
| HIGH | …it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py | 33 | else: # cell is 0 (undefined) | CODE |
| HIGH | …dataset/utils/megabench/scoring/latex_expr_equality.py | 36 | result = 1 if responses_expr.equals(targets_expr) else 0 | CODE |
| HIGH⚡ | …ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py | 69 | " cd VLM-3R && pip install -e .\n" | CODE |
| HIGH⚡ | …ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py | 70 | " cd CUT3R && pip install -r requirements.txt\n" | CODE |
| HIGH | …/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/cambrian_s.py | 66 | " cd cambrian-s && pip install -e .\n" | CODE |
| HIGH | …n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py | 154 | " cd Bagel && uv pip install -e .\n" | CODE |
| HIGH | …n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py | 155 | " OR: cd Bagel && pip install -e .\n" | CODE |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/vlm/spatial_mllm.py | 114 | "2. Install in development mode: cd Spatial-MLLM && uv pip install -e .\n" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 81 | ans = judge_model.generate(messages) | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py | 221 | # 'mcq_openended_bleu': 1.0, | COMMENT |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/reasonmap_plus.py | 1 | # ReasonMap Plus Dataset is an extension of the original ReasonMap dataset, | COMMENT |
| LOW | …valkit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py | 21 | # 3: parameter list (e.g., | COMMENT |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py | 1 | # flake8: noqa | COMMENT |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py | 21 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | COMMENT |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py | 41 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | COMMENT |
| LOW | …easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py | 61 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | COMMENT |
| LOW | …oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py | 801 | grid = response_list | COMMENT |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/dataset/utils/mmsci.py | 261 | COMMENT | |
| LOW | …ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py | 1 | # Copyright 2025 Garena Online Private Limited | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_evaluator.py | 121 | # def _get_prefix(self): | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_evaluator.py | 141 | # # name = func.__name__ | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_evaluator.py | 161 | # Axes.hist = log_function(Axes.hist) | COMMENT |
| LOW | …/dataset/utils/chartmimic/evaluator/color_evaluator.py | 21 | COMMENT | |
| LOW | …/dataset/utils/chartmimic/evaluator/color_evaluator.py | 201 | # similarities = [item[1] for item in similarities] | COMMENT |
| LOW | …ls/chartmimic/evaluator/chart_type_evaluator_prefix.py | 361 | # Axes.errorbar = log_function(Axes.errorbar) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 521 | Axes3D.plot_surface = log_function_for_3d(Axes3D.plot_surface) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 541 | # ax.axvline(0.5) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 561 | # loglog test | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 581 | # ax.fill_betweenx(x, y1, y2, cmap='viridis') | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 601 | # plt.show() | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 621 | # ax.set_xlabel('X-axis') | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 641 | # fig, ax = plt.subplots() | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 661 | # v = np.ones(10) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 681 | # ax.plot([0, 1], [0, 1], [1, 0], color='b') | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 701 | # dz = np.random.rand(10) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 721 | # plot in test | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 741 | # draw the pcolor plot, with a beautiful colormap | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 761 | # X, Y = np.meshgrid(x, y) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 781 | # x = np.linspace(0, 10, 100) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 821 | # fig, ax = plt.subplots(subplot_kw={'projection': 'polar'}) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 841 | # num_vars = len(values_pc1) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 861 | # ax.scatter(angles[:-1], values_pc2[:-1], color="red", s=50) | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 881 | # Line2D( | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 901 | # # Add legend and title | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 921 | # # =================== | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 941 | # ] | COMMENT |
| LOW | …set/utils/chartmimic/evaluator/chart_type_and_color.py | 961 | # ax.set_yticks([1, 2, 3]) | COMMENT |
| LOW | …kit/vlmeval/dataset/utils/mmif/function_and_compare.py | 401 | COMMENT | |
| LOW | …kit/vlmeval/dataset/utils/mmif/function_and_compare.py | 421 | # else: | COMMENT |
| LOW | …lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py | 561 | # initial_state: JSON string representing the grid structure | COMMENT |
| LOW | …lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py | 581 | # converted_answer = {} | COMMENT |
| LOW | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 1 | # flake8: noqa | COMMENT |
| LOW | …eval/dataset/utils/Ocrbench_v2/spotting_eval/script.py | 1 | # flake8: noqa | COMMENT |
| LOW | …3/reasoner/vlmevalkit/vlmeval/dataset/GUI/osworld_g.py | 341 | click_point = self.parse_response_func(prediction) | COMMENT |
| LOW | …ner/vlmevalkit/vlmeval/dataset/Omni3D/install_patch.sh | 61 | # echo "[INFO] Updating ~/.bashrc (tclsh + PYTHONPATH)..." | COMMENT |
| LOW | …valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py | 41 | if ord(char) > 127: # Non-ASCII characters | COMMENT |
| LOW | …valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py | 341 | # for match in inline_matches: | COMMENT |
| LOW | …valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py | 361 | # # 'category_type': 'equation_inline', | COMMENT |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py | 161 | # Second pass: Inline images (re-using the same images? Need to check if processor | COMMENT |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py | 181 | # the content structure. | COMMENT |
| LOW | …os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py | 201 | pass | COMMENT |
| LOW | …vlmevalkit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py | 1 | # Adapted from LLaVA code. | COMMENT |
| LOW | …awk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py | 1 | # -------------------------------------------------------- | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/utils.py | 1 | # Adopted from https://github.com/haotian-liu/LLaVA. Below is the original copyright: | COMMENT |
| LOW | …meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py | 461 | COMMENT | |
| LOW | …r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py | 1 | # Copyright (c) 2023-2024 DeepSeek. | COMMENT |
| LOW | …it/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py | 1 | # Copyright (c) 2023-2024 DeepSeek. | COMMENT |
| LOW | …lmevalkit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py | 241 | # if vision_feature_select_strategy == "default": | COMMENT |
| 17 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py | 74 | Calculate scores by category from evaluation results. Args: result_file: Path to the file | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py | 326 | Loads a video or image sequence, processes it, and returns the path to the processed video. Args: | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py | 399 | Builds a prompt for a language model based on the provided data and settings. Args: line ( | STRING |
| HIGH | …os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py | 447 | Evaluates the given evaluation file and generates ratings based on different dimensions. Args: | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py | 58 | Map question IDs to their categories based on the evaluation mode. Args: mode: Either "descriptive" or | STRING |
| HIGH | …cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py | 139 | Calculate scores by category from evaluation results. Args: result_file: Path to the file | STRING |
| HIGH | …asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py | 1077 | Load a single test from a JSON line string or JSON object. Args: data: Either a JSON string to parse o | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py | 48 | Load bbox_3d and label arrays from a JSON annotation file. Args: json_path: Path to JSON file containi | STRING |
| HIGH | …reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py | 71 | Normalize label by extracting base name and removing suffixes. Removes numeric suffixes and underscores (e.g., | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 53 | "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference image | CODE |
| MEDIUM | …os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py | 234 | # Save comprehensive results | COMMENT |
| MEDIUM | …asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py | 256 | """image is [good/usable/reject]". Please provide a comprehensive but concise assessment in """ | STRING |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/sitebench.py | 375 | # Use os.path.relpath for robust relative path extraction | COMMENT |
| MEDIUM | …meval/dataset/utils/mmhelix/evaluators/bridges_eval.py | 94 | # Various bridge patterns - more comprehensive | COMMENT |
| MEDIUM | …val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py | 314 | # Extract the predicted grid using robust parsing | COMMENT |
| MEDIUM | …val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py | 476 | # Extract predicted answer using the robust extract_answer method | COMMENT |
| MEDIUM | …meval/dataset/utils/mmhelix/evaluators/binario_eval.py | 518 | # More comprehensive patterns for matrix keywords | COMMENT |
| MEDIUM | …lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py | 32 | """Enhanced robust extraction of answer from model output with comprehensive patterns.""" | STRING |
| MEDIUM | …lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py | 101 | """Extract coordinate-value patterns with comprehensive regex.""" | STRING |
| MEDIUM | …lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py | 569 | # # If it's a string, try to extract the answer using robust parsing | COMMENT |
| LOW | …set/utils/megabench/scoring/xml_norm_point_distance.py | 36 | # Instead of normalizing by 1/sqrt(2), we just set it to 0 if the distance is above 1. | COMMENT |
| LOW | …lmeval/dataset/utils/megabench/parsing/common/utils.py | 137 | # the result overflows, simply return an empty string | COMMENT |
| MEDIUM | …cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py | 334 | # ThreeDAVGroundingBench: the harness writes a transposed 1-row DF as a flat | COMMENT |
| MEDIUM | …cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py | 355 | # AETCBench: harness writes a transposed 1-row DF as a flat dict — metric | COMMENT |
| MEDIUM | …cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py | 632 | # --- RynnScale harness adapter (layer-1: native <suite>.json -> standardized scores) --- | COMMENT |
| MEDIUM | …luation/cosmos3/generator/paibench_c/run_paibench_c.sh | 707 | # Use a robust python for printing results - prefer cosmos3 venv if functional, | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 179 | # Step 1: Find the last occurrence of 'summary:' | STRING |
| LOW⚡ | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 184 | # Step 2: Slice the string after 'summary:' and extract value | STRING |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 210 | # Step 1: Clean up punctuation from the response | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 235 | # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D) | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 243 | # Step 4: If no candidates, look for choices with a space after (A B C D) | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 250 | # Step 5: If no candidates and response has more than 5 tokens, try parsing based on content | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 258 | # Step 6: If still no candidates, randomly choose one | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 262 | # Step 7: If multiple candidates found, use the one appearing last | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 222 | # Step 2: If no candidates, look for choices with a period after (A. B. C. D.) | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 211 | # Step 1: Try template matching | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 216 | # Step 2: Fall back to LLM matching | COMMENT |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py | 74 | Predicted answer: ### Step 1: Calculate ( l_1 ) | CODE |
| LOW | …er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py | 55 | # Step 1: Reformatting the Solution | COMMENT |
| LOW | …er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py | 83 | # Step 2: Evaluating Correctness | COMMENT |
| LOW⚡ | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 355 | # Step 3: Validate segments format | COMMENT |
| LOW⚡ | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 359 | # Step 4: Validate using initial state and game rules | COMMENT |
| LOW | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 313 | # Step 1: Extract and validate predicted_answer | COMMENT |
| LOW | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 331 | # Step 2: Validate initial_state | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md | 5 | ## Step 0: Install LMDeploy | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md | 12 | ## Step 1: Start the Inference Service | COMMENT |
| LOW | …/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md | 23 | ## Step 2: Evaluation | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 179 | # Step 1: Find the last occurrence of 'summary:' | STRING |
| LOW⚡ | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py | 184 | # Step 2: Slice the string after 'summary:' and extract value | STRING |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 210 | # Step 1: Clean up punctuation from the response | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 235 | # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D) | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 243 | # Step 4: If no candidates, look for choices with a space after (A B C D) | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 250 | # Step 5: If no candidates and response has more than 5 tokens, try parsing based on content | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 258 | # Step 6: If still no candidates, randomly choose one | COMMENT |
| LOW⚡ | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 262 | # Step 7: If multiple candidates found, use the one appearing last | COMMENT |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py | 222 | # Step 2: If no candidates, look for choices with a period after (A. B. C. D.) | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 211 | # Step 1: Try template matching | COMMENT |
| LOW⚡ | …oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py | 216 | # Step 2: Fall back to LLM matching | COMMENT |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py | 74 | Predicted answer: ### Step 1: Calculate ( l_1 ) | CODE |
| LOW | …er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py | 55 | # Step 1: Reformatting the Solution | COMMENT |
| LOW | …er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py | 83 | # Step 2: Evaluating Correctness | COMMENT |
| LOW⚡ | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 355 | # Step 3: Validate segments format | COMMENT |
| LOW⚡ | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 359 | # Step 4: Validate using initial state and game rules | COMMENT |
| LOW | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 313 | # Step 1: Extract and validate predicted_answer | COMMENT |
| LOW | …eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py | 331 | # Step 2: Validate initial_state | COMMENT |
| LOW | …ataset/utils/mmhelix/evaluators/graph_problems_eval.py | 283 | # we need to check if the last node connects back to the first node | COMMENT |
| LOW | …ataset/utils/mmhelix/evaluators/graph_problems_eval.py | 508 | # If not, we need to check if the last node connects back to the first node | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py | 486 | __all__ = [ | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/dataset/utils/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | …meval/dataset/utils/SArena/video/viclip/viclip_text.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | …it/vlmeval/dataset/utils/SArena/video/viclip/viclip.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | …vlmevalkit/vlmeval/dataset/utils/megabench/__init__.py | 5 | __all__ = [AggregationType, MetricType, ResponseParseType] | CODE |
| LOW | …t/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py | 16 | logger = logging.getLogger(__name__) | CODE |
| LOW | …valkit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py | 5 | logger = logging.getLogger(__name__) | CODE |
| LOW | …er/vlmevalkit/vlmeval/dataset/metropolis2d/__init__.py | 5 | __all__ = ['Metropolis2DDetectionDataset', 'Metropolis2DGroundingDataset', 'Astro2DDetectionDataset'] | CODE |
| LOW | …/cosmos3/reasoner/vlmevalkit/vlmeval/utils/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …tion/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/points.py | 226 | def set_image_processor(self, dataset: str) -> None: | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/vlm/video_llm/__init__.py | 8 | __all__ = ['VideoLLaVA', 'VideoLLaVA_HF', 'Chatunivi', 'VideoChatGPT', 'LLaMAVID', 'VideoChat2_HD', 'PLLaVA'] | CODE |
| LOW | …/reasoner/vlmevalkit/vlmeval/vlm/xcomposer/__init__.py | 7 | __all__ = ['ShareCaptioner', 'XComposer', 'XComposer2', 'XComposer2_4KHD', 'XComposer2d5'] | CODE |
| LOW | …oner/vlmevalkit/vlmeval/vlm/granite_vision/__init__.py | 3 | __all__ = ['GraniteVision3'] | CODE |
| LOW | …s3/reasoner/vlmevalkit/vlmeval/vlm/qtunevl/__init__.py | 4 | __all__ = ['QTuneVL', 'QTuneVLChat'] | CODE |
| LOW | …3/reasoner/vlmevalkit/vlmeval/vlm/interns1/__init__.py | 3 | __all__ = ['InternS1Chat'] | CODE |
| LOW⚡ | …soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py | 105 | def set_dump_image(self, flag: bool) -> None: | CODE |
| LOW⚡ | …soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py | 108 | def set_batch_size(self, bs: int) -> None: | CODE |
| LOW⚡ | …soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py | 111 | def set_output_folder(self, out_dir: str) -> None: | CODE |
| LOW | …3/reasoner/vlmevalkit/vlmeval/vlm/internvl/__init__.py | 3 | __all__ = ['InternVLChat'] | CODE |
| LOW | …mos3/reasoner/vlmevalkit/vlmeval/vlm/llava/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/vlm/ovis/__init__.py | 3 | __all__ = ['Ovis', 'Ovis1_6', 'Ovis1_6_Plus', 'Ovis2', 'OvisU1', 'Ovis2_5'] | CODE |
| LOW | …meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py | 504 | def set_grad_checkpointing(self, enable: bool = True) -> None: | CODE |
| LOW | …lmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | …lmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py | 8 | __all__ = [ | CODE |
| LOW | …l/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | …3/reasoner/vlmevalkit/vlmeval/vlm/qwen3_vl/__init__.py | 3 | __all__ = [ | CODE |
| LOW | …r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py | 436 | def set_grad_checkpointing(self, enable: bool = True) -> None: | CODE |
| LOW | …ner/vlmevalkit/vlmeval/vlm/ursa/ursa_model/__init__.py | 20 | __all__ = [ | CODE |
| LOW | …s3/reasoner/vlmevalkit/vlmeval/api/sensechat_vision.py | 193 | def set_max_num(self, dataset: Optional[str] = None) -> None: | CODE |
| LOW | …on/cosmos3/reasoner/vlmevalkit/vlmeval/api/__init__.py | 41 | __all__ = [ | CODE |
| LOW | …3/reasoner/vlmevalkit/vlmeval/api/adapters/__init__.py | 8 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py | 53 | "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference image | CODE |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py | 43 | # I will dump all original keys as requested. | COMMENT |
| HIGH | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py | 158 | # Format as requested: dictionary composed of lists, organized into a pandas.DataFrame | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py | 2502 | # Once you get the permission, you can use the helper code here to download and extract necessary images: | COMMENT |
| MEDIUM | …osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py | 29 | Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solv | CODE |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py | 11 | Your task is to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's ho | CODE |
| MEDIUM | …smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py | 32 | Here's how you can accomplish the task: | CODE |
| MEDIUM | …easoner/vlmevalkit/vlmeval/dataset/utils/video_mmlu.py | 44 | Here's how you can accomplish the task: | CODE |
| MEDIUM | …oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py | 39 | Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solv | CODE |
| MEDIUM | …reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py | 33 | Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solv | CODE |
| LOW | …it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py | 36 | # Don't forget to add the last run if it exists | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 321 | CODE | |
| MEDIUM | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 331 | CODE | |
| MEDIUM | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 386 | CODE | |
| MEDIUM | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 397 | CODE | |
| MEDIUM | …val/dataset/utils/SArena/video/viclip/viclip_vision.py | 407 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | …easoner/vlmevalkit/vlmeval/vlm/video_llm/videochat2.py | 263 | seg_embs = [model.mistral_model.base_model.model.model.embed_tokens(seg_t) for seg_t in seg_tokens] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | cookbooks/cosmos3/README.md | 102 | ### System packages (required for Framework guardrails) | COMMENT |
| MEDIUM | cookbooks/cosmos3/README.md | 138 | license and set `HF_TOKEN` as in [Prerequisites](#prerequisites)). To disable guardrails | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py | 538 | self.data['question'] = ['placeholder'] * len(self.data) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py | 328 | def process_data(obj): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py | 1343 | # Example usage | STRING |