Home / NVIDIA / cosmos
AI Slop Code RIsk Monitor

NVIDIA/cosmos

NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.

28.0 Elevated pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for NVIDIA/cosmos, a Jupyter Notebook project with 12,009 GitHub stars. SynthScan examined 218,862 lines of code across 1010 source files, recording 4016 pattern matches distributed across 26 syntactic categories. The legacy adjusted pattern signal is 28.0 (Elevated pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

28.0
Adjusted Signal
28.0
Raw Pattern Score
100%
Time Factor
2026-10-08
Last Push
12.0K
Stars
Jupyter Notebook
Language
218.9K
Lines of Code
1.0K
Files
4.0K
Pattern Hits
2026-10-12
Scan Date
0.40
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 37.1/100 · MEDIUM

Can maintainers explain intent, assumptions, and trade-offs? 1860 finding(s).

Hallucinated Dependency & Supply Chain Risk 14.9/100 · LOW

Are imports declared and package/API names verified? 702 finding(s).

Agent Over-Engineering Ratio 59.8/100 · HIGH

Are wrappers, guards, and repeated structures necessary? 2080 finding(s).

PR Review Velocity Impact 78.6/100 · CRITICAL

Which changes need focused human review? 4016 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 1HIGH 407MEDIUM 342LOW 3266

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 4016 distinct pattern matches across 26 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Excessive Try-Catch Wrapping894 hits · 1013 pts
SeverityFileLineSnippetContext
LOW.github/workflows/validate-notebooks.yml85 except Exception as error:CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py24 except Exception:CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py540 except Exception as e:STRING
LOW⚡evaluation/cosmos3/reasoner/vlmevalkit/run_api.py350 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py377 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py391 except Exception as e:STRING
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py318 except Exception:STRING
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/inference_mt.py38 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py46 except Exception as e:CODE
LOW⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py67 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py205 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py222 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py240 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py257 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py402 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py452 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py489 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py574 except Exception as e:CODE
LOW⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py614 except Exception as e:CODE
LOW⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py623 except Exception as e:CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py644 except Exception as e:CODE
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py609def run_eval():CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py68 except Exception as err:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py82 except Exception as e:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py126 except Exception as e:CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py178 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py172 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py180 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py187 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py195 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py253 except Exception:CODE
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py263 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py92 except Exception:CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py136 except Exception as err:CODE
MEDIUM…valkit/vlmeval/dataset/medqbench_paired_description.py375 print(f"Error during judge evaluation: {e}")CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py71 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py85 except Exception as e:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py102 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py114 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py121 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py158 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py165 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py321 except Exception as e:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py374 except Exception as e:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py395 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py398 except Exception:CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py449 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py42 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py125 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py254 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py283 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py319 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py329 except Exception as e:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py351 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py416 except Exception:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py428 except Exception:CODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py410def str_to_dict(s):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py170 except Exception:STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py134 except Exception:STRING
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py149 except Exception:STRING
834 more matches not shown…
Cross-File Repetition194 hits · 970 pts
SeverityFileLineSnippetContext
HIGH…lmevalkit/vlmeval/dataset/avspecial_collision_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py0stub: dataset implementation not included in this public release.STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py0stub: dataset implementation not included in this public release.STRING
HIGH…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py0stub: dataset implementation not included in this public release.STRING
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py0stub: dataset implementation not included in this public release.STRING
HIGH…lmevalkit/vlmeval/dataset/sop_temporal_localization.py0stub: dataset implementation not included in this public release.STRING
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…evalkit/vlmeval/dataset/temporal_localization_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…oner/vlmevalkit/vlmeval/dataset/warehouse_near_miss.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_reason.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/intphys2.py0stub: dataset implementation not included in this public release.STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/cv_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…r/vlmevalkit/vlmeval/dataset/sop_action_recognition.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/robospatial_home.py0stub: dataset implementation not included in this public release.STRING
HIGH…evalkit/vlmeval/dataset/avspecial_environment_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/blink_depth.py0stub: dataset implementation not included in this public release.STRING
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/its_collision.py0stub: dataset implementation not included in this public release.STRING
HIGH…ner/vlmevalkit/vlmeval/dataset/warehouse_spatial_ai.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/camera_intrinsic.py0stub: dataset implementation not included in this public release.STRING
HIGH…soner/vlmevalkit/vlmeval/dataset/health_surgi_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/locate_anything_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_video.py0stub: dataset implementation not included in this public release.STRING
HIGH…ner/vlmevalkit/vlmeval/dataset/lvs_ai_hallucination.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/cosmos_erqa.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/lv_event_verification.py0stub: dataset implementation not included in this public release.STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/where2place.py0stub: dataset implementation not included in this public release.STRING
HIGH…easoner/vlmevalkit/vlmeval/dataset/cosmos_cab_image.py0stub: dataset implementation not included in this public release.STRING
HIGH…alkit/vlmeval/dataset/avspecial_ood_reasoning_bench.py0stub: dataset implementation not included in this public release.STRING
HIGH…oner/vlmevalkit/vlmeval/dataset/av_prompt_following.py0stub: dataset implementation not included in this public release.STRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/tailgating.py0stub: dataset implementation not included in this public release.STRING
HIGH…er/vlmevalkit/vlmeval/dataset/Omni3D/omni3d_dataset.py0stub: dataset implementation not included in this public release.STRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/IFBench/ifbench.py0stub: dataset implementation not included in this public release.STRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/simplevqa.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py0evaluate model predictions on the charxiv dataset. args: eval_file: path to the file containing model predictions **judgSTRING
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py0these are the frames of a video and the corresponding audio. \ select the best answer to the following multiple-choice qSTRING
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/chartmuseum.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py0build a prompt for the model from a data line. args: line: either an index into the dataset or a pandas series returns: STRING
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
HIGH…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
HIGH…reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py01. model response: 'rounded to two decimal places, the perimeter of the sector is approximately:\n\n(-2, 1)' extracted aSTRING
134 more matches not shown…
Empty Error Handlers182 hits · 910 pts
SeverityFileLineSnippetContext
HIGH…uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py31except ImportError:CODE
HIGH…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py180except Exception:CODE
HIGH…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py187except Exception:CODE
HIGH…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py195except Exception:CODE
HIGH…valkit/vlmeval/dataset/medqbench_paired_description.py114except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/medqbench_mcq.py97except ValueError:CODE
HIGH…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/moat.py121except Exception:CODE
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartbench.py273except (ValueError, SyntaxError):CODE
HIGH…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py175except Exception:CODE
HIGH…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py350except Exception:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py626except Exception:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py641except Exception:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py132except Exception:CODE
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py293except json.JSONDecodeError:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/omtgbench.py44except (ValueError, TypeError):CODE
HIGH…reasoner/vlmevalkit/vlmeval/dataset/refspatialbench.py167except Exception:CODE
HIGH…reasoner/vlmevalkit/vlmeval/dataset/refspatialbench.py123except Exception:CODE
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/spbench.py179except Exception:CODE
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py411except (json.JSONDecodeError, ValueError):CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py666except (json.JSONDecodeError, ValueError):CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py832except ValueError:CODE
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py201except BaseException:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py216except Exception:CODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/vcrbench.py254except Exception:CODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/vcrbench.py296except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py466except json.JSONDecodeError:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py461except json.JSONDecodeError:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py1624except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py3231except IndexError:CODE
HIGH…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py136except Exception:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/omnispatialbench.py212except Exception:CODE
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py583except (ValueError, SyntaxError):CODE
HIGH…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/eriq.py95except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/sparbench.py319except Exception:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/reasonmap_plus.py254except TypeError:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/viewspatialbench.py113except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/sitebench.py232except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/sitebench.py349except Exception:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py441except json.JSONDecodeError:CODE
HIGH…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py450except json.JSONDecodeError:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/mindcubebench.py110except Exception:CODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/miabench.py94except Exception:CODE
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/groundingme.py27except Exception:CODE
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/starebench.py208except Exception:CODE
HIGH…smos3/reasoner/vlmevalkit/vlmeval/dataset/mmsibench.py280except Exception:CODE
HIGH…alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py40except json.JSONDecodeError:CODE
HIGH…alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py35except json.JSONDecodeError:CODE
HIGH…alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py194except Exception:CODE
HIGH…valkit/vlmeval/dataset/SGI_Bench_1_0/dry_experiment.py530except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/seephys.py140except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/seephys.py148except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/seephys.py156except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/seephys.py162except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/seephys.py167except Exception:CODE
HIGH…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py363except OSError:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py213except Exception:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py342except Exception:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py349except Exception:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py137except:CODE
HIGH…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py164except:CODE
122 more matches not shown…
Deep Nesting887 hits · 824 pts
SeverityFileLineSnippetContext
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py227CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py31CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/run_api.py196CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py8CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py25CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py58CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/setup.py66CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py226CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py385CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py443CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py526CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py575CODE
LOWevaluation/cosmos3/reasoner/vlmevalkit/vlmeval/tools.py595CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py103CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py509CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/inference.py200CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/gobench.py73CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/image_base.py129CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py54CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py85CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py216CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py293CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py327CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py329CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py91CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py143CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py272CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py325CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py382CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py182CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py405CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py242CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/megabench.py354CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py64CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py208CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py243CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py85CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/image_shortqa.py92CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py124CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py276CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py309CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py472CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py59CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/dsrbench.py74CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py118CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py149CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py214CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py265CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py308CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py341CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py581CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py132CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py162CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py225CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py252CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py300CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vdc.py354CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/macbench.py150CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py21CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py64CODE
827 more matches not shown…
Unused Imports681 hits · 589 pts
SeverityFileLineSnippetContext
LOW…uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py38CODE
LOW…uation/cosmos3/reasoner/vlmevalkit/vlmeval/__init__.py30CODE
LOW…lmevalkit/vlmeval/dataset/avspecial_collision_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/causalvqa.py6CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lingoqa.py6CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mvpbench.py6CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/share_robot_bench.py6CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/blink_spatial.py6CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py575CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py576CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py577CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py578CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py578CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/odinw13.py6CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/spar_bench.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/motionbench.py6CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py7CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py11CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py15CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py17CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py19CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py19CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/mmoral_opg_closed.py6CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/mmoral_opg_open.py6CODE
LOW…ner/vlmevalkit/vlmeval/dataset/video_dataset_config.py3CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py2CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py7CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py10CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py11CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py13CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py15CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py17CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py19CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py19CODE
LOW…alkit/vlmeval/dataset/metropolis_event_verification.py7CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/av_3d_grounding.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py34CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/foxbench.py1CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py3CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py4CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py5CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py6CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py7CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py9CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/lvs.py6CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/vss_oobe_clips.py6CODE
LOW…lmevalkit/vlmeval/dataset/sop_temporal_localization.py6CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/dataset/camera_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/sat_bench.py6CODE
LOW…evalkit/vlmeval/dataset/temporal_localization_bench.py6CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py68CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py179CODE
LOW…alkit/vlmeval/dataset/avspecial_stop_behavior_bench.py6CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videophy2.py6CODE
621 more matches not shown…
Decorative Section Separators147 hits · 444 pts
SeverityFileLineSnippetContext
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py199 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py201 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py215 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py217 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py250 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py252 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py281 # ==============================================STRING
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run_api.py283 # ==============================================STRING
MEDIUM⚡evaluation/cosmos3/reasoner/vlmevalkit/run_api.py354 # ==============================================STRING
MEDIUM⚡evaluation/cosmos3/reasoner/vlmevalkit/run_api.py356 # ==============================================STRING
MEDIUM⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py74# ==========================================COMMENT
MEDIUM⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py76# ==========================================COMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py138# ==========================================COMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py140# ==========================================COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py38# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py40# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py55# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py57# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py168# ---------------------------------------------------------------------------STRING
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py170# ---------------------------------------------------------------------------STRING
MEDIUM⚡…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py222# ---------------------------------------------------------------------------COMMENT
MEDIUM⚡…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py224# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py343# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py345# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py511# ---------------------------------------------------------------------------COMMENT
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py513# ---------------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py558 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py560 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py613 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py615 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py732 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py734 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py763 # ------------------------------------------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py765 # ------------------------------------------------------------------COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py20# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py22# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py58# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py60# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py134# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py136# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py185# ============================================================================COMMENT
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py187# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py18# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py20# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py29# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py422# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py424# ============================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py498 # ========================================================================COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py500 # ========================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py62# ============================================================================COMMENT
MEDIUM…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py64# ============================================================================COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py25# ---------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py56# ---------------------------------COMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lvs.py279#--------------------------------------------------COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py835# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py837# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py851# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py853# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py906# # ===================COMMENT
MEDIUM…set/utils/chartmimic/evaluator/chart_type_and_color.py908# # ===================COMMENT
87 more matches not shown…
Hyper-Verbose Identifiers429 hits · 421 pts
SeverityFileLineSnippetContext
LOWevaluation/cosmos3/reasoner/vlmevalkit/run.py84def build_dataset_from_config(cfg, dataset_name):CODE
LOW⚡…n/cosmos3/reasoner/vlmevalkit/vlmeval/inference_api.py605 async def _run_eval_in_main_process(self, dataset_name: str):CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py50 def trans_greater_sign_to_interval(self, expr: str):CODE
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py91 def parse_score_from_response(resp):CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/medqbench_mcq.py84 def _convert_answer_to_letter(self, answer):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vladbench.py518def download_vladbench_from_hf():CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py168def _convert_single_page_pdf_to_png(pdf_path, output_path, dpi=350):CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py426 def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py13def format_model_answer_tolist(model_answer, task_gt):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ost_bench.py66 def longest_common_subsequence(str1, str2):STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py373 def _generate_tsv_from_annotations(self, s3, s3_path, local_dir, dataset_name):CODE
LOW…alkit/vlmeval/dataset/metropolis_event_verification.py319def test_metropolis_event_verification_dataset():CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py120def extract_score_from_direct_gpt_resp(raw_score):STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py158def extract_score_from_p_level_gpt_resp(raw_score):STRING
LOW⚡…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py178def extract_score_from_cmp_gpt_resp(response_text):STRING
LOW⚡…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py233def _regex_extract_frame_bboxes(CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/omtgbench.py122def compute_one_to_many_metrics(CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/spatial457.py149 def build_subtask_instruction(self, level):CODE
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py202def parse_multi_choice_response(response, all_choices, index2ans):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py701 def doc_to_text_perception_comprehension(self, doc):CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/av_speakerbench.py17def _parse_multi_choice_response(response, all_choices):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py20def evaluate_hipho_single_problem(judge_model, row, index, judge_kwargs, dataset_instance):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py594 def _evaluate_single_criterion(self, prediction, criterion, judge_model,CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py701 def _has_multiple_marking_sets(self, marking):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py715 def _evaluate_multiple_marking_sets(self, prediction, marking_sets, points, judge_model, question):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py736 def _evaluate_single_marking_set(self, prediction, marking, points, judge_model, question):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py796 def _extract_score_from_response(self, response):CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py111 def proportional_sample_from_lists(num_list, k):STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py123 def split_txt_based_on_images(text, image_paths):STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py166 def _resize_frames_to_fit_pixels(self, images, total_pixels):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py16def _find_first_bracketed_list(text: str) -> Optional[str]:CODE
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py69def _parse_python_list_from_text(text) -> Optional[List[int]]:CODE
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/ssi_bench.py89def _parse_all_python_int_lists_from_text(text) -> List[List[int]]:CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py129def _parse_subdataset_from_path(rel_path):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py301 def _extract_yesno_and_explanation(text):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py325 def _extract_letter_and_explanation(text):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py371 def _assert_nonempty_references(references, task_name):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py430 def _parse_timestamp_to_seconds(ts):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py470 def _eval_temporal_localization(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py518 def _eval_temporal_description(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py521 def _eval_video_summarization(self, data, **judge_kwargs):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/visfactor.py61 def extract_last_uppercase_letter(self, s):CODE
LOW⚡…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py114 def parse_score_from_response(resp):CODE
LOW⚡…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py100 def _save_images_and_update_df(self, df, base_image_dir):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py69 def _download_from_huggingface(self, data_path, image_dir):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py209 def parse_answer_from_response(response, category):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py495 def _process_graph_isomorphism_pairs(pairs_dict, results_dict, data_judged, results_by_category):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/longvideobench.py37def insert_subtitles_into_frames(CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py325 def load_into_video_and_process(self, line):STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py156 def _download_dataset_from_dss(self, dataset_name):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py212 def _generate_tsv_from_events(self, local_dir, tsv_path):CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py573 def compute_bert_scores_remote(candidates: List[str], references: List[str]) -> List[float]:CODE
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2499 def extract_content_in_braces(self, input_str):CODE
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2508 def compare_string_with_values(self, input_str, target_values):CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2594 def convert_to_vlmeval_format(example):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py15def get_multi_choice_prediction(response, all_choices, index2ans):CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py113def get_fill_blank_prediction(response, answer):CODE
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mvbench.py295 def load_into_video_and_process(self, line):STRING
LOW…lmeval/dataset/SGI_Bench_1_0/experimental_reasoning.py19def extract_answer_from_response(response):CODE
LOW…alkit/vlmeval/dataset/SGI_Bench_1_0/idea_generation.py223def calculate_semantic_repetition(text: str) -> float:CODE
369 more matches not shown…
Redundant / Tautological Comments135 hits · 196 pts
SeverityFileLineSnippetContext
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py201 # Check if two numerical values are equal within an allowed error rangeCOMMENT
LOW⚡…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py267 # Check if two equations are mathematically equivalentCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py101 # Check if two < or > in expressionCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py121 # Check if elements in both lists can be paired and are equalCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py158 # Set timeout to 30 seconds for is_equalCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py217 # Check if two expressions are mathematically equivalentCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmmath.py294 # Check if two intervals are mathematically equivalentCOMMENT
LOW…valkit/vlmeval/dataset/medqbench_paired_description.py333 # Check if prediction column exists, if not, evaluation cannot proceedCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py102 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/worldsense.py120 # Check if the member is a file (not a directory)STRING
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/videott.py100 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py482 # Check if py_root/v1 existsCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py219 # Check if video directory exists and has videosCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py504 # Check if any category name (lowercased) appears in the video nameCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py893 # Check if verbose is enabled in judge_kwargsCOMMENT
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py173 # Check if video directory exists and has videosCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py27 # Check if the options are already appended with lettersCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py503 # Check if the member is a file (not a directory)COMMENT
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py108 # Check if the member is a file (not a directory)STRING
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/videomme.py126 # Check if the member is a file (not a directory)STRING
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py113 # Check if TSV already existsCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/refspatial.py485 # Check if point is in mask (non-zero)COMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py199 # Check if this is HiPhO_ALL modeCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py219 # Check if this subset is already completedCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py510 # Check if API failedCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py622 # Check if it's an API failure messageCOMMENT
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py626 # Check if response is empty or invalidCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/video_base.py125 # Check if all frames already existCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py68 # Check if already extractedCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/da2k.py264 # Check if we need LLM judgeCOMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py336 # Check if prediction column exists, if not, evaluation cannot proceedCOMMENT
LOW⚡…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py109 # Check if image_list is emptyCOMMENT
LOW⚡…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/mmhelix.py292 # Check if image name is in reverse_xxx formatCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py54 # Check if TSV exists locallyCOMMENT
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/erqa.py110 # Check if question has <image> placeholdersCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py113 # Check if the data file existsSTRING
LOW…s3/reasoner/vlmevalkit/vlmeval/dataset/video_holmes.py98 # Check if the member is a file (not a directory)STRING
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py683 # Print results tableCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/asclepius.py126 # Check if evaluation results already existCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py949 # Check if dataset is cached and has integrityCOMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py184 # Check if this line has pipe characters (a table row indicator)COMMENT
LOW…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py287 # Check if this looks like a separator row (contains mostly dashes)COMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1003 # Check if a 2x2 block is formed by the same valueSTRING
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1013 # Check if the response is a 6x6 matrixSTRING
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1017 # Verify if the base matrix has been changedSTRING
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1025 # Check if 0s and 1s form connected componentsSTRING
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py1030 # Check if there are any 2x2 blocks of 0s or 1sSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py820 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py910 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py954 # Verify if the base matrix has been changedSTRING
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py992 # Check if all cells with the given value are visitedSTRING
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/tempcompass.py151 # Check if the chatgpt answer is the ground-truth answerSTRING
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py54 # Check if in main threadCOMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1027 # Check if judge model is availableCOMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py641 # Check if the expression is a power expressionCOMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py646 # Check if the base and the exponent are numbersCOMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py651 # Check if the exponent is greater than the thresholdCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/judge_util.py122 # Check if using NVIDIA APICOMMENT
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py329 # Check if there is an overlapCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vgrpbench/score.py110 # Check if cell is a number (not 0) or not a string/characterCOMMENT
75 more matches not shown…
AI Structural Patterns197 hits · 190 pts
SeverityFileLineSnippetContext
LOW…cosmos3/reasoner/vlmevalkit/vlmeval/inference_video.py100CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/mmlongbench.py432CODE
LOW…oner/vlmevalkit/vlmeval/dataset/metropolis_temporal.py122CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_vqa.py91CODE
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/vantage_sot.py552CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py219CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dream.py228CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/hipho.py713CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/mmsi_video.py132CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py527CODE
LOW…ion/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/vcr.py182CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py3586CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py683CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py958CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py1089CODE
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/slidevqa.py62CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/metropolis_dvc.py114CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2029CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2091CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2282CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py2422CODE
LOW…n/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/cmmmu.py325CODE
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/dataset/dude.py48CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/lens.py219CODE
LOW…reasoner/vlmevalkit/vlmeval/dataset/utils/mathvista.py167CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py47CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py93CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py145CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py187CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py210CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/yorn.py232CODE
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/logicvista.py151CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/omni3dbench.py100CODE
LOW…asoner/vlmevalkit/vlmeval/dataset/utils/videoholmes.py38CODE
LOW…oner/vlmevalkit/vlmeval/dataset/utils/ocr_reasoning.py173CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py107CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py142CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py165CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py513CODE
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/utils/cgbench.py539CODE
LOW…vlmeval/dataset/utils/ccocr_evaluator/kie_evaluator.py329CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py35CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py99CODE
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py187CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py49CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py115CODE
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py205CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py65CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py132CODE
LOW…t/utils/chartmimic/evaluator/color_evaluator_prefix.py232CODE
LOW…val/dataset/utils/SArena/video/viclip/viclip_vision.py335CODE
LOW…meval/dataset/utils/SArena/video/viclip/viclip_text.py216CODE
LOW…dataset/utils/megabench/scoring/latex_expr_equality.py75CODE
LOW…eval/dataset/utils/megabench/scoring/sacrebleu_bleu.py23CODE
LOW…eval/dataset/utils/megabench/parsing/common/parsers.py146CODE
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py338CODE
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py876CODE
LOW…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py426CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/vlm/cosmos_reason_hf.py47CODE
LOW…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py25CODE
137 more matches not shown…
Self-Referential Comments38 hits · 111 pts
SeverityFileLineSnippetContext
MEDIUMevaluation/cosmos3/reasoner/vlmevalkit/run.py529 # Create the symbolic links for the prediction filesSTRING
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py592 # Create a simple test PDF using matplotlibCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/vlrmbench.py150 # Create a copy of line to avoid SettingWithCopyWarningCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py24 # Define the option letters based on the number of optionsCOMMENT
MEDIUM…er/vlmevalkit/vlmeval/dataset/olmOCRBench/evaluator.py98 # Define an inner function to evaluate a single testCOMMENT
MEDIUM…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py216 # Create a new page.COMMENT
MEDIUM⚡…3/reasoner/vlmevalkit/vlmeval/dataset/utils/uni_svg.py430 # Create a base temporary directory for this evaluation runCOMMENT
MEDIUM⚡…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py430 # Create a dummy video on failure to prevent crash in metricsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py43 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/utils/tamperbench.py122 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py77 # Define the keys to locate in the textSTRING
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/mlvu.py111 # Define the keys to locate in the textSTRING
MEDIUM…meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py40 # Create a list of non-zero values in the rowCOMMENT
MEDIUM…meval/dataset/utils/vgrpbench/puzzles/oddevensudoku.py55 # Create a list of non-zero values in the columnCOMMENT
MEDIUM…taset/utils/vgrpbench/puzzles/common_puzzle_factory.py30 # Create a new board filled with zerosCOMMENT
MEDIUM…lmeval/dataset/utils/vgrpbench/puzzles/thermometers.py23 # Create a set of all thermometer positions for efficient lookupCOMMENT
MEDIUM…eval/dataset/utils/chartmimic/evaluator/color_utils.py24 # Create an sRGBColor object from RGB valuesCOMMENT
MEDIUM…ataset/utils/mmhelix/evaluators/graph_problems_eval.py526 # Create a new cycle with the first node repeated at the end for validationCOMMENT
MEDIUM⚡…meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py132 # Create a new simulator to test the ground truth solutionCOMMENT
MEDIUM⚡…meval/dataset/utils/mmhelix/evaluators/sokoban_eval.py142 # Define a Sokoban simulator class within the evaluatorCOMMENT
MEDIUM…vlmeval/dataset/utils/design2code/screenshot_single.py66 # Define the argumentsSTRING
MEDIUM…lmeval/dataset/utils/megabench/tools/analysis_utils.py178 meta = all_meta[task_name].copy() # Create a copy to avoid modifying originalCODE
MEDIUM…set/utils/megabench/scoring/general_numerical_match.py35 # Create a multiprocessing.Queue to receive the outputCOMMENT
MEDIUM…meval/dataset/utils/megabench/scoring/program_judge.py45 # Create a CodeTester instance with the response and the found test casesCOMMENT
MEDIUM…/dataset/utils/megabench/scoring/common/conversions.py221 # Create a new image with the calculated sizeCOMMENT
MEDIUM…vlmevalkit/vlmeval/dataset/utils/ssi_bench/__init__.py1# This file is intentionally left empty to make this directory a Python package.COMMENT
MEDIUM…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py376 # Create a dictionary for each combination of values.COMMENT
MEDIUM…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py388 # Create a dictionary for each combination of values.COMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py78 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/EgoExoBench/utils.py157 # Create the prediction column based on the Score and Answer columnsCOMMENT
MEDIUM…soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py1775 # Define a regex pattern to find Markdown tables with newlinesCOMMENT
MEDIUM…soner/vlmevalkit/vlmeval/dataset/OmniDocBench/utils.py1905 # Define a regex pattern to find Markdown tables with newlinesCOMMENT
MEDIUM…uation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vila.py173 # Create a unique temporary directory for this inference callCOMMENT
MEDIUM…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/long_vita.py129 # Create a new image with the target size and paste the resized image onto itCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py64# Define the temporary folder for processed imagesCOMMENT
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/vlm/thyme/sandbox.py211# Define a minimum dimension for the cropCOMMENT
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/vlm/qwen2_vl/model.py115 # Create a temporary file for the frameCOMMENT
MEDIUM…s3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/mm_utils.py69 # Create a new image with the target size and paste the resized image onto itCOMMENT
Cross-Language Confusion18 hits · 102 pts
SeverityFileLineSnippetContext
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/groundingme.py13Ensure that all details mentioned about the object are accurate. Provide at most one bounding box. If a matching object CODE
HIGH…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py264 return null;CODE
HIGH…vlmevalkit/vlmeval/dataset/olmOCRBench/katex/render.py310 list.push({CODE
HIGH…vlmevalkit/vlmeval/dataset/utils/physics_eval_utils.py83 sympy_result = sympy_result or expr1_sympy.equals(expr2_sympy)CODE
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py406 if (root && typeof root.pauseAnimations === 'function') root.pauseAnimations();CODE
HIGH…s3/reasoner/vlmevalkit/vlmeval/dataset/utils/sarena.py407 if (root && typeof root.setCurrentTime === 'function') root.setCurrentTime({t});CODE
HIGH…oner/vlmevalkit/vlmeval/dataset/utils/tablevqabench.py21 'If you have multiple answers, please separate them with || marks. Example: Apple||Banana||Tomato\n\n'CODE
HIGH…kit/vlmeval/dataset/utils/vgrpbench/puzzles/lightup.py81 # 2. We don't have enough potential lights (current + undefined) to reach the required numberCOMMENT
HIGH…lmeval/dataset/utils/vgrpbench/puzzles/fieldexplore.py37 # Check if current mines <= number <= potential mines (current + undefined)COMMENT
HIGH…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py14 # Convert line to runs of filled cells ('s' = filled, 'e' = empty, 0 = undefined)COMMENT
HIGH…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py33 else: # cell is 0 (undefined)CODE
HIGH…dataset/utils/megabench/scoring/latex_expr_equality.py36 result = 1 if responses_expr.equals(targets_expr) else 0CODE
HIGH⚡…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py69 " cd VLM-3R && pip install -e .\n"CODE
HIGH⚡…ation/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/vlm3r.py70 " cd CUT3R && pip install -r requirements.txt\n"CODE
HIGH…/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/cambrian_s.py66 " cd cambrian-s && pip install -e .\n"CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py154 " cd Bagel && uv pip install -e .\n"CODE
HIGH…n/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/bagel_umm.py155 " OR: cd Bagel && pip install -e .\n"CODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/vlm/spatial_mllm.py114 "2. Install in development mode: cd Spatial-MLLM && uv pip install -e .\n"CODE
Over-Commented Block87 hits · 76 pts
SeverityFileLineSnippetContext
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py81 ans = judge_model.generate(messages)COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/aetcbench.py221 # 'mcq_openended_bleu': 1.0,COMMENT
LOW…/reasoner/vlmevalkit/vlmeval/dataset/reasonmap_plus.py1# ReasonMap Plus Dataset is an extension of the original ReasonMap dataset,COMMENT
LOW…valkit/vlmeval/dataset/SGI_Bench_1_0/wet_experiment.py21 # 3: parameter list (e.g.,COMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py1# flake8: noqaCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py21# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py41# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…easoner/vlmevalkit/vlmeval/dataset/utils/bmmr_grade.py61# to use, copy, modify, merge, publish, distribute, sublicense, and/or sellCOMMENT
LOW…oner/vlmevalkit/vlmeval/dataset/utils/mme_reasoning.py801 grid = response_listCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/dataset/utils/mmsci.py261COMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1# Copyright 2025 Garena Online Private LimitedCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py121# def _get_prefix(self):COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py141# # name = func.__name__COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_evaluator.py161# Axes.hist = log_function(Axes.hist)COMMENT
LOW…/dataset/utils/chartmimic/evaluator/color_evaluator.py21COMMENT
LOW…/dataset/utils/chartmimic/evaluator/color_evaluator.py201 # similarities = [item[1] for item in similarities]COMMENT
LOW…ls/chartmimic/evaluator/chart_type_evaluator_prefix.py361# Axes.errorbar = log_function(Axes.errorbar)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py521Axes3D.plot_surface = log_function_for_3d(Axes3D.plot_surface)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py541# ax.axvline(0.5)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py561# loglog testCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py581# ax.fill_betweenx(x, y1, y2, cmap='viridis')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py601# plt.show()COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py621# ax.set_xlabel('X-axis')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py641# fig, ax = plt.subplots()COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py661# v = np.ones(10)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py681# ax.plot([0, 1], [0, 1], [1, 0], color='b')COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py701# dz = np.random.rand(10)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py721# plot in testCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py741# draw the pcolor plot, with a beautiful colormapCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py761# X, Y = np.meshgrid(x, y)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py781# x = np.linspace(0, 10, 100)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py821# fig, ax = plt.subplots(subplot_kw={'projection': 'polar'})COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py841# num_vars = len(values_pc1)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py861# ax.scatter(angles[:-1], values_pc2[:-1], color="red", s=50)COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py881# Line2D(COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py901# # Add legend and titleCOMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py921# # ===================COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py941# ]COMMENT
LOW…set/utils/chartmimic/evaluator/chart_type_and_color.py961# ax.set_yticks([1, 2, 3])COMMENT
LOW…kit/vlmeval/dataset/utils/mmif/function_and_compare.py401COMMENT
LOW…kit/vlmeval/dataset/utils/mmif/function_and_compare.py421# else:COMMENT
LOW…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py561 # initial_state: JSON string representing the grid structureCOMMENT
LOW…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py581 # converted_answer = {}COMMENT
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py1# flake8: noqaCOMMENT
LOW…eval/dataset/utils/Ocrbench_v2/spotting_eval/script.py1# flake8: noqaCOMMENT
LOW…3/reasoner/vlmevalkit/vlmeval/dataset/GUI/osworld_g.py341 click_point = self.parse_response_func(prediction)COMMENT
LOW…ner/vlmevalkit/vlmeval/dataset/Omni3D/install_patch.sh61# echo "[INFO] Updating ~/.bashrc (tclsh + PYTHONPATH)..."COMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py41 if ord(char) > 127: # Non-ASCII charactersCOMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py341# for match in inline_matches:COMMENT
LOW…valkit/vlmeval/dataset/OmniDocBench/data_preprocess.py361# # 'category_type': 'equation_inline',COMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py161 # Second pass: Inline images (re-using the same images? Need to check if processorCOMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py181 # the content structure.COMMENT
LOW…os3/reasoner/vlmevalkit/vlmeval/vlm/granite_docling.py201 passCOMMENT
LOW…vlmevalkit/vlmeval/vlm/hawk_vl/hawk/model/hawk_arch.py1# Adapted from LLaVA code.COMMENT
LOW…awk/model/vision_encoder/qwen_vit/modeling_qwen_vit.py1# --------------------------------------------------------COMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/vlm/ola/ola/utils.py1# Adopted from https://github.com/haotian-liu/LLaVA. Below is the original copyright:COMMENT
LOW…meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py461COMMENT
LOW…r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py1# Copyright (c) 2023-2024 DeepSeek.COMMENT
LOW…it/vlmeval/vlm/ursa/ursa_model/image_processing_vlm.py1# Copyright (c) 2023-2024 DeepSeek.COMMENT
LOW…lmevalkit/vlmeval/vlm/ursa/ursa_model/modeling_ursa.py241 # if vision_feature_select_strategy == "default":COMMENT
27 more matches not shown…
Docstring Block Structure9 hits · 45 pts
SeverityFileLineSnippetContext
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartqapro.py74 Calculate scores by category from evaluation results. Args: result_file: Path to the file STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py326 Loads a video or image sequence, processes it, and returns the path to the processed video. Args: STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py399 Builds a prompt for a language model based on the provided data and settings. Args: line (STRING
HIGH…os3/reasoner/vlmevalkit/vlmeval/dataset/tamperbench.py447 Evaluates the given evaluation file and generates ratings based on different dimensions. Args: STRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py58 Map question IDs to their categories based on the evaluation mode. Args: mode: Either "descriptive" orSTRING
HIGH…cosmos3/reasoner/vlmevalkit/vlmeval/dataset/charxiv.py139 Calculate scores by category from evaluation results. Args: result_file: Path to the file STRING
HIGH…asoner/vlmevalkit/vlmeval/dataset/olmOCRBench/tests.py1077 Load a single test from a JSON line string or JSON object. Args: data: Either a JSON string to parse oSTRING
HIGH…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py48 Load bbox_3d and label arrays from a JSON annotation file. Args: json_path: Path to JSON file containiSTRING
HIGH…reasoner/vlmevalkit/vlmeval/utils/eval_3d_grounding.py71 Normalize label by extracting base name and removing suffixes. Removes numeric suffixes and underscores (e.g.,STRING
Structural Annotation Overuse24 hits · 44 pts
SeverityFileLineSnippetContext
LOW⚡…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py179 # Step 1: Find the last occurrence of 'summary:'STRING
LOW⚡…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py184 # Step 2: Slice the string after 'summary:' and extract valueSTRING
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py210 # Step 1: Clean up punctuation from the responseCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py235 # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D)COMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py243 # Step 4: If no candidates, look for choices with a space after (A B C D)COMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py250 # Step 5: If no candidates and response has more than 5 tokens, try parsing based on contentCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py258 # Step 6: If still no candidates, randomly choose oneCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py262 # Step 7: If multiple candidates found, use the one appearing lastCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py222 # Step 2: If no candidates, look for choices with a period after (A. B. C. D.)COMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py211 # Step 1: Try template matchingCOMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py216 # Step 2: Fall back to LLM matchingCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py74Predicted answer: ### Step 1: Calculate ( l_1 )CODE
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py55# Step 1: Reformatting the SolutionCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py83# Step 2: Evaluating CorrectnessCOMMENT
LOW⚡…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py355 # Step 3: Validate segments formatCOMMENT
LOW⚡…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py359 # Step 4: Validate using initial state and game rulesCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py313 # Step 1: Extract and validate predicted_answerCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py331 # Step 2: Validate initial_stateCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md5## Step 0: Install LMDeployCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md12## Step 1: Start the Inference ServiceCOMMENT
LOW…/cosmos3/reasoner/vlmevalkit/docs/en/EvalByLMDeploy.md23## Step 2: EvaluationCOMMENT
LOW⚡…generator/action/finetune/action_fd_droid_posttrain.md82# Step 1: prepare Cosmos3-DROID -> $DATASET_PATH.COMMENT
LOW⚡…generator/action/finetune/action_fd_droid_posttrain.md88# Step 2: prepare the base DCP checkpoint and Wan2.2 VAE.COMMENT
LOW⚡…generator/action/finetune/action_fd_droid_posttrain.md97# Step 3: optionally choose the output root and launch.COMMENT
AI Slop Vocabulary17 hits · 41 pts
SeverityFileLineSnippetContext
MEDIUM…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py53 "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference imageCODE
MEDIUM…os3/reasoner/vlmevalkit/vlmeval/dataset/image_ccocr.py234 # Save comprehensive resultsCOMMENT
MEDIUM…asoner/vlmevalkit/vlmeval/dataset/medqbench_caption.py256 """image is [good/usable/reject]". Please provide a comprehensive but concise assessment in """STRING
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/sitebench.py375 # Use os.path.relpath for robust relative path extractionCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/bridges_eval.py94 # Various bridge patterns - more comprehensiveCOMMENT
MEDIUM…val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py314 # Extract the predicted grid using robust parsingCOMMENT
MEDIUM…val/dataset/utils/mmhelix/evaluators/futoshiki_eval.py476 # Extract predicted answer using the robust extract_answer methodCOMMENT
MEDIUM…meval/dataset/utils/mmhelix/evaluators/binario_eval.py518 # More comprehensive patterns for matrix keywordsCOMMENT
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py32 """Enhanced robust extraction of answer from model output with comprehensive patterns."""STRING
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py101 """Extract coordinate-value patterns with comprehensive regex."""STRING
MEDIUM…lmeval/dataset/utils/mmhelix/evaluators/kakuro_eval.py569 # # If it's a string, try to extract the answer using robust parsingCOMMENT
LOW…set/utils/megabench/scoring/xml_norm_point_distance.py36 # Instead of normalizing by 1/sqrt(2), we just set it to 0 if the distance is above 1.COMMENT
LOW…lmeval/dataset/utils/megabench/parsing/common/utils.py137 # the result overflows, simply return an empty stringCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py334 # ThreeDAVGroundingBench: the harness writes a transposed 1-row DF as a flatCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py355 # AETCBench: harness writes a transposed 1-row DF as a flat dict — metricCOMMENT
MEDIUM…cosmos3/reasoner/vlmevalkit/cosmos_eval/parse_score.py632# --- RynnScale harness adapter (layer-1: native <suite>.json -> standardized scores) ---COMMENT
MEDIUM…luation/cosmos3/generator/paibench_c/run_paibench_c.sh707# Use a robust python for printing results - prefer cosmos3 venv if functional,COMMENT
Verbosity Indicators20 hits · 36 pts
SeverityFileLineSnippetContext
LOW⚡…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py179 # Step 1: Find the last occurrence of 'summary:'STRING
LOW⚡…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmifeval.py184 # Step 2: Slice the string after 'summary:' and extract valueSTRING
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py210 # Step 1: Clean up punctuation from the responseCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py235 # Step 3: Look for choices with parentheses e.g., (A) (B) (C) (D)COMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py243 # Step 4: If no candidates, look for choices with a space after (A B C D)COMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py250 # Step 5: If no candidates and response has more than 5 tokens, try parsing based on contentCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py258 # Step 6: If still no candidates, randomly choose oneCOMMENT
LOW⚡…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py262 # Step 7: If multiple candidates found, use the one appearing lastCOMMENT
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/videommmu.py222 # Step 2: If no candidates, look for choices with a period after (A. B. C. D.)COMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py211 # Step 1: Try template matchingCOMMENT
LOW⚡…oner/vlmevalkit/vlmeval/dataset/utils/corecognition.py216 # Step 2: Fall back to LLM matchingCOMMENT
LOW…mos3/reasoner/vlmevalkit/vlmeval/dataset/utils/phyx.py74Predicted answer: ### Step 1: Calculate ( l_1 )CODE
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py55# Step 1: Reformatting the SolutionCOMMENT
LOW…er/vlmevalkit/vlmeval/dataset/utils/vcrbench/prompt.py83# Step 2: Evaluating CorrectnessCOMMENT
LOW⚡…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py355 # Step 3: Validate segments formatCOMMENT
LOW⚡…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py359 # Step 4: Validate using initial state and game rulesCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py313 # Step 1: Extract and validate predicted_answerCOMMENT
LOW…eval/dataset/utils/mmhelix/evaluators/shingoki_eval.py331 # Step 2: Validate initial_stateCOMMENT
LOW…ataset/utils/mmhelix/evaluators/graph_problems_eval.py283 # we need to check if the last node connects back to the first nodeCOMMENT
LOW…ataset/utils/mmhelix/evaluators/graph_problems_eval.py508 # If not, we need to check if the last node connects back to the first nodeCOMMENT
Modern Structural Boilerplate32 hits · 34 pts
SeverityFileLineSnippetContext
LOW…osmos3/reasoner/vlmevalkit/vlmeval/dataset/__init__.py486__all__ = [CODE
LOW…/reasoner/vlmevalkit/vlmeval/dataset/utils/__init__.py7__all__ = [CODE
LOW…val/dataset/utils/SArena/video/viclip/viclip_vision.py18logger = logging.getLogger(__name__)CODE
LOW…meval/dataset/utils/SArena/video/viclip/viclip_text.py18logger = logging.getLogger(__name__)CODE
LOW…it/vlmeval/dataset/utils/SArena/video/viclip/viclip.py16logger = logging.getLogger(__name__)CODE
LOW…vlmevalkit/vlmeval/dataset/utils/megabench/__init__.py5__all__ = [AggregationType, MetricType, ResponseParseType]CODE
LOW…t/vlmeval/dataset/utils/Ocrbench_v2/spotting_metric.py16logger = logging.getLogger(__name__)CODE
LOW…valkit/vlmeval/dataset/utils/Ocrbench_v2/vqa_metric.py5logger = logging.getLogger(__name__)CODE
LOW…er/vlmevalkit/vlmeval/dataset/metropolis2d/__init__.py5__all__ = ['Metropolis2DDetectionDataset', 'Metropolis2DGroundingDataset', 'Astro2DDetectionDataset']CODE
LOW…/cosmos3/reasoner/vlmevalkit/vlmeval/utils/__init__.py5__all__ = [CODE
LOW…tion/cosmos3/reasoner/vlmevalkit/vlmeval/vlm/points.py226 def set_image_processor(self, dataset: str) -> None:CODE
LOW…/reasoner/vlmevalkit/vlmeval/vlm/video_llm/__init__.py8__all__ = ['VideoLLaVA', 'VideoLLaVA_HF', 'Chatunivi', 'VideoChatGPT', 'LLaMAVID', 'VideoChat2_HD', 'PLLaVA']CODE
LOW…/reasoner/vlmevalkit/vlmeval/vlm/xcomposer/__init__.py7__all__ = ['ShareCaptioner', 'XComposer', 'XComposer2', 'XComposer2_4KHD', 'XComposer2d5']CODE
LOW…oner/vlmevalkit/vlmeval/vlm/granite_vision/__init__.py3__all__ = ['GraniteVision3']CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/vlm/qtunevl/__init__.py4__all__ = ['QTuneVL', 'QTuneVLChat']CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/interns1/__init__.py3__all__ = ['InternS1Chat']CODE
LOW⚡…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py105 def set_dump_image(self, flag: bool) -> None:CODE
LOW⚡…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py108 def set_batch_size(self, bs: int) -> None:CODE
LOW⚡…soner/vlmevalkit/vlmeval/vlm/interns1/interns1_chat.py111 def set_output_folder(self, out_dir: str) -> None:CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/internvl/__init__.py3__all__ = ['InternVLChat']CODE
LOW…mos3/reasoner/vlmevalkit/vlmeval/vlm/llava/__init__.py5__all__ = [CODE
LOW…smos3/reasoner/vlmevalkit/vlmeval/vlm/ovis/__init__.py3__all__ = ['Ovis', 'Ovis1_6', 'Ovis1_6_Plus', 'Ovis2', 'OvisU1', 'Ovis2_5']CODE
LOW…meval/vlm/ola/ola/model/multimodal_encoder/oryx_vit.py504 def set_grad_checkpointing(self, enable: bool = True) -> None:CODE
LOW…lmeval/vlm/ola/ola/model/speech_encoder/beats/BEATs.py26logger = logging.getLogger(__name__)CODE
LOW…lmeval/vlm/ola/ola/model/speech_encoder/beats/kaldi.py8__all__ = [CODE
LOW…l/vlm/ola/ola/model/speech_encoder/beats/Tokenizers.py27logger = logging.getLogger(__name__)CODE
LOW…3/reasoner/vlmevalkit/vlmeval/vlm/qwen3_vl/__init__.py3__all__ = [CODE
LOW…r/vlmevalkit/vlmeval/vlm/ursa/ursa_model/siglip_vit.py436 def set_grad_checkpointing(self, enable: bool = True) -> None:CODE
LOW…ner/vlmevalkit/vlmeval/vlm/ursa/ursa_model/__init__.py20__all__ = [CODE
LOW…s3/reasoner/vlmevalkit/vlmeval/api/sensechat_vision.py193 def set_max_num(self, dataset: Optional[str] = None) -> None:CODE
LOW…on/cosmos3/reasoner/vlmevalkit/vlmeval/api/__init__.py41__all__ = [CODE
LOW…3/reasoner/vlmevalkit/vlmeval/api/adapters/__init__.py8__all__ = [CODE
Synthetic Comment Markers3 hits · 20 pts
SeverityFileLineSnippetContext
HIGH…mos3/reasoner/vlmevalkit/vlmeval/dataset/chartmimic.py53 "instruction": "You are an excellent judge at evaluating visualization chart plots. The first image (reference imageCODE
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py43 # I will dump all original keys as requested.COMMENT
HIGH…osmos3/reasoner/vlmevalkit/vlmeval/dataset/chartcap.py158 # Format as requested: dictionary composed of lists, organized into a pandas.DataFrameCOMMENT
Slop Phrases8 hits · 16 pts
SeverityFileLineSnippetContext
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_vqa.py2502 # Once you get the permission, you can use the helper code here to download and extract necessary images:COMMENT
MEDIUM…osmos3/reasoner/vlmevalkit/vlmeval/dataset/mmreason.py29Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py11Your task is to compare the predicted answer with the correct answer and determine if they match meaningfully. Here's hoCODE
MEDIUM…smos3/reasoner/vlmevalkit/vlmeval/dataset/utils/vdc.py32Here's how you can accomplish the task:CODE
MEDIUM…easoner/vlmevalkit/vlmeval/dataset/utils/video_mmlu.py44Here's how you can accomplish the task:CODE
MEDIUM…oner/vlmevalkit/vlmeval/dataset/utils/olympiadbench.py39Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
MEDIUM…reasoner/vlmevalkit/vlmeval/dataset/utils/mathverse.py33Model response: 'As it stands, I cannot provide the correct option letter because there isn't enough information to solvCODE
LOW…it/vlmeval/dataset/utils/vgrpbench/puzzles/nonogram.py36 # Don't forget to add the last run if it existsCOMMENT
Dead Code5 hits · 10 pts
SeverityFileLineSnippetContext
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py321CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py331CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py386CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py397CODE
MEDIUM…val/dataset/utils/SArena/video/viclip/viclip_vision.py407CODE
Hallucination Indicators1 hit · 10 pts
SeverityFileLineSnippetContext
CRITICAL…easoner/vlmevalkit/vlmeval/vlm/video_llm/videochat2.py263 seg_embs = [model.mistral_model.base_model.model.model.embed_tokens(seg_t) for seg_t in seg_tokens]CODE
Modern AI Meta-Vocabulary4 hits · 9 pts
SeverityFileLineSnippetContext
MEDIUMcookbooks/cosmos3/README.md107### System packages (required for Framework guardrails)COMMENT
MEDIUMcookbooks/cosmos3/README.md142license and set `HF_TOKEN` as in [Prerequisites](#prerequisites)). To disable guardrailsCODE
MEDIUMcookbooks/cosmos3/nim/README.md184[Guardrails](operations.md#guardrails).CODE
MEDIUMcookbooks/cosmos3/nim/action.md415configuration; see [operations.md](operations.md#guardrails).CODE
Magic Placeholder Names1 hit · 5 pts
SeverityFileLineSnippetContext
HIGHevaluation/cosmos3/generator/rbench/README.md67export RBENCH_VLM_API_KEY=your-api-keyCODE
Fake / Example Data1 hit · 1 pts
SeverityFileLineSnippetContext
LOW…smos3/reasoner/vlmevalkit/vlmeval/dataset/image_mcq.py538 self.data['question'] = ['placeholder'] * len(self.data)CODE
Overly Generic Function Names1 hit · 1 pts
SeverityFileLineSnippetContext
LOW…alkit/vlmeval/dataset/utils/Ocrbench_v2/TEDS_metric.py328 def process_data(obj):CODE
Example Usage Blocks1 hit · 0 pts
SeverityFileLineSnippetContext
LOW…ner/vlmevalkit/vlmeval/dataset/utils/hipho_verifier.py1343 # Example usageSTRING