open-compass/opencompass

33.4

Adjusted Score

33.4

Raw Score

100%

Time Factor

2026-07-14

Last Push

7.2K

Stars

Python

Language

299.6K

Lines of Code

3.5K

Files

6.5K

Pattern Hits

2026-07-14

Scan Date

0.20

HC Hit Rate

What These Metrics Mean

Adjusted Score: Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score: The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor: The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits: Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate: High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files: Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 0HIGH 687MEDIUM 484LOW 5327

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 6498 distinct pattern matches across 22 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Cross-File Repetition633 hits · 3165 pts

Severity	File	Line	Snippet	Context
HIGH	tests/TESTING_GUIDE.md	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/datasets/test_beyondaime.py	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/datasets/test_aime2025.py	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/openicl/test_icl_chatml_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	tests/openicl/test_icl_gen_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	tests/openicl/test_icl_chat_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	docs/zh_cn/advanced_guides/prompt_attack.md	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	docs/en/advanced_guides/prompt_attack.md	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	…gs/datasets/promptbench/promptbench_wnli_gen_50662f.py	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	…/zh_cn/advanced_guides/objective_judgelm_evaluation.md	0	look at the following two expressions (answers to a math problem) and judge whether they are equivalent. only perform tr	STRING
HIGH	docs/en/advanced_guides/objective_judgelm_evaluation.md	0	look at the following two expressions (answers to a math problem) and judge whether they are equivalent. only perform tr	STRING
HIGH	examples/eval_math_llm_judge.py	0	look at the following two expressions (answers to a math problem) and judge whether they are equivalent. only perform tr	STRING
HIGH	examples/eval_cibench_api.py	0	you should directly give results based on history information.	STRING
HIGH	examples/eval_cibench.py	0	you should directly give results based on history information.	STRING
HIGH	opencompass/models/lagent.py	0	you should directly give results based on history information.	STRING
HIGH	…datasets/compassbench_v1_3/compassbench_v1_3_prompt.py	0	you should directly give results based on history information.	STRING
HIGH	examples/eval_cibench_api.py	0	\ you are an assistant who can utilize external tools. {tool_description} to use a tool, please response with the follow	STRING
HIGH	examples/eval_cibench.py	0	\ you are an assistant who can utilize external tools. {tool_description} to use a tool, please response with the follow	STRING
HIGH	…datasets/compassbench_v1_3/compassbench_v1_3_prompt.py	0	\ you are an assistant who can utilize external tools. {tool_description} to use a tool, please response with the follow	STRING
HIGH	examples/eval_cibench_api.py	0	\ it can run python code in a manner as jupyter notebook. the code must be a valid code that contains only python method	STRING
HIGH	examples/eval_cibench.py	0	\ it can run python code in a manner as jupyter notebook. the code must be a valid code that contains only python method	STRING
HIGH	…datasets/compassbench_v1_3/compassbench_v1_3_prompt.py	0	\ it can run python code in a manner as jupyter notebook. the code must be a valid code that contains only python method	STRING
HIGH	examples/eval_llm_judge.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…compass/configs/datasets/HLE/hle_verified_rawprompt.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…nfigs/datasets/HLE/hle_biomed_llm_verify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…mpass/configs/datasets/HLE/hle_llmverify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/HLE/hle_llmverify_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ts/OlymMATH/olymmath_llmverify_rawprompt_gen_9d3a8e.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/OlymMATH/olymmath_cascade_eval_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/OlymMATH/olymmath_llmverify_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/aime2026/aime2026_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…aime2026/aime2026_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…hmmt2026/hmmt2026_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/hmmt2026/hmmt2026_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/matbench/matbench_llm_judge_gen_0e9276.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…s/datasets/omni_math/omni_math_llmverify_gen_ccf9c0.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…atasets/omni_math/omni_math_cascade_eval_gen_ccf9c0.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	examples/eval_cascade_evaluator.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…figs/datasets/math/math_500_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…tasets/math/math_prm800k_500_0shot_cot_academic_gen.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ass/configs/datasets/math/math_llm_judge_gen_56606f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…h_prm800k_500_0shot_nocot_genericllmeval_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ets/math/math_500_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…th/math_prm800k_500_0shot_nocot_llmjudge_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…h_prm800k_500_0shot_nocot_genericllmeval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/configs/datasets/math/math_500_llmjudge_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/math/math_prm800k_500_llmverify_repeat4_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…datasets/math/math_prm800k_500_llmverify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…m800k_500_0shot_nocot_genericllmeval_xml_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…mpiadBench/OlympiadBench_0shot_llmverify_gen_be8b13.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…adBench/OlympiadBench_0shot_cascade_eval_gen_be8b13.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…dBench/OlympiadBenchMath_0shot_llmverify_gen_9c22f2.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/OlympiadBench_0shot_llmverify_rawprompt_gen_d3e9e4.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…CARDBiomedBench/CARDBiomedBench_llmjudge_gen_99a231.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…Bench/CARDBiomedBench_llmjudge_rawprompt_gen_b4d90c.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…tasets/R_Bench/rbench_llmjudge_rawprompt_gen_c24221.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…configs/datasets/R_Bench/rbench_llmjudge_gen_c89350.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…asets/BeyondAIME/beyondaime_cascade_eval_gen_5e9f4f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/configs/datasets/chatobj_custom/chatobj_custom_gen.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ivemathbench/livemathbench_hard_llmjudge_gen_71eaf5.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
573 more matches not shown…

Unused Imports3301 hits · 2938 pts

Severity	File	Line	Context
LOW	tests/datasets/test_local_datasets.py	24	CODE
LOW	tests/datasets/test_local_datasets.py	26	CODE
LOW	tests/datasets/test_local_datasets.py	28	CODE
LOW	tests/datasets/test_local_datasets.py	30	CODE
LOW	tests/datasets/test_local_datasets.py	32	CODE
LOW	tests/datasets/test_local_datasets.py	34	CODE
LOW	tests/datasets/test_local_datasets.py	36	CODE
LOW	tests/datasets/test_local_datasets.py	38	CODE
LOW	tests/datasets/test_local_datasets.py	39	CODE
LOW	tests/datasets/test_local_datasets.py	40	CODE
LOW	tests/datasets/test_local_datasets.py	42	CODE
LOW	tests/datasets/test_local_datasets.py	44	CODE
LOW	tests/datasets/test_local_datasets.py	45	CODE
LOW	tests/datasets/test_local_datasets.py	47	CODE
LOW	tests/datasets/test_local_datasets.py	49	CODE
LOW	tests/datasets/test_local_datasets.py	51	CODE
LOW	tests/datasets/test_local_datasets.py	53	CODE
LOW	tests/datasets/test_local_datasets.py	55	CODE
LOW	tests/datasets/test_local_datasets.py	57	CODE
LOW	tests/datasets/test_local_datasets.py	59	CODE
LOW	tests/datasets/test_local_datasets.py	61	CODE
LOW	tests/datasets/test_local_datasets.py	63	CODE
LOW	tests/datasets/test_local_datasets.py	65	CODE
LOW	tests/datasets/test_local_datasets.py	67	CODE
LOW	tests/datasets/test_local_datasets.py	68	CODE
LOW	tests/datasets/test_local_datasets.py	70	CODE
LOW	tests/datasets/test_local_datasets.py	72	CODE
LOW	tests/datasets/test_local_datasets.py	74	CODE
LOW	tests/datasets/test_local_datasets.py	76	CODE
LOW	tests/datasets/test_local_datasets.py	78	CODE
LOW	tests/datasets/test_local_datasets.py	80	CODE
LOW	tests/datasets/test_local_datasets.py	82	CODE
LOW	tests/datasets/test_local_datasets.py	84	CODE
LOW	tests/datasets/test_local_datasets.py	86	CODE
LOW	tests/datasets/test_local_datasets.py	88	CODE
LOW	tests/datasets/test_local_datasets.py	90	CODE
LOW	tests/datasets/test_local_datasets.py	92	CODE
LOW	tests/datasets/test_local_datasets.py	94	CODE
LOW	tests/datasets/test_local_datasets.py	96	CODE
LOW	tests/datasets/test_local_datasets.py	98	CODE
LOW	tests/datasets/test_local_datasets.py	100	CODE
LOW	tests/datasets/test_local_datasets.py	102	CODE
LOW	tests/datasets/test_local_datasets.py	104	CODE
LOW	tests/datasets/test_local_datasets.py	106	CODE
LOW	tests/datasets/test_local_datasets.py	108	CODE
LOW	tests/datasets/test_local_datasets.py	110	CODE
LOW	tests/datasets/test_local_datasets.py	111	CODE
LOW	tests/datasets/test_local_datasets.py	113	CODE
LOW	tests/datasets/test_local_datasets.py	115	CODE
LOW	tests/datasets/test_local_datasets.py	117	CODE
LOW	tests/datasets/test_local_datasets.py	119	CODE
LOW	tests/datasets/test_local_datasets.py	121	CODE
LOW	tests/datasets/test_local_datasets.py	123	CODE
LOW	tests/datasets/test_local_datasets.py	125	CODE
LOW	tests/datasets/test_local_datasets.py	127	CODE
LOW	tests/datasets/test_local_datasets.py	129	CODE
LOW	tests/datasets/test_local_datasets.py	131	CODE
LOW	tests/datasets/test_local_datasets.py	133	CODE
LOW	tests/datasets/test_local_datasets.py	135	CODE
LOW	tests/datasets/test_local_datasets.py	137	CODE
3241 more matches not shown…

Decorative Section Separators299 hits · 1139 pts

Severity	File	Line	Snippet	Context
MEDIUM	opencompass/datasets/zebralogic.py	155	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/zebralogic.py	157	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	38	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	53	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	94	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	96	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	163	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	165	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	220	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	222	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	291	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	293	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	112	# =============================================================================	COMMENT
MEDIUM⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	114	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	56	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	58	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	75	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	77	# =============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	53	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	55	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	60	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	62	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	128	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	130	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	134	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	136	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	143	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	145	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	231	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	233	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	251	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	253	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	13	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	15	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	115	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	117	# ============================================================================	COMMENT
MEDIUM⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	194	# ============================================================================	COMMENT
MEDIUM⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	196	# ============================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	84	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	86	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	217	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	219	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	381	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	383	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	576	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	578	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	891	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	893	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	961	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	963	# ========================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	17	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	19	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	83	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	85	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	507	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	509	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	812	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	814	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	1099	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	1101	# ============================================================================	COMMENT
239 more matches not shown…

Hyper-Verbose Identifiers745 hits · 718 pts

Severity	File	Line	Snippet	Context
LOW⚡	tools/analyze_repeat.py	71	def collect_predictions_by_benchmark(	STRING
LOW	tools/convert_alignmentbench.py	10	def extract_predictions_from_json(input_folder):	CODE
LOW	tests/TESTING_GUIDE.md	141	def test_load_returns_dataset(self):	CODE
LOW	tests/TESTING_GUIDE.md	186	def test_load_returns_dataset(self, mock_load_dataset):	CODE
LOW	tests/TESTING_GUIDE.md	236	def test_load_reads_jsonl_file(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE.md	258	def test_dataset_initialization(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE.md	296	def test_score_correct_prediction(self):	CODE
LOW	tests/TESTING_GUIDE.md	303	def test_score_incorrect_prediction(self):	CODE
LOW	tests/TESTING_GUIDE.md	354	def test_accuracy_calculation(self):	CODE
LOW	tests/TESTING_GUIDE.md	418	def test_result_statistics_consistency(self):	CODE
LOW	tests/TESTING_GUIDE.md	459	def test_details_count_matches_total_samples(self):	CODE
LOW	tests/TESTING_GUIDE.md	469	def test_result_metrics_completeness(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	143	def test_load_returns_dataset(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	188	def test_load_returns_dataset(self, mock_load_dataset):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	238	def test_load_reads_jsonl_file(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	260	def test_dataset_initialization(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	298	def test_score_correct_prediction(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	305	def test_score_incorrect_prediction(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	356	def test_accuracy_calculation(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	420	def test_result_statistics_consistency(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	461	def test_details_count_matches_total_samples(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	471	def test_result_metrics_completeness(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	24	def test_initialization_with_custom_keep_keys(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	31	def test_parse_model_dataset_args(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	46	def test_parse_model_dataset_args_with_combinations(self):	CODE
LOW	tests/partitioners/test_naive.py	35	def test_initialization_with_custom_n(self):	CODE
LOW	tests/partitioners/test_naive.py	41	def test_partition_creates_tasks(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	64	def test_partition_with_n_greater_than_one(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	95	def test_partition_skips_existing_files(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	128	def test_partition_with_add_cfg(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	48	def test_remaining_without_total(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	53	def test_remaining_with_total(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	60	def test_remaining_negative_protection(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	152	def test_initialization_with_defaults(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	174	def test_get_command_single_gpu(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	189	def test_get_command_multi_gpu(self, mock_randint):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	217	def test_get_command_with_backend(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	250	def test_inferencer_name_from_string(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	256	def test_inferencer_name_from_class(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	273	def test_set_default_value_existing(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	281	def test_build_inferencer_gen(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	306	def test_build_inferencer_chat(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	330	def test_build_inferencer_unsupported(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	366	def test_remaining_total_with_none(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	383	def test_run_dataset_task_success(self, mock_mkdir, mock_get_path,	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	439	def test_run_dataset_task_non_api_model(self, mock_build_dataset):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	469	def test_run_dataset_task_missing_template(self, mock_build_dataset):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	507	def test_run_skips_existing_output(self, mock_task_abbr, mock_get_path,	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	556	def test_run_with_max_workers_config(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	575	def test_build_inferencer_with_max_infer_workers(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	596	def test_build_inferencer_with_model_config(self, mock_registry):	CODE
LOW	tests/tasks/test_base_task.py	14	def deperacated_test_extract_role_pred_function_source(self):	CODE
LOW	tests/tasks/test_base_task.py	39	def deperacated_test_extract_role_pred_with_begin_and_end(self):	CODE
LOW⚡	tests/tasks/test_base_task.py	50	def deperacated_test_extract_role_pred_with_begin_only(self):	CODE
LOW⚡	tests/tasks/test_base_task.py	59	def deperacated_test_extract_role_pred_with_end_only(self):	CODE
LOW⚡	tests/tasks/test_base_task.py	68	def test_extract_role_pred_without_markers(self):	CODE
LOW⚡	tests/tasks/test_base_task.py	75	def test_extract_role_pred_with_whitespace_begin(self):	CODE
LOW	tests/tasks/test_openicl_eval_watch.py	52	def test_initialization_with_defaults(self):	CODE
LOW	tests/tasks/test_openicl_eval_watch.py	82	def test_get_command_single_gpu(self):	CODE
LOW	tests/tasks/test_openicl_eval_watch.py	97	def test_get_command_multi_gpu(self, mock_randint):	CODE
685 more matches not shown…

Deep Nesting571 hits · 560 pts

Severity	File	Line	Context
LOW	setup.py	20	CODE
LOW	setup.py	51	CODE
LOW	setup.py	85	CODE
LOW	setup.py	93	CODE
LOW	tools/analyze_repeat.py	71	CODE
LOW	tools/prompt_viewer.py	51	CODE
LOW	tools/collect_code_preds.py	130	CODE
LOW	tools/chatml_format_test.py	17	CODE
LOW	tools/chatml_format_test.py	37	CODE
LOW	tests/datasets/test_longbenchv2_config.py	7	CODE
LOW	tests/datasets/test_local_datasets.py	464	CODE
LOW	tests/datasets/test_local_datasets.py	345	CODE
LOW	autotest/utils/compare_results.py	60	CODE
LOW	autotest/utils/compare_results.py	132	CODE
LOW	autotest/utils/compare_results.py	190	CODE
LOW	autotest/utils/compare_results.py	455	CODE
LOW	autotest/utils/oc_score_assert.py	145	CODE
LOW	docs/zh_cn/statis.py	40	CODE
LOW	docs/en/statis.py	42	CODE
LOW	opencompass/metrics/mme_score.py	48	CODE
LOW	opencompass/evaluator/math_evaluator.py	167	CODE
LOW	opencompass/evaluator/cascade_evaluator.py	101	CODE
LOW	opencompass/evaluator/cascade_evaluator.py	122	CODE
LOW	opencompass/partitioners/sub_size.py	73	CODE
LOW	opencompass/partitioners/num_worker.py	55	CODE
LOW	opencompass/partitioners/sub_naive.py	52	CODE
LOW	opencompass/partitioners/sub_naive.py	157	CODE
LOW	opencompass/partitioners/naive.py	31	CODE
LOW	opencompass/partitioners/size.py	53	CODE
LOW	opencompass/partitioners/sub_num_worker.py	68	CODE
LOW	opencompass/partitioners/base.py	40	CODE
LOW	opencompass/partitioners/base.py	104	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	146	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	248	CODE
LOW	opencompass/tasks/openicl_eval.py	254	CODE
LOW	opencompass/summarizers/needlebench.py	150	CODE
LOW	opencompass/summarizers/needlebench.py	428	CODE
LOW	opencompass/summarizers/needlebench.py	610	CODE
LOW	opencompass/summarizers/needlebench.py	675	CODE
LOW	opencompass/summarizers/needlebench.py	692	CODE
LOW	opencompass/summarizers/default_subjective.py	73	CODE
LOW	opencompass/summarizers/default_subjective.py	175	CODE
LOW	opencompass/summarizers/default_subjective.py	272	CODE
LOW	opencompass/summarizers/summarizer_pretrain.py	34	CODE
LOW	opencompass/summarizers/multi_model.py	125	CODE
LOW	opencompass/summarizers/circular.py	22	CODE
LOW	opencompass/summarizers/default.py	67	CODE
LOW	opencompass/summarizers/default.py	137	CODE
LOW	opencompass/summarizers/default.py	258	CODE
LOW	opencompass/summarizers/llm_compression.py	27	CODE
LOW	opencompass/summarizers/subjective/compassbench_v13.py	57	CODE
LOW	opencompass/summarizers/subjective/compassbench_v13.py	124	CODE
LOW	opencompass/summarizers/subjective/wildbench.py	167	CODE
LOW	opencompass/summarizers/subjective/wildbench.py	237	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	446	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	516	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	648	CODE
LOW	opencompass/summarizers/subjective/charm.py	39	CODE
LOW	opencompass/summarizers/subjective/charm.py	138	CODE
LOW	opencompass/summarizers/subjective/fofo.py	62	CODE
511 more matches not shown…

Excessive Try-Catch Wrapping448 hits · 519 pts

Severity	File	Line	Snippet	Context
LOW⚡	tools/analyze_repeat.py	47	except Exception:	STRING
LOW⚡	tools/analyze_repeat.py	57	except Exception:	STRING
LOW⚡	tools/analyze_repeat.py	65	except Exception:	STRING
LOW	tools/convert_alignmentbench.py	27	except Exception as e:	CODE
LOW	tools/update_dataset_suffix.py	83	except Exception:	CODE
LOW	tools/chatml_format_test.py	53	except Exception as e:	CODE
LOW	tests/datasets/test_local_datasets.py	327	except Exception:	CODE
LOW	tests/datasets/test_local_datasets.py	358	except Exception as exception:	CODE
LOW	tests/datasets/test_local_datasets.py	418	except Exception as e:	CODE
LOW	autotest/all/chat_obj_fullbench_v2.py	485	except Exception:	CODE
LOW	autotest/all/chat_obj_fullbench_v1.py	342	except Exception:	CODE
LOW	opencompass/evaluator/math_evaluator.py	51	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	180	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	188	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	207	except Exception as e:	CODE
MEDIUM	opencompass/evaluator/math_evaluator.py	86	def score(self, predictions, references, test_set=None):	CODE
LOW	opencompass/partitioners/base.py	77	except Exception:	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	230	except Exception as e:	CODE
LOW	opencompass/tasks/openicl_eval.py	183	except Exception as e:	CODE
LOW	opencompass/tasks/openicl_eval.py	342	except Exception as e:	CODE
LOW	opencompass/summarizers/subjective/alpacaeval.py	46	except Exception as e:	STRING
LOW	opencompass/summarizers/subjective/alpacaeval.py	70	except Exception as e:	STRING
LOW	opencompass/summarizers/subjective/followbench.py	58	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	145	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	169	except Exception:	CODE
LOW	opencompass/datasets/srbench.py	234	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	252	except Exception as e:	CODE
MEDIUM	opencompass/datasets/srbench.py	161	def is_symbolically_equivalent(self, formula1, formula2, n_var=2):	CODE
LOW	opencompass/datasets/OpenSWI.py	48	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	251	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	263	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	442	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	447	except Exception:	CODE
LOW	opencompass/datasets/taco.py	185	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	351	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	396	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	472	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	504	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	550	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	580	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	613	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	624	except Exception:	CODE
LOW	opencompass/datasets/taco.py	635	except Exception:	CODE
LOW	opencompass/datasets/taco.py	652	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	676	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	689	except Exception as e:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	110	except Exception:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	126	except Exception:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	141	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	58	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	97	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	106	except Exception:	CODE
MEDIUM	opencompass/datasets/gsm8k.py	54	def is_equal(self, pred, refer):	CODE
MEDIUM	opencompass/datasets/gsm8k.py	93	def is_equal(self, pred, refer):	CODE
MEDIUM	opencompass/datasets/gsm8k.py	101	def soft_equal(self, pred, refer, step):	CODE
LOW⚡	opencompass/datasets/humanevalx.py	162	except Exception:	CODE
LOW⚡	opencompass/datasets/humanevalx.py	167	except Exception:	CODE
LOW⚡	opencompass/datasets/humanevalx.py	177	except Exception:	CODE
LOW	opencompass/datasets/zebralogic.py	208	except Exception:	CODE
LOW	opencompass/datasets/zebralogic.py	215	except Exception:	CODE
388 more matches not shown…

Modern AI Meta-Vocabulary68 hits · 168 pts

Severity	File	Line	Snippet	Context
MEDIUM	docs/zh_cn/get_started/faq.md	20	### OpenCompass 如何控制 few shot 评测的 shot 数目？	COMMENT
MEDIUM	docs/zh_cn/advanced_guides/evaluation_lmdeploy.md	70	# the max size of the context window	COMMENT
MEDIUM	docs/en/get_started/faq.md	20	### How does OpenCompass control the number of shots in few-shot evaluations?	COMMENT
MEDIUM	docs/en/advanced_guides/evaluation_lmdeploy.md	70	# the max size of the context window	COMMENT
MEDIUM	opencompass/datasets/CARDBiomedBench.py	29	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/nejmaibench.py	61	pass # TODO: Implement few-shot prompt handling	CODE
MEDIUM	opencompass/datasets/medmcqa.py	43	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/MedCalc_Bench.py	245	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/MedXpertQA.py	30	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/SciKnowEval.py	47	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/Medbullets.py	56	pass # TODO: Implement few-shot prompt handling	CODE
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	67	# process few-shot raw_prompts	COMMENT
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	240	# process demo once if it is few-shot-CoT	COMMENT
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	327	setting_name = 'zero-shot' # setting_name can be chosen from ["zero-shot", "zero-shot-CoT", "few-shot-CoT"]	CODE
MEDIUM	opencompass/datasets/medbench/post_process.py	171	# if setting_name == 'few-shot-CoT':	COMMENT
MEDIUM	opencompass/datasets/medbench/post_process.py	194	# all other datasets are QA problems with single answer and setting_name are few-shot	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	57	# BRICS / scaffold	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	198	# BRICS / scaffold	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	268	# BRICS / scaffold	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	236	# Murcko scaffold (3 different methods for different purposes)	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	240	), # Atom count in scaffold	CODE
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	248	), # SMILES string of scaffold	CODE
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	368	# BRICS and scaffold	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	761	# BRICS and scaffold positions	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1859	# Get the scaffold	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1864	# Find matching atoms between original molecule and scaffold	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1865	# We need to do a substructure match to map scaffold atoms to original indices	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1869	# If direct match fails, try to find the scaffold atoms differently	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1870	# Get the scaffold SMILES and match it back	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	94	# process few-shot raw_prompts	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	294	# process demo once if it is few-shot-CoT	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	396	setting_name = 'few-shot-CoT' # setting_name can be chosen from ["zero-shot", "zero-shot-CoT", "few-shot-CoT"]	CODE
MEDIUM	opencompass/datasets/agieval/post_process.py	192	# all other datasets are QA problems with single answer and setting_name are few-shot	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/bio_instruction.py	179	# Hugging Face zero-shot pipeline默认按分数高低排序返回结果	COMMENT
MEDIUM	…ompass/openicl/icl_evaluator/icl_korbench_evaluator.py	266	# evaluator = korbenchEvaluator(question_type='logic', mode='zero-shot')	COMMENT
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	30	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	54	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	78	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	102	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…CARDBiomedBench/CARDBiomedBench_llmjudge_gen_99a231.py	54	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…compass/configs/datasets/needlebench/atc/atc_choice.py	84	# Set few shot prompt number	COMMENT
MEDIUM	…datasets/needlebench/atc/atc_choice_50_en_reasoning.py	71	# Set few shot prompt number	COMMENT
MEDIUM	…pass/configs/datasets/needlebench/atc/atc_choice_20.py	83	# Set few shot prompt number	COMMENT
MEDIUM	…datasets/needlebench/atc/atc_choice_80_en_reasoning.py	47	# Set few shot prompt number	COMMENT
MEDIUM	…onfigs/datasets/advancedIF/advancedIF_rawprompt_gen.py	29	# Shared by CIF and CC subsets (zero-shot)	COMMENT
MEDIUM	…onfigs/datasets/advancedIF/advancedIF_rawprompt_gen.py	309	# System Steerability judge prompt (with few-shot examples)	COMMENT
MEDIUM	…ss/configs/datasets/llm_compression/llm_compression.py	27	# Calculates cross entropy loss for each batch based on a sliding context window	COMMENT
MEDIUM	…igs/datasets/Earth_Silver/Earth_Silver_llmjudge_gen.py	48	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…asets/Earth_Silver/Earth_Silver_llmjudge_gen_46140c.py	45	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…configs/datasets/SciKnowEval/SciKnowEval_gen_ebe47d.py	33	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	CODE
MEDIUM	…atasets/SciKnowEval/SciKnowEval_llmjudge_gen_ebe47d.py	59	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…compass/configs/datasets/agieval/agieval_gen_a0c741.py	68	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…configs/datasets/HealthBench/healthbench_gen_831613.py	22	prompt='{prompt_trans}', # prompt mode: zero-shot	CODE
MEDIUM	…blic/knowledge/compassbench_v1_knowledge_gen_bd74e0.py	44	# Set few shot prompt number	COMMENT
MEDIUM	…s/nejm_ai_benchmark/nejmaibench_llmjudge_gen_60c8f5.py	57	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…s/datasets/nejm_ai_benchmark/nejmaibench_gen_60c8f5.py	34	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	CODE
MEDIUM	…v1_1/knowledge/compassbench_v1_knowledge_gen_bd74e0.py	44	# Set few shot prompt number	COMMENT
MEDIUM	…s/configs/datasets/bbh/bbh_0shot_nocot_academic_gen.py	75	# For zero shot inference in bbh	STRING
MEDIUM	…ompass/configs/datasets/bbh/bbh_llmjudge_gen_b5bdf1.py	75	# For zero shot inference in bbh	STRING
MEDIUM	…nfigs/datasets/SciReasoner/retrosynthesis_USPTO_gen.py	37	type=ZeroRetriever, # For our trained model, use zero-shot	CODE
8 more matches not shown…

Docstring Block Structure29 hits · 145 pts

Severity	File	Line	Snippet	Context
HIGH	…datasets/moleculariq/moleculariq_core/_pools/loader.py	29	Load a molecule pool by name. This function provides access to molecule pools for training and development	STRING
HIGH	…atasets/moleculariq/moleculariq_core/_data/__init__.py	19	Get the full path to a data file. Args: filename: Name of the data file Returns: Path to	STRING
HIGH	opencompass/datasets/teval/utils/template.py	21	Return a dictionary whose keys are from input template and value is responding content from input_string. Args:	STRING
HIGH	…ss/datasets/calm/data_processing/generate_questions.py	9	Returns the appropriate prompt generation function based on the given task. Args: task (str): The name	STRING
HIGH	…ss/datasets/calm/data_processing/generate_questions.py	153	Generates a list of questions from the dataset based on the specified prompt style. Args: dataset_path	STRING
HIGH	opencompass/datasets/calm/evaluation/core_metrics.py	128	Loads the labeling and accuracy functions dynamically based on the specified task for core metric computation.	STRING
HIGH	opencompass/datasets/calm/evaluation/core_metrics.py	286	Computes core metrics for a given set of items based on the ground truth items. Args: items (list): The	STRING
HIGH	opencompass/datasets/phybench/latex_pre_process.py	111	Removes all occurrences of a specified LaTeX-style command from a string. This function searches for a	STRING
HIGH	opencompass/datasets/phybench/latex_pre_process.py	302	Converts LaTeX vector syntax to a standardized form. This function processes a given text string and ensures that L	STRING
HIGH	opencompass/datasets/phybench/EED.py	74	Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree	STRING
HIGH	opencompass/datasets/phybench/EED.py	147	Convert a SymPy expression into a tree structure. This function takes a SymPy expression and recursively conver	STRING
HIGH	opencompass/datasets/phybench/EED.py	254	Computes the similarity score and distance metrics between two LaTeX expressions. This function evaluates the equiva	STRING
HIGH	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	150	Removes all occurrences of a specified LaTeX-style command from a string using an iterative approach. This	STRING
HIGH	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	360	Converts LaTeX vector syntax to a standardized form. This function processes a given text string and ensures t	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	94	Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	235	Convert a SymPy expression into a tree structure. This function takes a SymPy expression and recursively co	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	621	SEED (Scalable Expression Edit Distance) - Enhanced version of EED NEW FEATURES in SEED vs EED: Multi-type	STRING
HIGH	…in_generation/omegafold/utils/protein_utils/aaframe.py	404	Apply the transformation on the input coordinates Args: pos (): the 3-D coordinates to tra	STRING
HIGH	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	252	Submit code to the evaluation service. Args: pid (str): Problem ID lang (str): Programm	STRING
HIGH	opencompass/utils/text_postprocessors.py	254	Extract content after the last reasoning tag from text. When only end token is present, returns content after the e	STRING
HIGH	opencompass/utils/prompt.py	83	Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be rep	STRING
HIGH	opencompass/utils/prompt.py	108	Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be rep	STRING
HIGH	opencompass/utils/fileio.py	30	Read a JSON file and return its contents as a dictionary. Args: file_path: Path to the JSON file	STRING
HIGH	opencompass/utils/fileio.py	55	Read a JSONL file and return its contents as a list of dictionaries. Args: file_path: Path to the J	STRING
HIGH	…ncompass/openicl/icl_inferencer/icl_base_inferencer.py	65	Perform In-Context Inference given a retriever and optional templates. Args: retriever (:ob	STRING
HIGH	opencompass/openicl/icl_evaluator/hf_metrics/rouge.py	52	Calculates average rouge scores for a list of hypotheses and references Args: predictions: list of predictions to s	STRING
HIGH	opencompass/openicl/icl_evaluator/hf_metrics/squad.py	40	Computes SQuAD scores (F1 and EM). Args: predictions: List of question-answers dictionaries with the following key-	STRING
HIGH	…ncompass/openicl/icl_evaluator/hf_metrics/sacrebleu.py	45	Produces BLEU scores along with its sufficient statistics from a source against one or more references. Args: pred	STRING
HIGH	…encompass/openicl/icl_evaluator/hf_metrics/accuracy.py	33	Args: predictions (`list` of `int`): Predicted labels. references (`list` of `int`): Ground truth labels. n	STRING

AI Structural Patterns128 hits · 128 pts

Severity	File	Line	Context
LOW	opencompass/evaluator/math_evaluator.py	127	CODE
LOW	opencompass/partitioners/sub_size.py	40	CODE
LOW	opencompass/partitioners/sub_num_worker.py	32	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	446	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	516	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	620	CODE
LOW	opencompass/datasets/ProteinLMBench.py	58	CODE
LOW	opencompass/datasets/simpleqa.py	79	CODE
LOW	opencompass/datasets/gpqa.py	63	CODE
LOW	opencompass/datasets/drop_simple_eval.py	82	CODE
LOW	opencompass/datasets/gsm8k.py	79	CODE
LOW	opencompass/datasets/gsm8k.py	151	CODE
LOW	opencompass/datasets/SeedBench.py	149	CODE
LOW	opencompass/datasets/ds1000_interpreter.py	75	CODE
LOW	opencompass/datasets/generic.py	60	CODE
LOW	opencompass/datasets/math401.py	30	CODE
LOW	opencompass/datasets/advancedIF.py	223	CODE
LOW	opencompass/datasets/nejmaibench.py	90	CODE
LOW	opencompass/datasets/medmcqa.py	75	CODE
LOW	opencompass/datasets/scicode.py	363	CODE
LOW	opencompass/datasets/game24.py	230	CODE
LOW	opencompass/datasets/MedCalc_Bench.py	323	CODE
LOW	opencompass/datasets/mmlu_pro.py	75	CODE
LOW	opencompass/datasets/mastermath2024v1.py	63	CODE
LOW	opencompass/datasets/MedXpertQA.py	55	CODE
LOW	opencompass/datasets/SciKnowEval.py	71	CODE
LOW	opencompass/datasets/math.py	229	CODE
LOW	opencompass/datasets/math.py	576	CODE
LOW	opencompass/datasets/OlympiadBench.py	762	CODE
LOW	opencompass/datasets/Medbullets.py	85	CODE
LOW	opencompass/datasets/math_intern.py	105	CODE
LOW	opencompass/datasets/medbench/post_process.py	89	CODE
LOW	opencompass/datasets/babilong/babilong.py	106	CODE
LOW	opencompass/datasets/needlebench_v2/atc_elder_only.py	253	CODE
LOW	opencompass/datasets/needlebench_v2/origin.py	212	CODE
LOW	opencompass/datasets/needlebench_v2/parallel.py	308	CODE
LOW	opencompass/datasets/PMMEval/flores.py	162	CODE
LOW	opencompass/datasets/PMMEval/mifeval.py	147	CODE
LOW	opencompass/datasets/needlebench/origin.py	298	CODE
LOW	opencompass/datasets/needlebench/parallel.py	334	CODE
LOW	…ompass/datasets/teval/evaluators/planning_evaluator.py	386	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-B.py	29	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-B.py	36	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-C.py	31	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-C.py	38	CODE
LOW	…ets/calm/evaluation/error/basic_adversarial/Natural.py	34	CODE
LOW	…ets/calm/evaluation/error/basic_adversarial/Natural.py	43	CODE
LOW	…atasets/calm/evaluation/error/basic_adversarial/ECI.py	33	CODE
LOW	…atasets/calm/evaluation/error/basic_adversarial/ECI.py	40	CODE
LOW	…ation/error/basic_adversarial/AC-B_causal_judgement.py	34	CODE
LOW	…ation/error/basic_adversarial/AC-B_causal_judgement.py	41	CODE
LOW	…/calm/evaluation/error/basic_adversarial/CR-C_CRASS.py	34	CODE
LOW	…/calm/evaluation/error/basic_adversarial/CR-C_CRASS.py	41	CODE
LOW	…tasets/calm/evaluation/error/basic_adversarial/CA-B.py	32	CODE
LOW	…tasets/calm/evaluation/error/basic_adversarial/CA-B.py	39	CODE
LOW	…datasets/calm/evaluation/error/basic_adversarial/AS.py	35	CODE
LOW	…datasets/calm/evaluation/error/basic_adversarial/AS.py	42	CODE
LOW	…alm/evaluation/error/basic_adversarial/AR-B_CaLM-AR.py	29	CODE
LOW	…alm/evaluation/error/basic_adversarial/AR-B_CaLM-AR.py	36	CODE
LOW	…calm/evaluation/error/basic_adversarial/Probability.py	42	CODE
68 more matches not shown…

Self-Referential Comments41 hits · 118 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/partitioners/test_base_partitioner.py	49	# Create a partitioner that supports model_dataset_combinations	COMMENT
MEDIUM	tests/datasets/test_beyondaime.py	50	# Create a mock dataset with 'problem' column	COMMENT
MEDIUM	tests/datasets/test_aime2025.py	51	# Create a mock reader instead of importing DatasetReader	COMMENT
MEDIUM⚡	opencompass/evaluator/cascade_evaluator.py	186	# Create a subset of the test_set for LLM evaluation	COMMENT
MEDIUM	opencompass/tasks/openicl_eval.py	204	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/tasks/openicl_eval.py	435	"""This function is designed for calculating the extraction rate.	STRING
MEDIUM	opencompass/tasks/openicl_eval.py	460	"""This function is responsible for formatting prediction details.	STRING
MEDIUM	opencompass/tasks/openicl_eval.py	513	"""This function is used to calculate the BPB (Bits Per Byte) for the	STRING
MEDIUM	opencompass/tasks/subjective_eval.py	187	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/tasks/subjective_eval.py	328	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/summarizers/subjective/compassbench_v13.py	182	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	…/summarizers/subjective/compass_arena_bradley_terry.py	974	# Create a copy of the ratings dataframe to avoid modifying the original	COMMENT
MEDIUM	opencompass/summarizers/subjective/subjective.py	78	# Create a DataFrame for each judgemodel with models as rows and datasets as columns	COMMENT
MEDIUM	opencompass/summarizers/subjective/subjective.py	83	# Create a DataFrame with models as index and datasets as columns	COMMENT
MEDIUM	opencompass/summarizers/subjective/compassbench.py	181	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	opencompass/summarizers/subjective/alignmentbench.py	304	# Creating a new dictionary with '总分' as the first item	COMMENT
MEDIUM	opencompass/summarizers/subjective/qacompassbench.py	173	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	opencompass/datasets/humaneval_multi.py	173	# Create a temporary directory using the tempfile module	COMMENT
MEDIUM	opencompass/datasets/medmcqa.py	241	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/MedXpertQA.py	219	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/Medbullets.py	237	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/mmlu_cf.py	36	# Create a DatasetDict and return it	COMMENT
MEDIUM	opencompass/datasets/NPHardEval/cmp_KSP.py	123	# Create a one-dimensional array to store intermediate solutions	COMMENT
MEDIUM⚡	opencompass/datasets/codecompass/evaluator.py	134	# Create a dictionary from the results list for easier lookup	COMMENT
MEDIUM	opencompass/datasets/codecompass/executor.py	33	"""This function is called in the child process right before exec."""	STRING
MEDIUM	opencompass/datasets/codecompass/executor.py	128	# Create a unique file to avoid race conditions in parallel execution	COMMENT
MEDIUM	opencompass/datasets/livecodebench/livecodebench.py	62	# Define the dataitem mapping logic	COMMENT
MEDIUM	opencompass/datasets/ruler/ruler_vt.py	79	# Create a list of the repeated noise	COMMENT
MEDIUM	opencompass/datasets/supergpqa/supergpqa.py	316	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/phybench/latex_pre_process.py	2	#This file is used to pre-process input latex expressions	COMMENT
MEDIUM	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	2	# This file is used to pre-process input latex expressions	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	3	# This file is adopted from DeepMind Technologies Limited.	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	441	# Define a restype name for all unknown residues.	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	642	# Create an ambiguous atoms mask. shape: (21, 14).	COMMENT
MEDIUM	…s/datasets/SciReasoner/opi/utils/accuracy4fold_type.py	23	# Create a set of test sequences	COMMENT
MEDIUM	…asets/SciReasoner/LLM4Chem/retrosynthesis_evaluator.py	280	# Create a partial function for multiprocessing	COMMENT
MEDIUM	…encompass/openicl/icl_evaluator/icl_toxic_evaluator.py	123	# Create a batch request. We will add a request to the batch request	COMMENT
MEDIUM	opencompass/configs/datasets/math/math_gen_943d32.py	22	# Define the variable	COMMENT
MEDIUM	opencompass/configs/datasets/math/math_gen_943d32.py	25	# Define the inequalities for the domain based on the expression	COMMENT
MEDIUM	opencompass/configs/datasets/SciReasoner/opi_gen.py	60	# Create the dataset dictionary for the current subtask	COMMENT
MEDIUM	…pass/configs/datasets/SciReasoner/opi_rawprompt_gen.py	60	# Create the dataset dictionary for the current subtask	COMMENT

Redundant / Tautological Comments68 hits · 108 pts

Severity	File	Line	Snippet	Context
LOW	tests/datasets/test_aime2025.py	247	# Check if COMPASS_DATA_CACHE is set	COMMENT
LOW	tests/datasets/test_aime2025.py	252	# Check if CustomDataset is available	COMMENT
LOW	tests/datasets/test_aime2025.py	281	# Check if COMPASS_DATA_CACHE is set	COMMENT
LOW	opencompass/evaluator/cascade_evaluator.py	203	# Check if results already exist to avoid re-evaluation	COMMENT
LOW	opencompass/tasks/openicl_eval.py	197	# Check if we're dealing with a list of lists (pred_list_flag)	COMMENT
LOW	opencompass/tasks/subjective_eval.py	304	# Check if the start index is provided	COMMENT
LOW	opencompass/tasks/subjective_eval.py	307	# Check if the end index is provided	COMMENT
LOW	opencompass/summarizers/needlebench.py	676	# Read the file without treating the first row as a header	COMMENT
LOW	opencompass/datasets/inference_ppl.py	19	# Check if file exists in the given path	COMMENT
LOW	opencompass/datasets/ds1000.py	290	# Check if in Windows https://stackoverflow.com/questions/1325581/how-do-i-check-if-im-running-on-windows-in	COMMENT
LOW	opencompass/datasets/ds1000.py	309	# Check if in Windows https://stackoverflow.com/questions/1325581/how-do-i-check-if-im-running-on-windows-in	COMMENT
LOW	opencompass/datasets/medmcqa.py	187	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/scicode.py	146	# Check if both dictionaries have the same keys	COMMENT
LOW	opencompass/datasets/scicode.py	150	# Check if the corresponding values are close	COMMENT
LOW	opencompass/datasets/arc_prize_public_evaluation.py	207	# Check if the result is a list of lists	COMMENT
LOW	opencompass/datasets/llm_compression.py	19	# Check if file exists in the given path	COMMENT
LOW	opencompass/datasets/MedXpertQA.py	165	# Check if the answer is correct (A means correct)	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	654	# Check if the expression is a power expression	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	659	# Check if the base and the exponent are numbers	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	665	# Check if the exponent is greater than the threshold	COMMENT
LOW	opencompass/datasets/OlympiadBench.py	742	# Check if answer is correct	COMMENT
LOW	opencompass/datasets/Medbullets.py	191	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/biodata.py	325	# Check if either pred or target is empty	COMMENT
LOW	opencompass/datasets/biodata.py	485	if not ec_list: # Check if ec_list is empty	CODE
LOW⚡	opencompass/datasets/NPHardEval/p_SPP.py	178	# Check if path starts and ends with the correct nodes	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/p_SPP.py	182	# Check if the path is continuous and calculate the cost	COMMENT
LOW	opencompass/datasets/NPHardEval/p_SPP.py	194	# Check if the calculated cost matches the total cost provided in the solution	COMMENT
LOW	opencompass/datasets/NPHardEval/cmp_KSP.py	175	# Check if the item weight exceeds the knapsack capacity	COMMENT
LOW	opencompass/datasets/NPHardEval/hard_GCP.py	123	# Check if all colors in the answer are valid	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	135	# Check if all meetings are scheduled within the available time slots	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	141	# Check if the meeting is scheduled	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	145	# Check if the meeting fits within the number of total time slots	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	149	# Check if the scheduled slots are contiguous and fit the meeting duration	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	154	# Check if all participants are available at the scheduled time	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	160	# Check if any participant is double-booked	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_TSP.py	165	# Check if tour is a cycle	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_TSP.py	169	# Check if all cities are visited	COMMENT
LOW	opencompass/datasets/NPHardEval/hard_TSP.py	180	# Check if the provided tour distance is equal to the greedy tour distance	COMMENT
LOW	…s/moleculariq/moleculariq_core/rewards/index_reward.py	46	# Check if it's a valid integer	COMMENT
LOW	…s/moleculariq/moleculariq_core/rewards/index_reward.py	170	# Check if single index task (target has exactly one key)	COMMENT
LOW⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	198	# Check if this is a string-valued property	COMMENT
LOW	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	593	# Check if it's a functional group with specific suffixes	COMMENT
LOW	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	670	# Check if it's a functional group	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	565	# Check if all atoms in ring are aromatic	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	627	# Check if ring contains heteroatom (non-C, non-H)	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	655	# Check if ring is saturated (only single bonds between ring atoms)	COMMENT
LOW	opencompass/datasets/supergpqa/supergpqa.py	262	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/cmphysbench/SEED/SEED.py	500	# Check if there's a top-level '(keys) = (values)' structure	STRING
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1108	# Check if either pred or target is empty	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1166	if not ec_list: # Check if ec_list is empty	CODE
LOW	…ts/SciReasoner/unconditional_protein_generation/UPG.py	62	# Check if the input is a string;	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	151	# Check if encountering blank line (two consecutive newlines)	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	162	# Check if encountering blank line	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	253	# Check if the ratio is 2:1	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	281	# Check if the ratio of sentence types is balanced	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	708	# Check if we just closed a group that reached 5+ depth	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	1552	# Check if questions get progressively longer	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	1594	# Check if we have at least 53 lines	COMMENT
LOW	…s/datasets/lawbench/utils/compare_m2_for_evaluation.py	35	# Print results	COMMENT
LOW	opencompass/utils/run.py	250	# Check if summarizer_arg contains '/'	COMMENT
8 more matches not shown…

Magic Placeholder Names21 hits · 100 pts

Severity	File	Line	Snippet	Context
HIGH	docs/zh_cn/user_guides/deepseek_r1.md	35	key='YOUR_API_KEY', # 需替换真实API Key	CODE
HIGH	docs/zh_cn/user_guides/interns1.md	20	key="YOUR_API_KEY", # 在此处填写模型服务的API KEY	CODE
HIGH	docs/zh_cn/user_guides/interns1.md	53	key='YOUR_API_KEY',	CODE
HIGH	docs/zh_cn/notes/academic.md	79	key='YOUR_API_KEY',	CODE
HIGH	docs/zh_cn/notes/rawprompttemplate.md	61	key='YOUR_API_KEY',	CODE
HIGH	docs/en/user_guides/deepseek_r1.md	35	key='YOUR_API_KEY', # Use real API key	CODE
HIGH	docs/en/user_guides/interns1.md	22	key="YOUR_API_KEY", # Fill in your API KEY here	CODE
HIGH	docs/en/user_guides/interns1.md	56	key='YOUR_API_KEY',	CODE
HIGH	docs/en/notes/academic.md	82	key='YOUR_API_KEY',	CODE
HIGH	docs/en/notes/rawprompttemplate.md	61	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_intern_s1_pro.py	191	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_eese_api_judge.py	23	key='your-api-key',	CODE
HIGH	examples/eval_internlm3_math500_thinking.py	26	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_model_rollout.py	44	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_corebench_2409_subjective.py	95	key='YOUR_API_KEY',	CODE
HIGH	…nfigs/datasets/atlas/atlas_val_rawprompt_gen_277bee.py	210	key='YOUR_API_KEY',	STRING
HIGH	opencompass/configs/datasets/atlas/README.md	14	key="YOUR_API_KEY",	CODE
HIGH	…compass/configs/datasets/atlas/atlas_val_gen_b2d1b6.py	217	key='YOUR_API_KEY',	STRING
HIGH	opencompass/configs/models/moonshot/kimi_k2.py	13	key='your-api-key-here', # Set your API key here	CODE
HIGH	…encompass/configs/models/moonshot/kimi_k2_streaming.py	13	key='your-api-key-here', # Set your API key here	CODE
HIGH	opencompass/configs/models/interns1/intern_s1.py	14	key='YOUR_API_KEY',	CODE

Over-Commented Block94 hits · 93 pts

Severity	File	Line	Snippet	Context
LOW	.pre-commit-config-zh-cn.yaml	121	- --root_folder	COMMENT
LOW	.pre-commit-config.yaml	121	# - repo: https://github.com/open-mmlab/pre-commit-hooks	COMMENT
LOW	tools/update_dataset_suffix.py	101	# new_name = os.path.basename(new_file)[:-3]	COMMENT
LOW	docs/zh_cn/conf.py	1	# flake8: noqa	COMMENT
LOW	docs/zh_cn/advanced_guides/circular_eval.md	61	# 为了与非循环评测版本做区分而进行改名	COMMENT
LOW	docs/en/conf.py	1	# flake8: noqa	COMMENT
LOW	docs/en/advanced_guides/circular_eval.md	61	# Renaming for differentiation from non-circular evaluation versions	COMMENT
LOW	examples/eval_academic_leaderboard_202407.py	41	mmlu_pro_summary_groups	COMMENT
LOW	examples/eval_charm_rea.py	1	from mmengine.config import read_base	COMMENT
LOW	examples/eval_charm_rea.py	21	# from opencompass.configs.models.yi.hf_yi_34b_chat import models as yi_34b_chat_model	COMMENT
LOW	examples/eval_charm_rea.py	41	# from opencompass.configs.models.qwen.hf_qwen1_5_7b_chat import models as qwen1_5_7b_chat_model	COMMENT
LOW	examples/eval_charm_rea.py	61	# charm-reason-Chinese_Translate-EN - naive_average gen 48.21	COMMENT
LOW	examples/eval_mmlu_pro.py	21	work_dir = 'outputs/debug/mmlu_pro'	COMMENT
LOW	examples/eval_hellobench.py	61	# path='THUDM/glm-4-9b-chat',	COMMENT
LOW	examples/eval_PMMEval.py	21	# datasets = PMMEval_HumanEvalXL_datasets	COMMENT
LOW	examples/eval_compassarena_subjectivebench.py	41	# dict(	COMMENT
LOW	examples/eval_deepseek_r1.py	101	# path='deepseek-ai/DeepSeek-R1-Distill-Qwen-14B',	COMMENT
LOW	examples/eval_deepseek_r1.py	121	# temperature=0.6,	COMMENT
LOW	examples/eval_edgellm_demo.py	41	# chid-dev 211ee7 accuracy gen 12.87	COMMENT
LOW	examples/eval_edgellm_demo.py	61	# -------------- --------- ---------- ------ -------------	COMMENT
LOW	examples/eval_charm_mem.py	21	# from opencompass.configs.models.chatglm.hf_chatglm3_6b_32k import models as chatglm3_6b_32k_model	COMMENT
LOW	examples/eval_llama3_instruct.py	21	# nq_open_1shot 01cf41 score gen 30.25	COMMENT
LOW	examples/eval_llama3_instruct.py	41	# cmmlu-social-science - naive_average gen 54.63	COMMENT
LOW	opencompass/datasets/medbench/evaluation.py	21	elif dataset_name in dataset_loader.math_output_datasets:	COMMENT
LOW	opencompass/datasets/medbench/evaluation.py	41	# if prediction == label:	COMMENT
LOW	opencompass/datasets/medbench/dataset_loader.py	21		COMMENT
LOW	opencompass/datasets/medbench/dataset_loader.py	41	# passage = line['passage'] if line['passage'] is not None else ''	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	141	# def __init__(self, num_workers=16) -> None:	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	161	# tempdir_name,	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	181	# time_limit = 60 # should not change the official time_limit	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	201	# for line in generated_code:	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	221	# return chdir_return(cwd, False)	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	241	# # Each process changes cwd, need to use multi-processing	COMMENT
LOW	opencompass/datasets/subjective/elbench.py	41	# ELBench ships its files under a data root that holds the four original	COMMENT
LOW	opencompass/datasets/IFEval/instructions.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/evaluation_main.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/instructions_util.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/instructions_registry.py	1	# Copyright 2023 The Google Research Authors.	COMMENT
LOW	…eculariq/moleculariq_core/_data/reaction_templates.txt	61	hydrazone_formation;condensation;[C:1](=[O:2])[#6,H:3]>>[C:1](=[N][NH2])[#6,H:3];Formation of hydrazone from carbonyl	COMMENT
LOW	opencompass/datasets/livecodebench/execute_utils.py	1	# Copyright 2020 The HuggingFace Datasets Authors and the	COMMENT
LOW	opencompass/datasets/livecodebench/testing_util.py	581	# tmp_result = set(frozenset(s) for s in output) == set(	COMMENT
LOW	opencompass/datasets/agieval/evaluation.py	21	elif dataset_name in dataset_loader.math_output_datasets:	COMMENT
LOW	opencompass/datasets/agieval/evaluation.py	41	# if prediction == label:	COMMENT
LOW	opencompass/datasets/ruler/ruler_qa.py	81	# 'http://curtis.ml.cmu.edu/datasets/hotpot/hotpot_dev_distractor_v1.json'	COMMENT
LOW	…ncompass/datasets/SciReasoner/bulk_modulus_material.py	21	@staticmethod	COMMENT
LOW	opencompass/datasets/SciReasoner/GUE.py	21	@staticmethod	COMMENT
LOW	…encompass/datasets/SciReasoner/composition_material.py	61		COMMENT
LOW	opencompass/datasets/SciReasoner/PEER.py	21	from opencompass.utils import get_data_path	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	241	return 'dont_know'	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1441		COMMENT
LOW	…unconditional_protein_generation/omegafold/omegaplm.py	1	# =============================================================================	COMMENT
LOW	…r/unconditional_protein_generation/omegafold/config.py	1	# =============================================================================	COMMENT
LOW	…conditional_protein_generation/omegafold/confidence.py	1	# =============================================================================	COMMENT
LOW	…nconditional_protein_generation/omegafold/geoformer.py	1	# =============================================================================	COMMENT
LOW	…unconditional_protein_generation/omegafold/__init__.py	1	# =============================================================================	COMMENT
LOW	…unconditional_protein_generation/omegafold/__init__.py	21	# =============================================================================	COMMENT
LOW	…nconditional_protein_generation/omegafold/embedders.py	1	# =============================================================================	COMMENT
LOW	…er/unconditional_protein_generation/omegafold/model.py	1	# =============================================================================	COMMENT
LOW	…unconditional_protein_generation/omegafold/pipeline.py	1	# =============================================================================	COMMENT
LOW	…/unconditional_protein_generation/omegafold/modules.py	1	# =============================================================================	COMMENT
34 more matches not shown…

Modern Structural Boilerplate23 hits · 24 pts

Severity	File	Line	Snippet	Context
LOW	autotest/__init__.py	3	__all__ = []	CODE
LOW	autotest/cluster/__init__.py	3	__all__ = []	CODE
LOW	autotest/all/__init__.py	3	__all__ = []	CODE
LOW	autotest/model/__init__.py	3	__all__ = []	CODE
LOW	autotest/eval/__init__.py	3	__all__ = []	CODE
LOW	opencompass/metrics/__init__.py	5	__all__ = ['DumpResults', 'SEEDBenchAcc', 'MMEMetric']	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	35	def set_total(self, total: int) -> None:	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	39	def set_completed(self, completed: int) -> None:	CODE
LOW	opencompass/datasets/inverse_ifeval.py	185	def _update_bucket(bucket: Dict[str, int], score: Optional[int]) -> None:	CODE
LOW	opencompass/datasets/codecompass/__init__.py	5	__all__ = [	CODE
LOW⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	116	__all__ = [	CODE
LOW	…tasets/moleculariq/moleculariq_core/_pools/__init__.py	14	__all__ = [	CODE
LOW	…asets/moleculariq/moleculariq_core/rewards/__init__.py	137	__all__ = [	CODE
LOW	…datasets/moleculariq/moleculariq_core/_nlp/__init__.py	21	__all__ = [	CODE
LOW	…tasets/moleculariq/moleculariq_core/solver/__init__.py	12	__all__ = [	CODE
LOW	…sets/moleculariq/moleculariq_core/_dynamic/__init__.py	10	__all__ = ["MolecularIQD"]	CODE
LOW	opencompass/datasets/teval/evaluators/__init__.py	5	__all__ = ['InstructEvaluator', 'PlanningEvaluator', 'ReviewEvaluator', 'ReasonRetrieveUnderstandEvaluator']	CODE
LOW	opencompass/datasets/mp20/__init__.py	3	__all__ = ['MP20Dataset', 'MP20Evaluator']	CODE
LOW⚡	…unconditional_protein_generation/omegafold/pipeline.py	59	def _set_precision(allow_tf32: bool) -> None:	CODE
LOW	opencompass/datasets/livecodebench_pro/__init__.py	4	__all__ = ['LCBProDataset', 'LCBProEvaluator']	CODE
LOW	opencompass/datasets/IFBench/instructions.py	53	logger = logging.getLogger(__name__)	CODE
LOW	opencompass/models/claude_api/__init__.py	3	__all__ = ['Claude']	CODE
LOW	opencompass/models/telechat_api/__init__.py	4	__all__ = ['TeleChat', 'TeleChatStream']	CODE

AI Slop Vocabulary7 hits · 19 pts

Severity	File	Line	Snippet	Context
MEDIUM	examples/eval_deepseek_r1.py	144	# to max-utilize the GPUs.	COMMENT
LOW	opencompass/datasets/babilong/babilong_utils.py	161	) # appends new sentences, can be updated to just return new sentences	CODE
MEDIUM	opencompass/datasets/codecompass/executor.py	151	# Normalize whitespace and line endings for robust comparison	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	953	# Add comprehensive aliases for better coverage	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	1078	# Merge comprehensive aliases	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/opi/config.py	123	# To facilitate evaluation, each semicolon is replaced by a dot.	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/LLM4Chem/config.py	141	# that separate different parts. To facilitate evaluation,	COMMENT

Cross-Language Confusion3 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	opencompass/datasets/humaneval.py	22	cd human-eval && pip install -e .'''	CODE
HIGH	opencompass/datasets/phybench/EED.py	135	return expr1.equals(expr2)	CODE
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	223	return expr1.equals(expr2)	CODE

Structural Annotation Overuse6 hits · 10 pts

Severity	File	Line	Snippet	Context
LOW	opencompass/evaluator/cascade_evaluator.py	141	# Step 1: Evaluate each sample individually using rule-based evaluation	COMMENT
LOW⚡	opencompass/evaluator/cascade_evaluator.py	182	# Step 2: If there are samples for LLM evaluation	COMMENT
LOW⚡	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	424	# Step 2: Poll for results	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	382	# Step 1: Extract code and submit	COMMENT
LOW	…s/configs/datasets/winogrande/winogrande_ppl_9307fd.py	7	# WARNING: This config cannot reproduce results in the paper.	COMMENT
LOW	…s/configs/datasets/winogrande/winogrande_ppl_55a66e.py	7	# WARNING: This config cannot reproduce results in the paper.	COMMENT

Slop Phrases4 hits · 8 pts

Severity	File	Line	Snippet	Context
LOW	examples/eval_hellobench.py	21	# make sure your models' generation parameters are set properly, for example, if you set temperature=0.8, make sure you	COMMENT
MEDIUM	opencompass/utils/menu.py	6	# you can try `windows_curses` module by yourself	COMMENT
MEDIUM	…compass/configs/models/openai/o1_preview_2024_09_12.py	19	max_completion_tokens=8192), # you can change it for large reasoning inference cost, according to: https://platf	CODE
MEDIUM	opencompass/configs/models/openai/o1_mini_2024_09_12.py	19	max_completion_tokens=8192), # you can change it for large reasoning inference cost, according to: https://platf	CODE

Verbosity Indicators4 hits · 8 pts

Severity	File	Line	Snippet	Context
LOW	opencompass/evaluator/cascade_evaluator.py	141	# Step 1: Evaluate each sample individually using rule-based evaluation	COMMENT
LOW⚡	opencompass/evaluator/cascade_evaluator.py	182	# Step 2: If there are samples for LLM evaluation	COMMENT
LOW⚡	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	424	# Step 2: Poll for results	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	382	# Step 1: Extract code and submit	COMMENT

Dead Code3 hits · 6 pts

Severity	File	Line	Context
MEDIUM	opencompass/models/base.py	509	CODE
MEDIUM	opencompass/models/base.py	510	CODE
MEDIUM	opencompass/models/base.py	515	CODE

Synthetic Comment Markers1 hit · 5 pts

Severity	File	Line	Snippet	Context
HIGH	opencompass/datasets/subjective/wildbench.py	19	the responses generated by AI models.	CODE

Example Usage Blocks1 hit · 2 pts

Severity	File	Line	Snippet	Context
LOW	…ompass/openicl/icl_evaluator/icl_korbench_evaluator.py	265	# Example usage:	COMMENT

Analysis Overview

What These Metrics Mean

Score History

Severity Breakdown

Directory Score Breakdown

Pattern Findings