open-compass/opencompass

Pattern Findings

The scanner identified 6848 distinct pattern matches across 23 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Cross-File Repetition644 hits · 3220 pts

Severity	File	Line	Snippet	Context
HIGH	tests/TESTING_GUIDE.md	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/datasets/test_beyondaime.py	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/datasets/test_aime2025.py	0	test that dataset can be initialized with proper configuration.	STRING
HIGH	tests/openicl/test_icl_chatml_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	tests/openicl/test_icl_gen_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	tests/openicl/test_icl_chat_inferencer_parallel.py	0	test _resolve_max_workers uses default calculation.	STRING
HIGH	docs/zh_cn/faq/prompt_attack.md	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	docs/en/faq/prompt_attack.md	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	…gs/datasets/promptbench/promptbench_wnli_gen_50662f.py	0	{adv_prompt} sentence 1: {sentence1} sentence 2: {sentence2} answer:	STRING
HIGH	docs/zh_cn/evaluation/llm_judge.md	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	docs/zh_cn/evaluation/cascade_evaluator.md	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	docs/en/evaluation/llm_judge.md	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	docs/en/evaluation/cascade_evaluator.md	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	examples/eval_llm_judge.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…compass/configs/datasets/HLE/hle_verified_rawprompt.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…nfigs/datasets/HLE/hle_biomed_llm_verify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…mpass/configs/datasets/HLE/hle_llmverify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/HLE/hle_llmverify_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ts/OlymMATH/olymmath_llmverify_rawprompt_gen_9d3a8e.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/OlymMATH/olymmath_cascade_eval_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/OlymMATH/olymmath_llmverify_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/aime2026/aime2026_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…aime2026/aime2026_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…hmmt2026/hmmt2026_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/hmmt2026/hmmt2026_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…igs/datasets/matbench/matbench_llm_judge_gen_0e9276.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…s/datasets/omni_math/omni_math_llmverify_gen_ccf9c0.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…atasets/omni_math/omni_math_cascade_eval_gen_ccf9c0.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	examples/eval_cascade_evaluator.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…figs/datasets/math/math_500_cascade_eval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…tasets/math/math_prm800k_500_0shot_cot_academic_gen.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ass/configs/datasets/math/math_llm_judge_gen_56606f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…h_prm800k_500_0shot_nocot_genericllmeval_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ets/math/math_500_cascade_eval_rawprompt_gen_0970dd.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…th/math_prm800k_500_0shot_nocot_llmjudge_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…h_prm800k_500_0shot_nocot_genericllmeval_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/configs/datasets/math/math_500_llmjudge_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/math/math_prm800k_500_llmverify_repeat4_gen_97b203.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…datasets/math/math_prm800k_500_llmverify_gen_6ff468.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…m800k_500_0shot_nocot_genericllmeval_xml_gen_63a000.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…mpiadBench/OlympiadBench_0shot_llmverify_gen_be8b13.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…adBench/OlympiadBench_0shot_cascade_eval_gen_be8b13.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…dBench/OlympiadBenchMath_0shot_llmverify_gen_9c22f2.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/OlympiadBench_0shot_llmverify_rawprompt_gen_d3e9e4.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…CARDBiomedBench/CARDBiomedBench_llmjudge_gen_99a231.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…Bench/CARDBiomedBench_llmjudge_rawprompt_gen_b4d90c.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…tasets/R_Bench/rbench_llmjudge_rawprompt_gen_c24221.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…configs/datasets/R_Bench/rbench_llmjudge_gen_c89350.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…asets/BeyondAIME/beyondaime_cascade_eval_gen_5e9f4f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/configs/datasets/chatobj_custom/chatobj_custom_gen.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ivemathbench/livemathbench_hard_llmjudge_gen_71eaf5.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…h/livemathbench_hard_custom_cascade_eval_gen_4bce59.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…bench_hard_custom_cascade_eval_rawprompt_gen_e1ce64.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ench/livemathbench_hard_custom_llmverify_gen_85d0ef.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…figs/datasets/aime2025/aime2025_llmjudge_gen_5e9f4f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…/datasets/aime2025/aime2025_cascade_eval_gen_5e9f4f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…aime2025/aime2025_cascade_eval_rawprompt_gen_2f2c96.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…ets/aime2024/aime2024_llmverify_repeat16_gen_bf7475.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…figs/datasets/aime2024/aime2024_llmjudge_gen_5e9f4f.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
HIGH	…sets/aime2024/aime2024_llmverify_repeat8_gen_e8fcee.py	0	please as a grading expert, judge whether the final answers given by the candidates below are consistent with the standa	STRING
584 more matches not shown…

Unused Imports3346 hits · 2974 pts

Severity	File	Line	Context
LOW	tests/datasets/test_local_datasets.py	24	CODE
LOW	tests/datasets/test_local_datasets.py	26	CODE
LOW	tests/datasets/test_local_datasets.py	28	CODE
LOW	tests/datasets/test_local_datasets.py	30	CODE
LOW	tests/datasets/test_local_datasets.py	32	CODE
LOW	tests/datasets/test_local_datasets.py	34	CODE
LOW	tests/datasets/test_local_datasets.py	36	CODE
LOW	tests/datasets/test_local_datasets.py	38	CODE
LOW	tests/datasets/test_local_datasets.py	39	CODE
LOW	tests/datasets/test_local_datasets.py	40	CODE
LOW	tests/datasets/test_local_datasets.py	42	CODE
LOW	tests/datasets/test_local_datasets.py	44	CODE
LOW	tests/datasets/test_local_datasets.py	45	CODE
LOW	tests/datasets/test_local_datasets.py	47	CODE
LOW	tests/datasets/test_local_datasets.py	49	CODE
LOW	tests/datasets/test_local_datasets.py	51	CODE
LOW	tests/datasets/test_local_datasets.py	53	CODE
LOW	tests/datasets/test_local_datasets.py	55	CODE
LOW	tests/datasets/test_local_datasets.py	57	CODE
LOW	tests/datasets/test_local_datasets.py	59	CODE
LOW	tests/datasets/test_local_datasets.py	61	CODE
LOW	tests/datasets/test_local_datasets.py	63	CODE
LOW	tests/datasets/test_local_datasets.py	65	CODE
LOW	tests/datasets/test_local_datasets.py	67	CODE
LOW	tests/datasets/test_local_datasets.py	68	CODE
LOW	tests/datasets/test_local_datasets.py	70	CODE
LOW	tests/datasets/test_local_datasets.py	72	CODE
LOW	tests/datasets/test_local_datasets.py	74	CODE
LOW	tests/datasets/test_local_datasets.py	76	CODE
LOW	tests/datasets/test_local_datasets.py	78	CODE
LOW	tests/datasets/test_local_datasets.py	80	CODE
LOW	tests/datasets/test_local_datasets.py	82	CODE
LOW	tests/datasets/test_local_datasets.py	84	CODE
LOW	tests/datasets/test_local_datasets.py	86	CODE
LOW	tests/datasets/test_local_datasets.py	88	CODE
LOW	tests/datasets/test_local_datasets.py	90	CODE
LOW	tests/datasets/test_local_datasets.py	92	CODE
LOW	tests/datasets/test_local_datasets.py	94	CODE
LOW	tests/datasets/test_local_datasets.py	96	CODE
LOW	tests/datasets/test_local_datasets.py	98	CODE
LOW	tests/datasets/test_local_datasets.py	100	CODE
LOW	tests/datasets/test_local_datasets.py	102	CODE
LOW	tests/datasets/test_local_datasets.py	104	CODE
LOW	tests/datasets/test_local_datasets.py	106	CODE
LOW	tests/datasets/test_local_datasets.py	108	CODE
LOW	tests/datasets/test_local_datasets.py	110	CODE
LOW	tests/datasets/test_local_datasets.py	111	CODE
LOW	tests/datasets/test_local_datasets.py	113	CODE
LOW	tests/datasets/test_local_datasets.py	115	CODE
LOW	tests/datasets/test_local_datasets.py	117	CODE
LOW	tests/datasets/test_local_datasets.py	119	CODE
LOW	tests/datasets/test_local_datasets.py	121	CODE
LOW	tests/datasets/test_local_datasets.py	123	CODE
LOW	tests/datasets/test_local_datasets.py	125	CODE
LOW	tests/datasets/test_local_datasets.py	127	CODE
LOW	tests/datasets/test_local_datasets.py	129	CODE
LOW	tests/datasets/test_local_datasets.py	131	CODE
LOW	tests/datasets/test_local_datasets.py	133	CODE
LOW	tests/datasets/test_local_datasets.py	135	CODE
LOW	tests/datasets/test_local_datasets.py	137	CODE
3286 more matches not shown…

Decorative Section Separators305 hits · 1149 pts

Severity	File	Line	Snippet	Context
MEDIUM	opencompass/datasets/zebralogic.py	155	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/zebralogic.py	157	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/SciReasoner1_5.py	611	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/SciReasoner1_5.py	620	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	38	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	53	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	94	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	96	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	163	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	165	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	220	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	222	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	291	# ---------------------------------------------------------------------------	COMMENT
MEDIUM	opencompass/datasets/subjective/elbench.py	293	# ---------------------------------------------------------------------------	COMMENT
MEDIUM⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	112	# =============================================================================	COMMENT
MEDIUM⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	114	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	56	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	58	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	75	# =============================================================================	COMMENT
MEDIUM	…pass/datasets/moleculariq/moleculariq_core/__init__.py	77	# =============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	53	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	55	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	60	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	62	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	128	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	130	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	134	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	136	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	143	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	145	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	231	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	233	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	251	# ============================================================================	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	253	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	13	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	15	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	115	# ============================================================================	COMMENT
MEDIUM	…iq/moleculariq_core/rewards/property_solver_mapping.py	117	# ============================================================================	COMMENT
MEDIUM⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	194	# ============================================================================	COMMENT
MEDIUM⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	196	# ============================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	84	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	86	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	217	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	219	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	381	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	383	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	576	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	578	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	891	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	893	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	961	# ========================================================================	COMMENT
MEDIUM	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	963	# ========================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	17	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	19	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	83	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	85	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	507	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	509	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	812	# ============================================================================	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	814	# ============================================================================	COMMENT
245 more matches not shown…

Hyper-Verbose Identifiers924 hits · 891 pts

Severity	File	Line	Snippet	Context
LOW⚡	tools/analyze_repeat.py	71	def collect_predictions_by_benchmark(	STRING
LOW	tools/convert_alignmentbench.py	10	def extract_predictions_from_json(input_folder):	CODE
LOW	tests/TESTING_GUIDE.md	141	def test_load_returns_dataset(self):	CODE
LOW	tests/TESTING_GUIDE.md	186	def test_load_returns_dataset(self, mock_load_dataset):	CODE
LOW	tests/TESTING_GUIDE.md	236	def test_load_reads_jsonl_file(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE.md	258	def test_dataset_initialization(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE.md	296	def test_score_correct_prediction(self):	CODE
LOW	tests/TESTING_GUIDE.md	303	def test_score_incorrect_prediction(self):	CODE
LOW	tests/TESTING_GUIDE.md	354	def test_accuracy_calculation(self):	CODE
LOW	tests/TESTING_GUIDE.md	418	def test_result_statistics_consistency(self):	CODE
LOW	tests/TESTING_GUIDE.md	459	def test_details_count_matches_total_samples(self):	CODE
LOW	tests/TESTING_GUIDE.md	469	def test_result_metrics_completeness(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	143	def test_load_returns_dataset(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	188	def test_load_returns_dataset(self, mock_load_dataset):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	238	def test_load_reads_jsonl_file(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	260	def test_dataset_initialization(self, mock_get_data_path):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	298	def test_score_correct_prediction(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	305	def test_score_incorrect_prediction(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	356	def test_accuracy_calculation(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	420	def test_result_statistics_consistency(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	461	def test_details_count_matches_total_samples(self):	CODE
LOW	tests/TESTING_GUIDE_zh-CN.md	471	def test_result_metrics_completeness(self):	CODE
LOW	tests/evaluator/test_cascade_evaluator.py	43	def test_sample_score_without_test_set_argument(self):	CODE
LOW	tests/evaluator/test_cascade_evaluator.py	54	def test_sample_score_with_test_set_argument(self):	CODE
LOW	tests/partitioners/test_size.py	39	def test_get_actual_size_handles_supported_test_range_types(self):	CODE
LOW	tests/partitioners/test_size.py	61	def test_get_cost_handles_non_string_test_range(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	24	def test_initialization_with_custom_keep_keys(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	31	def test_parse_model_dataset_args(self):	CODE
LOW	tests/partitioners/test_base_partitioner.py	46	def test_parse_model_dataset_args_with_combinations(self):	CODE
LOW	tests/partitioners/test_naive.py	35	def test_initialization_with_custom_n(self):	CODE
LOW	tests/partitioners/test_naive.py	41	def test_partition_creates_tasks(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	64	def test_partition_with_n_greater_than_one(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	95	def test_partition_skips_existing_files(self, mock_exists):	CODE
LOW	tests/partitioners/test_naive.py	128	def test_partition_with_add_cfg(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	48	def test_remaining_without_total(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	53	def test_remaining_with_total(self):	CODE
LOW⚡	tests/tasks/test_openicl_infer_concurrent.py	60	def test_remaining_negative_protection(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	152	def test_initialization_with_defaults(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	174	def test_get_command_single_gpu(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	189	def test_get_command_multi_gpu(self, mock_randint):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	217	def test_get_command_with_backend(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	250	def test_inferencer_name_from_string(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	256	def test_inferencer_name_from_class(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	273	def test_set_default_value_existing(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	281	def test_build_inferencer_gen(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	306	def test_build_inferencer_chat(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	330	def test_build_inferencer_unsupported(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	366	def test_remaining_total_with_none(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	383	def test_run_dataset_task_success(self, mock_mkdir, mock_get_path,	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	439	def test_run_dataset_task_non_api_model(self, mock_build_dataset):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	469	def test_run_dataset_task_missing_template(self, mock_build_dataset):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	507	def test_run_skips_existing_output(self, mock_task_abbr, mock_get_path,	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	557	def test_run_with_max_workers_config(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	577	def test_run_reports_failure_after_all_task_groups(self, mock_get_path,	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	608	def test_run_task_group_returns_failed_status(self):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	626	def test_build_inferencer_with_max_infer_workers(self, mock_registry):	CODE
LOW	tests/tasks/test_openicl_infer_concurrent.py	647	def test_build_inferencer_with_model_config(self, mock_registry):	CODE
LOW	tests/tasks/test_base_task.py	14	def deperacated_test_extract_role_pred_function_source(self):	CODE
LOW	tests/tasks/test_base_task.py	39	def deperacated_test_extract_role_pred_with_begin_and_end(self):	CODE
LOW⚡	tests/tasks/test_base_task.py	50	def deperacated_test_extract_role_pred_with_begin_only(self):	CODE
864 more matches not shown…

Deep Nesting583 hits · 570 pts

Severity	File	Line	Context
LOW	setup.py	20	CODE
LOW	setup.py	51	CODE
LOW	setup.py	85	CODE
LOW	setup.py	93	CODE
LOW	tools/analyze_repeat.py	71	CODE
LOW	tools/prompt_viewer.py	51	CODE
LOW	tools/collect_code_preds.py	130	CODE
LOW	tools/chatml_format_test.py	17	CODE
LOW	tools/chatml_format_test.py	37	CODE
LOW	tests/datasets/test_longbenchv2_config.py	7	CODE
LOW	tests/datasets/test_local_datasets.py	464	CODE
LOW	tests/datasets/test_local_datasets.py	345	CODE
LOW	tests/datasets/test_vlmevalkit.py	459	CODE
LOW	tests/datasets/test_vlmevalkit.py	604	CODE
LOW	autotest/utils/compare_results.py	74	CODE
LOW	autotest/utils/compare_results.py	214	CODE
LOW	autotest/utils/compare_results.py	272	CODE
LOW	autotest/utils/compare_results.py	565	CODE
LOW	autotest/utils/oc_score_assert.py	145	CODE
LOW	docs/zh_cn/statis.py	27	CODE
LOW	docs/en/statis.py	27	CODE
LOW	opencompass/metrics/mme_score.py	48	CODE
LOW	opencompass/evaluator/math_evaluator.py	167	CODE
LOW	opencompass/evaluator/cascade_evaluator.py	107	CODE
LOW	opencompass/evaluator/cascade_evaluator.py	128	CODE
LOW	opencompass/partitioners/sub_size.py	73	CODE
LOW	opencompass/partitioners/num_worker.py	55	CODE
LOW	opencompass/partitioners/sub_naive.py	52	CODE
LOW	opencompass/partitioners/sub_naive.py	157	CODE
LOW	opencompass/partitioners/naive.py	31	CODE
LOW	opencompass/partitioners/size.py	53	CODE
LOW	opencompass/partitioners/sub_num_worker.py	68	CODE
LOW	opencompass/partitioners/base.py	40	CODE
LOW	opencompass/partitioners/base.py	104	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	154	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	256	CODE
LOW	opencompass/tasks/openicl_eval.py	257	CODE
LOW	opencompass/summarizers/needlebench.py	150	CODE
LOW	opencompass/summarizers/needlebench.py	428	CODE
LOW	opencompass/summarizers/needlebench.py	610	CODE
LOW	opencompass/summarizers/needlebench.py	675	CODE
LOW	opencompass/summarizers/needlebench.py	692	CODE
LOW	opencompass/summarizers/default_subjective.py	73	CODE
LOW	opencompass/summarizers/default_subjective.py	176	CODE
LOW	opencompass/summarizers/default_subjective.py	273	CODE
LOW	opencompass/summarizers/summarizer_pretrain.py	34	CODE
LOW	opencompass/summarizers/multi_model.py	125	CODE
LOW	opencompass/summarizers/circular.py	22	CODE
LOW	opencompass/summarizers/default.py	67	CODE
LOW	opencompass/summarizers/default.py	138	CODE
LOW	opencompass/summarizers/default.py	259	CODE
LOW	opencompass/summarizers/llm_compression.py	27	CODE
LOW	opencompass/summarizers/subjective/compassbench_v13.py	57	CODE
LOW	opencompass/summarizers/subjective/compassbench_v13.py	124	CODE
LOW	opencompass/summarizers/subjective/wildbench.py	167	CODE
LOW	opencompass/summarizers/subjective/wildbench.py	237	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	446	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	516	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	648	CODE
LOW	opencompass/summarizers/subjective/charm.py	39	CODE
523 more matches not shown…

Excessive Try-Catch Wrapping461 hits · 532 pts

Severity	File	Line	Snippet	Context
LOW⚡	tools/analyze_repeat.py	47	except Exception:	STRING
LOW⚡	tools/analyze_repeat.py	57	except Exception:	STRING
LOW⚡	tools/analyze_repeat.py	65	except Exception:	STRING
LOW	tools/convert_alignmentbench.py	27	except Exception as e:	CODE
LOW	tools/update_dataset_suffix.py	83	except Exception:	CODE
LOW	tools/chatml_format_test.py	53	except Exception as e:	CODE
LOW	tests/datasets/test_local_datasets.py	327	except Exception:	CODE
LOW	tests/datasets/test_local_datasets.py	358	except Exception as exception:	CODE
LOW	tests/datasets/test_local_datasets.py	418	except Exception as e:	CODE
LOW	autotest/all/chat_obj_fullbench_v2.py	485	except Exception:	CODE
LOW	autotest/all/chat_obj_fullbench_v1.py	342	except Exception:	CODE
LOW	opencompass/evaluator/math_evaluator.py	51	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	180	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	188	except Exception as e:	CODE
LOW	opencompass/evaluator/math_evaluator.py	207	except Exception as e:	CODE
MEDIUM	opencompass/evaluator/math_evaluator.py	86	def score(self, predictions, references, test_set=None):	CODE
LOW	opencompass/partitioners/base.py	77	except Exception:	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	238	except Exception as e:	CODE
LOW	opencompass/tasks/openicl_eval.py	183	except Exception as e:	CODE
LOW	opencompass/tasks/openicl_eval.py	345	except Exception as e:	CODE
LOW	opencompass/summarizers/subjective/alpacaeval.py	46	except Exception as e:	STRING
LOW	opencompass/summarizers/subjective/alpacaeval.py	70	except Exception as e:	STRING
LOW	opencompass/summarizers/subjective/followbench.py	58	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	145	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	169	except Exception:	CODE
LOW	opencompass/datasets/srbench.py	234	except Exception as e:	CODE
LOW	opencompass/datasets/srbench.py	252	except Exception as e:	CODE
MEDIUM	opencompass/datasets/srbench.py	161	def is_symbolically_equivalent(self, formula1, formula2, n_var=2):	CODE
LOW	opencompass/datasets/OpenSWI.py	48	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	251	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	263	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	442	except Exception:	CODE
LOW	opencompass/datasets/ds1000.py	447	except Exception:	CODE
LOW	opencompass/datasets/taco.py	186	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	352	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	397	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	474	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	506	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	552	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	582	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	615	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	626	except Exception:	CODE
LOW	opencompass/datasets/taco.py	637	except Exception:	CODE
LOW	opencompass/datasets/taco.py	654	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	678	except Exception as e:	CODE
LOW	opencompass/datasets/taco.py	691	except Exception as e:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	110	except Exception:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	126	except Exception:	CODE
LOW	opencompass/datasets/s2_tomg_bench.py	141	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	58	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	97	except Exception:	CODE
LOW	opencompass/datasets/gsm8k.py	106	except Exception:	CODE
MEDIUM	opencompass/datasets/gsm8k.py	54	def is_equal(self, pred, refer):	CODE
MEDIUM	opencompass/datasets/gsm8k.py	93	def is_equal(self, pred, refer):	CODE
MEDIUM	opencompass/datasets/gsm8k.py	101	def soft_equal(self, pred, refer, step):	CODE
LOW⚡	opencompass/datasets/humanevalx.py	162	except Exception:	CODE
LOW⚡	opencompass/datasets/humanevalx.py	167	except Exception:	CODE
LOW⚡	opencompass/datasets/humanevalx.py	177	except Exception:	CODE
LOW	opencompass/datasets/zebralogic.py	208	except Exception:	CODE
LOW	opencompass/datasets/zebralogic.py	215	except Exception:	CODE
401 more matches not shown…

Empty Error Handlers78 hits · 390 pts

Severity	File	Line	Snippet	Context
HIGH	tools/analyze_repeat.py	47	except Exception:	CODE
HIGH	tools/analyze_repeat.py	57	except Exception:	CODE
HIGH	tools/analyze_repeat.py	65	except Exception:	CODE
HIGH	autotest/all/chat_obj_fullbench_v2.py	485	except Exception:	CODE
HIGH	autotest/all/chat_obj_fullbench_v1.py	342	except Exception:	CODE
HIGH	autotest/utils/compare_results.py	81	except json.JSONDecodeError:	CODE
HIGH	examples/eval_qwen3.py	75	except:	CODE
HIGH	opencompass/datasets/taco.py	755	except SystemExit:	CODE
HIGH	opencompass/datasets/taco.py	626	except Exception:	CODE
HIGH	opencompass/datasets/taco.py	637	except Exception:	CODE
HIGH	opencompass/datasets/gsm8k.py	58	except Exception:	CODE
HIGH	opencompass/datasets/gsm8k.py	97	except Exception:	CODE
HIGH	opencompass/datasets/gsm8k.py	106	except Exception:	CODE
HIGH	opencompass/datasets/humanevalx.py	177	except Exception:	CODE
HIGH	opencompass/datasets/zebralogic.py	208	except Exception:	CODE
HIGH	opencompass/datasets/math401.py	26	except Exception:	CODE
HIGH	opencompass/datasets/advancedIF.py	141	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/advancedIF.py	86	except (json.JSONDecodeError, TypeError):	CODE
HIGH	opencompass/datasets/apps.py	794	except SystemExit:	CODE
HIGH	opencompass/datasets/apps.py	665	except Exception:	CODE
HIGH	opencompass/datasets/apps.py	676	except Exception:	CODE
HIGH	opencompass/datasets/SciReasoner1_5.py	104	except Exception:	CODE
HIGH	opencompass/datasets/SciReasoner1_5.py	133	except Exception:	CODE
HIGH	opencompass/datasets/SciReasoner1_5.py	59	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/mmlu_pro.py	53	except Exception:	CODE
HIGH	opencompass/datasets/math.py	499	except Exception:	CODE
HIGH	opencompass/datasets/math.py	507	except Exception:	CODE
HIGH	opencompass/datasets/OlympiadBench.py	399	except Exception: # 处理具体异常	CODE
HIGH	opencompass/datasets/OlympiadBench.py	408	except Exception: # 处理具体异常	CODE
HIGH	opencompass/datasets/OlympiadBench.py	415	except Exception: # 处理具体异常	CODE
HIGH	opencompass/datasets/dingo.py	115	except Exception:	CODE
HIGH	opencompass/datasets/humaneval.py	221	except Exception:	CODE
HIGH	opencompass/datasets/LCBench.py	207	except Exception:	CODE
HIGH	opencompass/datasets/vlmevalkit/dataset.py	27	except (TypeError, ValueError):	CODE
HIGH	opencompass/datasets/TheoremQA/number_utils.py	58	except Exception:	CODE
HIGH	opencompass/datasets/PMMEval/humanevalxl.py	156	except Exception:	CODE
HIGH	opencompass/datasets/codecompass/executor.py	38	except Exception:	CODE
HIGH	opencompass/datasets/moleculariq/moleculariq.py	56	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/moleculariq/moleculariq.py	51	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/moleculariq/moleculariq.py	62	except json.JSONDecodeError:	CODE
HIGH	…eculariq/moleculariq_core/rewards/constraint_reward.py	96	except json.JSONDecodeError:	CODE
HIGH	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	193	except (json.JSONDecodeError, TypeError):	CODE
HIGH	opencompass/datasets/teval/utils/format_load.py	28	except Exception as e:	CODE
HIGH	opencompass/datasets/teval/utils/format_load.py	34	except Exception as e:	CODE
HIGH	opencompass/datasets/teval/utils/format_load.py	40	except Exception as e:	CODE
HIGH	opencompass/datasets/livecodebench/testing_util.py	724	except SystemExit as e: # noqa: F841	CODE
HIGH	opencompass/datasets/livecodebench/testing_util.py	164	except Exception as e: # noqa:	CODE
HIGH	opencompass/datasets/livecodebench/testing_util.py	519	except Exception as e: # noqa: F841	CODE
HIGH	opencompass/datasets/livecodebench/testing_util.py	539	except Exception as e: # noqa: F841	CODE
HIGH	opencompass/datasets/mp20/mp20.py	82	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/mp20/mp20.py	92	except Exception:	CODE
HIGH	opencompass/datasets/mp20/mp20.py	62	except json.JSONDecodeError:	CODE
HIGH	opencompass/datasets/matbench/post_process.py	113	except ValueError:	CODE
HIGH	opencompass/datasets/matbench/post_process.py	121	except ValueError:	CODE
HIGH	opencompass/datasets/matbench/post_process.py	129	except ValueError:	CODE
HIGH	…ncompass/datasets/SciReasoner/bulk_modulus_material.py	141	except Exception:	CODE
HIGH	…encompass/datasets/SciReasoner/composition_material.py	191	except Exception:	CODE
HIGH	opencompass/datasets/SciReasoner/LLM4Mat.py	115	except Exception:	CODE
HIGH	opencompass/datasets/SciReasoner/LLM4Mat.py	217	except Exception:	CODE
HIGH	opencompass/utils/datasets.py	86	except Exception as e: # noqa	CODE
18 more matches not shown…

Modern AI Meta-Vocabulary70 hits · 172 pts

Severity	File	Line	Snippet	Context
MEDIUM	docs/zh_cn/faq/index.md	20	### OpenCompass 如何控制 few shot 评测的 shot 数目？	COMMENT
MEDIUM	docs/zh_cn/faq/index.md	24	另一方面，in context 的样本也可以直接在数据集的模板中指定，在该情况下亦会搭配使用 `ZeroRetriever`，但此时的评测并不是 0-shot，而需要根据具体的模板来进行确定。具体请看[提示词模板中的 Few-shot 小节	CODE
MEDIUM	docs/zh_cn/faq/evaluation_lmdeploy.md	70	# the max size of the context window	COMMENT
MEDIUM	docs/en/faq/index.md	20	### How does OpenCompass control the shot count of few-shot evaluation?	COMMENT
MEDIUM	docs/en/faq/index.md	24	However, in-context examples can also be written directly in the dataset template. Such a configuration also uses `ZeroR	CODE
MEDIUM	docs/en/faq/evaluation_lmdeploy.md	70	# the max size of the context window	COMMENT
MEDIUM	opencompass/datasets/CARDBiomedBench.py	29	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/nejmaibench.py	61	pass # TODO: Implement few-shot prompt handling	CODE
MEDIUM	opencompass/datasets/medmcqa.py	43	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/MedCalc_Bench.py	245	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/MedXpertQA.py	30	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/SciKnowEval.py	47	pass # TODO: Implement few-shot prompt	CODE
MEDIUM	opencompass/datasets/Medbullets.py	56	pass # TODO: Implement few-shot prompt handling	CODE
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	67	# process few-shot raw_prompts	COMMENT
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	240	# process demo once if it is few-shot-CoT	COMMENT
MEDIUM	opencompass/datasets/medbench/dataset_loader.py	327	setting_name = 'zero-shot' # setting_name can be chosen from ["zero-shot", "zero-shot-CoT", "few-shot-CoT"]	CODE
MEDIUM	opencompass/datasets/medbench/post_process.py	171	# if setting_name == 'few-shot-CoT':	COMMENT
MEDIUM	opencompass/datasets/medbench/post_process.py	194	# all other datasets are QA problems with single answer and setting_name are few-shot	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	57	# BRICS / scaffold	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	198	# BRICS / scaffold	COMMENT
MEDIUM	…ss/datasets/moleculariq/moleculariq_core/properties.py	268	# BRICS / scaffold	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	236	# Murcko scaffold (3 different methods for different purposes)	COMMENT
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	240	), # Atom count in scaffold	CODE
MEDIUM⚡	…iq/moleculariq_core/rewards/property_solver_mapping.py	248	), # SMILES string of scaffold	CODE
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	368	# BRICS and scaffold	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	761	# BRICS and scaffold positions	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1859	# Get the scaffold	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1864	# Find matching atoms between original molecule and scaffold	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1865	# We need to do a substructure match to map scaffold atoms to original indices	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1869	# If direct match fails, try to find the scaffold atoms differently	COMMENT
MEDIUM⚡	…datasets/moleculariq/moleculariq_core/solver/solver.py	1870	# Get the scaffold SMILES and match it back	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	94	# process few-shot raw_prompts	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	294	# process demo once if it is few-shot-CoT	COMMENT
MEDIUM	opencompass/datasets/agieval/dataset_loader.py	396	setting_name = 'few-shot-CoT' # setting_name can be chosen from ["zero-shot", "zero-shot-CoT", "few-shot-CoT"]	CODE
MEDIUM	opencompass/datasets/agieval/post_process.py	192	# all other datasets are QA problems with single answer and setting_name are few-shot	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/bio_instruction.py	179	# Hugging Face zero-shot pipeline默认按分数高低排序返回结果	COMMENT
MEDIUM	…ompass/openicl/icl_evaluator/icl_korbench_evaluator.py	266	# evaluator = korbenchEvaluator(question_type='logic', mode='zero-shot')	COMMENT
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	30	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	54	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	78	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…mpass/configs/datasets/MedBench/medbench_gen_0b4fff.py	102	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…CARDBiomedBench/CARDBiomedBench_llmjudge_gen_99a231.py	54	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…compass/configs/datasets/needlebench/atc/atc_choice.py	84	# Set few shot prompt number	COMMENT
MEDIUM	…datasets/needlebench/atc/atc_choice_50_en_reasoning.py	71	# Set few shot prompt number	COMMENT
MEDIUM	…pass/configs/datasets/needlebench/atc/atc_choice_20.py	83	# Set few shot prompt number	COMMENT
MEDIUM	…datasets/needlebench/atc/atc_choice_80_en_reasoning.py	47	# Set few shot prompt number	COMMENT
MEDIUM	…onfigs/datasets/advancedIF/advancedIF_rawprompt_gen.py	29	# Shared by CIF and CC subsets (zero-shot)	COMMENT
MEDIUM	…onfigs/datasets/advancedIF/advancedIF_rawprompt_gen.py	309	# System Steerability judge prompt (with few-shot examples)	COMMENT
MEDIUM	…ss/configs/datasets/llm_compression/llm_compression.py	27	# Calculates cross entropy loss for each batch based on a sliding context window	COMMENT
MEDIUM	…igs/datasets/Earth_Silver/Earth_Silver_llmjudge_gen.py	48	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…asets/Earth_Silver/Earth_Silver_llmjudge_gen_46140c.py	45	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…configs/datasets/SciKnowEval/SciKnowEval_gen_ebe47d.py	33	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	CODE
MEDIUM	…atasets/SciKnowEval/SciKnowEval_llmjudge_gen_ebe47d.py	59	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…compass/configs/datasets/agieval/agieval_gen_a0c741.py	68	), # retriver 不起作用，以输入参数为准 (zero-shot / few-shot)	CODE
MEDIUM	…configs/datasets/HealthBench/healthbench_gen_831613.py	22	prompt='{prompt_trans}', # prompt mode: zero-shot	CODE
MEDIUM	…blic/knowledge/compassbench_v1_knowledge_gen_bd74e0.py	44	# Set few shot prompt number	COMMENT
MEDIUM	…s/nejm_ai_benchmark/nejmaibench_llmjudge_gen_60c8f5.py	57	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	STRING
MEDIUM	…s/datasets/nejm_ai_benchmark/nejmaibench_gen_60c8f5.py	34	prompt=ZERO_SHOT_PROMPT, # prompt mode: zero-shot	CODE
MEDIUM	…v1_1/knowledge/compassbench_v1_knowledge_gen_bd74e0.py	44	# Set few shot prompt number	COMMENT
MEDIUM	…s/configs/datasets/bbh/bbh_0shot_nocot_academic_gen.py	75	# For zero shot inference in bbh	STRING
10 more matches not shown…

Docstring Block Structure29 hits · 145 pts

Severity	File	Line	Snippet	Context
HIGH	…datasets/moleculariq/moleculariq_core/_pools/loader.py	29	Load a molecule pool by name. This function provides access to molecule pools for training and development	STRING
HIGH	…atasets/moleculariq/moleculariq_core/_data/__init__.py	19	Get the full path to a data file. Args: filename: Name of the data file Returns: Path to	STRING
HIGH	opencompass/datasets/teval/utils/template.py	21	Return a dictionary whose keys are from input template and value is responding content from input_string. Args:	STRING
HIGH	…ss/datasets/calm/data_processing/generate_questions.py	9	Returns the appropriate prompt generation function based on the given task. Args: task (str): The name	STRING
HIGH	…ss/datasets/calm/data_processing/generate_questions.py	153	Generates a list of questions from the dataset based on the specified prompt style. Args: dataset_path	STRING
HIGH	opencompass/datasets/calm/evaluation/core_metrics.py	128	Loads the labeling and accuracy functions dynamically based on the specified task for core metric computation.	STRING
HIGH	opencompass/datasets/calm/evaluation/core_metrics.py	286	Computes core metrics for a given set of items based on the ground truth items. Args: items (list): The	STRING
HIGH	opencompass/datasets/phybench/latex_pre_process.py	111	Removes all occurrences of a specified LaTeX-style command from a string. This function searches for a	STRING
HIGH	opencompass/datasets/phybench/latex_pre_process.py	302	Converts LaTeX vector syntax to a standardized form. This function processes a given text string and ensures that L	STRING
HIGH	opencompass/datasets/phybench/EED.py	74	Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree	STRING
HIGH	opencompass/datasets/phybench/EED.py	147	Convert a SymPy expression into a tree structure. This function takes a SymPy expression and recursively conver	STRING
HIGH	opencompass/datasets/phybench/EED.py	254	Computes the similarity score and distance metrics between two LaTeX expressions. This function evaluates the equiva	STRING
HIGH	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	150	Removes all occurrences of a specified LaTeX-style command from a string using an iterative approach. This	STRING
HIGH	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	360	Converts LaTeX vector syntax to a standardized form. This function processes a given text string and ensures t	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	94	Calculate the size of a subtree based on its total insertion cost. The function computes the size of a subtree	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	235	Convert a SymPy expression into a tree structure. This function takes a SymPy expression and recursively co	STRING
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	621	SEED (Scalable Expression Edit Distance) - Enhanced version of EED NEW FEATURES in SEED vs EED: Multi-type	STRING
HIGH	…in_generation/omegafold/utils/protein_utils/aaframe.py	404	Apply the transformation on the input coordinates Args: pos (): the 3-D coordinates to tra	STRING
HIGH	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	252	Submit code to the evaluation service. Args: pid (str): Problem ID lang (str): Programm	STRING
HIGH	opencompass/utils/text_postprocessors.py	254	Extract content after the last reasoning tag from text. When only end token is present, returns content after the e	STRING
HIGH	opencompass/utils/prompt.py	83	Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be rep	STRING
HIGH	opencompass/utils/prompt.py	108	Replaces all instances of 'src' in the PromptList with 'dst'. Args: src (str): The string to be rep	STRING
HIGH	opencompass/utils/fileio.py	30	Read a JSON file and return its contents as a dictionary. Args: file_path: Path to the JSON file	STRING
HIGH	opencompass/utils/fileio.py	55	Read a JSONL file and return its contents as a list of dictionaries. Args: file_path: Path to the J	STRING
HIGH	…ncompass/openicl/icl_inferencer/icl_base_inferencer.py	65	Perform In-Context Inference given a retriever and optional templates. Args: retriever (:ob	STRING
HIGH	opencompass/openicl/icl_evaluator/hf_metrics/rouge.py	52	Calculates average rouge scores for a list of hypotheses and references Args: predictions: list of predictions to s	STRING
HIGH	opencompass/openicl/icl_evaluator/hf_metrics/squad.py	40	Computes SQuAD scores (F1 and EM). Args: predictions: List of question-answers dictionaries with the following key-	STRING
HIGH	…ncompass/openicl/icl_evaluator/hf_metrics/sacrebleu.py	45	Produces BLEU scores along with its sufficient statistics from a source against one or more references. Args: pred	STRING
HIGH	…encompass/openicl/icl_evaluator/hf_metrics/accuracy.py	33	Args: predictions (`list` of `int`): Predicted labels. references (`list` of `int`): Ground truth labels. n	STRING

AI Structural Patterns131 hits · 130 pts

Severity	File	Line	Context
LOW	opencompass/evaluator/math_evaluator.py	127	CODE
LOW	opencompass/partitioners/sub_size.py	40	CODE
LOW	opencompass/partitioners/sub_num_worker.py	32	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	446	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	516	CODE
LOW	…/summarizers/subjective/compass_arena_bradley_terry.py	620	CODE
LOW	opencompass/datasets/ProteinLMBench.py	58	CODE
LOW	opencompass/datasets/simpleqa.py	79	CODE
LOW	opencompass/datasets/gpqa.py	63	CODE
LOW	opencompass/datasets/drop_simple_eval.py	82	CODE
LOW	opencompass/datasets/gsm8k.py	79	CODE
LOW	opencompass/datasets/gsm8k.py	151	CODE
LOW	opencompass/datasets/SeedBench.py	149	CODE
LOW	opencompass/datasets/ds1000_interpreter.py	75	CODE
LOW	opencompass/datasets/critpt.py	338	CODE
LOW	opencompass/datasets/generic.py	60	CODE
LOW	opencompass/datasets/math401.py	30	CODE
LOW	opencompass/datasets/advancedIF.py	223	CODE
LOW	opencompass/datasets/nejmaibench.py	90	CODE
LOW	opencompass/datasets/medmcqa.py	75	CODE
LOW	opencompass/datasets/scicode.py	363	CODE
LOW	opencompass/datasets/game24.py	230	CODE
LOW	opencompass/datasets/MedCalc_Bench.py	323	CODE
LOW	opencompass/datasets/mmlu_pro.py	75	CODE
LOW	opencompass/datasets/mastermath2024v1.py	63	CODE
LOW	opencompass/datasets/MedXpertQA.py	55	CODE
LOW	opencompass/datasets/SciKnowEval.py	71	CODE
LOW	opencompass/datasets/math.py	229	CODE
LOW	opencompass/datasets/math.py	576	CODE
LOW	opencompass/datasets/OlympiadBench.py	762	CODE
LOW	opencompass/datasets/Medbullets.py	85	CODE
LOW	opencompass/datasets/math_intern.py	105	CODE
LOW	opencompass/datasets/medbench/post_process.py	89	CODE
LOW	opencompass/datasets/babilong/babilong.py	106	CODE
LOW	opencompass/datasets/needlebench_v2/atc_elder_only.py	253	CODE
LOW	opencompass/datasets/needlebench_v2/origin.py	212	CODE
LOW	opencompass/datasets/needlebench_v2/parallel.py	308	CODE
LOW	opencompass/datasets/PMMEval/flores.py	162	CODE
LOW	opencompass/datasets/PMMEval/mifeval.py	147	CODE
LOW	opencompass/datasets/needlebench/origin.py	298	CODE
LOW	opencompass/datasets/needlebench/parallel.py	334	CODE
LOW	…ompass/datasets/teval/evaluators/planning_evaluator.py	386	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-B.py	29	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-B.py	36	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-C.py	31	CODE
LOW	…asets/calm/evaluation/error/basic_adversarial/PCD-C.py	38	CODE
LOW	…ets/calm/evaluation/error/basic_adversarial/Natural.py	34	CODE
LOW	…ets/calm/evaluation/error/basic_adversarial/Natural.py	43	CODE
LOW	…atasets/calm/evaluation/error/basic_adversarial/ECI.py	33	CODE
LOW	…atasets/calm/evaluation/error/basic_adversarial/ECI.py	40	CODE
LOW	…ation/error/basic_adversarial/AC-B_causal_judgement.py	34	CODE
LOW	…ation/error/basic_adversarial/AC-B_causal_judgement.py	41	CODE
LOW	…/calm/evaluation/error/basic_adversarial/CR-C_CRASS.py	34	CODE
LOW	…/calm/evaluation/error/basic_adversarial/CR-C_CRASS.py	41	CODE
LOW	…tasets/calm/evaluation/error/basic_adversarial/CA-B.py	32	CODE
LOW	…tasets/calm/evaluation/error/basic_adversarial/CA-B.py	39	CODE
LOW	…datasets/calm/evaluation/error/basic_adversarial/AS.py	35	CODE
LOW	…datasets/calm/evaluation/error/basic_adversarial/AS.py	42	CODE
LOW	…alm/evaluation/error/basic_adversarial/AR-B_CaLM-AR.py	29	CODE
LOW	…alm/evaluation/error/basic_adversarial/AR-B_CaLM-AR.py	36	CODE
71 more matches not shown…

Self-Referential Comments41 hits · 118 pts

Severity	File	Line	Snippet	Context
MEDIUM	tests/partitioners/test_base_partitioner.py	49	# Create a partitioner that supports model_dataset_combinations	COMMENT
MEDIUM	tests/datasets/test_beyondaime.py	50	# Create a mock dataset with 'problem' column	COMMENT
MEDIUM	tests/datasets/test_aime2025.py	51	# Create a mock reader instead of importing DatasetReader	COMMENT
MEDIUM⚡	opencompass/evaluator/cascade_evaluator.py	195	# Create a subset of the test_set for LLM evaluation	COMMENT
MEDIUM	opencompass/tasks/openicl_eval.py	207	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/tasks/openicl_eval.py	438	"""This function is designed for calculating the extraction rate.	STRING
MEDIUM	opencompass/tasks/openicl_eval.py	463	"""This function is responsible for formatting prediction details.	STRING
MEDIUM	opencompass/tasks/openicl_eval.py	516	"""This function is used to calculate the BPB (Bits Per Byte) for the	STRING
MEDIUM	opencompass/tasks/subjective_eval.py	187	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/tasks/subjective_eval.py	328	# Create a prompt template for role config parsing	COMMENT
MEDIUM	opencompass/summarizers/subjective/compassbench_v13.py	182	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	…/summarizers/subjective/compass_arena_bradley_terry.py	974	# Create a copy of the ratings dataframe to avoid modifying the original	COMMENT
MEDIUM	opencompass/summarizers/subjective/subjective.py	78	# Create a DataFrame for each judgemodel with models as rows and datasets as columns	COMMENT
MEDIUM	opencompass/summarizers/subjective/subjective.py	83	# Create a DataFrame with models as index and datasets as columns	COMMENT
MEDIUM	opencompass/summarizers/subjective/compassbench.py	181	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	opencompass/summarizers/subjective/alignmentbench.py	304	# Creating a new dictionary with '总分' as the first item	COMMENT
MEDIUM	opencompass/summarizers/subjective/qacompassbench.py	173	# Create a DataFrame for the category and concatenate with the main DataFrame	COMMENT
MEDIUM	opencompass/datasets/humaneval_multi.py	173	# Create a temporary directory using the tempfile module	COMMENT
MEDIUM	opencompass/datasets/medmcqa.py	241	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/MedXpertQA.py	219	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/Medbullets.py	237	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/mmlu_cf.py	36	# Create a DatasetDict and return it	COMMENT
MEDIUM	opencompass/datasets/NPHardEval/cmp_KSP.py	123	# Create a one-dimensional array to store intermediate solutions	COMMENT
MEDIUM⚡	opencompass/datasets/codecompass/evaluator.py	134	# Create a dictionary from the results list for easier lookup	COMMENT
MEDIUM	opencompass/datasets/codecompass/executor.py	33	"""This function is called in the child process right before exec."""	STRING
MEDIUM	opencompass/datasets/codecompass/executor.py	128	# Create a unique file to avoid race conditions in parallel execution	COMMENT
MEDIUM	opencompass/datasets/livecodebench/livecodebench.py	62	# Define the dataitem mapping logic	COMMENT
MEDIUM	opencompass/datasets/ruler/ruler_vt.py	79	# Create a list of the repeated noise	COMMENT
MEDIUM	opencompass/datasets/supergpqa/supergpqa.py	316	# Create a flattened key for the category	COMMENT
MEDIUM	opencompass/datasets/phybench/latex_pre_process.py	2	#This file is used to pre-process input latex expressions	COMMENT
MEDIUM	…compass/datasets/cmphysbench/SEED/latex_pre_process.py	2	# This file is used to pre-process input latex expressions	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	3	# This file is adopted from DeepMind Technologies Limited.	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	441	# Define a restype name for all unknown residues.	COMMENT
MEDIUM	…ion/omegafold/utils/protein_utils/residue_constants.py	642	# Create an ambiguous atoms mask. shape: (21, 14).	COMMENT
MEDIUM	…s/datasets/SciReasoner/opi/utils/accuracy4fold_type.py	23	# Create a set of test sequences	COMMENT
MEDIUM	…asets/SciReasoner/LLM4Chem/retrosynthesis_evaluator.py	280	# Create a partial function for multiprocessing	COMMENT
MEDIUM	…encompass/openicl/icl_evaluator/icl_toxic_evaluator.py	123	# Create a batch request. We will add a request to the batch request	COMMENT
MEDIUM	opencompass/configs/datasets/math/math_gen_943d32.py	22	# Define the variable	COMMENT
MEDIUM	opencompass/configs/datasets/math/math_gen_943d32.py	25	# Define the inequalities for the domain based on the expression	COMMENT
MEDIUM	opencompass/configs/datasets/SciReasoner/opi_gen.py	60	# Create the dataset dictionary for the current subtask	COMMENT
MEDIUM	…pass/configs/datasets/SciReasoner/opi_rawprompt_gen.py	60	# Create the dataset dictionary for the current subtask	COMMENT

Redundant / Tautological Comments67 hits · 107 pts

Severity	File	Line	Snippet	Context
LOW	tests/datasets/test_aime2025.py	247	# Check if COMPASS_DATA_CACHE is set	COMMENT
LOW	tests/datasets/test_aime2025.py	252	# Check if CustomDataset is available	COMMENT
LOW	tests/datasets/test_aime2025.py	281	# Check if COMPASS_DATA_CACHE is set	COMMENT
LOW	opencompass/evaluator/cascade_evaluator.py	212	# Check if results already exist to avoid re-evaluation	COMMENT
LOW	opencompass/tasks/openicl_eval.py	200	# Check if we're dealing with a list of lists (pred_list_flag)	COMMENT
LOW	opencompass/tasks/subjective_eval.py	304	# Check if the start index is provided	COMMENT
LOW	opencompass/tasks/subjective_eval.py	307	# Check if the end index is provided	COMMENT
LOW	opencompass/summarizers/needlebench.py	676	# Read the file without treating the first row as a header	COMMENT
LOW	opencompass/datasets/inference_ppl.py	19	# Check if file exists in the given path	COMMENT
LOW	opencompass/datasets/ds1000.py	290	# Check if in Windows https://stackoverflow.com/questions/1325581/how-do-i-check-if-im-running-on-windows-in	COMMENT
LOW	opencompass/datasets/ds1000.py	309	# Check if in Windows https://stackoverflow.com/questions/1325581/how-do-i-check-if-im-running-on-windows-in	COMMENT
LOW	opencompass/datasets/medmcqa.py	187	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/scicode.py	146	# Check if both dictionaries have the same keys	COMMENT
LOW	opencompass/datasets/scicode.py	150	# Check if the corresponding values are close	COMMENT
LOW	opencompass/datasets/arc_prize_public_evaluation.py	207	# Check if the result is a list of lists	COMMENT
LOW	opencompass/datasets/llm_compression.py	19	# Check if file exists in the given path	COMMENT
LOW	opencompass/datasets/MedXpertQA.py	165	# Check if the answer is correct (A means correct)	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	654	# Check if the expression is a power expression	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	659	# Check if the base and the exponent are numbers	COMMENT
LOW⚡	opencompass/datasets/OlympiadBench.py	665	# Check if the exponent is greater than the threshold	COMMENT
LOW	opencompass/datasets/OlympiadBench.py	742	# Check if answer is correct	COMMENT
LOW	opencompass/datasets/Medbullets.py	191	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/biodata.py	326	# Check if either pred or target is empty	COMMENT
LOW	opencompass/datasets/biodata.py	486	if not ec_list: # Check if ec_list is empty	CODE
LOW⚡	opencompass/datasets/NPHardEval/p_SPP.py	178	# Check if path starts and ends with the correct nodes	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/p_SPP.py	182	# Check if the path is continuous and calculate the cost	COMMENT
LOW	opencompass/datasets/NPHardEval/p_SPP.py	194	# Check if the calculated cost matches the total cost provided in the solution	COMMENT
LOW	opencompass/datasets/NPHardEval/cmp_KSP.py	175	# Check if the item weight exceeds the knapsack capacity	COMMENT
LOW	opencompass/datasets/NPHardEval/hard_GCP.py	123	# Check if all colors in the answer are valid	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	135	# Check if all meetings are scheduled within the available time slots	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	141	# Check if the meeting is scheduled	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	145	# Check if the meeting fits within the number of total time slots	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	149	# Check if the scheduled slots are contiguous and fit the meeting duration	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	154	# Check if all participants are available at the scheduled time	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_MSP.py	160	# Check if any participant is double-booked	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_TSP.py	165	# Check if tour is a cycle	COMMENT
LOW⚡	opencompass/datasets/NPHardEval/hard_TSP.py	169	# Check if all cities are visited	COMMENT
LOW	opencompass/datasets/NPHardEval/hard_TSP.py	180	# Check if the provided tour distance is equal to the greedy tour distance	COMMENT
LOW	…s/moleculariq/moleculariq_core/rewards/index_reward.py	46	# Check if it's a valid integer	COMMENT
LOW	…s/moleculariq/moleculariq_core/rewards/index_reward.py	170	# Check if single index task (target has exactly one key)	COMMENT
LOW⚡	…eculariq/moleculariq_core/rewards/constraint_reward.py	198	# Check if this is a string-valued property	COMMENT
LOW	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	593	# Check if it's a functional group with specific suffixes	COMMENT
LOW	…atasets/moleculariq/moleculariq_core/_nlp/formatter.py	670	# Check if it's a functional group	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	565	# Check if all atoms in ring are aromatic	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	627	# Check if ring contains heteroatom (non-C, non-H)	COMMENT
LOW	…datasets/moleculariq/moleculariq_core/solver/solver.py	655	# Check if ring is saturated (only single bonds between ring atoms)	COMMENT
LOW	opencompass/datasets/supergpqa/supergpqa.py	262	# Check if the answer is correct (A means correct)	COMMENT
LOW	opencompass/datasets/cmphysbench/SEED/SEED.py	500	# Check if there's a top-level '(keys) = (values)' structure	STRING
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1108	# Check if either pred or target is empty	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1166	if not ec_list: # Check if ec_list is empty	CODE
LOW	…ts/SciReasoner/unconditional_protein_generation/UPG.py	62	# Check if the input is a string;	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	151	# Check if encountering blank line (two consecutive newlines)	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	162	# Check if encountering blank line	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	253	# Check if the ratio is 2:1	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	281	# Check if the ratio of sentence types is balanced	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	708	# Check if we just closed a group that reached 5+ depth	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	1552	# Check if questions get progressively longer	COMMENT
LOW	opencompass/datasets/IFBench/instructions.py	1594	# Check if we have at least 53 lines	COMMENT
LOW	…s/datasets/lawbench/utils/compare_m2_for_evaluation.py	35	# Print results	COMMENT
LOW	opencompass/utils/run.py	258	# Check if summarizer_arg contains '/'	COMMENT
7 more matches not shown…

Over-Commented Block101 hits · 98 pts

Severity	File	Line	Snippet	Context
LOW	.pre-commit-config-zh-cn.yaml	121	- --root_folder	COMMENT
LOW	.pre-commit-config.yaml	121	# - repo: https://github.com/open-mmlab/pre-commit-hooks	COMMENT
LOW	README.md	101	- #### Install OpenCompass via pip	COMMENT
LOW	tools/update_dataset_suffix.py	101	# new_name = os.path.basename(new_file)[:-3]	COMMENT
LOW	docs/zh_cn/conf.py	1	# flake8: noqa	COMMENT
LOW	docs/zh_cn/faq/circular_eval.md	61	# 为了与非循环评测版本做区分而进行改名	COMMENT
LOW	docs/en/conf.py	1	# flake8: noqa	COMMENT
LOW	docs/en/faq/circular_eval.md	61	# Renaming for differentiation from non-circular evaluation versions	COMMENT
LOW	examples/eval_academic_leaderboard_202407.py	41	mmlu_pro_summary_groups	COMMENT
LOW	examples/eval_charm_rea.py	1	from mmengine.config import read_base	COMMENT
LOW	examples/eval_charm_rea.py	21	# from opencompass.configs.models.yi.hf_yi_34b_chat import models as yi_34b_chat_model	COMMENT
LOW	examples/eval_charm_rea.py	41	# from opencompass.configs.models.qwen.hf_qwen1_5_7b_chat import models as qwen1_5_7b_chat_model	COMMENT
LOW	examples/eval_charm_rea.py	61	# charm-reason-Chinese_Translate-EN - naive_average gen 48.21	COMMENT
LOW	examples/eval_mmlu_pro.py	21	work_dir = 'outputs/debug/mmlu_pro'	COMMENT
LOW	examples/eval_hellobench.py	61	# path='THUDM/glm-4-9b-chat',	COMMENT
LOW	examples/eval_PMMEval.py	21	# datasets = PMMEval_HumanEvalXL_datasets	COMMENT
LOW	examples/eval_compassarena_subjectivebench.py	41	# dict(	COMMENT
LOW	examples/eval_deepseek_r1.py	101	# path='deepseek-ai/DeepSeek-R1-Distill-Qwen-14B',	COMMENT
LOW	examples/eval_deepseek_r1.py	121	# temperature=0.6,	COMMENT
LOW	examples/eval_edgellm_demo.py	41	# chid-dev 211ee7 accuracy gen 12.87	COMMENT
LOW	examples/eval_edgellm_demo.py	61	# -------------- --------- ---------- ------ -------------	COMMENT
LOW	examples/eval_charm_mem.py	21	# from opencompass.configs.models.chatglm.hf_chatglm3_6b_32k import models as chatglm3_6b_32k_model	COMMENT
LOW	examples/eval_llama3_instruct.py	21	# nq_open_1shot 01cf41 score gen 30.25	COMMENT
LOW	examples/eval_llama3_instruct.py	41	# cmmlu-social-science - naive_average gen 54.63	COMMENT
LOW	opencompass/datasets/medbench/evaluation.py	21	elif dataset_name in dataset_loader.math_output_datasets:	COMMENT
LOW	opencompass/datasets/medbench/evaluation.py	41	# if prediction == label:	COMMENT
LOW	opencompass/datasets/medbench/dataset_loader.py	21		COMMENT
LOW	opencompass/datasets/medbench/dataset_loader.py	41	# passage = line['passage'] if line['passage'] is not None else ''	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	141	# def __init__(self, num_workers=16) -> None:	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	161	# tempdir_name,	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	181	# time_limit = 60 # should not change the official time_limit	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	201	# for line in generated_code:	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	221	# return chdir_return(cwd, False)	COMMENT
LOW	opencompass/datasets/subjective/followbench.py	241	# # Each process changes cwd, need to use multi-processing	COMMENT
LOW	opencompass/datasets/subjective/elbench.py	41	# ELBench ships its files under a data root that holds the four original	COMMENT
LOW	opencompass/datasets/IFEval/instructions.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/evaluation_main.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/instructions_util.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/IFEval/instructions_registry.py	1	# Copyright 2023 The Google Research Authors.	COMMENT
LOW	…eculariq/moleculariq_core/_data/reaction_templates.txt	61	hydrazone_formation;condensation;[C:1](=[O:2])[#6,H:3]>>[C:1](=[N][NH2])[#6,H:3];Formation of hydrazone from carbonyl	COMMENT
LOW	opencompass/datasets/livecodebench/execute_utils.py	1	# Copyright 2020 The HuggingFace Datasets Authors and the	COMMENT
LOW	opencompass/datasets/livecodebench/testing_util.py	581	output = output.split()	COMMENT
LOW	opencompass/datasets/livecodebench/testing_util.py	601	# )	COMMENT
LOW	opencompass/datasets/MultiIF/instructions.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/MultiIF/evaluation_main.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/MultiIF/instructions_util.py	1	# flake8: noqa	COMMENT
LOW	opencompass/datasets/MultiIF/instructions_registry.py	1	# Copyright 2023 The Google Research Authors.	COMMENT
LOW	opencompass/datasets/agieval/evaluation.py	21	elif dataset_name in dataset_loader.math_output_datasets:	COMMENT
LOW	opencompass/datasets/agieval/evaluation.py	41	# if prediction == label:	COMMENT
LOW	opencompass/datasets/ruler/ruler_qa.py	81	# 'http://curtis.ml.cmu.edu/datasets/hotpot/hotpot_dev_distractor_v1.json'	COMMENT
LOW	opencompass/datasets/fake_alignment/fake_alignment.py	1	# flake8: noqa	COMMENT
LOW	…ncompass/datasets/SciReasoner/bulk_modulus_material.py	21	@staticmethod	COMMENT
LOW	opencompass/datasets/SciReasoner/GUE.py	21	@staticmethod	COMMENT
LOW	…encompass/datasets/SciReasoner/composition_material.py	61		COMMENT
LOW	opencompass/datasets/SciReasoner/PEER.py	21	from opencompass.utils import get_data_path	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	241	return 'dont_know'	COMMENT
LOW	opencompass/datasets/SciReasoner/bio_instruction.py	1441		COMMENT
LOW	…unconditional_protein_generation/omegafold/omegaplm.py	1	# =============================================================================	COMMENT
LOW	…r/unconditional_protein_generation/omegafold/config.py	1	# =============================================================================	COMMENT
LOW	…conditional_protein_generation/omegafold/confidence.py	1	# =============================================================================	COMMENT
41 more matches not shown…

Magic Placeholder Names15 hits · 70 pts

Severity	File	Line	Snippet	Context
HIGH	docs/zh_cn/get_started/quick_start.md	17	key='your-api-key',	CODE
HIGH	docs/zh_cn/get_started/quick_start.md	26	export OPENCOMPASS_API_KEY="your-api-key"	CODE
HIGH	docs/en/get_started/quick_start.md	17	key='your-api-key',	CODE
HIGH	docs/en/get_started/quick_start.md	26	export OPENCOMPASS_API_KEY="your-api-key"	CODE
HIGH	examples/eval_intern_s1_pro.py	191	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_eese_api_judge.py	23	key='your-api-key',	CODE
HIGH	examples/eval_internlm3_math500_thinking.py	26	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_model_rollout.py	44	key='YOUR_API_KEY',	CODE
HIGH	examples/eval_corebench_2409_subjective.py	95	key='YOUR_API_KEY',	CODE
HIGH	…nfigs/datasets/atlas/atlas_val_rawprompt_gen_277bee.py	210	key='YOUR_API_KEY',	STRING
HIGH	opencompass/configs/datasets/atlas/README.md	14	key="YOUR_API_KEY",	CODE
HIGH	…compass/configs/datasets/atlas/atlas_val_gen_b2d1b6.py	217	key='YOUR_API_KEY',	STRING
HIGH	opencompass/configs/models/moonshot/kimi_k2.py	13	key='your-api-key-here', # Set your API key here	CODE
HIGH	…encompass/configs/models/moonshot/kimi_k2_streaming.py	13	key='your-api-key-here', # Set your API key here	CODE
HIGH	opencompass/configs/models/intern_s/intern_s1.py	14	key='YOUR_API_KEY',	CODE

Modern Structural Boilerplate24 hits · 25 pts

Severity	File	Line	Snippet	Context
LOW	autotest/__init__.py	3	__all__ = []	CODE
LOW	autotest/cluster/__init__.py	3	__all__ = []	CODE
LOW	autotest/all/__init__.py	3	__all__ = []	CODE
LOW	autotest/model/__init__.py	3	__all__ = []	CODE
LOW	autotest/eval/__init__.py	3	__all__ = []	CODE
LOW	opencompass/metrics/__init__.py	5	__all__ = ['DumpResults', 'SEEDBenchAcc', 'MMEMetric']	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	35	def set_total(self, total: int) -> None:	CODE
LOW	opencompass/tasks/openicl_infer_concurrent.py	39	def set_completed(self, completed: int) -> None:	CODE
LOW	opencompass/datasets/inverse_ifeval.py	185	def _update_bucket(bucket: Dict[str, int], score: Optional[int]) -> None:	CODE
LOW	opencompass/datasets/vlmevalkit/__init__.py	5	__all__ = [	CODE
LOW	opencompass/datasets/codecompass/__init__.py	5	__all__ = [	CODE
LOW⚡	…pass/datasets/moleculariq/moleculariq_core/__init__.py	116	__all__ = [	CODE
LOW	…tasets/moleculariq/moleculariq_core/_pools/__init__.py	14	__all__ = [	CODE
LOW	…asets/moleculariq/moleculariq_core/rewards/__init__.py	137	__all__ = [	CODE
LOW	…datasets/moleculariq/moleculariq_core/_nlp/__init__.py	21	__all__ = [	CODE
LOW	…tasets/moleculariq/moleculariq_core/solver/__init__.py	12	__all__ = [	CODE
LOW	…sets/moleculariq/moleculariq_core/_dynamic/__init__.py	10	__all__ = ["MolecularIQD"]	CODE
LOW	opencompass/datasets/teval/evaluators/__init__.py	5	__all__ = ['InstructEvaluator', 'PlanningEvaluator', 'ReviewEvaluator', 'ReasonRetrieveUnderstandEvaluator']	CODE
LOW	opencompass/datasets/mp20/__init__.py	3	__all__ = ['MP20Dataset', 'MP20Evaluator']	CODE
LOW⚡	…unconditional_protein_generation/omegafold/pipeline.py	59	def _set_precision(allow_tf32: bool) -> None:	CODE
LOW	opencompass/datasets/livecodebench_pro/__init__.py	4	__all__ = ['LCBProDataset', 'LCBProEvaluator']	CODE
LOW	opencompass/datasets/IFBench/instructions.py	53	logger = logging.getLogger(__name__)	CODE
LOW	opencompass/models/claude_api/__init__.py	3	__all__ = ['Claude']	CODE
LOW	opencompass/models/telechat_api/__init__.py	4	__all__ = ['TeleChat', 'TeleChatStream']	CODE

AI Slop Vocabulary7 hits · 19 pts

Severity	File	Line	Snippet	Context
MEDIUM	examples/eval_deepseek_r1.py	144	# to max-utilize the GPUs.	COMMENT
LOW	opencompass/datasets/babilong/babilong_utils.py	161	) # appends new sentences, can be updated to just return new sentences	CODE
MEDIUM	opencompass/datasets/codecompass/executor.py	151	# Normalize whitespace and line endings for robust comparison	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	953	# Add comprehensive aliases for better coverage	COMMENT
MEDIUM	…datasets/moleculariq/moleculariq_core/_nlp/mappings.py	1078	# Merge comprehensive aliases	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/opi/config.py	123	# To facilitate evaluation, each semicolon is replaced by a dot.	COMMENT
MEDIUM	opencompass/datasets/SciReasoner/LLM4Chem/config.py	141	# that separate different parts. To facilitate evaluation,	COMMENT

Cross-Language Confusion3 hits · 15 pts

Severity	File	Line	Snippet	Context
HIGH	opencompass/datasets/humaneval.py	23	cd human-eval && pip install -e .'''	CODE
HIGH	opencompass/datasets/phybench/EED.py	135	return expr1.equals(expr2)	CODE
HIGH	opencompass/datasets/cmphysbench/SEED/SEED.py	223	return expr1.equals(expr2)	CODE

Structural Annotation Overuse6 hits · 10 pts

Severity	File	Line	Snippet	Context
LOW	opencompass/evaluator/cascade_evaluator.py	147	# Step 1: Evaluate each sample individually using rule-based evaluation	COMMENT
LOW⚡	opencompass/evaluator/cascade_evaluator.py	191	# Step 2: If there are samples for LLM evaluation	COMMENT
LOW⚡	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	424	# Step 2: Poll for results	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	382	# Step 1: Extract code and submit	COMMENT
LOW	…s/configs/datasets/winogrande/winogrande_ppl_9307fd.py	7	# WARNING: This config cannot reproduce results in the paper.	COMMENT
LOW	…s/configs/datasets/winogrande/winogrande_ppl_55a66e.py	7	# WARNING: This config cannot reproduce results in the paper.	COMMENT

Slop Phrases4 hits · 8 pts

Severity	File	Line	Snippet	Context
LOW	examples/eval_hellobench.py	21	# make sure your models' generation parameters are set properly, for example, if you set temperature=0.8, make sure you	COMMENT
MEDIUM	opencompass/utils/menu.py	6	# you can try `windows_curses` module by yourself	COMMENT
MEDIUM	…compass/configs/models/openai/o1_preview_2024_09_12.py	19	max_completion_tokens=8192), # you can change it for large reasoning inference cost, according to: https://platf	CODE
MEDIUM	opencompass/configs/models/openai/o1_mini_2024_09_12.py	19	max_completion_tokens=8192), # you can change it for large reasoning inference cost, according to: https://platf	CODE

Verbosity Indicators4 hits · 8 pts

Severity	File	Line	Snippet	Context
LOW	opencompass/evaluator/cascade_evaluator.py	147	# Step 1: Evaluate each sample individually using rule-based evaluation	COMMENT
LOW⚡	opencompass/evaluator/cascade_evaluator.py	191	# Step 2: If there are samples for LLM evaluation	COMMENT
LOW⚡	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	424	# Step 2: Poll for results	COMMENT
LOW	…asets/livecodebench_pro/livecodebench_pro_evaluator.py	382	# Step 1: Extract code and submit	COMMENT

Dead Code3 hits · 6 pts

Severity	File	Line	Context
MEDIUM	opencompass/models/base.py	509	CODE
MEDIUM	opencompass/models/base.py	510	CODE
MEDIUM	opencompass/models/base.py	515	CODE

Synthetic Comment Markers1 hit · 5 pts

Severity	File	Line	Snippet	Context
HIGH	opencompass/datasets/subjective/wildbench.py	19	the responses generated by AI models.	CODE

Example Usage Blocks1 hit · 2 pts

Severity	File	Line	Snippet	Context
LOW	…ompass/openicl/icl_evaluator/icl_korbench_evaluator.py	265	# Example usage:	COMMENT

Maintenance and Governance Review

Risk Review · v3

What These Metrics Mean

Pattern-Signal History

Severity Breakdown

Directory Score Breakdown

Pattern Findings