Home / huggingface / open-r1
AI Slop Code RIsk Monitor

huggingface/open-r1

Fully open reproduction of DeepSeek-R1

36.5 High pattern signal View on GitHub

Maintenance and Governance Review

This report summarizes scanner evidence for huggingface/open-r1, a Python project with 26,478 GitHub stars. SynthScan examined 8,289 lines of code across 65 source files, recording 188 pattern matches distributed across 14 syntactic categories. The legacy adjusted pattern signal is 36.5 (High pattern signal); it is not an authorship estimate.

Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.

36.5
Adjusted Signal
36.5
Raw Pattern Score
100%
Time Factor
2026-10-02
Last Push
26.5K
Stars
Python
Language
8.3K
Lines of Code
65
Files
188
Pattern Hits
2026-10-12
Scan Date
0.31
HC Hit Rate

Risk Review · v3

Use each vector to guide manual verification of the underlying findings.

Cognitive Ownership Risk 58.8/100 · HIGH

Can maintainers explain intent, assumptions, and trade-offs? 108 finding(s).

Hallucinated Dependency & Supply Chain Risk 22.4/100 · LOW

Are imports declared and package/API names verified? 22 finding(s).

Agent Over-Engineering Ratio 64.8/100 · HIGH

Are wrappers, guards, and repeated structures necessary? 116 finding(s).

PR Review Velocity Impact 85.8/100 · CRITICAL

Which changes need focused human review? 188 finding(s).

What These Metrics Mean

Adjusted Signal
Legacy pattern score normalized per 1,000 scanned lines and multiplied by the repository-level time factor. It is not an estimate of AI authorship.
Raw Pattern Score
Sum of severity- and context-weighted pattern findings before applying the time factor.
Time Factor
A coarse multiplier based on repository last-push metadata. It does not identify when individual code was written.
Pattern Hits
Count of configured pattern findings. Review each location and surrounding code before acting.
HC Hit Rate
High- and critical-severity findings per scanned file. Severity is a configured detector weight, not proof of a vulnerability or authorship.
Lines / Files
Scan coverage denominators used by the legacy normalized score.

Pattern-Signal History

Run additional scans to build a history of the legacy pattern signal. Score changes are not a measure of AI adoption or code quality.

No multi-scan history yet — run the scanner again to build trend data.

Severity Breakdown

Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.

CRITICAL 0HIGH 20MEDIUM 18LOW 150

Directory Score Breakdown

This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.

Pattern Findings

The scanner identified 188 distinct pattern matches across 14 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.

Excessive Try-Catch Wrapping44 hits · 72 pts
SeverityFileLineSnippetContext
LOWscripts/generate_reasoning.py38 except Exception as e:CODE
MEDIUMscripts/generate_reasoning.py54 print(f"Error processing example")CODE
LOWscripts/generate_reasoning.py85 except Exception as e:CODE
MEDIUMscripts/generate_reasoning.py86 print(f"Error processing example: {e}")CODE
LOWscripts/generate_reasoning.py156 except Exception as e:CODE
LOWscripts/morph_router.py127 except Exception as e:CODE
LOWscripts/morph_router.py135 except Exception:CODE
LOWscripts/e2b_router.py124 except Exception as e:CODE
LOWscripts/e2b_router.py130 except Exception:CODE
LOWscripts/get_tensor_parallel_size.py15 except Exception as e:CODE
LOWsrc/open_r1/rewards.py73 except Exception as e:CODE
LOWsrc/open_r1/rewards.py395 except Exception as e:CODE
MEDIUMsrc/open_r1/rewards.py396 print(f"Error from {provider_type} worker: {e}")CODE
LOWsrc/open_r1/rewards.py448 except Exception as e:CODE
MEDIUMsrc/open_r1/rewards.py449 print(f"Error from Piston worker: {e}")CODE
MEDIUMsrc/open_r1/rewards.py392def run_catch_exceptions(task):CODE
MEDIUMsrc/open_r1/rewards.py445def run_catch_exceptions(task):CODE
LOWsrc/open_r1/utils/hub.py94 except Exception:CODE
LOWsrc/open_r1/utils/routed_morph.py113 except Exception as e:CODE
MEDIUM⚡src/open_r1/utils/code_providers.py110 print(f"Error from E2B executor: {e}")CODE
MEDIUM⚡src/open_r1/utils/code_providers.py120 print(f"Error from E2B executor async: {e}")CODE
MEDIUM⚡src/open_r1/utils/code_providers.py160 print(f"Error in `_run_script` from E2B sandbox ID {sandbox.sandbox_id} : {e}")CODE
MEDIUM⚡src/open_r1/utils/code_providers.py166 print(f"Error from E2B executor kill with sandbox ID {sandbox.sandbox_id} : {e}")CODE
MEDIUM⚡src/open_r1/utils/code_providers.py240 print(f"Error from MorphCloud router: {e}")CODE
MEDIUM⚡src/open_r1/utils/code_providers.py248 print(f"Error from MorphCloud executor: {e}")CODE
LOW⚡src/open_r1/utils/code_providers.py103 except Exception:CODE
LOW⚡src/open_r1/utils/code_providers.py109 except Exception as e:CODE
LOW⚡src/open_r1/utils/code_providers.py119 except Exception as e:CODE
LOW⚡src/open_r1/utils/code_providers.py159 except Exception as e:CODE
LOW⚡src/open_r1/utils/code_providers.py165 except Exception as e:CODE
LOW⚡src/open_r1/utils/code_providers.py239 except Exception as e:CODE
LOW⚡src/open_r1/utils/code_providers.py247 except Exception as e:CODE
LOWsrc/open_r1/utils/code_providers.py328 except Exception:CODE
LOWsrc/open_r1/utils/code_providers.py335 except Exception:CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py78 except Exception:CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py324 except Exception as e:CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py399 except Exception as e:CODE
LOW⚡src/open_r1/utils/competitive_programming/cf_scoring.py51 except Exception as e:CODE
MEDIUM⚡src/open_r1/utils/competitive_programming/cf_scoring.py52 print(f"Error scoring submission: {e}")CODE
LOW…open_r1/utils/competitive_programming/piston_client.py131 except Exception as e:CODE
MEDIUM…open_r1/utils/competitive_programming/piston_client.py132 print(f"Error checking endpoint {endpoint}, dropping it ({e})")CODE
LOW⚡…open_r1/utils/competitive_programming/piston_client.py188 except Exception as e:CODE
LOW⚡…open_r1/utils/competitive_programming/piston_client.py196 except Exception as e:CODE
MEDIUM⚡…open_r1/utils/competitive_programming/piston_client.py197 print(f"Error releasing endpoint {endpoint}: {e}")CODE
Hyper-Verbose Identifiers57 hits · 58 pts
SeverityFileLineSnippetContext
LOW⚡tests/test_rewards.py79 def test_accuracy_reward_correct_answer(self):CODE
LOW⚡tests/test_rewards.py86 def test_accuracy_reward_wrong_answer(self):CODE
LOW⚡tests/test_rewards.py93 def test_accuracy_reward_wrong_answer_no_latex(self):CODE
LOW⚡tests/test_rewards.py100 def test_format_reward_correct(self):CODE
LOW⚡tests/test_rewards.py106 def test_format_reward_incorrect(self):CODE
LOW⚡tests/test_rewards.py364 def test_three_word_completion(self):CODE
LOW⚡tests/test_rewards.py371 def test_three_word_repetition_completion(self):CODE
LOW⚡tests/test_rewards.py378 def test_four_word_completion_with_repetition(self):CODE
LOW⚡tests/test_rewards.py386 def test_five_word_completion_with_repetition(self):CODE
LOW⚡tests/test_rewards.py394 def test_six_word_completion_with_repetition(self):CODE
LOW⚡tests/test_rewards.py401 def test_long_completion_with_repetition(self):CODE
LOW⚡tests/test_rewards.py407 def test_long_completion_without_repetition(self):CODE
LOW⚡tests/test_rewards.py414 def test_tag_count_rewards_all_correct(self):CODE
LOW⚡tests/test_rewards.py420 def test_tag_count_rewards_missing_think_begin(self):CODE
LOW⚡tests/test_rewards.py426 def test_tag_count_rewards_missing_think_end(self):CODE
LOW⚡tests/test_rewards.py432 def test_tag_count_rewards_missing_answer_begin(self):CODE
LOW⚡tests/test_rewards.py438 def test_tag_count_rewards_missing_answer_end(self):CODE
LOW⚡tests/test_rewards.py444 def test_tag_count_rewards_missing_all_tags(self):CODE
LOW⚡tests/test_rewards.py450 def test_full_repetition_with_language(self):CODE
LOW⚡tests/test_rewards.py461 def test_soft_overlong_punishment_short_completion(self):CODE
LOW⚡tests/test_rewards.py469 def test_soft_overlong_punishment_long_completion(self):CODE
LOW⚡tests/test_rewards.py477 def test_soft_overlong_punishment_intermediate_completion(self):CODE
LOW⚡tests/test_rewards.py486 def test_correct_python_format(self):CODE
LOWtests/test_rewards.py121 def test_reasoning_steps_reward(self):CODE
LOWtests/test_rewards.py139 def test_multiple_completions(self):CODE
LOWtests/test_rewards.py152 def test_cosine_scaled_reward(self):CODE
LOWtests/test_rewards.py200 def test_format_reward_specific_multiline(self):CODE
LOWtests/test_rewards.py207 def test_same_length_responses(self):CODE
LOWtests/test_rewards.py218 def test_different_lengths_correct_answers(self):CODE
LOWtests/test_rewards.py230 def test_different_lengths_incorrect_answers(self):CODE
LOWtests/test_rewards.py270 def test_unparseable_solution(self):CODE
LOWtests/test_rewards.py284 def test_positive_max_penalty_raises_value_error(self):CODE
LOWtests/test_rewards.py313 def test_multiple_completions(self):CODE
LOWtests/test_rewards.py332 def test_different_ngram_size(self):CODE
LOWtests/test_rewards.py520 def test_multiple_code_blocks(self):CODE
LOWtests/slow/test_code_reward.py127 def test_python_code_reward_morph(self):CODE
LOWtests/slow/test_code_reward.py156 def test_morph_router_parallel(self):CODE
LOWtests/slow/test_code_reward.py183 def test_morph_router_run_code_success(self):CODE
LOWtests/slow/test_code_reward.py200 def test_morph_router_run_code_with_error(self):CODE
LOWtests/utils/test_data.py30 def test_dataset_and_config_name(self):CODE
LOWtests/utils/test_data.py69 def test_mixture_and_test_split(self):CODE
LOWtests/utils/test_data.py85 def test_mixture_column_selection(self):CODE
LOWtests/utils/test_data.py106 def test_mixture_with_mismatched_columns(self):CODE
LOWtests/utils/test_data.py122 def test_no_dataset_name_or_mixture(self):CODE
LOWscripts/pass_rate_filtering/compute_pass_rate.py137 def repeat_each_element_k_times(list_to_repeat: list, k: int) -> list:CODE
LOWsrc/open_r1/generate.py23def build_distilabel_pipeline(CODE
LOWsrc/open_r1/rewards.py285def get_repetition_penalty_reward(ngram_size: int, max_penalty: float, language: str = "en"):STRING
LOWsrc/open_r1/rewards.py321 def repetition_penalty_reward(completions, **kwargs) -> float:STRING
LOWsrc/open_r1/rewards.py620def get_soft_overlong_punishment(max_completion_len, soft_punish_cache):STRING
LOWsrc/open_r1/rewards.py630 def soft_overlong_punishment_reward(completion_ids: list[list[int]], **kwargs) -> list[float]:STRING
LOWsrc/open_r1/utils/hub.py70def check_hub_revision_exists(training_args: SFTConfig | GRPOConfig):CODE
LOWsrc/open_r1/utils/hub.py89def get_param_count_from_repo_id(repo_id: str) -> int:CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py337 async def _get_or_create_base_snapshot(self):CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py715def get_morph_client_from_env(session=None) -> MorphCloudExecutionClient:STRING
LOW⚡src/open_r1/utils/competitive_programming/cf_scoring.py59async def get_generated_contest_tests(contest_id: str) -> list[dict]:CODE
LOW…open_r1/utils/competitive_programming/piston_client.py17def get_piston_client_from_env(session=None):CODE
LOW⚡…open_r1/utils/competitive_programming/piston_client.py201def get_slurm_piston_endpoints():CODE
Docstring Block Structure8 hits · 40 pts
SeverityFileLineSnippetContext
HIGHscripts/e2b_router.py67 Creates and configures a FastAPI application instance. Args: args: An object containing configuration pSTRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py50 Prepare and start a MorphCloud instance. Args: snapshot_id: Optional snapshot ID to use. If STRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py83 Process files, determine problem ID, and prepare configuration. Args: data: Dictionary conSTRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py140 Upload all necessary files to the instance. Args: instance: The MorphCloud instance STRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py167 Compile the code on the instance. Args: instance: The MorphCloud instance ReturnsSTRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py187 Run tests and evaluate results. Args: instance: The MorphCloud instance data: STRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py223Execute code using a prepared instance. Args: instance: Ready MorphCloud instance data:STRING
HIGH…/open_r1/utils/competitive_programming/morph_client.py251 Internal implementation of execute with no retry logic. Args: data: Dictionary containing STRING
Empty Error Handlers7 hits · 35 pts
SeverityFileLineSnippetContext
HIGHscripts/morph_router.py135except Exception:CODE
HIGHscripts/e2b_router.py130except Exception:CODE
HIGHsrc/open_r1/utils/code_providers.py321except (ValueError, AttributeError):CODE
HIGHsrc/open_r1/utils/code_providers.py335except Exception:CODE
HIGHsrc/open_r1/utils/code_providers.py319except ValueError:CODE
HIGHsrc/open_r1/utils/code_providers.py312except ValueError:CODE
HIGH…/open_r1/utils/competitive_programming/morph_client.py78except Exception:CODE
Cross-Language Confusion5 hits · 25 pts
SeverityFileLineSnippetContext
HIGH…/open_r1/utils/competitive_programming/morph_client.py201 run_command = f"cd /workspace && timeout {hard_timeout}s ./run"CODE
HIGH…/open_r1/utils/competitive_programming/morph_client.py382 await temp_instance.aexec("apt-get update && apt-get install -y build-essential cmake g++")CODE
HIGH…/open_r1/utils/competitive_programming/morph_client.py386 "mkdir -p /workspace && mkdir -p /workspace/graders && chmod 777 /workspace"CODE
HIGH…/open_r1/utils/competitive_programming/morph_client.py536 kill -9 "$pid" 2>/dev/null || trueCODE
HIGH…/open_r1/utils/competitive_programming/morph_client.py608 if diff -bq <(echo "$CORRECT_OUTPUT") "$SOLUTION_OUTPUT" >/dev/null; thenCODE
Over-Commented Block25 hits · 24 pts
SeverityFileLineSnippetContext
LOWsetup.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWslurm/piston/launch_single_piston.sh1#!/bin/bashCOMMENT
LOWtests/test_rewards.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWtests/slow/test_code_reward.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWtests/utils/test_data.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWscripts/benchmark_e2b.py1# coding=utf-8COMMENT
LOWscripts/decontaminate.py1#!/usr/bin/env pythonCOMMENT
LOWscripts/upload_details.py1# coding=utf-8COMMENT
LOWscripts/morph_router.py1# coding=utf-8COMMENT
LOWscripts/e2b_router.py1# coding=utf-8COMMENT
LOWscripts/run_benchmarks.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWscripts/pass_rate_filtering/compute_pass_rate.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/configs.py1# coding=utf-8COMMENT
LOWsrc/open_r1/generate.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/__init__.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/grpo.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/rewards.py1# coding=utf-8COMMENT
LOWsrc/open_r1/sft.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/utils/routed_sandbox.py1# coding=utf-8COMMENT
LOWsrc/open_r1/utils/hub.py1#!/usr/bin/env pythonCOMMENT
LOWsrc/open_r1/utils/callbacks.py1#!/usr/bin/env pythonCOMMENT
LOWsrc/open_r1/utils/routed_morph.py1# coding=utf-8COMMENT
LOWsrc/open_r1/utils/import_utils.py1# Copyright 2025 The HuggingFace Team. All rights reserved.COMMENT
LOWsrc/open_r1/utils/code_providers.py1# coding=utf-8COMMENT
LOW…/open_r1/utils/competitive_programming/code_patcher.py21 # Fix imp module deprecation (deprecated in 3.4)COMMENT
Unused Imports17 hits · 17 pts
SeverityFileLineSnippetContext
LOWsrc/open_r1/utils/evaluation.py8CODE
LOWsrc/open_r1/utils/evaluation.py8CODE
LOWsrc/open_r1/utils/evaluation.py8CODE
LOWsrc/open_r1/utils/__init__.py1CODE
LOWsrc/open_r1/utils/__init__.py2CODE
LOWsrc/open_r1/utils/__init__.py2CODE
LOWsrc/open_r1/utils/__init__.py3CODE
LOWsrc/open_r1/utils/__init__.py3CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py1CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py2CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py3CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py3CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py3CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py4CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py5CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py6CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py6CODE
Deep Nesting9 hits · 8 pts
SeverityFileLineSnippetContext
LOWscripts/generate_reasoning.py104CODE
LOWsrc/open_r1/utils/hub.py70CODE
LOWsrc/open_r1/utils/hub.py89CODE
LOWsrc/open_r1/utils/code_providers.py211CODE
LOWsrc/open_r1/utils/code_providers.py273CODE
LOWsrc/open_r1/utils/data.py12CODE
LOW…/open_r1/utils/competitive_programming/morph_client.py337CODE
LOW…c/open_r1/utils/competitive_programming/ioi_scoring.py110CODE
LOW…open_r1/utils/competitive_programming/piston_client.py137CODE
Modern Structural Boilerplate7 hits · 7 pts
SeverityFileLineSnippetContext
LOWscripts/pass_rate_filtering/compute_pass_rate.py35logger = logging.getLogger(__name__)CODE
LOWsrc/open_r1/grpo.py32logger = logging.getLogger(__name__)CODE
LOWsrc/open_r1/sft.py52logger = logging.getLogger(__name__)CODE
LOWsrc/open_r1/utils/__init__.py6__all__ = ["get_tokenizer", "is_e2b_available", "is_morph_available", "get_model", "get_dataset"]CODE
LOWsrc/open_r1/utils/hub.py36logger = logging.getLogger(__name__)CODE
LOWsrc/open_r1/utils/data.py9logger = logging.getLogger(__name__)CODE
LOWsrc/open_r1/utils/competitive_programming/__init__.py9__all__ = [CODE
Self-Referential Comments2 hits · 6 pts
SeverityFileLineSnippetContext
MEDIUMsrc/open_r1/utils/hub.py42 # Create a repo if it doesn't exist yetCOMMENT
MEDIUM…/open_r1/utils/competitive_programming/morph_client.py524# Create a temporary file for solution outputCOMMENT
Redundant / Tautological Comments3 hits · 4 pts
SeverityFileLineSnippetContext
LOW…/open_r1/utils/competitive_programming/morph_client.py265 # Set timeouts to ensure we don't block indefinitelyCOMMENT
LOW…/open_r1/utils/competitive_programming/morph_client.py493# Check if grader_config.json existsCOMMENT
LOW…/open_r1/utils/competitive_programming/morph_client.py601 # Check if there's a custom checkerCOMMENT
AI Slop Vocabulary2 hits · 4 pts
SeverityFileLineSnippetContext
MEDIUMrecipes/OpenR1-Distill-7B/sft/config_distill.yaml9chat_template: "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0]['role'] == 'system' %}\n CODE
LOWscripts/pass_rate_filtering/compute_pass_rate.py39 # we can be lazy and just use the same script args as GRPOCOMMENT
TODO Padding1 hit · 2 pts
SeverityFileLineSnippetContext
LOWsrc/open_r1/rewards.py552 # TODO: implement a proper validator to compare against ground truth. For now we just check for exact stringCOMMENT
AI Structural Patterns1 hit · 1 pts
SeverityFileLineSnippetContext
LOWsrc/open_r1/generate.py23CODE