Democratizing Reinforcement Learning for LLMs
This report summarizes scanner evidence for rllm-org/rllm, a Python project with 5,870 GitHub stars. SynthScan examined 109,442 lines of code across 652 source files, recording 3218 pattern matches distributed across 24 syntactic categories. The legacy adjusted pattern signal is 54.2 (High pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 1449 finding(s).
Are imports declared and package/API names verified? 443 finding(s).
Are wrappers, guards, and repeated structures necessary? 2403 finding(s).
Which changes need focused human review? 3218 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 3218 distinct pattern matches across 24 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_store.py | 11 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_store.py | 13 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 71 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 73 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 116 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 118 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 168 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_store.py | 170 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 11 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 13 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 73 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 75 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 95 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 97 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 138 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 140 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 160 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 162 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 202 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 204 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 228 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 230 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_remote_engine.py | 23 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_remote_engine.py | 25 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_remote_engine.py | 66 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_remote_engine.py | 68 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_sampling_e2e.py | 24 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_sampling_e2e.py | 26 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_sampling_e2e.py | 98 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/integration/test_sampling_e2e.py | 101 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/integration/test_sampling_e2e.py | 130 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/integration/test_sampling_e2e.py | 132 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/integration/test_sampling_e2e.py | 158 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/integration/test_sampling_e2e.py | 160 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 145 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 147 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 199 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 201 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 267 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 269 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 323 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 325 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 361 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_tinker_transform.py | 363 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 21 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 23 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 78 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 80 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 464 | # ============================================================================= | COMMENT |
| MEDIUM | tests/unified_trainer/test_tinker_transform.py | 466 | # ============================================================================= | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 105 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 107 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 196 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 198 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 262 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 264 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 302 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/unified_trainer/test_config_sync.py | 304 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/harnesses/test_cli_harness.py | 79 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/harnesses/test_cli_harness.py | 82 | # --------------------------------------------------------------------------- | COMMENT |
| 602 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_logging.py | 28 | def test_get_log_level_from_env_uses_default(monkeypatch): | CODE |
| LOW⚡ | tests/test_logging.py | 35 | def test_get_log_level_from_env_reads_level(monkeypatch): | CODE |
| LOW⚡ | tests/test_logging.py | 41 | def test_get_log_level_from_env_falls_back_for_invalid_level(monkeypatch): | CODE |
| LOW⚡ | tests/test_logging.py | 47 | def test_configure_logging_from_env_sets_rllm_logger(monkeypatch): | CODE |
| LOW⚡ | tests/test_logging.py | 55 | def test_configure_logging_from_env_initializes_basic_logging(monkeypatch): | CODE |
| LOW | tests/test_logging.py | 68 | def test_import_configures_rllm_logger_from_env(monkeypatch): | CODE |
| LOW | tests/test_verl_imports.py | 47 | def test_verl_module_paths_exist(module_path: str, name: str) -> None: | CODE |
| LOW | tests/test_verl_metrics.py | 24 | def test_calculate_debug_metrics_compat_passes_through_upstream_metrics(monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | tests/test_verl_metrics.py | 49 | def test_calculate_debug_metrics_compat_keeps_newer_upstream_empty_mask_fallback( | CODE |
| LOW | tests/test_verl_metrics.py | 72 | def test_calculate_debug_metrics_compat_single_token_std_is_zero(monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | tests/test_verl_metrics.py | 90 | def test_calculate_debug_metrics_compat_backfills_legacy_empty_mask_error( | CODE |
| LOW | tests/test_verl_metrics.py | 113 | def test_calculate_debug_metrics_compat_reraises_unrelated_runtime_errors( | CODE |
| LOW | tests/test_store.py | 27 | def test_get_missing_key_returns_default(self, store): | CODE |
| LOW⚡ | tests/test_store.py | 64 | def test_stores_arbitrary_types(self, store): | CODE |
| LOW⚡ | tests/test_store.py | 77 | def test_concurrent_writes_no_corruption(self, store): | CODE |
| LOW | tests/test_store.py | 94 | def test_concurrent_reads_during_writes(self, store): | CODE |
| LOW⚡ | tests/test_store.py | 122 | def test_two_instances_share_state(self, store): | CODE |
| LOW | tests/test_store.py | 138 | def test_interleaved_read_write(self, store): | CODE |
| LOW⚡ | tests/test_store.py | 174 | def test_workflow_receives_store(self): | CODE |
| LOW⚡ | tests/test_store.py | 192 | def test_workflow_without_store(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 17 | def test_bare_values_unchanged(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 79 | def test_unknown_loss_triggers_warning(self, caplog): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 88 | def test_known_losses_complete(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 108 | def test_override_applied_and_restored(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 130 | def test_no_override_passes_through(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 151 | def test_default_loss_is_known(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 166 | def test_roles_with_same_loss_grouped(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 208 | def test_accumulated_data_tracks_group_roles(self): | CODE |
| LOW⚡ | tests/test_verl_policy_loss.py | 234 | def test_no_loss_fn_map_means_no_splitting(self): | CODE |
| LOW | tests/test_verl_policy_loss.py | 53 | def test_invalid_tuple_length(self): | CODE |
| LOW | tests/test_verl_policy_loss.py | 62 | def test_backward_compat_no_tuples(self): | CODE |
| LOW | tests/test_verl_policy_loss.py | 179 | def test_all_same_loss_single_group(self): | CODE |
| LOW | tests/test_verl_policy_loss.py | 190 | def test_each_role_different_loss(self): | CODE |
| LOW | tests/test_verl_policy_loss.py | 245 | def test_advantage_computation_unaffected(self): | CODE |
| LOW | tests/tasks/test_loader_memory.py | 8 | def test_vlm_instruction_encodes_images_lazily(tmp_path, monkeypatch): | CODE |
| LOW | tests/integration/test_remote_engine.py | 105 | async def test_batch_tasks_session_correlation(self, engine): | CODE |
| LOW | tests/integration/test_remote_engine.py | 130 | async def test_timeout_produces_error_episode(self, engine): | CODE |
| LOW⚡ | tests/integration/test_sampling_e2e.py | 135 | def test_eval_run_dataset_enforces_sampling(mock_upstream): | CODE |
| LOW⚡ | tests/integration/test_sampling_e2e.py | 163 | def test_train_rollout_enforces_train_sampling(mock_upstream): | CODE |
| LOW | tests/integration/test_minimax_integration.py | 27 | def test_minimax_litellm_config_generation(self): | CODE |
| LOW | tests/integration/test_minimax_integration.py | 41 | def test_minimax_config_save_and_load(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/integration/test_minimax_integration.py | 60 | def test_minimax_litellm_completion(self): | CODE |
| LOW | tests/integration/test_tinker_gateway_integration.py | 124 | def test_vllm_fields_stripped(self, tinker_gateway): | CODE |
| LOW | tests/integration/test_tinker_gateway_integration.py | 147 | def test_trace_has_token_data(self, tinker_gateway): | CODE |
| LOW | tests/integration/test_tinker_gateway_integration.py | 177 | def test_trace_round_trips_to_step(self, tinker_gateway): | CODE |
| LOW | tests/integration/test_tinker_gateway_integration.py | 211 | def test_streaming_delivers_content_and_trace(self, tinker_gateway): | CODE |
| LOW | tests/integration/test_tinker_gateway_integration.py | 265 | def test_tool_calling_multi_turn(self, tinker_gateway): | CODE |
| LOW⚡ | tests/unified_trainer/test_async_batch_size_override.py | 17 | def test_override_when_async_and_batch_size_not_one(): | CODE |
| LOW⚡ | tests/unified_trainer/test_async_batch_size_override.py | 26 | def test_no_override_when_already_one(): | CODE |
| LOW⚡ | tests/unified_trainer/test_async_batch_size_override.py | 32 | def test_no_override_when_async_disabled(): | CODE |
| LOW⚡ | tests/unified_trainer/test_async_batch_size_override.py | 40 | def test_no_override_when_key_absent(): | CODE |
| LOW | tests/unified_trainer/test_warm_queue_wiring.py | 15 | def test_detached_warm_queue_detaches_and_restores(): | CODE |
| LOW | tests/unified_trainer/test_warm_queue_wiring.py | 53 | def test_start_warm_queue_guard_returns_none(warm_queue_size, sandbox_backend, hooks_present): | CODE |
| LOW | tests/unified_trainer/test_trajectory_grouping.py | 31 | def test_default_episode_keeps_bare_uuid_behavior(): | CODE |
| LOW | tests/unified_trainer/test_trajectory_grouping.py | 51 | def test_trajectory_group_id_parsing(group_id, task_id, group_role): | CODE |
| LOW | tests/unified_trainer/test_trajectory_grouping.py | 59 | def test_transform_groups_by_full_task_id_and_trajectory_name(names): | CODE |
| LOW | tests/unified_trainer/test_trajectory_grouping.py | 91 | def test_default_grouping_rejects_colon_in_trajectory_name(name): | CODE |
| LOW | tests/unified_trainer/test_trajectory_grouping.py | 104 | def test_grpo_advantages_are_centered_per_hierarchical_task(use_role_override): | CODE |
| LOW⚡ | tests/unified_trainer/test_algorithm_config.py | 48 | def test_norm_adv_by_std_in_grpo_true_from_algorithm(): | CODE |
| LOW⚡ | tests/unified_trainer/test_algorithm_config.py | 55 | def test_norm_adv_by_std_in_grpo_false_from_algorithm(): | CODE |
| 988 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/conftest.py | 22 | except Exception: | CODE |
| LOW | tests/conftest.py | 31 | except Exception: | CODE |
| LOW | tests/conftest.py | 51 | except Exception: | CODE |
| LOW | tests/conftest.py | 121 | except Exception: | CODE |
| MEDIUM | tests/tools/test_tools.py | 143 | print(f"Error in tool execution: {result.error}") | CODE |
| MEDIUM | tests/tools/test_tools.py | 175 | print(f"Error: {result.error}") | CODE |
| MEDIUM | tests/tools/test_tools.py | 204 | print(f"Error: {result.error}") | CODE |
| MEDIUM | tests/tools/test_tools.py | 245 | print(f"Error: {result.error}") | CODE |
| LOW | tests/parser/test_harmony_chat_parser.py | 17 | except Exception as exc: | CODE |
| LOW | tests/parser/test_chat_parser.py | 111 | except Exception as e: | CODE |
| LOW | rllm/hooks.py | 237 | except Exception: | CODE |
| LOW | rllm/hooks.py | 267 | except Exception as e: | CODE |
| LOW | rllm/hooks.py | 277 | except Exception: | CODE |
| LOW | rllm/hooks.py | 288 | except Exception: | CODE |
| LOW⚡ | …ental/rllm_telemetry/examples/experiment_comparison.py | 107 | except Exception as e: | CODE |
| LOW | …erimental/rllm_telemetry/src/rllm_telemetry/scorers.py | 250 | except Exception as exc: | CODE |
| LOW⚡ | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 50 | except Exception: | CODE |
| LOW⚡ | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 55 | except Exception: | CODE |
| LOW⚡ | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 59 | except Exception: | CODE |
| MEDIUM | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 425 | def _flush_loop(self) -> None: | CODE |
| LOW⚡ | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 230 | except Exception: | CODE |
| LOW⚡ | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 378 | except Exception as exc: | CODE |
| LOW⚡ | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 385 | except Exception as exc: | CODE |
| LOW⚡ | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 394 | except Exception: | CODE |
| LOW⚡ | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 646 | except Exception as exc: | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 407 | except Exception: | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 457 | except Exception as exc: | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 474 | except Exception: | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 563 | except Exception as exc: | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 584 | except Exception as exc: | CODE |
| MEDIUM | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 63 | def wrapper(self, **kwargs): | CODE |
| LOW⚡ | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 66 | except Exception: | CODE |
| LOW⚡ | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 159 | except Exception: | CODE |
| LOW⚡ | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 163 | except Exception: | CODE |
| LOW | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 91 | except Exception: | CODE |
| LOW | …experimental/rllm_telemetry/src/rllm_telemetry/eval.py | 285 | except Exception as exc: | CODE |
| LOW | …experimental/rllm_telemetry/src/rllm_telemetry/eval.py | 300 | except Exception as exc: | CODE |
| LOW | rllm/experimental/fully_async/runner.py | 246 | except Exception as e: | CODE |
| LOW | rllm/experimental/fully_async/runner.py | 254 | except Exception as e: | CODE |
| LOW | rllm/experimental/fully_async/rollout_executor.py | 283 | except Exception as e: | STRING |
| LOW | rllm/experimental/fully_async/rollout_executor.py | 303 | except Exception as e: | STRING |
| LOW | rllm/experimental/fully_async/rollout_executor.py | 310 | except Exception: | STRING |
| LOW | rllm/experimental/fully_async/rollout_executor.py | 388 | except Exception: | CODE |
| LOW | rllm/experimental/fully_async/utils.py | 209 | except Exception: | CODE |
| LOW | rllm/tasks/loader.py | 210 | except Exception: | CODE |
| LOW | rllm/tasks/loader.py | 385 | except Exception: | CODE |
| LOW | rllm/tools/utils.py | 208 | except Exception as e: | CODE |
| MEDIUM | rllm/tools/utils.py | 213 | print(f"Error in test case {{i+1}}: {{str(e)}}") | CODE |
| LOW | rllm/tools/utils.py | 284 | except Exception as e: | CODE |
| LOW | rllm/tools/mcp_tool.py | 37 | except Exception as e: | CODE |
| MEDIUM | rllm/tools/mcp_tool.py | 20 | def async_forward(self, **kwargs) -> ToolOutput: | CODE |
| LOW | rllm/tools/tool_base.py | 122 | except Exception as e: | CODE |
| LOW | rllm/tools/code_tools/e2b_tool.py | 39 | except Exception as e: | CODE |
| MEDIUM | rllm/tools/code_tools/e2b_tool.py | 40 | print(f"Error killing sandbox: {e}") | CODE |
| LOW | rllm/tools/code_tools/e2b_tool.py | 77 | except Exception: | CODE |
| LOW | rllm/tools/code_tools/together_tool.py | 90 | except Exception as e: | CODE |
| LOW | rllm/tools/code_tools/lcb_tool.py | 52 | except Exception as e: | CODE |
| LOW | rllm/tools/code_tools/lcb_tool.py | 99 | except Exception as e: | CODE |
| LOW | rllm/tools/code_tools/lcb_tool.py | 110 | except Exception: | CODE |
| LOW | rllm/tools/code_tools/lcb_tool.py | 203 | except Exception as e: | CODE |
| 354 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …erimental/rllm_telemetry/src/rllm_telemetry/scorers.py | 147 | except (json.JSONDecodeError, KeyError, TypeError): | CODE |
| HIGH | …erimental/rllm_telemetry/src/rllm_telemetry/scorers.py | 156 | except (json.JSONDecodeError, KeyError, TypeError): | CODE |
| HIGH | …erimental/rllm_telemetry/src/rllm_telemetry/scorers.py | 164 | except ValueError: | CODE |
| HIGH | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 50 | except Exception: | CODE |
| HIGH | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 55 | except Exception: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 184 | except asyncio.CancelledError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 192 | except RuntimeError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 430 | except asyncio.CancelledError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 437 | except RuntimeError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 623 | except RuntimeError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 166 | except asyncio.CancelledError: | CODE |
| HIGH | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 362 | except asyncio.CancelledError: | CODE |
| HIGH | …perimental/rllm_telemetry/src/rllm_telemetry/plugin.py | 159 | except Exception: | CODE |
| HIGH | rllm/experimental/fully_async/rollout_executor.py | 398 | except asyncio.CancelledError: | CODE |
| HIGH | rllm/experimental/fully_async/rollout_executor.py | 403 | except asyncio.CancelledError: | CODE |
| HIGH | rllm/tasks/loader.py | 385 | except Exception: | CODE |
| HIGH | rllm/tools/web_tools/gsearch_tool.py | 110 | except Exception: | CODE |
| HIGH | rllm/tools/web_tools/firecrawl_tool.py | 10 | except ImportError: | CODE |
| HIGH | rllm/rewards/math_utils/utils.py | 431 | except Exception: | CODE |
| HIGH | rllm/rewards/math_utils/utils.py | 320 | except Exception: | CODE |
| HIGH | rllm/rewards/math_utils/utils.py | 408 | except Exception: | CODE |
| HIGH | rllm/rewards/math_utils/utils.py | 412 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/swebench.py | 135 | except: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 72 | except ValueError: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 542 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 549 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 611 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 284 | except SystemExit: | CODE |
| HIGH | rllm/rewards/code_utils/taco.py | 642 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/codeforces.py | 489 | except Exception as e: | CODE |
| HIGH | rllm/rewards/code_utils/codeforces.py | 496 | except Exception as e: | CODE |
| HIGH | rllm/rewards/code_utils/codeforces.py | 263 | except SystemExit as e: | CODE |
| HIGH | rllm/rewards/code_utils/livecodebench.py | 84 | except Exception: | CODE |
| HIGH | rllm/rewards/code_utils/livecodebench.py | 138 | except SystemExit: | CODE |
| HIGH | rllm/utils/tracking.py | 313 | except Exception: | CODE |
| HIGH | rllm/utils/tracking.py | 116 | except Exception: | CODE |
| HIGH | rllm/utils/tracking.py | 388 | except Exception: | CODE |
| HIGH | rllm/utils/tracking.py | 417 | except Exception: | CODE |
| HIGH | rllm/cli/_ui.py | 199 | except ValueError: | CODE |
| HIGH | rllm/cli/model_cmd.py | 57 | except OSError: | CODE |
| HIGH | rllm/cli/_sampling.py | 100 | except ValueError: | CODE |
| HIGH | rllm/cli/train.py | 350 | except (KeyError, ImportError): | CODE |
| HIGH | rllm/cli/login.py | 38 | except Exception: | CODE |
| HIGH | rllm/cli/eval.py | 293 | except (KeyError, ImportError): | CODE |
| HIGH | rllm/parser/chat_template_parser.py | 491 | except json.JSONDecodeError: | CODE |
| HIGH | rllm/parser/chat_template_parser.py | 300 | except json.JSONDecodeError: | CODE |
| HIGH | rllm/sandbox/backends/daytona.py | 340 | except DaytonaNotFoundError: | CODE |
| HIGH | rllm/sandbox/backends/daytona.py | 262 | except OSError: | CODE |
| HIGH | rllm/sandbox/backends/modal_backend.py | 254 | except Exception: | CODE |
| HIGH | rllm/sandbox/backends/docker.py | 122 | except Exception: | CODE |
| HIGH | rllm/sandbox/backends/docker.py | 118 | except Exception: | CODE |
| HIGH | rllm/eval/proxy.py | 189 | except Exception: | CODE |
| HIGH | rllm/eval/proxy.py | 195 | except (ValueError, OSError): | CODE |
| HIGH | rllm/eval/proxy.py | 243 | except requests.RequestException: | CODE |
| HIGH | rllm/eval/proxy.py | 271 | except OSError: | CODE |
| HIGH | rllm/eval/script_evaluator.py | 70 | except Exception: | CODE |
| HIGH | rllm/eval/reward_fns/ifeval.py | 147 | except (json.JSONDecodeError, ValueError): | CODE |
| HIGH | rllm/eval/reward_fns/ifeval.py | 154 | except (json.JSONDecodeError, ValueError): | CODE |
| HIGH | rllm/eval/reward_fns/bfcl.py | 56 | except (json.JSONDecodeError, ValueError): | CODE |
| HIGH | rllm/eval/reward_fns/llm_judge.py | 105 | except (json.JSONDecodeError, ValueError, KeyError): | CODE |
| 33 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_verl_metrics.py | 3 | CODE | |
| LOW | tests/test_store.py | 3 | CODE | |
| LOW | tests/test_verl_policy_loss.py | 3 | CODE | |
| LOW | tests/tasks/test_loader_memory.py | 1 | CODE | |
| LOW | tests/integration/test_sampling_e2e.py | 10 | CODE | |
| LOW | tests/harnesses/test_cli_harness.py | 13 | CODE | |
| LOW | tests/cli/test_sampling_config.py | 3 | CODE | |
| LOW | tests/cli/test_sampling_resolution.py | 3 | CODE | |
| LOW | tests/sandbox/test_warm_queue.py | 9 | CODE | |
| LOW | tests/sandbox/test_sandboxed_flow.py | 3 | CODE | |
| LOW | tests/sandbox/test_snapshot.py | 10 | CODE | |
| LOW | tests/sandbox/test_daytona_backend.py | 8 | CODE | |
| LOW | tests/eval/test_unified_runner.py | 17 | CODE | |
| LOW | tests/eval/test_pull.py | 3 | CODE | |
| LOW | tests/eval/test_evaluator_loader.py | 3 | CODE | |
| LOW | tests/eval/test_agent_loader.py | 3 | CODE | |
| LOW | tests/eval/test_env_join.py | 12 | CODE | |
| LOW | tests/eval/test_script_module_evaluators.py | 11 | CODE | |
| LOW | tests/eval/test_agentic.py | 9 | CODE | |
| LOW | tests/eval/test_vlm_agents.py | 3 | CODE | |
| LOW | tests/eval/test_eval_engine.py | 16 | CODE | |
| LOW | tests/eval/test_ifeval.py | 9 | CODE | |
| LOW | tests/eval/test_rollout_decorator.py | 3 | CODE | |
| LOW | tests/eval/test_vlm_transforms.py | 3 | CODE | |
| LOW | tests/eval/test_transforms.py | 3 | CODE | |
| LOW | tests/eval/test_materialize.py | 13 | CODE | |
| LOW | tests/eval/test_resolver.py | 8 | CODE | |
| LOW | tests/eval/test_eval_types.py | 11 | CODE | |
| LOW | tests/eval/test_mcq.py | 9 | CODE | |
| LOW | tests/eval/test_dockerfile_replay.py | 9 | CODE | |
| LOW | tests/data/test_swesmith_builder.py | 3 | CODE | |
| LOW | tests/data/test_skillsbench_builder.py | 8 | CODE | |
| LOW | rllm/hooks.py | 24 | CODE | |
| LOW | rllm/env.py | 19 | CODE | |
| LOW | rllm/paths.py | 18 | CODE | |
| LOW | rllm/types.py | 18 | CODE | |
| LOW | …erimental/rllm_telemetry/src/rllm_telemetry/scorers.py | 3 | CODE | |
| LOW | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 19 | CODE | |
| LOW | …perimental/rllm_telemetry/src/rllm_telemetry/config.py | 3 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 13 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 3 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 3 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 5 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 6 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 6 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 6 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 7 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 9 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 9 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 9 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 9 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 9 | CODE | |
| 317 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/harnesses/test_cli_harness.py | 98 | invocations like ``cd /work && claude ...`` are compound commands and | STRING |
| HIGH | tests/eval/test_dockerfile_replay.py | 28 | "FROM python:3.11-slim\nRUN apt-get update && apt-get install -y \\\n python3-pip \\\n && rm -rf /var/lib/ | CODE |
| HIGH | rllm/harnesses/qwen_code.py | 31 | -o Acquire::Check-Valid-Until=false 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/qwen_code.py | 92 | f". $HOME/.nvm/nvm.sh 2>/dev/null; " | CODE |
| HIGH | rllm/harnesses/zeroclaw.py | 46 | apt-get update -qq 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/zeroclaw.py | 47 | apt-get install -y -qq --no-install-recommends curl ca-certificates tar coreutils 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/zeroclaw.py | 146 | f"mkdir -p $HOME/.zeroclaw && cat > {_CONFIG_PATH} << 'ZC_CONFIG_EOF'\n{content}\nZC_CONFIG_EOF", | CODE |
| HIGH | rllm/harnesses/codex.py | 44 | -o Acquire::Check-Valid-Until=false 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/codex.py | 155 | f". $HOME/.nvm/nvm.sh 2>/dev/null; " | CODE |
| HIGH | rllm/harnesses/kimi_cli.py | 42 | -o Acquire::Check-Valid-Until=false 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/kimi_cli.py | 158 | f"done; kill 0 2>/dev/null)" | CODE |
| HIGH | rllm/harnesses/mini_swe_agent.py | 55 | apt-get update -qq && apt-get install -y -qq curl ca-certificates git | CODE |
| HIGH | rllm/harnesses/mini_swe_agent.py | 147 | f"mkdir -p $HOME/.config/mini-swe-agent && cat > {path} << 'MSWEA_DOTENV_EOF'\n{content}\nMSWEA_DOTENV_EOF", | CODE |
| HIGH | rllm/harnesses/aider.py | 44 | -o Acquire::Check-Valid-Until=false 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/opencode.py | 47 | apt-get update -qq && apt-get install -y -qq curl ca-certificates | CODE |
| HIGH | rllm/harnesses/opencode.py | 148 | cmd = f"mkdir -p $HOME/.config/opencode && cat > $HOME/.config/opencode/opencode.json << '{marker}'\n{content}\n | CODE |
| HIGH | rllm/harnesses/opencode.py | 168 | f". $HOME/.nvm/nvm.sh 2>/dev/null; " | CODE |
| HIGH | rllm/harnesses/cli_harness.py | 83 | invocations like ``cd /workspace && claude ...`` are compound | STRING |
| HIGH | rllm/harnesses/terminus2.py | 63 | apt-get update -qq && apt-get install -y -qq tmux util-linux procps curl ca-certificates git | CODE |
| HIGH | rllm/harnesses/claude_code.py | 65 | apt-get "${_APT_OPTS[@]}" update -qq 2>/dev/null || true | CODE |
| HIGH | rllm/harnesses/tools/submit_tool.py | 31 | diff = _safe_exec(sandbox, "git diff 2>/dev/null || echo 'No git repo'") | CODE |
| HIGH⚡ | rllm/eval/script_evaluator.py | 119 | check = sandbox.exec(f"test -f {path} && echo yes || echo no", timeout=10, user=user).strip() | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 370 | if (v == null) return ""; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 372 | try { return JSON.stringify(v, null, 2); } catch (e) { return String(v); } | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 375 | if (s == null) return "text-gray-400"; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 472 | if (av == null && bv == null) return 0; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 472 | if (av == null && bv == null) return 0; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 473 | if (av == null) return 1; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 474 | if (bv == null) return -1; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 722 | if (!task || typeof task !== "object") return fieldBox("task", task); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 724 | const metadata = task.metadata && Object.keys(task.metadata).length ? task.metadata : null; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 724 | const metadata = task.metadata && Object.keys(task.metadata).length ? task.metadata : null; | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 737 | for (const [ti, traj] of (ep.trajectories || []).entries()) parts.push(renderTrajectory(traj, ti)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 738 | if (ep.artifacts && Object.keys(ep.artifacts).length) parts.push(fieldBox("artifacts", ep.artifacts)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 738 | if (ep.artifacts && Object.keys(ep.artifacts).length) parts.push(fieldBox("artifacts", ep.artifacts)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 739 | if (ep.metrics && Object.keys(ep.metrics).length) parts.push(fieldBox("metrics", ep.metrics)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 739 | if (ep.metrics && Object.keys(ep.metrics).length) parts.push(fieldBox("metrics", ep.metrics)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 740 | if (ep.metadata && Object.keys(ep.metadata).length) parts.push(fieldBox("metadata", ep.metadata)); | CODE |
| HIGH⚡ | rllm/eval/visualizer.py | 740 | if (ep.metadata && Object.keys(ep.metadata).length) parts.push(fieldBox("metadata", ep.metadata)); | CODE |
| HIGH | rllm/eval/visualizer.py | 524 | let runState = { id:null, meta:null, index:[], filter:"all", search:"" }; | CODE |
| HIGH | rllm/eval/visualizer.py | 556 | if (summary.total != null) { | CODE |
| HIGH | rllm/eval/visualizer.py | 620 | if (r == null) return ""; | CODE |
| HIGH | rllm/eval/visualizer.py | 645 | if (v == null) return ""; | CODE |
| HIGH | rllm/eval/visualizer.py | 677 | const meta = step.metadata && Object.keys(step.metadata).length ? step.metadata : null; | CODE |
| HIGH | rllm/eval/visualizer.py | 677 | const meta = step.metadata && Object.keys(step.metadata).length ? step.metadata : null; | CODE |
| HIGH | rllm/data/r2egym_builder.py | 291 | bash "$RUN_TESTS" > /tmp/test_output.txt 2>&1 || log "run_tests.sh exited non-zero (parser inspects log)" | CODE |
| HIGH | rllm/data/r2egym_builder.py | 296 | PYTHONWARNINGS='ignore::UserWarning,ignore::SyntaxWarning' "$PY" -W ignore -m pytest -rA r2e_tests > /tmp/test_outpu | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 313 | git add -A . >/dev/null 2>&1 || true | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 314 | git diff --cached --binary "$BASE_COMMIT" > "$MODEL_PATCH" 2>/dev/null || true | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 315 | git reset >/dev/null 2>&1 || true | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 317 | PATCH_BYTES=$(wc -c < "$MODEL_PATCH" 2>/dev/null || echo 0) | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 334 | git reset --hard "$BASE_COMMIT" >/dev/null 2>&1 || log "git reset --hard failed (continuing)" | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 335 | git checkout "$BASE_COMMIT" >/dev/null 2>&1 || log "git checkout failed (continuing)" | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 336 | git clean -fd >/dev/null 2>&1 || log "git clean -fd failed (continuing)" | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 370 | bash /tests/run_script.sh "$SELECTED" > /tmp/stdout.log 2> /tmp/stderr.log || log "run_script.sh exited non-zero (parser | CODE |
| HIGH⚡ | rllm/data/swebench_pro_builder.py | 373 | python3 /tests/parser.py /tmp/stdout.log /tmp/stderr.log /tmp/output.json 2>>/tmp/stderr.log || log "parser.py failed" | CODE |
| HIGH | rllm/data/swebench_pro_builder.py | 289 | git config --global --add safe.directory "$(pwd)" 2>/dev/null || true | CODE |
| HIGH | rllm/data/swebench_pro_builder.py | 169 | - ``requirements``: explicit deliverables (sometimes null) | STRING |
| HIGH | rllm/data/swebench_pro_builder.py | 456 | "git config --global --add safe.directory /app 2>/dev/null || true\n" | STRING |
| HIGH | rllm/trainer/fireworks/fireworks_backend.py | 269 | raise ValueError(f"rllm.algorithm.loss_agg_mode must be null, 'token-mean', 'seq-mean-token-sum', or 'seq-me | CODE |
| 1 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/hooks.py | 168 | CODE | |
| LOW | rllm/hooks.py | 245 | CODE | |
| LOW | …lm/experimental/rllm_telemetry/examples/hello_world.py | 69 | CODE | |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/exporter.py | 440 | CODE | |
| LOW | …erimental/rllm_telemetry/src/rllm_telemetry/dataset.py | 48 | CODE | |
| LOW | …experimental/rllm_telemetry/src/rllm_telemetry/eval.py | 159 | CODE | |
| LOW | …erimental/rllm_telemetry/src/rllm_telemetry/compare.py | 88 | CODE | |
| LOW | …erimental/rllm_telemetry/src/rllm_telemetry/compare.py | 165 | CODE | |
| LOW | rllm/experimental/fully_async/runner.py | 228 | CODE | |
| LOW | rllm/experimental/fully_async/metric_utils.py | 122 | CODE | |
| LOW | rllm/experimental/fully_async/rollout_executor.py | 326 | CODE | |
| LOW | rllm/experimental/fully_async/utils.py | 251 | CODE | |
| LOW | rllm/experimental/fully_async/utils.py | 350 | CODE | |
| LOW | rllm/experimental/fully_async/utils.py | 480 | CODE | |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 166 | CODE | |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 524 | CODE | |
| LOW | rllm/tasks/loader.py | 169 | CODE | |
| LOW | rllm/tools/utils.py | 43 | CODE | |
| LOW | rllm/tools/mcp_tool.py | 20 | CODE | |
| LOW | rllm/tools/code_tools/together_tool.py | 40 | CODE | |
| LOW | rllm/tools/code_tools/lcb_tool.py | 58 | CODE | |
| LOW | rllm/tools/code_tools/lcb_tool.py | 122 | CODE | |
| LOW | rllm/harnesses/tool_calling.py | 33 | CODE | |
| LOW | rllm/harnesses/tools/file_editor_tool.py | 61 | CODE | |
| LOW | rllm/rewards/code_reward.py | 425 | CODE | |
| LOW | rllm/rewards/countdown_reward.py | 156 | CODE | |
| LOW | rllm/rewards/math_utils/utils.py | 42 | CODE | |
| LOW | rllm/rewards/math_utils/utils.py | 496 | CODE | |
| LOW | rllm/rewards/math_utils/utils.py | 43 | CODE | |
| LOW | rllm/rewards/code_utils/taco.py | 79 | CODE | |
| LOW | rllm/rewards/code_utils/taco.py | 214 | CODE | |
| LOW | rllm/rewards/code_utils/taco.py | 292 | CODE | |
| LOW | rllm/rewards/code_utils/taco.py | 598 | CODE | |
| LOW | rllm/rewards/code_utils/codeforces.py | 61 | CODE | |
| LOW | rllm/rewards/code_utils/codeforces.py | 193 | CODE | |
| LOW | rllm/rewards/code_utils/codeforces.py | 271 | CODE | |
| LOW | rllm/rewards/code_utils/livecodebench.py | 268 | CODE | |
| LOW | rllm/rewards/code_utils/livecodebench.py | 382 | CODE | |
| LOW | rllm/rewards/code_utils/humanevalplus.py | 16 | CODE | |
| LOW | rllm/workflows/distillation_workflow.py | 43 | CODE | |
| LOW | rllm/workflows/simple_workflow.py | 32 | CODE | |
| LOW | rllm/workflows/timing_mixin.py | 134 | CODE | |
| LOW | rllm/utils/visualization.py | 69 | CODE | |
| LOW | rllm/utils/tracking.py | 442 | CODE | |
| LOW | rllm/utils/tracking.py | 487 | CODE | |
| LOW | rllm/utils/tracking.py | 608 | CODE | |
| LOW | rllm/cli/_ui.py | 74 | CODE | |
| LOW | rllm/cli/model_cmd.py | 35 | CODE | |
| LOW | rllm/cli/dataset.py | 47 | CODE | |
| LOW | rllm/cli/dataset.py | 224 | CODE | |
| LOW | rllm/cli/train.py | 158 | CODE | |
| LOW | rllm/cli/_pull.py | 153 | CODE | |
| LOW | rllm/cli/_pull.py | 316 | CODE | |
| LOW | rllm/cli/eval.py | 80 | CODE | |
| LOW | rllm/cli/eval.py | 578 | CODE | |
| LOW | rllm/parser/chat_template_parser.py | 87 | CODE | |
| LOW | rllm/parser/chat_template_parser.py | 206 | CODE | |
| LOW | rllm/parser/chat_template_parser.py | 263 | CODE | |
| LOW | rllm/parser/chat_template_parser.py | 319 | CODE | |
| LOW | rllm/parser/chat_template_parser.py | 396 | CODE | |
| 109 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | rllm/hooks.py | 42 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/hooks.py | 345 | __all__ = [ | CODE |
| LOW | rllm/__init__.py | 10 | __all__ = ["BaseAgent", "Action", "Step", "Trajectory", "Episode", "rollout", "evaluator", "Task"] | CODE |
| LOW | …rimental/rllm_telemetry/src/rllm_telemetry/__init__.py | 34 | __all__ = [ | CODE |
| LOW | rllm/experimental/fully_async/param_sync.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/experimental/fully_async/message_queue.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/tasks/__init__.py | 16 | __all__ = [ | CODE |
| LOW | rllm/tasks/loader.py | 38 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/tools/__init__.py | 25 | __all__ = ["PythonInterpreter", "GoogleSearchTool", "FirecrawlTool", "TavilyExtractTool", "TavilySearchTool", "ToolRegis | CODE |
| LOW | rllm/tools/mcp_tool.py | 6 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/tools/code_tools/__init__.py | 6 | __all__ = [ | CODE |
| LOW | rllm/tools/web_tools/__init__.py | 5 | __all__ = ["GoogleSearchTool", "FirecrawlTool", "TavilyExtractTool", "TavilySearchTool"] | CODE |
| LOW | rllm/environments/__init__.py | 3 | __all__ = ["BaseEnv"] | CODE |
| LOW | rllm/environments/base/__init__.py | 3 | __all__ = ["BaseEnv"] | CODE |
| LOW | rllm/harnesses/react.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/bash.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/cli_harness.py | 41 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/terminus2.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/oracle.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/tool_calling.py | 17 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/harnesses/tools/__init__.py | 7 | __all__ = ["BashTool", "FileEditorTool", "SubmitTool"] | CODE |
| LOW | rllm/rewards/__init__.py | 6 | __all__ = ["RewardInput", "RewardOutput", "RewardType", "RewardConfig", "RewardFunction", "zero_reward"] | CODE |
| LOW | rllm/rewards/countdown_reward.py | 8 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/rewards/math_utils/__init__.py | 11 | __all__ = ["extract_answer", "grade_answer_sympy", "grade_answer_mathd"] | CODE |
| LOW | rllm/rewards/code_utils/pyext2.py | 5 | __all__ = [ | CODE |
| LOW | rllm/workflows/__init__.py | 8 | __all__ = [ | CODE |
| LOW | rllm/agents/__init__.py | 13 | __all__ = [ | CODE |
| LOW | rllm/agents/agent.py | 29 | __all__ = [ | CODE |
| LOW | rllm/utils/__init__.py | 8 | __all__ = ["EpisodeLogger", "compute_pass_at_k", "extract_source_metadata", "visualize_trajectories", "VisualizationConf | CODE |
| LOW | rllm/cli/_pull.py | 12 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/cli/eval.py | 24 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/parser/chat_template_parser.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/parser/__init__.py | 3 | __all__ = [ | CODE |
| LOW⚡ | rllm/integrations/harbor/atif_trajectory_bridge.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/integrations/harbor/runtime.py | 24 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/integrations/harbor/dataset_loader.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/integrations/harbor/evaluator.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/integrations/harbor/trial_helper.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/train_schedule.py | 55 | __all__ = ["build_train_schedule"] | CODE |
| LOW | rllm/sandbox/sandboxed_flow.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/warm_queue.py | 48 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/warm_queue.py | 258 | __all__ = ["WarmQueue"] | CODE |
| LOW | rllm/sandbox/snapshot.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/snapshot.py | 429 | __all__ = ["env_key", "env_key_for", "install_script_for", "keys_for_tasks", "get_sandbox", "SnapshotRegistry"] | CODE |
| LOW | rllm/sandbox/backends/daytona.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/backends/modal_backend.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/backends/local.py | 11 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/sandbox/backends/docker.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/module_evaluator.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/vlm_utils.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | rllm/eval/rollout_decorator.py | 21 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | rllm/eval/rollout_decorator.py | 31 | __all__ = ["_coerce_to_episode"] | CODE |
| LOW | rllm/eval/runner.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/proxy.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/proxy.py | 192 | def set_limits() -> None: | CODE |
| LOW | rllm/eval/script_evaluator.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/materialize.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW⚡ | rllm/eval/_resolution.py | 37 | logger = logging.getLogger(__name__) | CODE |
| LOW | rllm/eval/_resolution.py | 329 | def _setup_task_environment(task: Task, sandbox: Sandbox) -> None: | CODE |
| LOW | rllm/eval/reward_fns/llm_equality.py | 19 | logger = logging.getLogger(__name__) | CODE |
| 86 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/test_store.py | 184 | # Create a minimal concrete subclass. | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 116 | # Create a mock actor instance with a config | COMMENT |
| MEDIUM⚡ | tests/test_verl_policy_loss.py | 121 | # Create a mock DataProto with override | COMMENT |
| MEDIUM | tests/data/test_registry_migration.py | 44 | # Create a fake v1 registry at the new location | COMMENT |
| MEDIUM | tests/data/test_registry_migration.py | 48 | # Create a fake parquet file | COMMENT |
| MEDIUM | rllm/experimental/fully_async/runner.py | 291 | # Create a configured TaskRunner class with rollout_fn baked in | COMMENT |
| MEDIUM | rllm/tools/__init__.py | 21 | # Create the singleton registry instance and register all default tools | COMMENT |
| MEDIUM | rllm/tools/utils.py | 161 | # Create an underlying bytes buffer initialized with the encoded input. | COMMENT |
| MEDIUM | rllm/tools/code_tools/e2b_tool.py | 83 | # Create a CodeToolOutput object instead of a dictionary | COMMENT |
| MEDIUM | rllm/tools/code_tools/together_tool.py | 114 | # Create a StringIO object with some test data | COMMENT |
| MEDIUM | rllm/tools/code_tools/lcb_tool.py | 211 | # Create a Python interpreter instance | COMMENT |
| MEDIUM | rllm/rewards/code_reward.py | 113 | # Create a new dict with only the selected test cases | COMMENT |
| MEDIUM⚡ | rllm/rewards/countdown_reward.py | 49 | # Define a regex pattern that only allows numbers, operators, parentheses, and whitespace | COMMENT |
| MEDIUM | rllm/rewards/code_utils/kodcode.py | 18 | # Create a preexec_fn function to set resource limits | COMMENT |
| MEDIUM | rllm/rewards/code_utils/humanevalplus.py | 45 | # Create a preexec_fn function to set resource limits | COMMENT |
| MEDIUM | rllm/utils/compute_pass_at_k.py | 11 | # Create a map to store correct answers per problem | COMMENT |
| MEDIUM⚡ | rllm/utils/tracking.py | 865 | # Create a new table with same columns and existing data | COMMENT |
| MEDIUM⚡ | rllm/trainer/verl/train_agent_ppo.py | 38 | # Create a remote instance of the TaskRunner class, and | COMMENT |
| MEDIUM⚡ | rllm/trainer/verl/train_agent_ppo.py | 48 | # This file is used for performance analysis | COMMENT |
| MEDIUM | examples/archive/deepresearch/deepresearch_workflow.py | 54 | # Create the DeepResearch agent | COMMENT |
| MEDIUM | examples/archive/deepresearch/deepresearch_workflow.py | 103 | # Create a failed episode | COMMENT |
| MEDIUM | examples/countdown/run_countdown.py | 53 | # Create a map to store correct answers per problem | COMMENT |
| MEDIUM⚡ | scripts/data/download_gaia.py | 7 | # Define the Google Drive file IDs for the Gaia files | COMMENT |
| MEDIUM⚡ | scripts/data/download_gaia.py | 16 | # Define the destination paths | COMMENT |
| MEDIUM⚡ | scripts/data/download_gaia.py | 22 | # Create the necessary directories | COMMENT |
| MEDIUM | scripts/data/download_datasets.py | 8 | # Define the Google Drive file IDs for the JSON files | COMMENT |
| MEDIUM | scripts/data/download_datasets.py | 21 | # Define the destination paths | COMMENT |
| MEDIUM | scripts/data/download_datasets.py | 31 | # Create the necessary directories | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | rllm/experimental/fully_async/utils.py | 481 | Assemble gen_batch_output from TrajectoryGroup objects. Optimized version: pre-allocates tensors and fills in s | STRING |
| HIGH | rllm/tools/registry.py | 123 | Get a tool class by name using dictionary-like syntax. Args: name: The name of the tool to | STRING |
| HIGH | rllm/environments/base/base_env.py | 60 | Creates an environment instance from a dictionary. This method should be implemented by concrete subclasses to | STRING |
| HIGH | rllm/parser/chat_template_parser.py | 51 | Verify that parsing messages together is equivalent to parsing them individually. Args: messages (l | STRING |
| HIGH | rllm/parser/chat_template_parser.py | 88 | Factory method to get the appropriate parser based on a string identifier. Args: parser_type (str): | STRING |
| HIGH | rllm/parser/tool_parser.py | 21 | Factory method to get the appropriate tool parser based on a string identifier. Args: tokenizer: Th | STRING |
| HIGH | rllm/data/dataset.py | 136 | Load dataset directly from a file path. Args: path: Path to the dataset file Returns: | STRING |
| HIGH | rllm/data/utils.py | 43 | Load a dataset from a JSON file based on the dataset enum. This function takes a dataset enum value and loads the c | STRING |
| HIGH | rllm/engine/unified_workflow_engine.py | 121 | Process a single task rollout with retry logic based on termination reasons. Args: task: Task dicti | STRING |
| HIGH | rllm/engine/agent_workflow_engine.py | 99 | Process a single task rollout with retry logic based on termination reasons. Args: task: Task dicti | STRING |
| HIGH | rllm/engine/rollout/completer.py | 25 | Basic completer that takes in messages and returns a single step. Args: rollout_engine: The rollout en | STRING |
| HIGH | rllm/trainer/distill/alignment.py | 211 | Align teacher logprobs to student tokens using byte-level alignment. Maps teacher logprobs to student tokens b | STRING |
| HIGH | examples/fully_async/deepresearch/refine_agent.py | 311 | Parse out the <think>...</think> tags and return only the result part. Args: content: The raw model re | STRING |
| HIGH | examples/fully_async/deepresearch/refine_agent.py | 340 | Parse out the content inside <information>...</information> tags. Args: content: The model response th | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | rllm/rewards/code_utils/taco.py | 0 | if test(generated_code) is not none it'll try to run the code. otherwise it'll just return an input and output pair. | STRING |
| HIGH | rllm/rewards/code_utils/codeforces.py | 0 | if test(generated_code) is not none it'll try to run the code. otherwise it'll just return an input and output pair. | STRING |
| HIGH | rllm/rewards/code_utils/livecodebench.py | 0 | if test(generated_code) is not none it'll try to run the code. otherwise it'll just return an input and output pair. | STRING |
| HIGH | rllm/rewards/code_utils/taco.py | 0 | this disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bo | STRING |
| HIGH | rllm/rewards/code_utils/codeforces.py | 0 | this disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bo | STRING |
| HIGH | rllm/rewards/code_utils/livecodebench.py | 0 | this disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bo | STRING |
| HIGH | rllm/rewards/code_utils/kodcode.py | 0 | this disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bo | STRING |
| HIGH | rllm/rewards/code_utils/humanevalplus.py | 0 | this disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bo | STRING |
| HIGH | scripts/data/gaia_dataset.py | 0 | create a mapping function to process dataset examples. args: split: dataset split name ('train' or 'test') returns: func | STRING |
| HIGH | scripts/data/code_dataset.py | 0 | create a mapping function to process dataset examples. args: split: dataset split name ('train' or 'test') returns: func | STRING |
| HIGH | scripts/data/deepscaler_dataset.py | 0 | create a mapping function to process dataset examples. args: split: dataset split name ('train' or 'test') returns: func | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/harnesses/test_cli_harness.py | 258 | # Cross-harness: ANTHROPIC_BASE_URL must not end in /v1 | COMMENT |
| LOW | rllm/tools/web_tools/firecrawl_tool.py | 130 | # For now, just call the synchronous version | COMMENT |
| MEDIUM | rllm/harnesses/zeroclaw.py | 158 | # startup before any LLM call in testing. The symlink is robust.) | COMMENT |
| MEDIUM | rllm/harnesses/claude_code.py | 127 | # harness doesn't pollute $HOME between runs. | COMMENT |
| MEDIUM | rllm/rewards/code_utils/swebench.py | 49 | # from swebench.harness.reporting import make_run_report | COMMENT |
| MEDIUM | rllm/cli/train.py | 327 | # Harbor datasets on an rllm-native harness skip ``harbor_reward_fn`` | COMMENT |
| MEDIUM | rllm/cli/eval.py | 162 | # Resolve agent name (display + harness lookup) | COMMENT |
| MEDIUM | rllm/cli/eval.py | 276 | # harbor-sourced dataset through an rllm-native harness (oracle, | COMMENT |
| MEDIUM | rllm/data/skillsbench_builder.py | 240 | # materialization time; the harness-layer hook is the right place for them. | COMMENT |
| MEDIUM⚡ | rllm/data/swebench_pro_builder.py | 310 | # regardless of what the harness or the image left HEAD pointing at. | COMMENT |
| MEDIUM | rllm/data/swebench_pro_builder.py | 648 | # solution/ (gold patch + apply script for the oracle harness) | COMMENT |
| LOW | rllm/engine/rollout/tinker_engine.py | 305 | sampling_params.pop("max_tokens", None) # TODO(listar2000): later we should simply use this as the max_tokens | CODE |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 195 | # No rejection, just return filtered groups with metrics | COMMENT |
| LOW | rllm/trainer/verl/transform.py | 184 | # The padding is done after the transform (in `_pad_dataproto_to_world_size`), so we simply set all to False her | COMMENT |
| MEDIUM | rllm/trainer/verl/transform.py | 600 | # Match advantages to batch entries by step_id (robust to batch reordering and padding) | COMMENT |
| MEDIUM⚡ | rllm/trainer/verl/verl_backend.py | 841 | # Async hook methods - leverage RayPPOTrainer utilities where possible | COMMENT |
| MEDIUM | examples/fully_async/deepresearch/train.py | 91 | # Be robust: if structure is unexpected, fall back to zero | COMMENT |
| MEDIUM | examples/fully_async/deepresearch/train.py | 147 | # Be robust: if structure is unexpected, fall back to zero | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/experimental/fully_async/utils.py | 418 | # Check if all correct or all incorrect | COMMENT |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 235 | # Check if the ESI (Elastic Server Instance)/training plan is close to expiration. | COMMENT |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 240 | # Check if the conditions for saving a checkpoint are met. | COMMENT |
| LOW | rllm/rewards/code_reward.py | 62 | # Check if we're out of the block (less indentation) | COMMENT |
| LOW⚡ | rllm/rewards/countdown_reward.py | 36 | # Check if all numbers in equation are available | COMMENT |
| LOW | rllm/rewards/countdown_reward.py | 180 | # Check if all numbers used are from the available set | COMMENT |
| LOW | rllm/rewards/code_utils/swebench.py | 203 | # Check if git diff changed after running eval script | COMMENT |
| LOW | rllm/cli/agent.py | 45 | # Check if it's a plugin agent | COMMENT |
| LOW | rllm/cli/login.py | 21 | # Check if already logged in | COMMENT |
| LOW | rllm/cli/eval.py | 491 | # Print results | COMMENT |
| LOW | rllm/parser/chat_template_parser.py | 73 | # Check if results are equivalent | COMMENT |
| LOW | rllm/integrations/harbor/dataset_loader.py | 176 | # Check if it's a local dataset directory | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 201 | # Check if we have enough partial solves | COMMENT |
| LOW | rllm/trainer/verl/agent_workflow_trainer.py | 228 | # Check if all episodes are correct or incorrect | COMMENT |
| LOW⚡ | rllm/trainer/verl/train_agent_ppo.py | 29 | # Check if Ray is not initialized | COMMENT |
| LOW | rllm/trainer/deprecated/tinker_data_processor.py | 466 | # Check if current step extends previous sequence | COMMENT |
| LOW | rllm/trainer/deprecated/tinker_policy_trainer.py | 74 | # Check if a Tinker model ID is provided in config | COMMENT |
| LOW | examples/archive/sft_tinker/train_norobots_tinker.sh | 17 | # Check if TINKER_API_KEY is set | COMMENT |
| LOW | examples/archive/deepresearch/deepresearch_tools.py | 634 | # Check if the module or its parent is allowed | COMMENT |
| LOW | examples/archive/deepresearch/deepresearch_agent.py | 568 | # Check if we've exceeded call limit | COMMENT |
| LOW | examples/archive/deepresearch/evaluate_hle.py | 355 | # Check if model is O3/O1 (use model_name which is already determined above) | COMMENT |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 326 | # Check if there's a <think> tag | COMMENT |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 330 | # Check if there's a </think> tag | COMMENT |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 355 | # Check if there's an <information> tag | COMMENT |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 359 | # Check if there's a </information> tag | COMMENT |
| LOW | examples/fully_async/deepresearch/search_agent.py | 338 | # Check if duplicate search was detected - if so, set reward to 0 | COMMENT |
| LOW | examples/fully_async/deepresearch/search_agent.py | 347 | # Check if excessive parallel tool calls detected (>= 5) - if so, set reward to 0 | COMMENT |
| LOW | examples/fully_async/deepresearch/tool.py | 116 | self._maybe_refresh_urls() # Check if URLs need refreshing | CODE |
| LOW | examples/fully_async/deepresearch/rag/launch_rag.sh | 61 | # Check if data directory exists | COMMENT |
| LOW⚡ | scripts/benchmark/cf_elo_calc.py | 212 | # Display results | COMMENT |
| LOW⚡ | cookbooks/workflow/solver_judge/solver_judge_flow.py | 118 | # Assign rewards to solver trajectories | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 280 | # Step 1: prompt=[1,2], response=[3,4] | COMMENT |
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 287 | # Step 2: prompt=[1,2,3,4,5] (extends [1,2,3,4]), response=[6,7] | COMMENT |
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 418 | # Step 1: [img_chunk, 1, 2] -> response [3, 4] | COMMENT |
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 424 | # Step 2: extends with [img_chunk, 1, 2, 3, 4, 5] -> response [6, 7] | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 303 | # Step 1: capture the agent's diff vs base_commit. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 320 | # Step 2: reset to base_commit so the patch applies on a clean tree. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 338 | # Step 3: re-apply the agent's patch. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 347 | # Step 4: run the last non-blank line of before_repo_set_cmd. Upstream | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 362 | # Step 5: run the per-instance run_script.sh with the selected test files. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 372 | # Step 6: parse the run_script.sh output into the standard JSON shape. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 375 | # Step 7: compare passed set against fail_to_pass ∪ pass_to_pass. | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 186 | # Step 1: Filter groups and episodes based on config | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 190 | # Step 2: Compute episode-level correctness metrics (always, for logging) | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 193 | # Step 3: Apply mode-specific logic (TODO(listar2000): implement a group-level rejection sampling) | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 245 | # Step 1: Name imputation | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 253 | # Step 2: Invoke the trajectory grouping hook | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 256 | # Step 3: Get metrics | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 228 | # Step 1: Process to datums (includes filtering and advantage computation) | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 236 | # Step 3: Remove mask from datums (not needed by forward_backward) | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 239 | # Step 4: Forward-backward pass | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 245 | # Step 5: Optimizer step | COMMENT |
| LOW⚡ | cookbooks/workflow/solver_judge/solver_judge_flow.py | 115 | # Step 1: Solver generates multiple solutions in parallel | COMMENT |
| LOW⚡ | cookbooks/workflow/solver_judge/solver_judge_flow.py | 127 | # Step 2: Judge selects the best solution | COMMENT |
| LOW⚡ | cookbooks/solver_judge_flow/solver_judge_flow.py | 27 | # Step 1: Solver generates N solutions in parallel | COMMENT |
| LOW⚡ | cookbooks/solver_judge_flow/solver_judge_flow.py | 30 | # Step 2: Judge selects the best solution | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 280 | # Step 1: prompt=[1,2], response=[3,4] | COMMENT |
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 287 | # Step 2: prompt=[1,2,3,4,5] (extends [1,2,3,4]), response=[6,7] | COMMENT |
| LOW⚡ | tests/unified_trainer/test_tinker_transform.py | 424 | # Step 2: extends with [img_chunk, 1, 2, 3, 4, 5] -> response [6, 7] | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 303 | # Step 1: capture the agent's diff vs base_commit. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 320 | # Step 2: reset to base_commit so the patch applies on a clean tree. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 338 | # Step 3: re-apply the agent's patch. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 347 | # Step 4: run the last non-blank line of before_repo_set_cmd. Upstream | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 362 | # Step 5: run the per-instance run_script.sh with the selected test files. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 372 | # Step 6: parse the run_script.sh output into the standard JSON shape. | COMMENT |
| LOW⚡ | rllm/data/swebench_pro_builder.py | 375 | # Step 7: compare passed set against fail_to_pass ∪ pass_to_pass. | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 186 | # Step 1: Filter groups and episodes based on config | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 190 | # Step 2: Compute episode-level correctness metrics (always, for logging) | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/rejection_sampling.py | 193 | # Step 3: Apply mode-specific logic (TODO(listar2000): implement a group-level rejection sampling) | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 245 | # Step 1: Name imputation | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 253 | # Step 2: Invoke the trajectory grouping hook | COMMENT |
| LOW⚡ | rllm/trainer/algorithms/transform.py | 256 | # Step 3: Get metrics | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 228 | # Step 1: Process to datums (includes filtering and advantage computation) | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 236 | # Step 3: Remove mask from datums (not needed by forward_backward) | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 239 | # Step 4: Forward-backward pass | COMMENT |
| LOW⚡ | rllm/trainer/deprecated/tinker_policy_trainer.py | 245 | # Step 5: Optimizer step | COMMENT |
| LOW⚡ | cookbooks/workflow/solver_judge/solver_judge_flow.py | 115 | # Step 1: Solver generates multiple solutions in parallel | COMMENT |
| LOW⚡ | cookbooks/workflow/solver_judge/solver_judge_flow.py | 127 | # Step 2: Judge selects the best solution | COMMENT |
| LOW⚡ | cookbooks/solver_judge_flow/solver_judge_flow.py | 27 | # Step 1: Solver generates N solutions in parallel | COMMENT |
| LOW⚡ | cookbooks/solver_judge_flow/solver_judge_flow.py | 30 | # Step 2: Judge selects the best solution | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/tools/test_tools.py | 121 | "google_search": [ | COMMENT |
| LOW | rllm/experimental/fully_async/runner.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/metric_utils.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/param_sync.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/message_queue.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/inference_manager.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 1 | # Copyright 2025 Meituan Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 241 | # The conditions include a mandatory condition (1) and | COMMENT |
| LOW | rllm/experimental/fully_async/fully_async_trainer.py | 601 | if rollout_corr_config is not None and "rollout_log_probs" in batch.batch: | COMMENT |
| LOW | rllm/harnesses/zeroclaw.py | 141 | content = self._config_toml(config) | COMMENT |
| LOW | rllm/harnesses/kimi_cli.py | 121 | self._exec_agent(sandbox, self._heredoc_write(_KIMI_CONFIG_PATH, content), env=env) | COMMENT |
| LOW | rllm/harnesses/claude_code.py | 21 | """ | COMMENT |
| LOW | rllm/harnesses/claude_code.py | 121 | "ANTHROPIC_MODEL": model, | COMMENT |
| LOW | rllm/rewards/code_utils/taco.py | 421 | # result = subprocess.run(['python', temp_program_path], stdin=inputs_tmp_file, text=True, captu | COMMENT |
| LOW | rllm/rewards/code_utils/codeforces.py | 361 | exec_code = -2 | COMMENT |
| LOW | rllm/utils/tracking.py | 1 | # Copyright 2024 Bytedance Ltd. and/or its affiliates | COMMENT |
| LOW | rllm/cli/eval.py | 261 | # catalog reward_fn > None. When ``evaluator`` is None ``SandboxTaskHooks`` | COMMENT |
| LOW | rllm/data/r2egym_builder.py | 261 | # R2E-Gym's grader runs the repo's own ``run_tests.sh`` from the repo root. | COMMENT |
| LOW | rllm/data/skillsbench_builder.py | 221 | _SKILLS_COPY_MARKER = "# rllm-skillsbench: skills injection" | COMMENT |
| LOW | rllm/data/swebench_pro_builder.py | 321 | # | COMMENT |
| LOW | rllm/trainer/tinker/transform.py | 161 | datums = [] | COMMENT |
| LOW | rllm/trainer/fireworks/fireworks_backend.py | 21 | from training.provision import FireworksProvisionInfra, init_fireworks_infra | COMMENT |
| LOW | rllm/trainer/algorithms/advantage.py | 241 | COMMENT | |
| LOW | rllm/trainer/algorithms/advantage.py | 261 | # rewards bucketed sensibly rather than silently dropped. | COMMENT |
| LOW | rllm/trainer/verl/transform.py | 301 | # ------------------------------------------------------------------ | COMMENT |
| LOW | rllm/trainer/sft/config/fireworks.yaml | 1 | # Native config for the Fireworks SFT backend. | COMMENT |
| LOW | examples/archive/sft_tinker/train_norobots_tinker.sh | 1 | #!/bin/bash | COMMENT |
| LOW | examples/fully_async/deepresearch/rag/launch_rag.sh | 1 | #!/bin/bash | COMMENT |
| LOW | examples/verifiers_env/train_verifiers_tinker.sh | 1 | #!/bin/bash | COMMENT |
| LOW | .github/workflows/claude.yml | 41 | COMMENT | |
| LOW | cookbooks/math/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/frozenlake/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/deepcoder/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/agent_frameworks/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/agent_frameworks/train_tinker.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/agent_frameworks/train_verl.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/finqa/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/workflow/solver_judge/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/workflow/geo3k/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/geo3k/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/math_tool_agent/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | cookbooks/solver_judge_flow/train_fireworks.sh | 1 | #!/usr/bin/env bash | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_store.py | 134 | CODE | |
| LOW | rllm/cli/eval.py | 80 | CODE | |
| LOW | rllm/integrations/harbor/runtime.py | 44 | CODE | |
| LOW | rllm/integrations/harbor/trial_helper.py | 107 | CODE | |
| LOW | rllm/integrations/harbor/trial_helper.py | 303 | CODE | |
| LOW | rllm/eval/runner.py | 29 | CODE | |
| LOW | rllm/data/claw_eval_builder.py | 92 | CODE | |
| LOW | rllm/data/r2egym_builder.py | 532 | CODE | |
| LOW | rllm/data/skillsbench_builder.py | 372 | CODE | |
| LOW | rllm/data/swebench_pro_builder.py | 667 | CODE | |
| LOW | rllm/data/deepswe_builder.py | 99 | CODE | |
| LOW | rllm/data/swesmith_builder.py | 174 | CODE | |
| LOW | rllm/engine/rollout/openai_engine.py | 21 | CODE | |
| LOW | rllm/engine/rollout/fireworks_engine.py | 89 | CODE | |
| LOW | rllm/engine/rollout/tinker_engine.py | 150 | CODE | |
| LOW | rllm/trainer/unified_trainer.py | 142 | CODE | |
| LOW | rllm/trainer/unified_trainer.py | 1002 | CODE | |
| LOW | rllm/trainer/distill/advantage.py | 77 | CODE | |
| LOW | examples/archive/ocr/process_ocr_data.py | 24 | CODE | |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 336 | CODE | |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 371 | CODE | |
| LOW | examples/fully_async/deepresearch/search_agent.py | 68 | CODE | |
| LOW | examples/fully_async/deepresearch/tool.py | 15 | CODE | |
| LOW | examples/fully_async/deepresearch/rag/rag_server.py | 59 | CODE | |
| LOW | scripts/data/webarena_dataset.py | 56 | CODE | |
| LOW | scripts/data/miniwob_dataset.py | 57 | CODE | |
| LOW | scripts/data/gaia_dataset.py | 59 | CODE | |
| LOW | scripts/data/code_dataset.py | 67 | CODE | |
| LOW | scripts/data/deepscaler_dataset.py | 66 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …rllm_telemetry/src/rllm_telemetry/trajectory_export.py | 11 | # After the agentic loop finishes: | STRING |
| MEDIUM⚡ | rllm/cli/eval.py | 114 | # whenever --agent is set to anything other than a harbor scaffold | COMMENT |
| MEDIUM | rllm/cli/eval.py | 251 | # plus user-registered + plugin agents; ``harbor:<scaffold>`` resolves | COMMENT |
| MEDIUM | rllm/integrations/harbor/trial_helper.py | 164 | # agent scaffolds read. Each scaffold checks a different one: | COMMENT |
| MEDIUM | rllm/engine/remote_runtime/protocol.py | 92 | # Pass-through to harbor AgentConfig.kwargs (scaffold-specific flags). | COMMENT |
| MEDIUM | rllm/trainer/config/rllm/base.yaml | 140 | agent: mini-swe-agent # scaffold name (mini-swe-agent | aider | openhands | swe-agent | qwen-coder | ...) | CODE |
| MEDIUM | rllm/trainer/config/rllm/base.yaml | 142 | agent_kwargs: {} # pass-through to harbor's AgentConfig.kwargs (scaffold-specific flags) | CODE |
| MEDIUM | rllm/trainer/algorithms/advantage.py | 300 | # but role-keyed for multi-agent correctness (single-agent -> "default"). | COMMENT |
| MEDIUM | rllm/trainer/verl/verl_backend.py | 446 | # and can grow up to the full context window - so use max_total_length to | COMMENT |
| MEDIUM | rllm/trainer/deprecated/tinker_data_processor.py | 705 | # Group by (task_id, trajectory_name) - for multi-agent workflows like solver-judge | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | examples/archive/sft_tinker/README_NOROBOTS.md | 162 | export TINKER_API_KEY=your_api_key | CODE |
| HIGH | examples/archive/sft_tinker/train_norobots_tinker.sh | 20 | echo "Please set it with: export TINKER_API_KEY=your_api_key" | CODE |
| HIGH | examples/fully_async/deepresearch/search_agent.py | 8 | export OPENAI_API_KEY="your-api-key" | STRING |
| HIGH | examples/verifiers_env/README.md | 77 | echo 'TINKER_API_KEY="your-api-key"' > examples/verifiers_env/.env | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/data/utils.py | 111 | # Example usage | COMMENT |
| LOW | examples/fully_async/deepresearch/refine_agent.py | 501 | # Example usage | COMMENT |
| LOW | examples/fully_async/deepresearch/rag/launch_rag.sh | 8 | # Usage: | COMMENT |
| LOW | cookbooks/agent_frameworks/train_fireworks.sh | 4 | # Usage: | COMMENT |
| LOW | cookbooks/agent_frameworks/train_tinker.sh | 4 | # Usage: | COMMENT |
| LOW | cookbooks/agent_frameworks/train_verl.sh | 4 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/trainer/verl/dataclass.py | 93 | if step_data.advantage is not None: # make sure to not add None to the list | CODE |
| MEDIUM | rllm/trainer/verl/train_agent_ppo.py | 111 | # TODO Here you can use the new registration method to support dynamic registration of roles | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/engine/rollout/verl_engine.py | 140 | # TODO: implement parse_completion for the standard parser | COMMENT |
| LOW | rllm/trainer/verl/agent_workflow_trainer.py | 530 | # TODO: implement actual tflpo and theoretical tflpo | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | examples/fully_async/deepresearch/search_agent.py | 105 | async def helper(tool_call): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | rllm/data/dataset.py | 730 | "prompt": [{"role": "user", "content": "placeholder"}], | CODE |