Train transformer language models with reinforcement learning.
This report presents the forensic synthetic code analysis of huggingface/trl, a Python project with 19,171 GitHub stars. SynthScan v2.0 examined 144,691 lines of code across 491 source files, recording 2321 pattern matches distributed across 20 syntactic categories. The overall adjusted score of 19.3 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 2321 distinct pattern matches across 20 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_rloo_trainer.py | 107 | def test_reward_func_wrong_number_of_rewards(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 132 | def test_train_dataset_format(self, config_name): | CODE |
| LOW | tests/test_rloo_trainer.py | 207 | def test_train_with_num_generations_eval(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 231 | def test_train_with_iterable_dataset(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 266 | def test_init_with_train_dataset(self, train_dataset_type): | CODE |
| LOW | tests/test_rloo_trainer.py | 318 | def test_init_with_eval_dataset(self, eval_dataset_type): | CODE |
| LOW | tests/test_rloo_trainer.py | 352 | def test_iterable_dataset_requires_dispatch_batches_false(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 367 | def test_iterable_dataset_forces_num_workers_zero(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 382 | def test_iterable_eval_keeps_map_style_train_workers(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 420 | def test_evaluate_with_eval_dataset(self, eval_dataset_type): | CODE |
| LOW | tests/test_rloo_trainer.py | 459 | def test_train_multiple_iterations(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 527 | def test_train_peft_and_quantization(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 609 | def test_train_moe_peft_model(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 657 | def test_train_peft_with_gradient_checkpointing(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 693 | def test_train_different_reward_model(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 732 | def test_train_reward_func_standard(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 766 | def test_train_reward_func_conversational(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 801 | def test_train_multiple_reward_funcs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 839 | def test_train_sync_and_async_reward_funcs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 888 | def non_applicable_reward_func(completions, **kwargs): | CODE |
| LOW | tests/test_rloo_trainer.py | 924 | def test_train_multiple_reward_funcs_with_weights(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 968 | def test_reward_metric_reflects_reward_weights(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1005 | def test_train_multiple_mixed_reward_funcs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1039 | def test_train_reward_func_additional_column(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1079 | def test_train_with_sync_ref_model(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1144 | def test_train_with_pad_to_multiple_of(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1224 | def test_train_vllm_structured_outputs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1256 | def test_train_with_additional_generation_kwargs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1293 | def test_train_vllm_with_additional_generation_kwargs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1329 | def test_train_with_normalized_advantages(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1359 | def test_train_with_clipped_rewards(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1390 | def test_train_with_mask_truncated_completions(self, mock_generate): | CODE |
| LOW | tests/test_rloo_trainer.py | 1439 | def test_train_with_mask_truncated_completions_all_masked(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1477 | def test_warning_raised_all_rewards_none(self, caplog): | CODE |
| LOW | tests/test_rloo_trainer.py | 1506 | def test_train_num_generations_larger_than_batch_size(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1536 | def test_train_multiple_dataloader_workers(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1577 | def test_train_with_generation_kwargs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1608 | def test_train_with_reward_func_accessing_trainer_state(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1633 | def test_train_reward_func_with_log_extra(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1659 | def test_train_reward_func_with_log_metric(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1686 | def test_prepare_input_called_with_correct_data(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1737 | def test_train_with_chat_template_kwargs(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1767 | def test_mismatched_reward_processing_classes_length(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1793 | def test_correct_reward_processing_classes_list(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1824 | def test_single_reward_model_with_single_processing_class(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1927 | def test_train_vlm_with_pad_to_multiple_of(self): | CODE |
| LOW | tests/test_rloo_trainer.py | 1970 | def test_train_vlm_beta_non_zero(self, model_id): | CODE |
| LOW | tests/test_rloo_trainer.py | 2060 | def test_train_vlm_peft_and_quantization(self, model_id): | CODE |
| LOW | tests/test_rloo_trainer.py | 2169 | def test_train_vlm_multi_image(self, model_id): | CODE |
| LOW | tests/test_rloo_trainer.py | 2211 | def test_train_vlm_log_multimodal_false(self): | CODE |
| LOW⚡ | tests/test_utils.py | 185 | def test_pad_to_multiple_of_side_left(self): | CODE |
| LOW⚡ | tests/test_utils.py | 193 | def test_pad_to_multiple_of_no_extra_padding(self): | CODE |
| LOW⚡ | tests/test_utils.py | 203 | def test_hash_module_deterministic_across_order(self): | CODE |
| LOW⚡ | tests/test_utils.py | 220 | def test_hash_module_changes_with_value(self): | CODE |
| LOW⚡ | tests/test_utils.py | 228 | def test_hash_module_includes_dtype(self): | CODE |
| LOW⚡ | tests/test_utils.py | 236 | def test_hash_module_tiny_model_twice(self): | CODE |
| LOW⚡ | tests/test_utils.py | 242 | def test_hash_module_tiny_model_change_layer(self): | CODE |
| LOW⚡ | tests/test_utils.py | 574 | def test_drops_incomplete_batch(self): | CODE |
| LOW⚡ | tests/test_utils.py | 580 | def test_preserves_all_columns(self): | CODE |
| LOW⚡ | tests/test_utils.py | 590 | def test_repeats_are_independent_objects(self): | CODE |
| 1047 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | pyproject.toml | 201 | # On Python 3.14+ the same deprecation is reworded to "is not supported in Python 3.14+ and may break" | COMMENT |
| LOW | pyproject.toml | 221 | # Upstream issue: https://github.com/triton-lang/triton/issues/10981 | COMMENT |
| LOW | pyproject.toml | 241 | COMMENT | |
| LOW | tests/test_rloo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/conftest.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_callbacks.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_sft_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/testing_constants.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_chat_template_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_grpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_model_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/testing_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_data_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_examples_index.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/__init__.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_vllm_client_server.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_rich_progress_callback.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_activation_offloading.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_rewards.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_cli_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_kto_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_dpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_distillation_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_cli.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/test_reward_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/_openreward_echo_env.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_async_distillation_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | …xperimental/test_self_distillation_trainer_behavior.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_utils.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_a2po_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_gmpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_gkd_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_nash_md_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_server_distillation_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_minillm_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_tpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_gspo_token_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_sdft_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/__init__.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_sdpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_xpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_iw_opd_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_gold_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_prm_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_online_dpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_openreward.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_openreward.py | 201 | # If a task spec ever shipped a `prompt` key, the metadata loop must | COMMENT |
| LOW | tests/experimental/test_async_grpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_async_grpo_trainer.py | 1081 | # Forks add rows, hence fixed-size optimizer steps: 3x rows per conversation must take strictly more | COMMENT |
| LOW | tests/experimental/test_ssd_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_harbor.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_orpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_bco_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_cpo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_modeling_value_head.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/experimental/test_merge_model_callback.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| LOW | tests/invariant/__init__.py | 1 | # Copyright 2020-2026 The HuggingFace Team. All rights reserved. | COMMENT |
| 325 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | AGENTS.md | 0 | brief one-line description of what this does. args: param1 (`str`): description of required param. param2 (`int`, *optio | STRING |
| HIGH | CLAUDE.md | 0 | brief one-line description of what this does. args: param1 (`str`): description of required param. param2 (`int`, *optio | STRING |
| HIGH | .cursor/BUGBOT.md | 0 | brief one-line description of what this does. args: param1 (`str`): description of required param. param2 (`int`, *optio | STRING |
| HIGH | .ai/AGENTS.md | 0 | brief one-line description of what this does. args: param1 (`str`): description of required param. param2 (`int`, *optio | STRING |
| HIGH | tests/test_rloo_trainer.py | 0 | test that training works with additional generation kwargs. | STRING |
| HIGH | tests/test_grpo_trainer.py | 0 | test that training works with additional generation kwargs. | STRING |
| HIGH | tests/test_distillation_trainer.py | 0 | test that training works with additional generation kwargs. | STRING |
| HIGH | tests/test_grpo_trainer.py | 0 | asynchronously multiplies two integers. args: a: the first integer. b: the second integer. returns: the product of the t | STRING |
| HIGH | tests/test_vllm_client_server.py | 0 | asynchronously multiplies two integers. args: a: the first integer. b: the second integer. returns: the product of the t | STRING |
| HIGH | tests/test_distillation_trainer.py | 0 | asynchronously multiplies two integers. args: a: the first integer. b: the second integer. returns: the product of the t | STRING |
| HIGH | docs/source/grpo_trainer.md | 0 | asynchronously multiplies two integers. args: a: the first integer. b: the second integer. returns: the product of the t | STRING |
| HIGH | docs/source/distillation_trainer.md | 0 | asynchronously multiplies two integers. args: a: the first integer. b: the second integer. returns: the product of the t | STRING |
| HIGH | docs/source/openenv.md | 0 | make a guess in the wordle environment. args: guess: the guessed word, formatted as '[abcde]' returns: the feedback mess | STRING |
| HIGH | examples/grpo_multi_env/grpo_multi_env.py | 0 | make a guess in the wordle environment. args: guess: the guessed word, formatted as '[abcde]' returns: the feedback mess | STRING |
| HIGH | examples/grpo_wordle/grpo_wordle.py | 0 | make a guess in the wordle environment. args: guess: the guessed word, formatted as '[abcde]' returns: the feedback mess | STRING |
| HIGH | examples/grpo_carla/carla_vlm_gemma.py | 0 | change lane to avoid obstacles. args: direction: direction to change lane, either "left" or "right". returns: the scene | STRING |
| HIGH | examples/grpo_carla/carla_vlm.py | 0 | change lane to avoid obstacles. args: direction: direction to change lane, either "left" or "right". returns: the scene | STRING |
| HIGH | examples/grpo_carla/grpo_carla.py | 0 | change lane to avoid obstacles. args: direction: direction to change lane, either "left" or "right". returns: the scene | STRING |
| HIGH | examples/datasets/tldr.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/deepmath_103k.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/lm-human-preferences-sentiment.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/prm800k.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | …mples/datasets/lm-human-preferences-descriptiveness.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/hh-rlhf-helpful-base.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/rlaif-v.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/math_shepherd.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/llava_instruct_mix.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/ultrafeedback-prompt.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | examples/datasets/tldr_preference.py | 0 | arguments for the script. args: push_to_hub (`bool`, *optional*, defaults to `false`): whether to push the dataset to th | STRING |
| HIGH | scripts/generate_zen_multi_image_dataset.py | 0 | arguments for the script. args: test_size (`float`, *optional*, defaults to `0.1`): fraction of the dataset to include i | STRING |
| HIGH | scripts/generate_harmony_dataset.py | 0 | arguments for the script. args: test_size (`float`, *optional*, defaults to `0.1`): fraction of the dataset to include i | STRING |
| HIGH | scripts/generate_zen_image_dataset.py | 0 | arguments for the script. args: test_size (`float`, *optional*, defaults to `0.1`): fraction of the dataset to include i | STRING |
| HIGH | scripts/generate_zen_dataset.py | 0 | arguments for the script. args: test_size (`float`, *optional*, defaults to `0.1`): fraction of the dataset to include i | STRING |
| HIGH | scripts/generate_toolcall_dataset.py | 0 | arguments for the script. args: test_size (`float`, *optional*, defaults to `0.1`): fraction of the dataset to include i | STRING |
| HIGH | trl/experimental/iw_opd/iw_opd_trainer.py | 0 | trim padded batch tensors into per-sample (prompt+completion) sequences for teacher-server requests. | STRING |
| HIGH | …tal/server_distillation/server_distillation_trainer.py | 0 | trim padded batch tensors into per-sample (prompt+completion) sequences for teacher-server requests. | STRING |
| HIGH | trl/experimental/sdpo/sdpo_trainer.py | 0 | trim padded batch tensors into per-sample (prompt+completion) sequences for teacher-server requests. | STRING |
| HIGH | trl/experimental/sdft/sdft_trainer.py | 0 | trim padded batch tensors into per-sample (prompt+completion) sequences for teacher-server requests. | STRING |
| HIGH | trl/experimental/iw_opd/iw_opd_trainer.py | 0 | backbone forward used by the liger jsd path (skips lm_head to save memory). | STRING |
| HIGH | trl/experimental/gkd/gkd_trainer.py | 0 | backbone forward used by the liger jsd path (skips lm_head to save memory). | STRING |
| HIGH | trl/experimental/gold/gold_trainer.py | 0 | backbone forward used by the liger jsd path (skips lm_head to save memory). | STRING |
| HIGH | trl/experimental/cpo/cpo_trainer.py | 0 | compute the kto loss and other metrics for the given batch of inputs for train or test. | STRING |
| HIGH | trl/experimental/bco/bco_trainer.py | 0 | compute the kto loss and other metrics for the given batch of inputs for train or test. | STRING |
| HIGH | trl/experimental/orpo/orpo_trainer.py | 0 | compute the kto loss and other metrics for the given batch of inputs for train or test. | STRING |
| HIGH | trl/trainer/kto_trainer.py | 0 | compute the kto loss and other metrics for the given batch of inputs for train or test. | STRING |
| HIGH | trl/experimental/cpo/cpo_trainer.py | 0 | generate samples from the model and reference model for the given batch of inputs. | STRING |
| HIGH | trl/experimental/bco/bco_trainer.py | 0 | generate samples from the model and reference model for the given batch of inputs. | STRING |
| HIGH | trl/experimental/orpo/orpo_trainer.py | 0 | generate samples from the model and reference model for the given batch of inputs. | STRING |
| HIGH | trl/experimental/cpo/cpo_trainer.py | 0 | overriding built-in evaluation loop to store metrics for each batch. prediction/evaluation loop, shared by `trainer.eval | STRING |
| HIGH | trl/experimental/bco/bco_trainer.py | 0 | overriding built-in evaluation loop to store metrics for each batch. prediction/evaluation loop, shared by `trainer.eval | STRING |
| HIGH | trl/experimental/orpo/orpo_trainer.py | 0 | overriding built-in evaluation loop to store metrics for each batch. prediction/evaluation loop, shared by `trainer.eval | STRING |
| HIGH | trl/experimental/cpo/cpo_trainer.py | 0 | log `logs` on the various objects watching training, including stored metrics. args: logs (`dict[str, float]`): the valu | STRING |
| HIGH | trl/experimental/bco/bco_trainer.py | 0 | log `logs` on the various objects watching training, including stored metrics. args: logs (`dict[str, float]`): the valu | STRING |
| HIGH | trl/experimental/orpo/orpo_trainer.py | 0 | log `logs` on the various objects watching training, including stored metrics. args: logs (`dict[str, float]`): the valu | STRING |
| HIGH | trl/trainer/distillation_trainer.py | 0 | tokenize prompts and extract images/multimodal fields for generation. | STRING |
| HIGH | trl/trainer/rloo_trainer.py | 0 | tokenize prompts and extract images/multimodal fields for generation. | STRING |
| HIGH | trl/trainer/grpo_trainer.py | 0 | tokenize prompts and extract images/multimodal fields for generation. | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/conftest.py | 26 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 28 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 73 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 75 | # ============================================================================ | COMMENT |
| MEDIUM | tests/conftest.py | 87 | # ============================================================================ | COMMENT |
| MEDIUM | examples/grpo_harbor/harnesses/terminal_notes/env.py | 60 | # ── shell toolkit ─────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | examples/grpo_harbor/harnesses/terminal_notes/env.py | 170 | # ── note toolkit (in-env state) ────────────────────────────────────────── | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/async_grpo_opencode.py | 95 | # ============================================================================================================ | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/async_grpo_opencode.py | 97 | # ------------------------------------------------------------------------------------------------------------ | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/async_grpo_opencode.py | 102 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 260 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 262 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 390 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 392 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 487 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 489 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 536 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 538 | # ============================================================================================================ | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/opencode_hf_sandbox.py | 111 | # ============================================================================================================ | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/opencode_hf_sandbox.py | 113 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 241 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 243 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 280 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 282 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 330 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 332 | # ============================================================================================================ | COMMENT |
| MEDIUM | examples/grpo_browsergym/browsergym_llm.py | 237 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_browsergym/browsergym_llm.py | 239 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | examples/grpo_browsergym/browsergym_llm.py | 256 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | examples/grpo_browsergym/browsergym_llm.py | 258 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | examples/grpo_browsergym/browsergym_llm.py | 266 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | examples/grpo_browsergym/browsergym_llm.py | 268 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 109 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 111 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 191 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 193 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 343 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 345 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 373 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sudoku/grpo_sudoku.py | 375 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 184 | # ------------------------ | COMMENT |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 186 | # ------------------------ | COMMENT |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 231 | # ------------------------ | COMMENT |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 233 | # ------------------------ | COMMENT |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 260 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 262 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 267 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 269 | # ------------------------ | STRING |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 284 | # ------------------------ | STRING |
| MEDIUM | examples/grpo_sql_agent/grpo_sql_agent.py | 286 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 298 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 300 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 310 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 312 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 315 | # ------------------------ | STRING |
| MEDIUM⚡ | examples/grpo_sql_agent/grpo_sql_agent.py | 317 | # ------------------------ | STRING |
| MEDIUM | examples/gold_qwen3_vl/gold_qwen3_vl.py | 110 | # ────────────────────────────────────────────── | COMMENT |
| MEDIUM | examples/gold_qwen3_vl/gold_qwen3_vl.py | 112 | # ────────────────────────────────────────────── | COMMENT |
| MEDIUM | examples/gold_qwen3_vl/gold_qwen3_vl.py | 130 | # ────────────────────────────────────────────── | COMMENT |
| MEDIUM | examples/gold_qwen3_vl/gold_qwen3_vl.py | 132 | # ────────────────────────────────────────────── | COMMENT |
| 33 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/grpo_harbor/harnesses/__init__.py | 15 | CODE | |
| LOW | examples/grpo_harbor/harnesses/__init__.py | 16 | CODE | |
| LOW | examples/grpo_harbor/harnesses/__init__.py | 17 | CODE | |
| LOW | examples/grpo_harbor/harnesses/bash/__init__.py | 15 | CODE | |
| LOW | …mples/grpo_harbor/harnesses/terminal_notes/__init__.py | 15 | CODE | |
| LOW | examples/grpo_harbor/harnesses/jupyter/kernel_server.py | 29 | CODE | |
| LOW | examples/grpo_harbor/harnesses/jupyter/__init__.py | 15 | CODE | |
| LOW | examples/grpo_harbor/harnesses/jupyter/run_cell.py | 25 | CODE | |
| LOW | examples/async_grpo_opencode/async_grpo_opencode.py | 59 | CODE | |
| LOW | examples/async_grpo_opencode/opencode_hf_sandbox.py | 78 | CODE | |
| LOW | examples/grpo_browsergym/grpo_browsergym.py | 53 | CODE | |
| LOW | examples/grpo_browsergym/browsergym_llm.py | 67 | CODE | |
| LOW | examples/grpo_sudoku/grpo_sudoku.py | 87 | CODE | |
| LOW | examples/ssd_codegen/ssd_eval.py | 59 | CODE | |
| LOW | trl/__init__.py | 19 | CODE | |
| LOW | trl/__init__.py | 78 | CODE | |
| LOW | trl/__init__.py | 78 | CODE | |
| LOW | trl/__init__.py | 78 | CODE | |
| LOW | trl/__init__.py | 78 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 84 | CODE | |
| LOW | trl/__init__.py | 98 | CODE | |
| LOW | trl/__init__.py | 99 | CODE | |
| LOW | trl/__init__.py | 99 | CODE | |
| LOW | trl/__init__.py | 99 | CODE | |
| LOW | trl/__init__.py | 99 | CODE | |
| LOW | trl/__init__.py | 99 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/__init__.py | 100 | CODE | |
| LOW | trl/experimental/utils.py | 55 | CODE | |
| 135 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_utils.py | 949 | CODE | |
| LOW | tests/test_utils.py | 980 | CODE | |
| LOW | tests/test_grpo_trainer.py | 4806 | CODE | |
| LOW | tests/test_grpo_trainer.py | 4953 | CODE | |
| LOW | tests/test_kto_trainer.py | 1026 | CODE | |
| LOW | tests/test_dpo_trainer.py | 1033 | CODE | |
| LOW | tests/test_distillation_trainer.py | 850 | CODE | |
| LOW | examples/grpo_catch/grpo_catch.py | 209 | CODE | |
| LOW | examples/datasets/math_shepherd.py | 52 | CODE | |
| LOW | …les/sdft_privileged_context/sdft_privileged_context.py | 119 | CODE | |
| LOW | examples/sdpo_math/sdpo_math.py | 162 | CODE | |
| LOW | examples/grpo_multi_env/grpo_multi_env.py | 127 | CODE | |
| LOW | examples/sft_gemma3_vision/sft_gemma3_vision.py | 77 | CODE | |
| LOW | examples/sft_gemma3_vision/sft_gemma3_vision.py | 96 | CODE | |
| LOW | examples/grpo_sudoku/grpo_sudoku.py | 212 | CODE | |
| LOW | examples/grpo_sudoku/grpo_sudoku.py | 294 | CODE | |
| LOW | examples/grpo_sudoku/grpo_sudoku.py | 378 | CODE | |
| LOW | examples/grpo_sudoku/grpo_sudoku.py | 545 | CODE | |
| LOW | examples/grpo_sql_agent/grpo_sql_agent.py | 50 | CODE | |
| LOW | examples/grpo_sql_agent/grpo_sql_agent.py | 145 | CODE | |
| LOW | scripts/log_reports.py | 27 | CODE | |
| LOW | scripts/generate_tiny_models/_common.py | 256 | CODE | |
| LOW | trl/chat_template_utils.py | 656 | CODE | |
| LOW | trl/data_utils.py | 33 | CODE | |
| LOW | trl/data_utils.py | 127 | CODE | |
| LOW | trl/data_utils.py | 975 | CODE | |
| LOW | trl/_compat.py | 86 | CODE | |
| LOW | trl/import_utils.py | 29 | CODE | |
| LOW | trl/experimental/merge_model_callback.py | 82 | CODE | |
| LOW | trl/experimental/merge_model_callback.py | 260 | CODE | |
| LOW | trl/experimental/utils.py | 191 | CODE | |
| LOW | trl/experimental/utils.py | 934 | CODE | |
| LOW | trl/experimental/utils.py | 954 | CODE | |
| LOW | trl/experimental/utils.py | 73 | CODE | |
| LOW | trl/experimental/utils.py | 470 | CODE | |
| LOW | trl/experimental/utils.py | 487 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 233 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 393 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 714 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 866 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 908 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 965 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 1130 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 1830 | CODE | |
| LOW | trl/experimental/sdpo/sdpo_trainer.py | 177 | CODE | |
| LOW | trl/experimental/sdpo/sdpo_trainer.py | 353 | CODE | |
| LOW | trl/experimental/sdpo/sdpo_trainer.py | 1174 | CODE | |
| LOW | trl/experimental/cpo/cpo_trainer.py | 138 | CODE | |
| LOW | trl/experimental/cpo/cpo_trainer.py | 480 | CODE | |
| LOW | trl/experimental/cpo/cpo_trainer.py | 613 | CODE | |
| LOW | trl/experimental/cpo/cpo_trainer.py | 677 | CODE | |
| LOW | trl/experimental/bco/bco_trainer.py | 425 | CODE | |
| LOW | trl/experimental/bco/bco_trainer.py | 1084 | CODE | |
| LOW | trl/experimental/bco/bco_trainer.py | 1287 | CODE | |
| LOW | trl/experimental/bco/bco_trainer.py | 1416 | CODE | |
| LOW | trl/experimental/harbor/_spec.py | 65 | CODE | |
| LOW | trl/experimental/orpo/orpo_trainer.py | 148 | CODE | |
| LOW | trl/experimental/orpo/orpo_trainer.py | 466 | CODE | |
| LOW | trl/experimental/orpo/orpo_trainer.py | 609 | CODE | |
| LOW | trl/experimental/online_dpo/online_dpo_trainer.py | 177 | CODE | |
| 100 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_rloo_trainer.py | 1777 | # Create a single processing class (tokenizer) | COMMENT |
| MEDIUM | tests/test_rloo_trainer.py | 1831 | # Create a single processing class (tokenizer) | COMMENT |
| MEDIUM⚡ | tests/test_grpo_trainer.py | 93 | # Create a mock trainer with minimal setup | COMMENT |
| MEDIUM⚡ | tests/test_grpo_trainer.py | 96 | # Create a mock accelerator | COMMENT |
| MEDIUM⚡ | tests/test_grpo_trainer.py | 100 | # Create a minimal trainer instance just to access the method | COMMENT |
| MEDIUM | tests/test_grpo_trainer.py | 3864 | # Create a single processing class (tokenizer) | COMMENT |
| MEDIUM | tests/test_grpo_trainer.py | 3918 | # Create a single processing class (tokenizer) | COMMENT |
| MEDIUM | tests/test_grpo_trainer.py | 4833 | # Create a VLM processor | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 202 | # Create the main parser | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 208 | # Create a subparser for a specific command | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 228 | # Create the main parser | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 234 | # Create a subparser for a specific command | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 251 | # Create the main parser | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 257 | # Create a subparser for a specific command | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 273 | # Create the main parser | COMMENT |
| MEDIUM⚡ | tests/test_cli_utils.py | 279 | # Create a subparser for a specific command | COMMENT |
| MEDIUM | tests/test_cli.py | 108 | # Create a temporary config file | COMMENT |
| MEDIUM | tests/experimental/test_prm_trainer.py | 126 | # Define the input features | COMMENT |
| MEDIUM | tests/experimental/test_prm_trainer.py | 150 | # Define the input features | COMMENT |
| MEDIUM | tests/experimental/test_prm_trainer.py | 173 | # Define the input features | COMMENT |
| MEDIUM | tests/experimental/test_prm_trainer.py | 197 | # Define the input features | COMMENT |
| MEDIUM | tests/experimental/test_prm_trainer.py | 221 | # Define the input features | COMMENT |
| MEDIUM | scripts/generate_zen_multi_image_dataset.py | 76 | # Create the images | STRING |
| MEDIUM | scripts/generate_zen_multi_image_dataset.py | 106 | # Create the images | STRING |
| MEDIUM | scripts/generate_zen_multi_image_dataset.py | 157 | # Create the images | STRING |
| MEDIUM | scripts/generate_zen_multi_image_dataset.py | 229 | # Create the images | STRING |
| MEDIUM | scripts/generate_zen_multi_image_dataset.py | 260 | # Create the images | STRING |
| MEDIUM | trl/data_utils.py | 794 | # Create a new bin | COMMENT |
| MEDIUM | trl/experimental/merge_model_callback.py | 37 | # Create the repository if it doesn't exist | COMMENT |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 118 | # Create the merge configuration dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 128 | # Create the MergeConfiguration from the dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 137 | # Create the TIES merge configuration dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 172 | # Create the MergeConfiguration from the dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 181 | # Create the DARE TIES merge configuration dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 216 | # Create the MergeConfiguration from the dictionary | STRING |
| MEDIUM⚡ | trl/experimental/merge_model_callback.py | 226 | # Create the SLERP merge configuration dictionary | STRING |
| MEDIUM | trl/experimental/merge_model_callback.py | 255 | # Create the MergeConfiguration from the dictionary | STRING |
| MEDIUM | trl/experimental/utils.py | 15 | # This file contains utility classes and functions that are used across more than one experimental trainer or feature. | COMMENT |
| MEDIUM | trl/experimental/cpo/cpo_trainer.py | 184 | # Initialize this variable to False. This helps tracking the case when `peft_module_casting_to_bf16` | COMMENT |
| MEDIUM | trl/experimental/bco/bco_trainer.py | 493 | # Initialize this variable to False. This helps tracking the case when `peft_module_casting_to_bf16` | COMMENT |
| MEDIUM | trl/experimental/orpo/orpo_trainer.py | 194 | # Initialize this variable to False. This helps tracking the case when `peft_module_casting_to_bf16` | COMMENT |
| MEDIUM | trl/experimental/online_dpo/online_dpo_trainer.py | 394 | # Define the collator if not provided | COMMENT |
| MEDIUM | trl/experimental/online_dpo/online_dpo_trainer.py | 906 | # Create the output dict with required fields | COMMENT |
| MEDIUM | trl/experimental/prm/prm_trainer.py | 339 | # Create the label | STRING |
| MEDIUM | trl/experimental/ppo/ppo_trainer.py | 386 | # Define the collator if not provided | COMMENT |
| MEDIUM | trl/trainer/utils.py | 164 | # Create an output tensor filled with the padding value | COMMENT |
| MEDIUM | trl/trainer/utils.py | 175 | # Define the slices | COMMENT |
| MEDIUM | trl/trainer/utils.py | 1104 | # This function is intentionally not used internally. It is provided as a utility for users whose datasets contain | COMMENT |
| MEDIUM | trl/trainer/utils.py | 764 | self.generator = torch.Generator() # Create a local random generator | CODE |
| MEDIUM | trl/trainer/callbacks.py | 220 | # Create a table per group | COMMENT |
| MEDIUM | trl/trainer/callbacks.py | 697 | # Create a new instance and load state_dict | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_sft_trainer.py | 2497 | except Exception: | CODE |
| LOW | tests/test_grpo_trainer.py | 4920 | except Exception as e: | CODE |
| LOW | tests/test_grpo_trainer.py | 4987 | except Exception as e: | CODE |
| MEDIUM | tests/test_rewards.py | 229 | def target(): | CODE |
| LOW | tests/test_rewards.py | 232 | except Exception as e: | CODE |
| LOW | tests/test_distillation_trainer.py | 890 | except Exception as e: | CODE |
| MEDIUM | tests/experimental/test_gold_trainer.py | 43 | def openr1_examples(): | CODE |
| MEDIUM | tests/experimental/test_gold_trainer.py | 56 | def countdown_examples(): | CODE |
| MEDIUM | tests/experimental/test_gold_trainer.py | 306 | def vlm_dataset(): | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 50 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 63 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 313 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 3371 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 3427 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 3478 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW | tests/experimental/test_gold_trainer.py | 3528 | except Exception as exc: # pragma: no cover - network/environment dependent | CODE |
| LOW⚡ | tests/experimental/test_online_dpo_trainer.py | 257 | except Exception: | CODE |
| LOW | docs/source/openenv.md | 406 | except Exception: | CODE |
| LOW | docs/source/openenv.md | 417 | except Exception: | CODE |
| LOW | examples/grpo_catch/grpo_catch.py | 166 | except Exception as e: | CODE |
| LOW | examples/grpo_carla/carla_vlm_gemma.py | 165 | except Exception as e: | CODE |
| LOW | examples/grpo_harbor/harnesses/jupyter/kernel_server.py | 70 | except Exception: | CODE |
| MEDIUM | examples/grpo_harbor/harnesses/jupyter/kernel_server.py | 63 | def do_POST(self): | CODE |
| LOW | examples/grpo_harbor/harnesses/jupyter/run_cell.py | 55 | except Exception as exc: | CODE |
| LOW | …les/sdft_privileged_context/sdft_privileged_context.py | 199 | except Exception: | CODE |
| LOW | examples/async_grpo_opencode/async_grpo_opencode.py | 215 | except Exception: | CODE |
| LOW | examples/async_grpo_opencode/async_grpo_opencode.py | 332 | except Exception: | CODE |
| LOW | examples/async_grpo_opencode/async_grpo_opencode.py | 458 | except Exception: | CODE |
| LOW | examples/async_grpo_opencode/opencode_hf_sandbox.py | 183 | except Exception: | CODE |
| LOW | examples/grpo_multi_env/grpo_multi_env.py | 136 | except Exception: | CODE |
| LOW | examples/grpo_multi_env/grpo_multi_env.py | 147 | except Exception: | CODE |
| LOW | examples/sft_gemma3_vision/sft_gemma3_vision.py | 106 | except Exception as e: | CODE |
| MEDIUM | examples/sft_gemma3_vision/sft_gemma3_vision.py | 107 | print(f"Error processing image {img_path}: {e}") | CODE |
| LOW | scripts/log_reports.py | 50 | except Exception as e: | CODE |
| MEDIUM | scripts/add_copyrights.py | 48 | print(f"Error fetching tracked files: {e}") | CODE |
| LOW | trl/_lazy_module.py | 72 | except Exception as e: | CODE |
| LOW | trl/_compat.py | 45 | except Exception as e: | CODE |
| LOW | trl/_compat.py | 69 | except Exception as e: | CODE |
| LOW | trl/_compat.py | 126 | except Exception as e: | CODE |
| LOW | trl/_compat.py | 155 | except Exception as e: | CODE |
| LOW | trl/experimental/harbor/_spec.py | 103 | except Exception: # noqa: BLE001 | CODE |
| LOW | trl/experimental/harbor/_env.py | 194 | except Exception: # noqa: BLE001 — best-effort teardown | CODE |
| MEDIUM | trl/experimental/harbor/_env.py | 191 | def __del__(self): | CODE |
| LOW | trl/experimental/openreward/_spec.py | 175 | except Exception: # noqa: BLE001 | CODE |
| LOW | trl/experimental/openreward/_spec.py | 282 | except Exception as e: # noqa: BLE001 | CODE |
| LOW | trl/experimental/openreward/_spec.py | 312 | except Exception as e: # noqa: BLE001 | CODE |
| LOW | trl/experimental/openreward/environment.py | 261 | except Exception as e: # noqa: BLE001 | CODE |
| LOW | trl/experimental/openreward/environment.py | 273 | except Exception as e: # noqa: BLE001 | CODE |
| LOW | trl/experimental/async_distillation/weight_transfer.py | 105 | except Exception as exc: | CODE |
| LOW | trl/experimental/async_distillation/weight_transfer.py | 139 | except Exception as exc: | CODE |
| LOW | …xperimental/async_distillation/async_rollout_worker.py | 145 | except Exception: | CODE |
| LOW | …xperimental/async_distillation/async_rollout_worker.py | 282 | except Exception: | CODE |
| LOW | …ental/async_distillation/async_distillation_trainer.py | 564 | except Exception as e: # noqa: BLE001 | CODE |
| LOW | trl/experimental/async_grpo/async_grpo_trainer.py | 283 | except Exception as e: # noqa: BLE001 | CODE |
| LOW⚡ | trl/experimental/async_grpo/openenv_harness.py | 171 | except Exception: | CODE |
| LOW | trl/experimental/async_grpo/openenv_harness.py | 202 | except Exception: | CODE |
| LOW | trl/experimental/async_grpo/openenv_harness.py | 260 | except Exception: | CODE |
| LOW | trl/experimental/async_grpo/openenv_harness.py | 267 | except Exception: | CODE |
| MEDIUM | trl/experimental/async_grpo/openenv_harness.py | 168 | def _close(session): | CODE |
| LOW | trl/experimental/async_grpo/weight_transfer.py | 113 | except Exception as exc: | CODE |
| 14 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | CONTRIBUTING.md | 268 | Replicate a string `n` times with a separator. Args: string (`str`): String to replicate. | STRING |
| HIGH | AGENTS.md | 71 | Brief one-line description of what this does. Args: param1 (`str`): Description of require | STRING |
| HIGH | CLAUDE.md | 71 | Brief one-line description of what this does. Args: param1 (`str`): Description of require | STRING |
| HIGH | .cursor/BUGBOT.md | 71 | Brief one-line description of what this does. Args: param1 (`str`): Description of require | STRING |
| HIGH | trl/chat_template_utils.py | 657 | Adds the appropriate response template (or legacy schema) to the given tokenizer based on its chat template. A | STRING |
| HIGH | trl/data_utils.py | 35 | Convert messages into a structured multimodal format and inject the provided images into the message contents. | STRING |
| HIGH | trl/data_utils.py | 161 | Check if the example is in a conversational format. Args: example (`dict[str, Any]`): A si | STRING |
| HIGH | trl/data_utils.py | 343 | If the example is in a conversational format, apply a chat template to it. Args: example (`dict[str, l | STRING |
| HIGH | trl/data_utils.py | 558 | Extracts the shared prompt from a preference data example, where the prompt is implicit within both the chosen and | STRING |
| HIGH | trl/data_utils.py | 936 | Check if the example is in a conversational format (from/value). Note that this format isn't recommended. Prefer | STRING |
| HIGH | trl/experimental/gold/gold_trainer.py | 554 | Merge probabilities based on alignment groups, using either the "observed" or "bayesian" strategy (`sel | STRING |
| HIGH | trl/scripts/vllm_serve.py | 194 | Build the `vllm serve` command line that serves `script_args` the way TRL trainers expect. Beyond the plain ar | STRING |
| HIGH | trl/trainer/utils.py | 123 | Pads a list of tensors to the same shape along the first dimension. Args: tensors (`list[torch.Tensor] | STRING |
| HIGH | .ai/AGENTS.md | 71 | Brief one-line description of what this does. Args: param1 (`str`): Description of require | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_chat_template_utils.py | 47 | # Check if special tokens are correctly set | COMMENT |
| LOW | tests/test_chat_template_utils.py | 114 | # Check if special tokens are correctly set | COMMENT |
| LOW | tests/test_grpo_trainer.py | 4900 | # Check if signature columns were set properly | COMMENT |
| LOW | tests/experimental/test_gkd_trainer.py | 54 | # Set temperature to 0 for deterministic output | COMMENT |
| LOW | tests/experimental/test_gkd_trainer.py | 71 | # Check if the generated texts start with the original prompts | COMMENT |
| LOW | tests/experimental/test_gkd_trainer.py | 82 | # Check if the two generations are identical | COMMENT |
| LOW | tests/experimental/test_online_dpo_trainer.py | 286 | # Check if training loss is available | COMMENT |
| LOW⚡ | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 45 | # Check if the buffer contains the correct number of elements | COMMENT |
| LOW⚡ | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 48 | # Check if the buffer maintains the min-heap property | COMMENT |
| LOW | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 67 | # Check if the buffer contains the correct number of elements | COMMENT |
| LOW | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 70 | # Check if the buffer maintains the min-heap property | COMMENT |
| LOW | …s/experimental/test_grpo_with_replay_buffer_trainer.py | 90 | # Check if the sampled elements are from the buffer | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 750 | # Check if critic weights have been updated | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 757 | # Check if policy weights have been updated | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 812 | # Check if critic weights have been updated | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 819 | # Check if policy weights have been updated - for PEFT we check the LoRA weights | COMMENT |
| LOW | tests/experimental/test_ppo_trainer.py | 191 | # Check if the weights are the same | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 206 | # Check if the weights are the same | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 224 | # Check if the weights are the same | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 245 | # Check if the weights are the same | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 251 | # Check if the trl model has the same keys as the transformers model | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 290 | # Check if the outputs are of the right size - here | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 303 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 315 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 320 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 405 | # Check if the outputs are of the right size - here | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 418 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 430 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | tests/experimental/test_ppo_trainer.py | 435 | # Check if v head of the model has the same dropout as the config | STRING |
| LOW | scripts/add_copyrights.py | 61 | # Check if the exact copyright header exists | COMMENT |
| LOW | .github/workflows/tests.yml | 95 | if: github.ref == 'refs/heads/main' && always() # Check if the branch is main | CODE |
| LOW | .github/workflows/tests.yml | 150 | if: github.ref == 'refs/heads/main' && always() # Check if the branch is main | CODE |
| LOW | .github/workflows/tests.yml | 200 | if: github.ref == 'refs/heads/main' && always() # Check if the branch is main | CODE |
| LOW | .github/workflows/tests.yml | 254 | if: github.ref == 'refs/heads/main' && always() # Check if the branch is main | CODE |
| LOW | .github/workflows/tests.yml | 306 | if: github.ref == 'refs/heads/main' && always() # Check if the branch is main | CODE |
| LOW | trl/experimental/utils.py | 881 | # Check if model is sharded (FSDP/DS-Zero3) | COMMENT |
| LOW | trl/experimental/tpo/tpo_trainer.py | 591 | # Check if the tokenized prompt starts with the tokenized prompt+completion | COMMENT |
| LOW⚡ | trl/models/activation_offloading.py | 235 | # Check if tensor is on CPU (skip offloading) | COMMENT |
| LOW⚡ | trl/models/activation_offloading.py | 240 | # Check if tensor is too small | COMMENT |
| LOW⚡ | trl/models/activation_offloading.py | 245 | # Check if tensor is a parameter or buffer | COMMENT |
| LOW⚡ | trl/models/activation_offloading.py | 252 | # Check if tensor is an FP8 tensor (TorchAO) - skip offloading as they're already compressed | COMMENT |
| LOW⚡ | trl/models/activation_offloading.py | 258 | # Check if tensor storage is a model parameter (for FSDP compatibility) | COMMENT |
| LOW | trl/models/activation_offloading.py | 301 | # Check if tensor has broadcast dimensions (stride == 0) | COMMENT |
| LOW | trl/models/activation_offloading.py | 511 | # Check if tensor still exists (might have been cleaned up by a previous node) | COMMENT |
| LOW | trl/models/utils.py | 283 | # Check if the model is already a FSDP model due to `Manual Wrapping` and if so, don't wrap it again | COMMENT |
| LOW | trl/trainer/sft_trainer.py | 1542 | # Check if the tokenized prompt starts with the tokenized prompt+completion | COMMENT |
| LOW | trl/trainer/dpo_trainer.py | 1074 | # Check if the tokenized prompt starts with the tokenized prompt+completion | COMMENT |
| LOW | trl/generation/vllm_client.py | 275 | # Check if the total timeout duration has passed | COMMENT |
| LOW | trl/extras/profiling.py | 208 | # Check if self is a Trainer-like object with required attributes | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_grpo_trainer.py | 2118 | return ratios[len(ratios) // 2] # median, robust to per-step noise | CODE |
| MEDIUM | tests/test_grpo_trainer.py | 2118 | return ratios[len(ratios) // 2] # median, robust to per-step noise | CODE |
| LOW | tests/test_kto_trainer.py | 699 | # purposes we will just use the same checkpoint | COMMENT |
| LOW | tests/test_dpo_trainer.py | 617 | # purposes we will just just use the same checkpoint | COMMENT |
| MEDIUM | examples/grpo_harbor/harnesses/bash/__init__.py | 15 | from trl.experimental.harbor import HarborBashEnv as BashEnv # the built-in bash harness | CODE |
| MEDIUM⚡ | examples/async_grpo_opencode/async_grpo_opencode.py | 98 | # OpenEnv's opencode harness only ships an E2B (cloud) backend, and a cloud sandbox can't reach a local vLLM. | COMMENT |
| MEDIUM⚡ | examples/async_grpo_opencode/async_grpo_opencode.py | 100 | # harness bakes the prefix `/home/user` into several paths, so each sandbox REMAPS that prefix to its own dir and | COMMENT |
| MEDIUM | examples/async_grpo_opencode/async_grpo_opencode.py | 592 | reward_funcs=[], # reward comes from the harness verifier via rollout_reward_fn, not reward_funcs | CODE |
| MEDIUM⚡ | examples/async_grpo_opencode/opencode_hf_sandbox.py | 104 | # harness path (`workdir`, `.opencode/bin`, `proxy/`) hang off `/root`. This is the only path difference from the | COMMENT |
| MEDIUM | examples/async_grpo_opencode/opencode_hf_sandbox.py | 401 | reward_funcs=[], # reward comes from the harness verifier via rollout_reward_fn, not reward_funcs | CODE |
| MEDIUM | trl/experimental/cpo/cpo_trainer.py | 498 | # 2. https://github.com/EleutherAI/lm-evaluation-harness/pull/531#issuecomment-1595586257 | COMMENT |
| MEDIUM | trl/experimental/bco/bco_trainer.py | 270 | # 2. https://github.com/EleutherAI/lm-evaluation-harness/pull/531#issuecomment-1595586257 | COMMENT |
| MEDIUM | trl/experimental/harbor/_env.py | 49 | #: Extra guidance appended to the task instruction by the harness subclass. | COMMENT |
| MEDIUM | trl/experimental/harbor/_env.py | 122 | await self._setup() # harness-specific sandbox prep (e.g. start a Jupyter kernel) | CODE |
| MEDIUM | trl/experimental/harbor/_env.py | 230 | #: import path / file path resolving to one) for a custom harness. | COMMENT |
| MEDIUM | trl/experimental/orpo/orpo_trainer.py | 484 | # 2. https://github.com/EleutherAI/lm-evaluation-harness/pull/531#issuecomment-1595586257 | COMMENT |
| MEDIUM⚡ | trl/experimental/async_grpo/openenv_harness.py | 155 | # OpenEnv's harness layer is synchronous, so run the whole session on the pool. | COMMENT |
| MEDIUM | trl/generation/vllm_client.py | 212 | # This is not strictly required for correctness, but it helps make training more robust to rare, transient | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/gold_chatbot_arena/gold_chatbot_arena.py | 70 | logger = logging.getLogger(__name__) | CODE |
| LOW | examples/grpo_harbor/harnesses/__init__.py | 20 | __all__ = ["BashEnv", "JupyterEnv", "TerminalNotesEnv"] | CODE |
| LOW | examples/grpo_harbor/harnesses/bash/__init__.py | 18 | __all__ = ["BashEnv"] | CODE |
| LOW | …mples/grpo_harbor/harnesses/terminal_notes/__init__.py | 18 | __all__ = ["TerminalNotesEnv"] | CODE |
| LOW | examples/grpo_harbor/harnesses/jupyter/env.py | 52 | async def _setup(self) -> None: | CODE |
| LOW | examples/grpo_harbor/harnesses/jupyter/__init__.py | 18 | __all__ = ["JupyterEnv"] | CODE |
| LOW | trl/experimental/merge_model_callback.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/iw_opd/__init__.py | 19 | __all__ = ["IWOPDConfig", "IWOPDTrainer"] | CODE |
| LOW | trl/experimental/ssd/__init__.py | 19 | __all__ = ["SSDConfig", "SSDTrainer"] | CODE |
| LOW | trl/experimental/sdpo/__init__.py | 19 | __all__ = ["SDPOConfig", "SDPOTrainer"] | CODE |
| LOW | trl/experimental/sdpo/teacher_sync.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/sdpo/sdpo_trainer.py | 700 | def _setup_teacher_model(self) -> None: | CODE |
| LOW | trl/experimental/cpo/__init__.py | 19 | __all__ = ["CPOConfig", "CPOTrainer"] | CODE |
| LOW | trl/experimental/harbor/__init__.py | 32 | __all__ = ["AGENTS", "HarborBashEnv", "HarborEnv", "HarborSpec"] | CODE |
| LOW | trl/experimental/harbor/_env.py | 155 | async def _setup(self) -> None: | CODE |
| LOW | trl/experimental/orpo/__init__.py | 19 | __all__ = ["ORPOConfig", "ORPOTrainer"] | CODE |
| LOW | trl/experimental/online_dpo/__init__.py | 19 | __all__ = ["OnlineDPOConfig", "OnlineDPOTrainer"] | CODE |
| LOW | trl/experimental/bema_for_ref_model/callback.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/xpo/__init__.py | 19 | __all__ = ["XPOConfig", "XPOTrainer"] | CODE |
| LOW | trl/experimental/sdft/__init__.py | 19 | __all__ = ["SDFTConfig", "SDFTTrainer"] | CODE |
| LOW | trl/experimental/sdft/teacher_sync.py | 29 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/sdft/sdft_trainer.py | 488 | def _setup_teacher_model(self) -> None: | CODE |
| LOW | trl/experimental/distillation/__init__.py | 22 | __all__ = ["DistillationConfig", "DistillationTrainer"] | CODE |
| LOW | trl/experimental/openreward/__init__.py | 18 | __all__ = ["OpenRewardSpec"] | CODE |
| LOW | trl/experimental/openreward/_spec.py | 51 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/openreward/environment.py | 32 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/experimental/gkd/__init__.py | 19 | __all__ = ["GKDConfig", "GKDTrainer"] | CODE |
| LOW | …xperimental/async_distillation/async_rollout_worker.py | 670 | def update_model_version(self, model_version: int) -> None: | CODE |
| LOW | trl/experimental/async_distillation/vllm_client.py | 106 | def update_weights(self, update_info: dict, timeout: int = 1800) -> None: | CODE |
| LOW | …ental/async_distillation/async_distillation_trainer.py | 446 | def update_model_version(self, model_version: int) -> None: ... | CODE |
| LOW | trl/experimental/gold/__init__.py | 19 | __all__ = ["GOLDConfig", "GOLDTrainer"] | CODE |
| LOW | trl/experimental/prm/__init__.py | 19 | __all__ = ["PRMConfig", "PRMTrainer"] | CODE |
| LOW | trl/experimental/async_grpo/async_grpo_trainer.py | 135 | def update_model_version(self, model_version: int) -> None: | CODE |
| LOW | trl/experimental/async_grpo/async_rollout_worker.py | 1146 | def update_model_version(self, model_version: int) -> None: | CODE |
| LOW | trl/experimental/async_grpo/vllm_client.py | 118 | def update_weights(self, update_info: dict, timeout: int = 1800) -> None: | CODE |
| LOW | trl/experimental/nash_md/__init__.py | 19 | __all__ = ["NashMDConfig", "NashMDTrainer"] | CODE |
| LOW | trl/experimental/ppo/__init__.py | 24 | __all__ = [ | CODE |
| LOW | trl/experimental/minillm/__init__.py | 19 | __all__ = ["MiniLLMConfig", "MiniLLMTrainer"] | CODE |
| LOW | trl/experimental/tpo/__init__.py | 19 | __all__ = ["TPOConfig", "TPOTrainer"] | CODE |
| LOW | trl/experimental/kto/__init__.py | 22 | __all__ = ["KTOConfig", "KTOTrainer"] | CODE |
| LOW | trl/experimental/server_distillation/__init__.py | 19 | __all__ = ["ServerDistillationConfig", "ServerDistillationTrainer", "build_teacher_request_inputs"] | CODE |
| LOW | trl/cli/__init__.py | 18 | __all__ = ["main"] | CODE |
| LOW | trl/cli/commands/__init__.py | 36 | __all__ = ["Command", "get_commands"] | CODE |
| LOW | trl/scripts/utils.py | 36 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/trainer/callbacks.py | 61 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/generation/vllm_generation.py | 47 | logger = logging.getLogger(__name__) | CODE |
| LOW | trl/generation/__init__.py | 20 | __all__ = [] | CODE |
| LOW | trl/generation/vllm_client.py | 53 | logger = logging.getLogger(__name__) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | trl/experimental/ppo/modeling_value_head.py | 713 | self.v_head.summary.weight.data.normal_(mean=0.0, std=initializer_range) | STRING |
| CRITICAL | trl/experimental/ppo/modeling_value_head.py | 714 | self.v_head.summary.bias.data.zero_() | STRING |
| CRITICAL | trl/experimental/ppo/modeling_value_head.py | 979 | self.v_head.summary.weight.data.normal_(mean=0.0, std=initializer_range) | STRING |
| CRITICAL | trl/experimental/ppo/modeling_value_head.py | 980 | self.v_head.summary.bias.data.zero_() | STRING |
| CRITICAL⚡ | trl/models/activation_offloading.py | 413 | torch.autograd.variable.Variable._execution_engine.queue_callback(wait_and_del_remaining_references) | CODE |
| CRITICAL | trl/generation/vllm_generation.py | 511 | self.llm.llm_engine.model_executor.driver_worker.model_runner.model.load_weights([(name, param)]) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_distillation_trainer.py | 61 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 795 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 1857 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2190 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2288 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2384 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2573 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2638 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2738 | CODE | |
| LOW | tests/experimental/test_gold_trainer.py | 2802 | CODE | |
| LOW | trl/experimental/iw_opd/iw_opd_trainer.py | 393 | CODE | |
| LOW | trl/experimental/sdpo/sdpo_trainer.py | 353 | CODE | |
| LOW | trl/experimental/cpo/cpo_trainer.py | 138 | CODE | |
| LOW | trl/experimental/bco/bco_trainer.py | 425 | CODE | |
| LOW | trl/experimental/orpo/orpo_trainer.py | 148 | CODE | |
| LOW | trl/experimental/online_dpo/online_dpo_trainer.py | 177 | CODE | |
| LOW | trl/experimental/bema_for_ref_model/callback.py | 130 | CODE | |
| LOW | trl/experimental/xpo/xpo_trainer.py | 113 | CODE | |
| LOW | trl/experimental/openreward/_spec.py | 115 | CODE | |
| LOW | trl/experimental/openreward/environment.py | 145 | CODE | |
| LOW | trl/experimental/gkd/gkd_trainer.py | 115 | CODE | |
| LOW | …xperimental/async_distillation/async_rollout_worker.py | 191 | CODE | |
| LOW | trl/experimental/gold/gold_trainer.py | 785 | CODE | |
| LOW | trl/experimental/prm/prm_trainer.py | 153 | CODE | |
| LOW | trl/experimental/async_grpo/async_grpo_trainer.py | 778 | CODE | |
| LOW | trl/experimental/async_grpo/async_rollout_worker.py | 296 | CODE | |
| LOW | trl/experimental/nash_md/nash_md_trainer.py | 172 | CODE | |
| LOW | trl/experimental/ppo/modeling_value_head.py | 644 | CODE | |
| LOW | trl/experimental/minillm/minillm_trainer.py | 166 | CODE | |
| LOW | trl/experimental/tpo/tpo_trainer.py | 289 | CODE | |
| LOW | …tal/server_distillation/server_distillation_trainer.py | 142 | CODE | |
| LOW | trl/trainer/distillation_trainer.py | 163 | CODE | |
| LOW | trl/trainer/distillation_trainer.py | 396 | CODE | |
| LOW | trl/trainer/distillation_trainer.py | 1513 | CODE | |
| LOW | trl/trainer/reward_trainer.py | 329 | CODE | |
| LOW | trl/trainer/sft_trainer.py | 916 | CODE | |
| LOW | trl/trainer/callbacks.py | 641 | CODE | |
| LOW | trl/trainer/rloo_trainer.py | 235 | CODE | |
| LOW | trl/trainer/rloo_trainer.py | 892 | CODE | |
| LOW | trl/trainer/kto_trainer.py | 568 | CODE | |
| LOW | trl/trainer/grpo_trainer.py | 303 | CODE | |
| LOW | trl/trainer/grpo_trainer.py | 1448 | CODE | |
| LOW | trl/trainer/dpo_trainer.py | 517 | CODE | |
| LOW | trl/generation/vllm_generation.py | 216 | CODE | |
| LOW | trl/generation/vllm_client.py | 334 | CODE | |
| LOW | trl/generation/vllm_client.py | 468 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | examples/grpo_harbor/harnesses/terminal_notes/env.py | 79 | f"mkfifo {pipe} 2>/dev/null; ( nohup setsid bash -c {shlex.quote(command)} <{pipe} >{log} 2>&1 ) & echo $!", | CODE |
| HIGH | examples/grpo_harbor/harnesses/jupyter/env.py | 56 | await self._env.exec("which curl >/dev/null 2>&1 || apt-get install -y curl", timeout_sec=120) | CODE |
| HIGH | examples/async_grpo_opencode/async_grpo_opencode.py | 326 | cmd = "ulimit -v 2000000 2>/dev/null; timeout %d python3 solution.py 2>/dev/null | head -c 2000000" % PER_TEST_TIMEO | CODE |
| HIGH | examples/async_grpo_opencode/async_grpo_opencode.py | 345 | r = sandbox.exec("cd /home/user/workdir && python3 _run_tests.py", timeout=PER_TEST_TIMEOUT * len(tests) + 30) | CODE |
| HIGH | examples/async_grpo_opencode/opencode_hf_sandbox.py | 177 | cmd = "ulimit -v 2000000 2>/dev/null; timeout %d python3 solution.py 2>/dev/null | head -c 2000000" % PER_TEST_TIMEO | CODE |
| HIGH | examples/grpo_wordle/grpo_wordle.py | 80 | uv venv && source .venv/bin/activate | STRING |
| HIGH | trl/data_utils.py | 690 | elif not first_column_offsets.equals(column.offsets): | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_data_utils.py | 752 | # chosen keeps its reasoning trace, rejected doesn't: the common prefix of the two rendered strings ends | STRING |
| MEDIUM | tests/experimental/test_bco_trainer.py | 358 | # Get embedding model | COMMENT |
| MEDIUM | docs/source/openenv.md | 3 | [OpenEnv](https://github.com/huggingface/OpenEnv) is an open-source framework for defining, deploying, and interacting w | CODE |
| MEDIUM | docs/source/bco_trainer.md | 64 | # customize this function depending on your embedding model | COMMENT |
| MEDIUM | docs/source/harbor.md | 3 | [Harbor](https://www.harborframework.com) is a framework for running agentic tasks in sandboxes. It decouples a **task** | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | docs/source/customization.md | 128 | # In this example: the hinge loss from https://huggingface.co/papers/2309.06657 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | trl/chat_template_utils.py | 122 | dummy_token = AddedToken(f"<extra_id_{idx}>") | CODE |
| LOW⚡ | trl/chat_template_utils.py | 123 | is_added = tokenizer.add_tokens(dummy_token) | CODE |
| LOW⚡ | trl/chat_template_utils.py | 126 | added_tokens.append(dummy_token) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | trl/generation/vllm_client.py | 839 | # Example usage | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_utils.py | 287 | def test_function(self): | CODE |