Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
This report summarizes scanner evidence for MakazhanAlpamys/Soup, a Python project with 8,547 GitHub stars. SynthScan examined 851,694 lines of code across 2232 source files, recording 25447 pattern matches distributed across 22 syntactic categories. The legacy adjusted pattern signal is 43.2 (High pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 18813 finding(s).
Are imports declared and package/API names verified? 1568 finding(s).
Are wrappers, guards, and repeated structures necessary? 23102 finding(s).
Which changes need focused human review? 25447 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 25447 distinct pattern matches across 22 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_v0403_part_c.py | 60 | def test_kept_when_chosen_clearly_wins(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 67 | def test_dropped_when_diff_below_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 74 | def test_dropped_when_rejected_wins(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 80 | def test_judge_exception_counts_as_error(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 88 | def test_lower_threshold_keeps_borderline(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 96 | def test_threshold_zero_keeps_ties(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 101 | def test_rejects_non_numeric_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 106 | def test_rejects_bool_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 111 | def test_rejects_threshold_above_one(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 116 | def test_rejects_threshold_below_zero(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 121 | def test_rejects_nan_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 126 | def test_too_many_pairs_rejected(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 132 | def test_empty_pairs_returns_empty(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 138 | def test_uses_judge_rubric_scale(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 144 | def test_report_is_frozen_dataclass(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 154 | def test_degenerate_scale_drops_all_at_positive_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 162 | def test_degenerate_scale_keeps_all_at_zero_threshold(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 169 | def test_lazy_materialisation_does_not_buffer_full_generator(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 292 | def test_appends_multiple_lines(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 300 | def test_outside_cwd_rejected(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 307 | def test_null_byte_path_rejected(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 317 | def test_cap_below_min_rejected(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 322 | def test_cap_above_max_rejected(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 332 | def test_non_string_path_rejected(self): | CODE |
| LOW⚡ | tests/test_v0403_part_c.py | 336 | def test_rotation_when_cap_exceeded(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 198 | def test_judge_flag_filters_pairs(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 244 | def test_judge_provider_invalid_rejected_early(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 362 | def test_secret_redaction_in_prompt(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 376 | def test_secret_redaction_in_response(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 390 | def test_secret_redaction_preserves_trailing_period(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 406 | def test_secret_redaction_in_extra_dict(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 419 | def test_secret_redaction_redacts_sk_keys(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 434 | def test_rotation_refuses_symlink_backup(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 451 | def test_unserialisable_entry_dropped_silently(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 486 | def test_extra_does_not_override_core_fields(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 525 | def test_serve_help_lists_trace_log(self): | CODE |
| LOW | tests/test_v0403_part_c.py | 534 | def test_serve_help_lists_trace_log_cap_mb(self): | CODE |
| LOW | tests/test_v0403_part_c.py | 545 | def test_create_app_accepts_trace_log_writer(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_v0403_part_c.py | 569 | def test_create_app_default_writer_is_none(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 78 | def test_none_fires_the_head_at_the_last_layer_going_forward(self): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 83 | def test_layer_zero_fires_the_head_at_layer_zero_going_forward_only(self): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 87 | def test_a_middle_layer_fires_once_and_never_in_the_backward_walk(self): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 91 | def test_the_backward_tail_embed_prefetch_is_unchanged_by_the_head_timing(self): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 96 | def test_a_second_forward_after_prime_fires_again(self): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 128 | def test_a_valid_assignment_is_accepted_and_read_back(self, good): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 289 | def test_the_head_load_comes_at_the_configured_layer(self, tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 295 | def test_the_real_load_sequence_and_count_are_the_same_in_both_timings( | CODE |
| LOW⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 348 | def test_the_head_refill_waits_for_the_embedding_lookup_still_queued(): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 110 | def test_an_out_of_range_layer_is_refused(self, bad): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 117 | def test_an_assignment_is_checked_like_the_constructor_and_leaves_the_value(self, bad): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 243 | def test_the_comparison_is_not_noise(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 250 | def test_tied_has_no_large_pool_so_the_switch_does_nothing(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 261 | def test_an_untied_model_defaults_to_the_earliest_point(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 270 | def test_a_tied_model_has_nothing_to_move(self, tmp_path, monkeypatch): | CODE |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 306 | def test_an_eval_forward_then_a_training_step(self, tmp_path, monkeypatch, head_layer): | CODE |
| LOW⚡ | tests/test_issue1350_reward_synth_numeric_parser.py | 59 | def test_a_synthesized_verifier_pays_every_correct_completion(verifier, gold, completion): | CODE |
| LOW⚡ | tests/test_issue1350_reward_synth_numeric_parser.py | 65 | def test_a_synthesized_verifier_still_refuses_a_wrong_answer(verifier, gold, completion): | CODE |
| LOW⚡ | tests/test_issue1350_reward_synth_numeric_parser.py | 69 | def test_grouped_references_are_detected_as_numeric(): | CODE |
| LOW⚡ | tests/test_issue1350_reward_synth_numeric_parser.py | 85 | def test_the_issue_reference_set_is_accepted_whole(tmp_path, monkeypatch): | CODE |
| LOW⚡ | tests/test_issue1350_reward_synth_numeric_parser.py | 93 | def test_a_refusing_calibration_names_every_rejected_reference(): | CODE |
| 17904 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_v0403_part_c.py | 26 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v0403_part_c.py | 28 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v0403_part_c.py | 267 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v0403_part_c.py | 269 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 136 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 138 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 281 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 283 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 336 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_issue1174_head_prefetch_after_lookup.py | 338 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_issue1174_head_prefetch_after_lookup.py | 33 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_issue1174_head_prefetch_after_lookup.py | 35 | # -------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_training_intelligence.py | 80 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_training_intelligence.py | 82 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_training_intelligence.py | 318 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_training_intelligence.py | 320 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_training_intelligence.py | 5 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_training_intelligence.py | 7 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_training_intelligence.py | 190 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_training_intelligence.py | 192 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_cans.py | 69 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_cans.py | 71 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_cans.py | 160 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_cans.py | 162 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_cans.py | 13 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_cans.py | 15 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_cans.py | 202 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_cans.py | 204 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 74 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 76 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 138 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 140 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 209 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 211 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 361 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 363 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 416 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 418 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 463 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 465 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 541 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0660_part_a.py | 543 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v0660_part_a.py | 35 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v0660_part_a.py | 37 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v07133.py | 34 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v07133.py | 36 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 419 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 421 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 676 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 678 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 872 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 874 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 1215 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | tests/test_v07133.py | 1217 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 21 | # ---------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 23 | # ---------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 86 | # ---------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 88 | # ---------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 247 | # ---------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | tests/test_v0538.py | 249 | # ---------------------------------------------------------------------- | COMMENT |
| 3935 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_issue1449_trust_remote_code_dropped.py | 370 | except Exception: | CODE |
| HIGH | tests/test_issue1449_trust_remote_code_dropped.py | 398 | except Exception: | CODE |
| HIGH | tests/test_issue1449_trust_remote_code_dropped.py | 428 | except Exception: | CODE |
| HIGH | tests/test_issue1449_trust_remote_code_dropped.py | 458 | except Exception: | CODE |
| HIGH | tests/test_issue681_uld_id_compat.py | 185 | except Exception: | CODE |
| HIGH | tests/test_issue691_sft_packing.py | 38 | except Exception: | CODE |
| HIGH | tests/test_outbound_endpoint_ip_literals.py | 259 | except _RequestAttemptedError: | CODE |
| HIGH | tests/test_outbound_endpoint_ip_literals.py | 272 | except _RequestAttemptedError: | CODE |
| HIGH | tests/test_outbound_endpoint_ip_literals.py | 289 | except _RequestAttemptedError: | CODE |
| HIGH | tests/test_awq_gptq_export.py | 1075 | except ValueError: | CODE |
| HIGH | tests/test_v0580.py | 788 | except (OSError, ValueError): | CODE |
| HIGH | tests/test_vllm_serve.py | 128 | except (SystemExit, Exception): | CODE |
| HIGH | tests/test_v0537.py | 1113 | except ImportError: | CODE |
| HIGH | tests/test_issue1631_mxfp4_preflight.py | 101 | except typer.Exit: | CODE |
| HIGH | tests/test_issue1070_moe_target_modules.py | 595 | except Exception: # noqa: BLE001 -- setup carries on past the attach | CODE |
| HIGH | tests/test_v0531_139.py | 227 | except OSError: | CODE |
| HIGH | tests/test_v0531_139.py | 254 | except OSError: | CODE |
| HIGH | tests/test_v0450.py | 936 | except OSError: | CODE |
| HIGH | tests/test_part_d.py | 113 | except Exception: # tokenizer mock approximation may explode in decode | CODE |
| HIGH | tests/test_issue865_langfuse_pull_bounds.py | 125 | except (BrokenPipeError, ConnectionResetError): | CODE |
| HIGH | tests/test_issue1652_preflight_prequantized.py | 260 | except typer.Exit: | CODE |
| HIGH | tests/test_v0660_followups.py | 301 | except OSError: | CODE |
| HIGH | tests/test_issue204_langfuse_pull.py | 358 | except (BrokenPipeError, ConnectionResetError): | CODE |
| HIGH | benchmarks/bench_validator.py | 108 | except Exception: | CODE |
| HIGH | benchmarks/harness/issue1702_arena_pack_gate.py | 376 | except torch.OutOfMemoryError: | CODE |
| HIGH | benchmarks/harness/stream_probe.py | 718 | except (OSError, ValueError, subprocess.SubprocessError): | CODE |
| HIGH | benchmarks/harness/issue361_nf4_throughput.py | 274 | except Exception: | CODE |
| HIGH | benchmarks/harness/issue395_second_stack_vram.py | 155 | except Exception: # noqa: BLE001 - teardown must not mask the measurement | CODE |
| HIGH | src/soup_cli/cli.py | 901 | except Exception: # noqa: BLE001 — audit must never crash the CLI | CODE |
| HIGH | src/soup_cli/cli.py | 968 | except Exception: # noqa: BLE001 — telemetry must never crash the CLI | CODE |
| HIGH | src/soup_cli/cli.py | 1016 | except Exception: | CODE |
| HIGH | src/soup_cli/cli.py | 638 | except ImportError: | CODE |
| HIGH | src/soup_cli/cli.py | 649 | except ImportError: | CODE |
| HIGH | src/soup_cli/ui/app.py | 275 | except Exception: | CODE |
| HIGH | src/soup_cli/ui/app.py | 84 | except ValueError: | CODE |
| HIGH | src/soup_cli/ui/app.py | 606 | except (ValueError, TypeError): | CODE |
| HIGH | src/soup_cli/ui/app.py | 1231 | except (ValueError, IndexError, KeyError): | CODE |
| HIGH | src/soup_cli/autodistill/mlx_worker.py | 488 | except Exception: | CODE |
| HIGH | src/soup_cli/plugins/__init__.py | 166 | except OSError: | CODE |
| HIGH | src/soup_cli/plugins/__init__.py | 523 | except RuntimeError: | CODE |
| HIGH | src/soup_cli/utils/liger.py | 129 | except (ImportError, AttributeError): | CODE |
| HIGH | src/soup_cli/utils/liger.py | 77 | except (ImportError, AttributeError, NotImplementedError, KeyError, ValueError): | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 178 | except FileNotFoundError: | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 190 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 257 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 261 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 238 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/advise_history.py | 246 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/ci_workflow.py | 244 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/data_forge.py | 493 | except FileNotFoundError: | CODE |
| HIGH | src/soup_cli/utils/data_forge.py | 516 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/ra_dit_run.py | 131 | except Exception: # noqa: BLE001 — best-effort cleanup. | CODE |
| HIGH | src/soup_cli/utils/ra_dit_run.py | 251 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/mole_routing.py | 413 | except OSError: | CODE |
| HIGH | src/soup_cli/utils/data_doctor.py | 804 | except Exception: # noqa: BLE001 — fall through to per-id decode | CODE |
| HIGH | src/soup_cli/utils/reward_hack_control.py | 1014 | except Exception: # noqa: BLE001 | CODE |
| HIGH | src/soup_cli/utils/reward_hack_control.py | 777 | except FileNotFoundError: | CODE |
| HIGH | src/soup_cli/utils/reward_hack_control.py | 1019 | except Exception: # noqa: BLE001 | CODE |
| HIGH | src/soup_cli/utils/reward_hack_control.py | 1074 | except Exception: # noqa: BLE001 | CODE |
| HIGH | src/soup_cli/utils/reward_hack_control.py | 1009 | except Exception: # noqa: BLE001 — never crash training | CODE |
| 220 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_v0403_part_c.py | 3 | CODE | |
| LOW | tests/test_v0403_part_c.py | 519 | CODE | |
| LOW | tests/test_issue1174_head_prefetch_after_lookup.py | 22 | CODE | |
| LOW | tests/test_issue1350_reward_synth_numeric_parser.py | 8 | CODE | |
| LOW | tests/test_issue1247_grpo_beta_zero.py | 3 | CODE | |
| LOW | tests/test_issue819_plugin_discovery.py | 3 | CODE | |
| LOW | tests/test_issue821_toxicity_truthfulness.py | 3 | CODE | |
| LOW | tests/test_cans.py | 3 | CODE | |
| LOW | tests/test_peft_patches.py | 8 | CODE | |
| LOW | tests/test_v0660_part_a.py | 20 | CODE | |
| LOW | tests/test_issue1218_parquet_parity.py | 9 | CODE | |
| LOW | tests/test_v07133.py | 12 | CODE | |
| LOW | tests/test_v0538.py | 11 | CODE | |
| LOW | tests/test_refusal_multilingual.py | 11 | CODE | |
| LOW | tests/test_issue359_deepspeed_guard_coverage.py | 16 | CODE | |
| LOW | tests/test_batch_probe.py | 8 | CODE | |
| LOW | tests/test_terminal_output_hygiene.py | 35 | CODE | |
| LOW | tests/test_recipe_count_is_synced.py | 20 | CODE | |
| LOW | tests/test_issue1440_from_traces_reads_ingest_signal.py | 12 | CODE | |
| LOW | tests/test_issue549_bon_resume.py | 3 | CODE | |
| LOW | tests/test_issue1437_memorization_long_rows.py | 3 | CODE | |
| LOW | tests/test_issue1449_trust_remote_code_dropped.py | 14 | CODE | |
| LOW | tests/test_issue823_stale_feature_descriptions.py | 3 | CODE | |
| LOW | tests/test_issue580_autodistill_mlx_worker.py | 3 | CODE | |
| LOW | tests/test_v07117.py | 15 | CODE | |
| LOW | tests/test_ui_metrics.py | 91 | CODE | |
| LOW | tests/test_issue533_offline_hf_materialization.py | 3 | CODE | |
| LOW | tests/test_issue1731_fp8_needs_transformers_5_17.py | 15 | CODE | |
| LOW | tests/test_issue1445_early_destination_validation.py | 17 | CODE | |
| LOW | tests/test_sync_recipe_count.py | 16 | CODE | |
| LOW | tests/test_v0570_part_c.py | 3 | CODE | |
| LOW | tests/test_issue721_ppo_config_forwarding.py | 3 | CODE | |
| LOW | tests/test_issue1162_quest_base_identity.py | 3 | CODE | |
| LOW | tests/test_v0600_part_c.py | 10 | CODE | |
| LOW | tests/test_v07123.py | 21 | CODE | |
| LOW | tests/test_registry.py | 3 | CODE | |
| LOW | tests/test_windows_encoding.py | 11 | CODE | |
| LOW | tests/test_mcp_snapshot_run_directory.py | 11 | CODE | |
| LOW | tests/test_issue702_streaming_over_val_leak.py | 21 | CODE | |
| LOW | tests/test_issue1450_atomic_replace.py | 24 | CODE | |
| LOW | tests/test_v0650_part_b.py | 6 | CODE | |
| LOW | tests/test_issue1236_packing_loss_mask.py | 12 | CODE | |
| LOW | tests/test_issue805_optimizer_settings.py | 3 | CODE | |
| LOW | tests/test_issue836_bench_train_contract.py | 19 | CODE | |
| LOW | tests/test_issue835_fp8_gate.py | 18 | CODE | |
| LOW | tests/test_graddiff_acceptance.py | 1 | CODE | |
| LOW | tests/test_issue1433_expect_judge.py | 7 | CODE | |
| LOW | tests/test_part_a_wave1.py | 12 | CODE | |
| LOW | tests/test_serve_non_loopback_notice.py | 17 | CODE | |
| LOW | tests/test_issue599_telemetry_dns_ssrf.py | 12 | CODE | |
| LOW | tests/test_v0640_followups.py | 20 | CODE | |
| LOW | tests/test_issue1266_mole_gate_fp32_master.py | 35 | CODE | |
| LOW | tests/test_v0430_part_b.py | 2 | CODE | |
| LOW | tests/test_issue1195_ui_auth_non_ascii.py | 13 | CODE | |
| LOW | tests/test_issue1011_streamed_save_guard.py | 13 | CODE | |
| LOW | tests/test_code_review_medium_low.py | 3 | CODE | |
| LOW | tests/test_v0660_part_e.py | 25 | CODE | |
| LOW | tests/test_issue649_batch_probe_peak_gate.py | 13 | CODE | |
| LOW | tests/test_issue812_eval_against_results.py | 3 | CODE | |
| LOW | tests/test_issue1216_canary_formats.py | 9 | CODE | |
| 1319 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH⚡ | tests/test_cans.py | 80 | eid = store.push( | CODE |
| HIGH⚡ | tests/test_cans.py | 172 | eid = store.push( | CODE |
| HIGH | tests/test_cans.py | 215 | eid = store.push( | CODE |
| HIGH | tests/test_cans.py | 269 | eid = store.push(name="recipe", tag="v1", base_model="llama", | CODE |
| HIGH | tests/test_issue549_bon_resume.py | 366 | '{"index":1,"sft":{},"dpo":null}\n', | CODE |
| HIGH⚡ | tests/test_issue1162_quest_base_identity.py | 302 | '{"auto_map":{"AutoTokenizer":["custom.Tokenizer",null]}}', encoding="utf-8" | CODE |
| HIGH⚡ | tests/test_registry.py | 215 | e1 = store.push(name="model-a", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 217 | e2 = store.push(name="model-a", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 232 | store.push(name=f"m{i}", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 240 | store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 242 | store.push(name="m2", tag="v2", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 249 | store.push(name="m1", tag="v1", base_model="llama-3.1-8b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 251 | store.push(name="m2", tag="v1", base_model="qwen-3-8b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 259 | store.push(name="medical-chat", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 261 | store.push(name="legal-chat", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 269 | store.push(name="Medical-Chat", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 276 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 284 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 290 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 296 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 320 | store.push(name="m1", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 322 | store.push(name="m2", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 333 | store.push(name="m1", tag="v1", base_model="", | CODE |
| HIGH⚡ | tests/test_registry.py | 340 | store.push(name="m1", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 348 | eid = store.push( | CODE |
| HIGH⚡ | tests/test_registry.py | 360 | store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 362 | store.push(name="m2", tag="v1", base_model="b", task="dpo", | CODE |
| HIGH⚡ | tests/test_registry.py | 372 | store.push(name=f"m{i}", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 380 | store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 399 | parent = store.push(name="parent", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 401 | child = store.push(name="child", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 437 | eid = store.push(name="m1", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 451 | store.push(name="m1", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 468 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 487 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 501 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 520 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 533 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 553 | parent = store.push(name="base", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 555 | child = store.push(name="fork", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 570 | a = store.push(name="a", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 572 | b = store.push(name="b", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 582 | a = store.push(name="a", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 593 | a = store.push(name="a", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 595 | b = store.push(name="b", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 607 | parent = store.push(name="p", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 618 | child = store.push(name="c", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 860 | eid = store.push(name="m1", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 875 | eid = store.push(name="m1", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 911 | eid = store.push(name="m1", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 953 | parent = store.push(name="base", tag="v1", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 955 | child = store.push(name="base", tag="v2", base_model="llama", | CODE |
| HIGH⚡ | tests/test_registry.py | 986 | eid = store.push(name="m1", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 1005 | store.push(name="clean", tag="v1", base_model="b", task="sft", | CODE |
| HIGH⚡ | tests/test_registry.py | 1020 | store.push(name="alpha", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 1022 | store.push(name="beta", tag="v1", base_model="b", | CODE |
| HIGH⚡ | tests/test_registry.py | 1034 | store.push(name="m1", tag="v1", base_model="b", | CODE |
| HIGH | tests/test_registry.py | 193 | entry_id = store.push( | CODE |
| HIGH | tests/test_registry.py | 809 | eid = store.push(name="m1", tag="v1", base_model="llama", | CODE |
| HIGH | tests/test_registry.py | 830 | store.push(name="medical-chat", tag="v1", base_model="llama", | CODE |
| 108 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | tests/test_issue359_deepspeed_guard_coverage.py | 0 | strip trailing comments so prose about the call is not read as a call. | STRING |
| HIGH | tests/test_issue725_lorafa_wiring_coverage.py | 0 | strip trailing comments so prose about the call is not read as a call. | STRING |
| HIGH | tests/test_issue724_loraplus_wiring_coverage.py | 0 | strip trailing comments so prose about the call is not read as a call. | STRING |
| HIGH | tests/test_issue359_deepspeed_guard_coverage.py | 0 | without this, a moved source tree turns every check below into a vacuous pass over an empty file list. | STRING |
| HIGH | tests/test_issue725_lorafa_wiring_coverage.py | 0 | without this, a moved source tree turns every check below into a vacuous pass over an empty file list. | STRING |
| HIGH | tests/test_issue724_loraplus_wiring_coverage.py | 0 | without this, a moved source tree turns every check below into a vacuous pass over an empty file list. | STRING |
| HIGH | tests/test_device_map_distributed.py | 0 | without this, a moved source tree turns every check below into a vacuous pass over an empty file list. | STRING |
| HIGH | tests/test_serve_non_loopback_notice.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_issue814_reward_stress.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_autopilot.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_migrate.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_train_gpus_exit_code.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_issue808_staged_config_fields.py | 0 | ansi-stripped, whitespace-collapsed cli output, safe to substring-match. | STRING |
| HIGH | tests/test_issue1222_refuse_int8_qat.py | 0 | rich colours per character and wraps, so compare stripped + collapsed. | STRING |
| HIGH | tests/test_issue1240_refuse_longlora.py | 0 | rich colours per character and wraps, so compare stripped + collapsed. | STRING |
| HIGH | tests/test_issue1613_early_stripe_roots.py | 0 | rich colours per character and wraps, so compare stripped + collapsed. | STRING |
| HIGH | tests/test_issue1230_refuse_ebft.py | 0 | rich colours per character and wraps, so compare stripped + collapsed. | STRING |
| HIGH | tests/test_eval_platform.py | 0 | remove ansi escape codes from rich-formatted output. | STRING |
| HIGH | tests/test_speculative_decoding.py | 0 | remove ansi escape codes from rich-formatted output. | STRING |
| HIGH | tests/test_synth_data_pro.py | 0 | remove ansi escape codes from rich-formatted output. | STRING |
| HIGH | tests/test_inference_advanced.py | 0 | remove ansi escape codes from rich-formatted output. | STRING |
| HIGH | tests/test_issue724_loraplus_optimizer.py | 0 | a real config-shaped object (not a mock): missing attributes raise. | STRING |
| HIGH | tests/test_issue745_ppo_loraplus_scheduler.py | 0 | a real config-shaped object (not a mock): missing attributes raise. | STRING |
| HIGH | tests/test_issue725_lorafa_optimizer.py | 0 | a real config-shaped object (not a mock): missing attributes raise. | STRING |
| HIGH | tests/test_pretrain.py | 0 | test the runtimeerror guard when train() is called before setup(). | STRING |
| HIGH | tests/test_kto.py | 0 | test the runtimeerror guard when train() is called before setup(). | STRING |
| HIGH | tests/test_orpo.py | 0 | test the runtimeerror guard when train() is called before setup(). | STRING |
| HIGH | tests/test_embedding.py | 0 | test the runtimeerror guard when train() is called before setup(). | STRING |
| HIGH | tests/test_pretrain.py | 0 | calling train() before setup() should raise runtimeerror. | STRING |
| HIGH | tests/test_kto.py | 0 | calling train() before setup() should raise runtimeerror. | STRING |
| HIGH | tests/test_embedding.py | 0 | calling train() before setup() should raise runtimeerror. | STRING |
| HIGH | tests/test_pretrain.py | 0 | test the result dict returned by train() using a mocked trainer. | STRING |
| HIGH | tests/test_kto.py | 0 | test the result dict returned by train() using a mocked trainer. | STRING |
| HIGH | tests/test_orpo.py | 0 | test the result dict returned by train() using a mocked trainer. | STRING |
| HIGH | tests/test_embedding.py | 0 | test the result dict returned by train() using a mocked trainer. | STRING |
| HIGH | tests/test_pretrain.py | 0 | train() result dict must contain all expected keys. | STRING |
| HIGH | tests/test_kto.py | 0 | train() result dict must contain all expected keys. | STRING |
| HIGH | tests/test_embedding.py | 0 | train() result dict must contain all expected keys. | STRING |
| HIGH | tests/test_pretrain.py | 0 | initial_loss and final_loss should come from trainer log_history. | STRING |
| HIGH | tests/test_kto.py | 0 | initial_loss and final_loss should come from trainer log_history. | STRING |
| HIGH | tests/test_embedding.py | 0 | initial_loss and final_loss should come from trainer log_history. | STRING |
| HIGH | tests/test_pretrain.py | 0 | when log_history has no 'loss' entries, losses should be 0. | STRING |
| HIGH | tests/test_kto.py | 0 | when log_history has no 'loss' entries, losses should be 0. | STRING |
| HIGH | tests/test_embedding.py | 0 | when log_history has no 'loss' entries, losses should be 0. | STRING |
| HIGH | tests/test_pretrain.py | 0 | total_steps should match trainer.state.global_step. | STRING |
| HIGH | tests/test_kto.py | 0 | total_steps should match trainer.state.global_step. | STRING |
| HIGH | tests/test_embedding.py | 0 | total_steps should match trainer.state.global_step. | STRING |
| HIGH | tests/test_pretrain.py | 0 | output_dir in result should match wrapper._output_dir. | STRING |
| HIGH | tests/test_kto.py | 0 | output_dir in result should match wrapper._output_dir. | STRING |
| HIGH | tests/test_embedding.py | 0 | output_dir in result should match wrapper._output_dir. | STRING |
| HIGH | tests/test_pretrain.py | 0 | train() should call trainer.save_model with output_dir. | STRING |
| HIGH | tests/test_kto.py | 0 | train() should call trainer.save_model with output_dir. | STRING |
| HIGH | tests/test_embedding.py | 0 | train() should call trainer.save_model with output_dir. | STRING |
| HIGH | tests/test_pretrain.py | 0 | train() should call tokenizer.save_pretrained with output_dir. | STRING |
| HIGH | tests/test_kto.py | 0 | train() should call tokenizer.save_pretrained with output_dir. | STRING |
| HIGH | tests/test_embedding.py | 0 | train() should call tokenizer.save_pretrained with output_dir. | STRING |
| HIGH | tests/test_pretrain.py | 0 | train() should forward resume_from_checkpoint to trainer.train(). | STRING |
| HIGH | tests/test_kto.py | 0 | train() should forward resume_from_checkpoint to trainer.train(). | STRING |
| HIGH | tests/test_embedding.py | 0 | train() should forward resume_from_checkpoint to trainer.train(). | STRING |
| HIGH | tests/test_pretrain.py | 0 | long durations (>=1h) should produce 'xh ym' format. | STRING |
| 49 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_issue1449_trust_remote_code_dropped.py | 458 | except Exception: | CODE |
| LOW | tests/test_issue1449_trust_remote_code_dropped.py | 370 | except Exception: | CODE |
| LOW | tests/test_issue1449_trust_remote_code_dropped.py | 398 | except Exception: | CODE |
| LOW | tests/test_issue1449_trust_remote_code_dropped.py | 428 | except Exception: | CODE |
| LOW | tests/test_issue681_uld_id_compat.py | 185 | except Exception: | CODE |
| LOW⚡ | tests/test_v07204.py | 37 | except Exception: # pragma: no cover - torch always present in CI | CODE |
| LOW⚡ | tests/conftest.py | 25 | except Exception: | CODE |
| LOW⚡ | tests/conftest.py | 29 | except Exception: # noqa: BLE001 — broken CUDA install or driver | CODE |
| LOW | tests/conftest.py | 57 | except Exception: # noqa: BLE001 — a broken CUDA install counts as no CUDA | CODE |
| LOW | tests/conftest.py | 71 | except Exception: # noqa: BLE001 — a broken torch install counts as no MPS | CODE |
| LOW⚡ | tests/test_v05311.py | 1528 | except Exception as exc: # noqa: BLE001 — a broken trl is not a failure | CODE |
| LOW | tests/test_v05311.py | 1239 | except Exception as exc: # noqa: BLE001 — a broken trl is not a failure | CODE |
| MEDIUM | tests/test_issue775_path_containment_ratchet.py | 314 | def _validate(target, base): | CODE |
| LOW⚡ | tests/test_issue775_path_containment_ratchet.py | 383 | except Exception: | CODE |
| LOW | tests/test_issue1099_moe_lora_remaining_tasks.py | 215 | except Exception as exc: # noqa: BLE001 — setup continues past the LoRA attach | CODE |
| LOW | tests/test_issue691_sft_packing.py | 38 | except Exception: | CODE |
| LOW | tests/test_trl_preference_config_contract.py | 298 | except Exception as exc: # noqa: BLE001 - reported, not swallowed | CODE |
| MEDIUM | tests/test_v0532.py | 256 | def _torch_or_skip(): | CODE |
| LOW⚡ | tests/test_v0532.py | 261 | except Exception: # pragma: no cover - CI without torch | CODE |
| LOW | tests/test_errors.py | 293 | except Exception as caught: | CODE |
| LOW | tests/test_why.py | 101 | except Exception: | CODE |
| LOW | tests/test_issue968_bnb_nf4_path_divergence.py | 60 | except Exception: | CODE |
| MEDIUM | tests/test_issue968_bnb_nf4_path_divergence.py | 55 | def _bnb_version() -> str: | CODE |
| LOW | tests/test_v0537.py | 1128 | except Exception as e: | CODE |
| LOW | tests/test_issue1441_ppo_refuses_unfed_rl_flags.py | 221 | except Exception as exc: # noqa: BLE001 — anything but the refusal | CODE |
| LOW | tests/test_issue1070_moe_target_modules.py | 595 | except Exception: # noqa: BLE001 -- setup carries on past the attach | CODE |
| LOW | tests/test_v0712.py | 487 | except Exception as exc: # noqa: BLE001 | CODE |
| LOW | tests/test_part_d.py | 113 | except Exception: # tokenizer mock approximation may explode in decode | CODE |
| LOW⚡ | tests/test_recipes.py | 87 | except Exception as exc: | CODE |
| LOW | tests/test_issue1208_simpo_zero_token_completion.py | 305 | except Exception as exc: # noqa: BLE001 — a broken trl is not a failure | CODE |
| LOW | tests/test_issue798_moe_flags_per_task.py | 191 | except Exception as exc: # noqa: BLE001 — setup continues past the LoRA attach | CODE |
| LOW | tests/test_v0540.py | 971 | except Exception as exc: # noqa: BLE001 | CODE |
| LOW | tests/test_trl_version_compat.py | 244 | except Exception as exc: # noqa: BLE001 - reported, not swallowed | CODE |
| LOW | tests/test_issue1361_auth_status_text_chain.py | 58 | except Exception: | CODE |
| LOW⚡ | tests/test_issue336_deepspeed_lora.py | 363 | except Exception: # noqa: BLE001 - any other refusal still counts as strict | CODE |
| LOW | tests/test_issue839_fast_lora_single_projection.py | 400 | except Exception as exc: # noqa: BLE001 | CODE |
| MEDIUM | tests/test_issue839_fast_lora_single_projection.py | 397 | def run(): | CODE |
| LOW | benchmarks/bench_validator.py | 108 | except Exception: | CODE |
| LOW | benchmarks/harness/two_drive_stripe_gate.py | 245 | except Exception as exc: # a diagnostic must never fail the arm | CODE |
| LOW | benchmarks/harness/two_drive_stripe_gate.py | 316 | except Exception as exc: # a diagnostic must never fail the arm | CODE |
| LOW⚡ | benchmarks/harness/issue1702_arena_pack_gate.py | 252 | except Exception as exc: # the stamp is context, never a reason to stop a series | CODE |
| LOW | benchmarks/harness/issue1702_arena_pack_gate.py | 471 | except Exception as exc: | CODE |
| LOW | benchmarks/harness/issue1702_arena_pack_gate.py | 475 | except Exception as exc: | CODE |
| LOW⚡ | benchmarks/harness/issue1702_arena_pack_gate.py | 488 | except Exception as exc: | CODE |
| LOW | benchmarks/harness/bitexact.py | 56 | except Exception: | CODE |
| LOW | benchmarks/harness/bitexact.py | 656 | except Exception as exc: | CODE |
| MEDIUM | benchmarks/harness/bitexact.py | 51 | def cuda_available() -> bool: | CODE |
| LOW | benchmarks/harness/graddiff.py | 404 | except Exception as exc: | CODE |
| LOW | benchmarks/harness/graddiff.py | 739 | except Exception as exc: | CODE |
| LOW | benchmarks/harness/fixture_window_cpu.py | 308 | except Exception: | CODE |
| LOW | benchmarks/harness/fixture_window_cpu.py | 335 | except Exception: | CODE |
| MEDIUM | benchmarks/harness/fixture_window_cpu.py | 326 | def _cpu_gemv_registered() -> bool | None: | CODE |
| MEDIUM | benchmarks/harness/stream_probe.py | 62 | def cuda_available() -> bool: | CODE |
| LOW | benchmarks/harness/stream_probe.py | 67 | except Exception: | CODE |
| LOW | benchmarks/harness/stream_probe.py | 830 | except Exception as close_exc: # the OOM is the result; a failed close must not hide it | CODE |
| LOW | benchmarks/harness/stream_probe.py | 1180 | except Exception as exc: # re-raised unless it is an out-of-memory error | CODE |
| LOW | benchmarks/harness/stream_probe.py | 1201 | except Exception as exc: # re-raised unless it is an out-of-memory error | CODE |
| LOW | benchmarks/harness/stream_probe.py | 1261 | except Exception as exc: # re-raised unless it is an out-of-memory error | CODE |
| LOW | benchmarks/harness/synth_checkpoint.py | 104 | except Exception: # the generator needs only torch | CODE |
| LOW | benchmarks/harness/issue901_stream_step_probe.py | 193 | except Exception as exc: # catching the OOM is the point | CODE |
| 455 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_v07133.py | 939 | CODE | |
| LOW | tests/test_v0538.py | 509 | CODE | |
| LOW | tests/test_terminal_output_hygiene.py | 69 | CODE | |
| LOW | tests/test_terminal_output_hygiene.py | 89 | CODE | |
| LOW | tests/test_issue823_stale_feature_descriptions.py | 131 | CODE | |
| LOW | tests/test_issue1266_mole_gate_fp32_master.py | 99 | CODE | |
| LOW | tests/test_issue1195_ui_auth_non_ascii.py | 160 | CODE | |
| LOW | tests/test_issue896_no_duplicate_terminal_sanitizer.py | 62 | CODE | |
| LOW | tests/test_readme_translation_sync.py | 193 | CODE | |
| LOW | tests/test_readme_translation_sync.py | 229 | CODE | |
| LOW | tests/test_readme_translation_sync.py | 247 | CODE | |
| LOW | tests/test_issue1124_sft_audio_stream_unsloth_fp8.py | 181 | CODE | |
| LOW | tests/test_issue1005_peft021_canonical_names.py | 199 | CODE | |
| LOW | tests/test_v07132.py | 160 | CODE | |
| LOW | tests/test_v07201.py | 87 | CODE | |
| LOW | tests/test_awq_gptq_export.py | 315 | CODE | |
| LOW | tests/test_awq_gptq_export.py | 393 | CODE | |
| LOW | tests/test_awq_gptq_export.py | 557 | CODE | |
| LOW | tests/test_awq_gptq_export.py | 705 | CODE | |
| LOW | tests/test_issue1547_outbound_http_sinks.py | 287 | CODE | |
| LOW | tests/test_issue1547_outbound_http_sinks.py | 319 | CODE | |
| LOW | tests/test_issue1547_outbound_http_sinks.py | 342 | CODE | |
| LOW | tests/test_issue1547_outbound_http_sinks.py | 493 | CODE | |
| LOW | tests/test_issue1547_outbound_http_sinks.py | 612 | CODE | |
| LOW | tests/test_issue616_net_guard_is_shared.py | 290 | CODE | |
| LOW | tests/test_build_dag.py | 21 | CODE | |
| LOW | tests/test_issue1219_classifier_data_loader.py | 94 | CODE | |
| LOW | tests/test_issue1219_classifier_data_loader.py | 112 | CODE | |
| LOW | tests/test_issue1480_machine_output_rich_json.py | 168 | CODE | |
| LOW | tests/test_issue755_backend_support_registry.py | 151 | CODE | |
| LOW | tests/test_issue755_backend_support_registry.py | 555 | CODE | |
| LOW | tests/test_issue724_loraplus_wiring_coverage.py | 156 | CODE | |
| LOW | tests/test_issue602_qwen4_ple_streaming.py | 69 | CODE | |
| LOW | tests/test_issue752_cli_commands_called_as_functions.py | 83 | CODE | |
| LOW | tests/test_issue1522_judge_down.py | 103 | CODE | |
| LOW | tests/test_issue802_callback_kwargs_in_trainers.py | 65 | CODE | |
| LOW | tests/test_issue802_callback_kwargs_in_trainers.py | 183 | CODE | |
| LOW | tests/test_issue1220_resume_prm_moe.py | 66 | CODE | |
| LOW | tests/test_issue1220_resume_prm_moe.py | 88 | CODE | |
| LOW | tests/test_issue1497_freeze_refused_outside_sft.py | 197 | CODE | |
| LOW | tests/test_issue844_notebook_pin_guard.py | 141 | CODE | |
| LOW | tests/test_issue674_postfit_evidence.py | 39 | CODE | |
| LOW | tests/test_commands_md_covers_registered_commands.py | 41 | CODE | |
| LOW | tests/test_onnx_tensorrt_export.py | 80 | CODE | |
| LOW | tests/test_issue1232_grpo_variant_kl.py | 140 | CODE | |
| LOW | tests/test_ci_full_matrix_on_approval.py | 248 | CODE | |
| LOW | tests/test_issue979_documented_cli_flags_exist.py | 60 | CODE | |
| LOW | tests/test_issue865_langfuse_pull_bounds.py | 90 | CODE | |
| LOW | tests/test_issue865_langfuse_pull_bounds.py | 99 | CODE | |
| LOW | tests/test_trainer_coverage_v035.py | 42 | CODE | |
| LOW | tests/test_trainer_coverage_v035.py | 226 | CODE | |
| LOW | tests/test_issue550_bon_offline.py | 446 | CODE | |
| LOW | tests/test_issue748_config_fields_reach_a_consumer.py | 134 | CODE | |
| LOW | tests/test_issue748_config_fields_reach_a_consumer.py | 231 | CODE | |
| LOW | tests/test_issue748_config_fields_reach_a_consumer.py | 898 | CODE | |
| LOW | tests/test_issue748_config_fields_reach_a_consumer.py | 938 | CODE | |
| LOW | tests/test_issue781_inference_bos.py | 641 | CODE | |
| LOW | tests/test_config_regex_limits.py | 235 | CODE | |
| LOW | tests/test_issue1465_documented_invocations_exist.py | 83 | CODE | |
| LOW | tests/test_issue1465_documented_invocations_exist.py | 136 | CODE | |
| 340 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_issue815_canary_runtime.py | 21 | def set_adapter(self, name: str) -> None: | CODE |
| LOW⚡ | tests/test_issue971_async_disk_source.py | 1040 | def _settle(source, timeout: float = 10.0) -> None: | CODE |
| LOW | benchmarks/harness/two_drive_aspm_probe.py | 98 | def set_aspm_ac(index: int) -> None: | CODE |
| LOW | benchmarks/harness/fsdp_sharding_probe.py | 61 | __all__ = [ | CODE |
| LOW | benchmarks/harness/head_prefetch_ab.py | 283 | def set_arm(arm: Optional[int]) -> None: | CODE |
| LOW | src/soup_cli/ui/app.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/ui/app.py | 305 | def set_auth_token(token: str) -> None: | CODE |
| LOW | src/soup_cli/autodistill/__init__.py | 27 | __all__ = [ | CODE |
| LOW | src/soup_cli/config/unknown_keys.py | 49 | __all__ = [ | CODE |
| LOW | src/soup_cli/config/staged_fields.py | 17 | __all__ = [ | CODE |
| LOW | src/soup_cli/plugins/__init__.py | 41 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/plugins/__init__.py | 532 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/tool_outputs.py | 138 | def set_output(self, text: str) -> None: | CODE |
| LOW | src/soup_cli/utils/tool_outputs.py | 143 | def set_error(self, text: str) -> None: | CODE |
| LOW | src/soup_cli/utils/prune_prompt.py | 440 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/seeding.py | 39 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/ci_workflow.py | 250 | __all__ = ["render_soup_gate_workflow", "write_soup_gate_workflow"] | CODE |
| LOW | src/soup_cli/utils/data_forge.py | 793 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/local_rl_scheduler.py | 301 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/fast_lora_mlp.py | 24 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/fast_lora_mlp.py | 31 | __all__ = ["patch_fast_lora_mlp", "unpatch_fast_lora_mlp"] | CODE |
| LOW | src/soup_cli/utils/ra_dit_run.py | 46 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/prompt_distill.py | 346 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/data_doctor.py | 942 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/reward_hack_control.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/prompt_compile.py | 482 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/tracing.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/persona_hub.py | 268 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/layer_shard.py | 50 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/mix_proxy.py | 43 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/curriculum_dynamic.py | 35 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/vllm.py | 10 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/magpie.py | 577 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/data_pipeline.py | 1006 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/embed.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/apple_adapter.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/apple_adapter.py | 564 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/reward_hacking.py | 44 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/drift_alarm.py | 243 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/anthropic_messages.py | 210 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/deploy_autopilot.py | 384 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/prm_reward.py | 490 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/hardware_fit.py | 347 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/fast_lora_qkv.py | 39 | __all__ = ["patch_fast_lora_qkv", "unpatch_fast_lora_qkv"] | CODE |
| LOW | src/soup_cli/utils/multipack_trainer.py | 34 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/server_tools.py | 169 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/license_advisor.py | 321 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/tunability.py | 620 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/interference_live.py | 163 | __all__ = ["measure_interference_losses"] | CODE |
| LOW | src/soup_cli/utils/terraform_plan.py | 413 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/grpo_long_context.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/dpo_variants.py | 37 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/final_answer.py | 81 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/peft_wiring.py | 19 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/mii.py | 15 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/ingest_pull.py | 583 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/ngram_spec.py | 65 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/fsdp_consolidate.py | 30 | logger = logging.getLogger(__name__) | CODE |
| LOW | src/soup_cli/utils/completions.py | 294 | __all__ = [ | CODE |
| LOW | src/soup_cli/utils/build_dag.py | 1025 | __all__ = [ | CODE |
| 107 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | pyproject.toml | 136 | # Aider's normal CLI. Its Polyglot harness is source-only, so `soup eval aider` | COMMENT |
| LOW⚡ | tests/test_registry.py | 1007 | # SQL injection attempt — should simply return 0 hits, not crash | COMMENT |
| LOW | tests/test_issue443_interleave_wiring.py | 446 | # the real process cwd — must actually chdir, not just pass tmp_path. | COMMENT |
| MEDIUM | tests/test_issue814_reward_stress.py | 128 | # pinned. (`accuracy` itself is now robust: tests/test_issue1226_reward_gold_parsing.py.) | STRING |
| MEDIUM | tests/test_issue814_reward_stress.py | 211 | # robust families (0% acceptance). Under pooled aggregate denominator this would | STRING |
| MEDIUM⚡ | tests/test_issue683_mlx_response_masking.py | 529 | # live in `mlx_sft.py`, where the feature is connected. The fake-MLX harness | COMMENT |
| MEDIUM⚡ | tests/test_issue683_mlx_response_masking.py | 609 | # `CacheDataset` is identity in the fake harness, so the dataset that | COMMENT |
| MEDIUM⚡ | tests/test_issue23_mlx_harness_bridge.py | 142 | # ansi-ok: harness bridge formatted text, not CLI console | COMMENT |
| MEDIUM⚡ | tests/test_issue23_mlx_harness_bridge.py | 144 | # ansi-ok: harness bridge formatted text, not CLI console | COMMENT |
| MEDIUM⚡ | tests/test_issue968_bnb_nf4_path_divergence.py | 78 | # harness — deliberately a copy of the neighbour's, not an import of it. | COMMENT |
| LOW | tests/test_doctor.py | 1010 | # We mock mp.spawn to just set a value in the return_dict instead of actually running processes. | COMMENT |
| MEDIUM | tests/test_v0401_part_d.py | 1 | """v0.40.1 Part D — CLI UX consistency tests (highest-leverage subset). | STRING |
| MEDIUM | tests/test_issue716_harness_step_guard.py | 1 | """#716: the MLX benchmark harness refuses a run whose step counts it did not pin. | STRING |
| LOW⚡ | tests/test_inference_advanced.py | 482 | # Should not raise, just return None + log | COMMENT |
| MEDIUM⚡ | tests/test_v0713.py | 33 | # Rich's FORCE_COLOR splitting on CI (mirrors the v0.71.1 ANSI-robust fix: | COMMENT |
| MEDIUM | tests/test_issue1265_ab_nig.py | 1 | """#1265 — `soup ab` decides with a variance-robust statistic and needs no burn-in. | STRING |
| MEDIUM | tests/test_issue1265_ab_nig.py | 1 | """#1265 — `soup ab` decides with a variance-robust statistic and needs no burn-in. | STRING |
| LOW | tests/test_bugfixes.py | 946 | # resume is None/falsy so it should just call train() | COMMENT |
| MEDIUM⚡ | tests/test_issue1702_arena_pack_gate.py | 256 | # Registered first, so the harness's process-wide switch is undone after the test. | COMMENT |
| MEDIUM | tests/test_v0404_part_a.py | 27 | # Strip ANSI so substring assertions are robust. Mirrors the helper in | COMMENT |
| MEDIUM⚡ | tests/test_trust_remote_code.py | 19 | # ANSI so substring assertions are robust. Mirrors the helper in | COMMENT |
| MEDIUM | tests/test_issue23_mlx_val_loss_producer.py | 320 | # a measured value to carry. (The default harness order is mlx-lm's | COMMENT |
| MEDIUM | tests/test_log_level.py | 23 | # substring assertions are robust across all CI matrix jobs. | COMMENT |
| MEDIUM⚡ | tests/test_issue686_mlx_optimizer_schedule.py | 306 | # design is that one expression, and nothing reached it. Same fake-MLX harness | COMMENT |
| MEDIUM⚡ | tests/test_issue686_mlx_optimizer_schedule.py | 313 | # The shared fake-MLX harness, imported rather than re-implemented -- the same | COMMENT |
| MEDIUM | tests/test_issue686_mlx_optimizer_schedule.py | 407 | # fake-MLX harness, so they run on Linux and Windows too. | COMMENT |
| MEDIUM | tests/test_issue379_variant2_gate.py | 1 | """#379: publish the STEP 14 control-versus-repair measurement harness. | STRING |
| MEDIUM⚡ | tests/test_v0718.py | 44 | # Tiny real-torch harness for activation capture (#215 / #217 / #219). | COMMENT |
| MEDIUM | tests/test_issue361_nf4_throughput.py | 37 | # The harness imports its sibling ``source_provenance`` (#1387), as it does | COMMENT |
| MEDIUM | tests/test_issue361_nf4_throughput.py | 38 | # when run as a script from benchmarks/harness. | COMMENT |
| MEDIUM⚡ | tests/test_issue361_nf4_throughput.py | 412 | headline predates the #331 repair and name this harness as the protocol, | STRING |
| MEDIUM | tests/test_v07203.py | 2734 | # Omitted rather than defaulted, so the harness exercises the production | COMMENT |
| MEDIUM⚡ | tests/test_v07141.py | 179 | # zip short-circuits) — must be a hard error, never a silent 0/1=robust. | COMMENT |
| MEDIUM | tests/test_v07141.py | 347 | # false "robust" exit 0. | STRING |
| MEDIUM | tests/test_issue1387_source_provenance.py | 1 | """#1387: one ``source_sha()`` for every benchmark harness that records its tree. | STRING |
| MEDIUM | benchmarks/harness/fixture_window_cpu.py | 235 | # A broken control is a harness/Soup failure on every host; do not hide it | COMMENT |
| MEDIUM | benchmarks/harness/stream_probe.py | 82 | # entries at the same physical drive is on the operator running this harness. | COMMENT |
| MEDIUM | benchmarks/harness/issue361_nf4_throughput.py | 6 | reproduction) — was measured BEFORE the #331 repair. This harness re-runs that | STRING |
| MEDIUM | benchmarks/harness/two_drive_sustained.py | 240 | out["windows_unix"] = None # a harness without per-step start times | CODE |
| MEDIUM⚡ | benchmarks/harness/mlx_sft_smoke.py | 193 | # This harness backs a published throughput record, so it pins the setting | COMMENT |
| MEDIUM⚡ | benchmarks/harness/mlx_sft_smoke.py | 198 | # here, with the reason, because this harness backs a published throughput | COMMENT |
| MEDIUM⚡ | benchmarks/harness/mlx_sft_smoke.py | 200 | # record depend on a value the harness never mentions, which is #716. | COMMENT |
| MEDIUM | benchmarks/harness/bnb_repro.py | 263 | # A successful reproduction here would mean the harness is not | COMMENT |
| MEDIUM | benchmarks/harness/bnb_repro.py | 282 | # is removed or bypassed, the comparison becomes equal and the harness | COMMENT |
| MEDIUM | .github/workflows/ci.yml | 279 | # benchmarks/harness/ carries real parsing logic — mlx_sft_smoke.py | COMMENT |
| MEDIUM | src/soup_cli/cli.py | 352 | # v0.63.0 Part D — mSPRT A/B harness. | COMMENT |
| MEDIUM | src/soup_cli/utils/data_clean.py | 103 | # Strip prefix boilerplate (loop to catch chained preambles like "Certainly! As an AI...") | COMMENT |
| MEDIUM | src/soup_cli/utils/curriculum_dynamic.py | 263 | # Median is robust to outliers; matches Axolotl curriculum policy. | COMMENT |
| MEDIUM | src/soup_cli/utils/capability_suite.py | 249 | # Lazy-load the harness via importlib so the module has no heavy | COMMENT |
| MEDIUM | src/soup_cli/utils/async_disk_source.py | 796 | Still the reader's ONE call per layer read, the seam #974's harness brackets. Since | STRING |
| MEDIUM | src/soup_cli/utils/layer_stream_runtime.py | 1277 | # point. It is a property, so a harness can flip it between steps and | COMMENT |
| MEDIUM | src/soup_cli/utils/freeze.py | 117 | # (quantization='none') normally prevents this, but stay robust. | COMMENT |
| MEDIUM | src/soup_cli/utils/preference_combine.py | 52 | #: attached without a config — an existing test harness, say — lands on the | COMMENT |
| MEDIUM | src/soup_cli/commands/reward.py | 333 | """Adversarially probe a reward verifier for gameability (exit 0 robust / 2 gameable).""" | STRING |
| MEDIUM | src/soup_cli/commands/doctor.py | 1195 | # Per-iteration timing (10 samples). Median is more robust to | COMMENT |
| MEDIUM | src/soup_cli/commands/doctor.py | 1163 | _NCCL_BENCHMARK_TIMED_ITERS = 10 # 10 timed runs; report the median (robust to outliers) | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .pre-commit-config.yaml | 1 | # Pre-commit hooks for Soup CLI. | COMMENT |
| LOW | .pre-commit-config.yaml | 21 | # 2. run `ruff format src/soup_cli/ tests/` and commit the result alone, | COMMENT |
| LOW | pyproject.toml | 41 | # Soup supports both the 5.x and 6.x surfaces through plotext_compat (#522). | COMMENT |
| LOW | pyproject.toml | 61 | "torch>=2.6.0", | COMMENT |
| LOW | pyproject.toml | 81 | # 0.27.0 - 0.27.2 yes NO yes yes yes | COMMENT |
| LOW | pyproject.toml | 101 | # DPO/GRPO modules from importing (#503). trl 0.29 vendors its own probe and | COMMENT |
| LOW | pyproject.toml | 121 | # Soup now declares what its trainers need instead of inheriting it. The | COMMENT |
| LOW | pyproject.toml | 181 | data-pro = ["langdetect>=1.0.9", "presidio-analyzer>=2.2.0"] | COMMENT |
| LOW | pyproject.toml | 201 | # lazy-imported (only src/soup_cli/mcp_server/server.py touches it), so the CLI | COMMENT |
| LOW | tests/test_v07200.py | 2581 | assert attempts == [True, False] | COMMENT |
| LOW | tests/test_issue1449_trust_remote_code_dropped.py | 281 | return SimpleNamespace(pad_token="x", eos_token="x") | COMMENT |
| LOW | tests/test_issue459_interleave_streaming_hub.py | 561 | streaming=True, | COMMENT |
| LOW | tests/test_issue683_mlx_response_masking.py | 381 | COMMENT | |
| LOW | tests/test_issue755_backend_support_registry.py | 421 | COMMENT | |
| LOW | tests/test_rewind_hf.py | 241 | COMMENT | |
| LOW | tests/test_issue723_sequence_gspo.py | 121 | # s_0 = exp(0.1) ~ 1.1051709 | COMMENT |
| LOW | tests/test_issue844_notebook_pin_guard.py | 201 | # huggingface_hub >= 1.32 keeps Xet blobs in a cache-wide store that | COMMENT |
| LOW | tests/test_issue1070_moe_target_modules.py | 401 | COMMENT | |
| LOW | tests/test_recipe_sizes.py | 1 | # Copyright 2026 Soup Authors. | COMMENT |
| LOW | tests/test_v0640_part_c.py | 721 | monkeypatch.chdir(tmp_path) | COMMENT |
| LOW | tests/test_issue748_config_fields_reach_a_consumer.py | 321 | # #807: a read inside a function nothing calls is not consumption, for | COMMENT |
| LOW | tests/test_issue686_mlx_optimizer_schedule.py | 301 | COMMENT | |
| LOW | tests/test_v07202.py | 1221 | #: branch taken only when ``not self.training and not x.requires_grad`` -- it | COMMENT |
| LOW | tests/test_v07202.py | 1241 | #: measured constant from one stack to another is the same mistake this project | COMMENT |
| LOW | tests/test_v07203.py | 41 | dict(label="Qwen2.5-0.5B B2 S512", batch=2, seq=512, peak=2592245248, **_QWEN), | COMMENT |
| LOW | tests/test_v07203.py | 601 | def test_rejects_a_non_finite_tflops(self): | COMMENT |
| LOW | tests/test_v07203.py | 2001 | #: which forced the tier by describing a box with a kilobyte of free RAM; | COMMENT |
| LOW | tests/test_cli_help_assertions_are_ansi_safe.py | 301 | # --------------------------------------------------------------------------- | COMMENT |
| LOW | docs/training.md | 441 | # Wasserstein-distance distillation between two models that SHARE a | COMMENT |
| LOW | docs/training.md | 461 | # Config-only: there is no --minillm-* flag beyond --minillm-on-policy below | COMMENT |
| LOW | docs/evaluation.md | 721 | # Custom eval tasks from JSONL | COMMENT |
| LOW | examples/configs/rlhf_step3_ppo.yaml | 1 | # RLHF Step 3: PPO Training | COMMENT |
| LOW | examples/configs/sft_basic.yaml | 1 | # SFT Basic Example | COMMENT |
| LOW | examples/configs/rlhf_step1_sft.yaml | 1 | # RLHF Step 1: Supervised Fine-Tuning (SFT) | COMMENT |
| LOW | examples/configs/grpo_reasoning.yaml | 1 | # GRPO Reasoning Example | COMMENT |
| LOW | examples/configs/vision_llama.yaml | 1 | # TEMPLATE — NOT runnable as-is. Bring your own images and JSONL. | COMMENT |
| LOW | examples/configs/rlhf_step2_reward.yaml | 1 | # RLHF Step 2: Train Reward Model | COMMENT |
| LOW | benchmarks/harness/mlx_sft_smoke.py | 181 | base: {model} | COMMENT |
| LOW | benchmarks/harness/mlx_sft_smoke.py | 201 | epochs: {epochs} | COMMENT |
| LOW | .github/FUNDING.yml | 1 | # Sponsor button shown in the repo sidebar and on every issue / PR. | COMMENT |
| LOW | .github/workflows/merge-freshness.yml | 1 | # #1017 — re-evaluate a PR against live main without writing to the fork. | COMMENT |
| LOW | .github/workflows/recipe-repo-ids.yml | 1 | # Recipe repo-id resolution (#677). | COMMENT |
| LOW | .github/workflows/dependency-drift.yml | 1 | # Dependency drift (#323). | COMMENT |
| LOW | .github/workflows/recipe-attach.yml | 1 | # Recipe attach ratchet (#1116). | COMMENT |
| LOW | .github/workflows/recipe-attach.yml | 81 | # and nothing from the test extras. | COMMENT |
| LOW | .github/workflows/issue-triage.yml | 1 | # Keep the triage labels of an issue consistent: | COMMENT |
| LOW | .github/workflows/recipe-validation.yml | 41 | # against a service rate of 4.6, i.e. the queue diverged rather than backed up. | COMMENT |
| LOW | .github/workflows/recipe-validation.yml | 61 | # that morning: 24 of 70 queued runs were full matrices for main commits (about | COMMENT |
| LOW | .github/workflows/codspeed.yml | 1 | name: CodSpeed | COMMENT |
| LOW | .github/workflows/codspeed.yml | 21 | # nothing to authenticate with. | COMMENT |
| LOW | .github/workflows/codspeed.yml | 41 | # variables > Actions > Variables). Remove or unset it to turn it off again. | COMMENT |
| LOW | .github/workflows/ci.yml | 1 | name: CI | COMMENT |
| LOW | .github/workflows/ci.yml | 21 | - cron: '23 3 * * *' | COMMENT |
| LOW | .github/workflows/ci.yml | 41 | # tagged, and that now survives a later push instead of depending on a freeze. | COMMENT |
| LOW | .github/workflows/ci.yml | 61 | # group a RUNNING main run still finishes (cancel-in-progress stays PR-only), and | COMMENT |
| LOW | .github/workflows/ci.yml | 81 | # matrix, before the platform intervenes at all. | COMMENT |
| LOW | .github/workflows/ci.yml | 101 | # job median max -> limit | COMMENT |
| LOW | .github/workflows/ci.yml | 121 | COMMENT | |
| LOW | .github/workflows/ci.yml | 141 | # request run | COMMENT |
| LOW | .github/workflows/ci.yml | 161 | # The pull request's current labels decide, not the event, so every run on a | COMMENT |
| 114 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | src/soup_cli/utils/fsdp.py | 64 | Get FSDP config dict by preset name. Args: preset: One of 'full_shard', 'shard_grad', 'full_offload'. | STRING |
| HIGH | src/soup_cli/utils/fsdp.py | 104 | Mutate and return ``training_kwargs`` with FSDP + optional torch.compile. Centralizes the "FSDP-block" logic from t | STRING |
| HIGH | src/soup_cli/utils/data_forge.py | 594 | Build a ``judge(prompt) -> {'text': str}`` callable for ``provider``. Args: provider: one of ``"ollama"`` / | STRING |
| HIGH | src/soup_cli/utils/ra_dit_run.py | 300 | Chain the two RA-DIT stages: retriever → auto-link → generator. Args: retriever_config: Path (under cwd) to | STRING |
| HIGH | src/soup_cli/utils/mix_proxy.py | 264 | Run one short proxy training and return the observed eval loss. Args: weights: Per-dataset interleave proba | STRING |
| HIGH | src/soup_cli/utils/fp8.py | 228 | Convert eligible linear layers to FP8 for training. Uses torchao's ``convert_to_float8_training`` with a scaling re | STRING |
| HIGH | src/soup_cli/utils/kernel_picker.py | 260 | Pick the fastest kernel combo given benchmarked timings. Args: candidates: List of dicts each with ``name`` | STRING |
| HIGH | src/soup_cli/utils/multipack_sampler.py | 160 | First-Fit-Decreasing bin packing. Args: lengths: per-sample sequence lengths. max_len: maximum sum | STRING |
| HIGH | src/soup_cli/utils/ebft_gdpo.py | 311 | Energy-Based Fine-Tuning loss kernel (v0.53.2 #135). EBFT treats per-token logits as energies (lower = more probabl | STRING |
| HIGH | src/soup_cli/utils/lr_finder.py | 155 | Run an in-process LR-sweep training loop (#56, v0.33.0). For each LR in ``schedule``, pulls the next batch from ``d | STRING |
| HIGH | src/soup_cli/utils/gradient_ckpt.py | 102 | Install selective / medium gradient-checkpoint hooks on transformer blocks (#44, v0.33.0). Iterates the model's | STRING |
| HIGH | src/soup_cli/utils/hubs.py | 686 | Snapshot ``base`` from ``hub`` into a cwd-contained cache + return path. v0.53.10 #152 — shared helper extracted fr | STRING |
| HIGH | src/soup_cli/utils/jinja_analyzer.py | 55 | Walk ``template``'s Jinja AST and return the set of message fields. Detects both attribute-style ``{{ m.role }}`` a | STRING |
| HIGH | src/soup_cli/utils/recipe_run.py | 639 | Execute a validated :class:`RecipeDAG` end-to-end (v0.53.7 live). Per-node handlers dispatch on ``RecipeNode.kind`` | STRING |
| HIGH | src/soup_cli/utils/longlora.py | 264 | Apply the LongLoRA S² shift to half the attention heads. Pure math kernel used by :func:`apply_longlora_forward_ove | STRING |
| HIGH | src/soup_cli/utils/strict_safetensors.py | 216 | Refuse pickle / PyTorch-classic weights when ``strict=True``. Args: model_dir: cwd-contained model / adapte | STRING |
| HIGH | src/soup_cli/utils/neat_packing.py | 37 | Convert document boundaries into per-token 1-indexed segment IDs. Example: ``[0, 3, 5, 9]`` (3 docs of length 3, 2, | STRING |
| HIGH | src/soup_cli/utils/neat_packing.py | 107 | Build a ``(B, 1, S, S)`` additive attention mask from segment IDs. Args: seq_pos_ids: integer array of shap | STRING |
| HIGH | src/soup_cli/utils/data_mix.py | 190 | Validate dataset paths: containment + dedup + bounds. Args: raw: Sequence of dataset paths (relative or abs | STRING |
| HIGH | src/soup_cli/utils/freeze.py | 77 | Freeze every parameter, then unfreeze those matching ``patterns``. This is the Spectrum (#266) targeted-training me | STRING |
| HIGH | src/soup_cli/utils/trust_remote.py | 94 | Decide whether to pass ``trust_remote_code=True`` to HF loaders. Args: model_name: HF repo id or local path | STRING |
| HIGH | src/soup_cli/utils/multipack.py | 47 | Build a :class:`MultipackBatchSampler` from per-sample token lengths. Args: lengths: token counts for each | STRING |
| HIGH | src/soup_cli/utils/ollama.py | 160 | Generate Ollama Modelfile content. Args: gguf_path: Path to the GGUF model file. template: Template | STRING |
| HIGH | src/soup_cli/utils/cross_doc_attn.py | 26 | Build a block-diagonal causal attention mask. Args: boundaries: Sorted document boundary indices in ``[0, s | STRING |
| HIGH | src/soup_cli/data/loss_mask.py | 281 | Build labels where only assistant tokens contribute to loss. Args: messages: Chat messages list (``{"role": | STRING |
| HIGH | src/soup_cli/data/providers/anthropic.py | 24 | Generate examples using Anthropic Claude API. Args: prompt: User-provided topic/instructions. count | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/test_v07129.py | 451 | # Task 4 — commands/shrink.py prune orchestration + CLI registration | COMMENT |
| MEDIUM⚡ | tests/test_v0580.py | 499 | # Part C — budget guardrails | COMMENT |
| MEDIUM⚡ | tests/test_v0715.py | 187 | profile = self._profile() # factual_lookup + high variance → RAG | CODE |
| MEDIUM⚡ | tests/test_v0715.py | 196 | profile = self._profile() # would be RAG by default | CODE |
| MEDIUM | tests/test_v0535.py | 510 | # `subprocess.run` and asserts orchestration instead.) | COMMENT |
| MEDIUM | tests/test_v0680_part_e.py | 434 | # scaffold and exits 0 (no systemctl call). | COMMENT |
| MEDIUM | docs/training.md | 47 | - [RAFT — Retrieval-Augmented Fine-Tuning](#raft--retrieval-augmented-fine-tuning) | CODE |
| MEDIUM | docs/training.md | 48 | - [RA-DIT — Retrieval-Augmented Dual Instruction Tuning](#ra-dit--retrieval-augmented-dual-instruction-tuning) | CODE |
| MEDIUM⚡ | docs/evaluation.md | 74 | # Choice: SFT (or PROMPT_ENG / RAG / DPO / GRPO) | COMMENT |
| MEDIUM⚡ | docs/evaluation.md | 79 | # Optional ROI probe (offline heuristic: zero/few-shot + RAG + LoRA estimate). | COMMENT |
| MEDIUM⚡ | docs/evaluation.md | 79 | # Optional ROI probe (offline heuristic: zero/few-shot + RAG + LoRA estimate). | COMMENT |
| MEDIUM⚡ | docs/evaluation.md | 82 | # LIVE ROI probe (v0.71.7): loads the model for zero/few-shot token-F1, a short | COMMENT |
| MEDIUM | src/soup_cli/config/schema.py | 8365 | # Fine-tune a model for chain-of-thought reasoning with GRPO | COMMENT |
| MEDIUM | src/soup_cli/config/schema.py | 8664 | # Extend model context window for long-document understanding | COMMENT |
| MEDIUM | src/soup_cli/config/schema.py | 8697 | # Fine-tune a sentence embedding model (BGE, E5, GTE, etc.) | COMMENT |
| MEDIUM⚡ | src/soup_cli/utils/advise.py | 944 | # Heuristic: shorter outputs → easier zero-shot; long-form → harder. | COMMENT |
| MEDIUM⚡ | src/soup_cli/utils/advise.py | 946 | # Few-shot beats zero-shot by a small margin when input length is moderate. | COMMENT |
| MEDIUM⚡ | src/soup_cli/utils/advise.py | 948 | # RAG only helps when there's lookup-style variance. | COMMENT |
| MEDIUM | src/soup_cli/utils/advise.py | 719 | # Marginal RAG → SFT flip: strong SFT track record, no comparable RAG | COMMENT |
| MEDIUM | src/soup_cli/utils/advise.py | 720 | # track record. RAG is the marginal call (it fired on a heuristic | COMMENT |
| MEDIUM | src/soup_cli/utils/advise.py | 1063 | "# RAG is outside Soup core — wire your data into a vector store " | CODE |
| MEDIUM | src/soup_cli/utils/advise.py | 910 | a LIVE probe (#161): it generates zero-shot and few-shot completions on a | STRING |
| MEDIUM | src/soup_cli/utils/ra_dit_run.py | 1 | """v0.71.10 #200 — live RA-DIT two-stage orchestration + retriever auto-link. | STRING |
| MEDIUM | src/soup_cli/utils/prm_reward.py | 226 | # Cap the assembled length to the PRM's own context window (bounded by a | COMMENT |
| MEDIUM | src/soup_cli/utils/asr_metrics.py | 154 | # pure insertions so a hallucination on an empty-ref row is NOT | COMMENT |
| MEDIUM | src/soup_cli/utils/compile_tools.py | 33 | # -> str`` callable so the parse -> iterate -> write orchestration is | COMMENT |
| MEDIUM⚡ | src/soup_cli/utils/spectrum_scan.py | 568 | # Resolve + scan orchestration | COMMENT |
| MEDIUM | src/soup_cli/templates/embedding.yaml | 2 | # Fine-tune a sentence embedding model (BGE, E5, GTE, etc.) | COMMENT |
| MEDIUM | src/soup_cli/templates/reasoning.yaml | 2 | # Fine-tune a model for chain-of-thought reasoning with GRPO | COMMENT |
| MEDIUM | src/soup_cli/templates/longcontext.yaml | 2 | # Extend model context window for long-document understanding | COMMENT |
| MEDIUM | src/soup_cli/commands/advise.py | 298 | # Pick the best of zero-shot vs few-shot as the "prompt_eng" baseline. | COMMENT |
| MEDIUM | src/soup_cli/commands/local_rl.py | 218 | # Render the scheduler scaffold; never run systemctl / launchctl. | COMMENT |
| MEDIUM | src/soup_cli/data/formats.py | 793 | _MAX_RAFT_FIELD_LEN = 65_536 # 64 KiB per document — generous for legal/RAG corpora. | CODE |
| MEDIUM | src/soup_cli/trainer/asr.py | 271 | # teaches hallucination), so count both and warn once after the map. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/test_registry.py | 755 | # Create a finished run first | COMMENT |
| MEDIUM | tests/test_sweep.py | 270 | # Create a minimal config | COMMENT |
| MEDIUM | tests/test_v0620_part_c.py | 344 | # Create a tiny pairs JSONL to pass the path-containment check. | COMMENT |
| MEDIUM⚡ | tests/test_v0580.py | 1312 | # Create the dir then monkeypatch os.listdir to raise — simulates a | COMMENT |
| MEDIUM⚡ | tests/test_vllm_serve.py | 71 | # Create a fake model dir | COMMENT |
| MEDIUM⚡ | tests/test_v0537.py | 293 | # Create a tiny module with an upper-casing transform | COMMENT |
| MEDIUM | tests/test_ui.py | 315 | # Create a run | COMMENT |
| MEDIUM⚡ | tests/test_sglang_serve.py | 219 | # Create a dummy model path | COMMENT |
| MEDIUM | tests/test_templates_yaml.py | 115 | # Create a file OUTSIDE the templates dir | COMMENT |
| MEDIUM | tests/test_adapters.py | 28 | # Create a small dummy safetensors file (just for size detection) | COMMENT |
| MEDIUM | tests/test_bugfixes.py | 855 | # Create a real callable with no params (like experimental PPOTrainer.train) | COMMENT |
| MEDIUM | tests/test_bugfixes.py | 893 | # Create a real callable that accepts resume_from_checkpoint | COMMENT |
| MEDIUM | tests/test_part_f_hardening.py | 87 | # Create a broken symlink as 'checkpoint-300' | COMMENT |
| MEDIUM⚡ | tests/test_v0660_followups.py | 236 | # Create a real adapter dir | COMMENT |
| MEDIUM | tests/test_dataset_registry.py | 133 | # Create a dummy data file | COMMENT |
| MEDIUM⚡ | tests/test_cli.py | 59 | # Create a file (not a directory) | COMMENT |
| MEDIUM⚡ | tests/test_cli.py | 69 | # Create an empty directory (no adapter_config.json or config.json) | COMMENT |
| MEDIUM | src/soup_cli/templates/eu-ai-act.yaml | 4 | # This file is a normal training config — the compliance controls are Soup | COMMENT |
| MEDIUM | src/soup_cli/templates/hipaa.yaml | 4 | # This file is a normal training config — the compliance controls are Soup | COMMENT |
| MEDIUM | src/soup_cli/templates/soc2.yaml | 4 | # This file is a normal training config — the compliance controls are Soup | COMMENT |
| MEDIUM | src/soup_cli/templates/sr-11-7.yaml | 4 | # This file is a normal training config — the compliance controls are Soup | COMMENT |
| MEDIUM | src/soup_cli/trainer/embedding.py | 495 | # Create a custom Trainer subclass dynamically to inject compute_loss | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_v07127.py | 59 | CODE | |
| LOW | tests/test_v07122.py | 626 | CODE | |
| LOW | tests/test_issue745_ppo_loraplus_scheduler.py | 214 | CODE | |
| LOW | tests/test_v07203.py | 2680 | CODE | |
| LOW | benchmarks/harness/cuda_graph_decode_compare.py | 74 | CODE | |
| LOW | benchmarks/harness/issue725_lorafa_operating_point.py | 37 | CODE | |
| LOW | benchmarks/harness/mechanism.py | 459 | CODE | |
| LOW | src/soup_cli/utils/reward_hack_control.py | 813 | CODE | |
| LOW | src/soup_cli/utils/layer_shard.py | 1167 | CODE | |
| LOW | src/soup_cli/utils/vllm.py | 226 | CODE | |
| LOW | src/soup_cli/utils/magpie.py | 473 | CODE | |
| LOW | src/soup_cli/utils/long_context.py | 413 | CODE | |
| LOW | src/soup_cli/utils/adapter_pr.py | 243 | CODE | |
| LOW | src/soup_cli/utils/layer_stream_runtime.py | 2847 | CODE | |
| LOW | src/soup_cli/utils/layer_stream_runtime.py | 3458 | CODE | |
| LOW | src/soup_cli/utils/launcher.py | 189 | CODE | |
| LOW | src/soup_cli/utils/layer_stream.py | 1534 | CODE | |
| LOW | src/soup_cli/utils/fast_lora.py | 343 | CODE | |
| LOW | src/soup_cli/utils/diagnose/live.py | 406 | CODE | |
| LOW | src/soup_cli/experiment/tracker.py | 345 | CODE | |
| LOW | src/soup_cli/commands/distill_prompt.py | 22 | CODE | |
| LOW | src/soup_cli/commands/data_forge.py | 27 | CODE | |
| LOW | src/soup_cli/commands/data_doctor.py | 128 | CODE | |
| LOW | src/soup_cli/commands/ingest.py | 44 | CODE | |
| LOW | src/soup_cli/commands/merge.py | 16 | CODE | |
| LOW | src/soup_cli/commands/data_clean.py | 48 | CODE | |
| LOW | src/soup_cli/commands/steer.py | 59 | CODE | |
| LOW | src/soup_cli/commands/deploy.py | 202 | CODE | |
| LOW | src/soup_cli/commands/deploy.py | 571 | CODE | |
| LOW | src/soup_cli/commands/shrink.py | 224 | CODE | |
| LOW | src/soup_cli/commands/generate.py | 22 | CODE | |
| LOW | src/soup_cli/commands/generate.py | 493 | CODE | |
| LOW | src/soup_cli/commands/ci.py | 24 | CODE | |
| LOW | src/soup_cli/commands/tunability.py | 48 | CODE | |
| LOW | src/soup_cli/commands/serve.py | 72 | CODE | |
| LOW | src/soup_cli/commands/serve.py | 1601 | CODE | |
| LOW | src/soup_cli/commands/serve.py | 1694 | CODE | |
| LOW | src/soup_cli/commands/serve.py | 2583 | CODE | |
| LOW | src/soup_cli/commands/edit.py | 131 | CODE | |
| LOW | src/soup_cli/commands/edit.py | 377 | CODE | |
| LOW | src/soup_cli/commands/export.py | 67 | CODE | |
| LOW | src/soup_cli/commands/bench.py | 36 | CODE | |
| LOW | src/soup_cli/commands/diagnose.py | 262 | CODE | |
| LOW | src/soup_cli/commands/chat.py | 16 | CODE | |
| LOW | src/soup_cli/commands/chat.py | 293 | CODE | |
| LOW | src/soup_cli/commands/bom.py | 30 | CODE | |
| LOW | src/soup_cli/commands/data_mix.py | 44 | CODE | |
| LOW | src/soup_cli/commands/train.py | 494 | CODE | |
| LOW | src/soup_cli/commands/infer.py | 56 | CODE | |
| LOW | src/soup_cli/commands/diff.py | 16 | CODE | |
| LOW | src/soup_cli/commands/sweep.py | 55 | CODE | |
| LOW | src/soup_cli/commands/ship.py | 1188 | CODE | |
| LOW | src/soup_cli/commands/eval.py | 166 | CODE | |
| LOW | src/soup_cli/commands/attest.py | 38 | CODE | |
| LOW | src/soup_cli/commands/local_rl.py | 161 | CODE | |
| LOW | src/soup_cli/commands/reward.py | 115 | CODE | |
| LOW | src/soup_cli/commands/reward.py | 304 | CODE | |
| LOW | src/soup_cli/commands/draft.py | 552 | CODE | |
| LOW | src/soup_cli/commands/draft.py | 729 | CODE | |
| LOW | src/soup_cli/commands/_eval_v0650.py | 257 | CODE | |
| 12 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_issue1226_reward_gold_parsing.py | 65 | "G27 heading before a phrase": "#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", | CODE |
| LOW | tests/test_issue1226_reward_gold_parsing.py | 111 | "C31 heading before a phrase": "#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", | CODE |
| LOW | tests/test_issue1226_reward_gold_parsing.py | 729 | ("#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", "42"), | CODE |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 89 | # Step 1: NaN (transient overflow) | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 91 | # Step 2: finite (GradScaler skipped step, backoff recovered) | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 93 | # Step 3: finite | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 95 | # Step 4: finite | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 97 | # Step 5: finite (triggers check, counter is 0) | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8810 | # Step 1: soup train --config soup_sft.yaml # SFT warmup | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8811 | # Step 2: soup train --config soup_rm.yaml # Train reward model | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8812 | # Step 3: soup train --config soup_ppo.yaml # PPO with reward model | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 5 | # Step 1: soup train --config soup_sft.yaml # SFT warmup | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 6 | # Step 2: soup train --config soup_rm.yaml # Train reward model | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 7 | # Step 3: soup train --config soup_ppo.yaml # PPO with reward model | COMMENT |
| LOW | src/soup_cli/commands/export.py | 385 | # Step 1: Convert HF model to GGUF (f16) | COMMENT |
| LOW | src/soup_cli/commands/export.py | 854 | # Step 1: Convert HF model to TensorRT-LLM checkpoint | COMMENT |
| LOW | src/soup_cli/commands/export.py | 880 | # Step 2: Build TensorRT engine | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_issue1226_reward_gold_parsing.py | 65 | "G27 heading before a phrase": "#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", | CODE |
| LOW | tests/test_issue1226_reward_gold_parsing.py | 111 | "C31 heading before a phrase": "#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", | CODE |
| LOW | tests/test_issue1226_reward_gold_parsing.py | 729 | ("#### Step 1: multiply\n6*7 = 42\nThe answer is 42.", "42"), | CODE |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 89 | # Step 1: NaN (transient overflow) | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 91 | # Step 2: finite (GradScaler skipped step, backoff recovered) | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 93 | # Step 3: finite | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 95 | # Step 4: finite | COMMENT |
| LOW⚡ | tests/test_issue887_distill_broken_teacher_warning.py | 97 | # Step 5: finite (triggers check, counter is 0) | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8810 | # Step 1: soup train --config soup_sft.yaml # SFT warmup | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8811 | # Step 2: soup train --config soup_rm.yaml # Train reward model | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8812 | # Step 3: soup train --config soup_ppo.yaml # PPO with reward model | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 5 | # Step 1: soup train --config soup_sft.yaml # SFT warmup | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 6 | # Step 2: soup train --config soup_rm.yaml # Train reward model | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 7 | # Step 3: soup train --config soup_ppo.yaml # PPO with reward model | COMMENT |
| LOW | src/soup_cli/commands/export.py | 385 | # Step 1: Convert HF model to GGUF (f16) | COMMENT |
| LOW | src/soup_cli/commands/export.py | 854 | # Step 1: Convert HF model to TensorRT-LLM checkpoint | COMMENT |
| LOW | src/soup_cli/commands/export.py | 880 | # Step 2: Build TensorRT engine | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | tests/test_refusal_multilingual.py | 68 | assert looks_like_refusal("As an AI language model, I refuse to answer.") | CODE |
| MEDIUM | tests/test_issue1228_behavior_scoring.py | 29 | "I'm unable to help with that.", | CODE |
| MEDIUM⚡ | tests/test_issue1228_behavior_scoring.py | 73 | "I'm unable to help with that.", | CODE |
| MEDIUM⚡ | tests/test_v0560.py | 280 | assert looks_like_refusal("As an AI language model, I refuse to answer.") | CODE |
| MEDIUM⚡ | tests/test_data_clean.py | 115 | text = "Certainly! As an AI language model, I can explain that. Quantum computing uses qubits." | CODE |
| MEDIUM | tests/test_data_clean.py | 55 | "Certainly! As an AI language model, here is the function:\n\n" | CODE |
| MEDIUM | tests/test_data_clean.py | 163 | "Sure! As an AI language model, here is the function:\n\n" | CODE |
| MEDIUM | src/soup_cli/utils/diagnose/refusal.py | 37 | # "i am unable" branch, so "I'm unable to help with that." read as an answer. | COMMENT |
| MEDIUM | src/soup_cli/eval/gate_suites.py | 535 | else "I'm unable to help with that." | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_issue459_interleave_streaming_hub.py | 540 | (tmp_path / f"a{suffix}").write_text("placeholder", encoding="utf-8") | CODE |
| LOW⚡ | tests/test_issue459_interleave_streaming_hub.py | 541 | (tmp_path / f"b{suffix}").write_text("placeholder", encoding="utf-8") | CODE |
| LOW⚡ | tests/test_v07127.py | 1388 | prompt = "Please summarize the following article: " + ("lorem ipsum " * 5) | CODE |
| LOW⚡ | tests/test_v07127.py | 1396 | prompt = "Please summarize the following article: " + ("lorem ipsum " * 5) | CODE |
| LOW | tests/test_v07127.py | 1523 | leak_prompt = "Please summarize the following unique article: " + ("lorem ipsum " * 5) | CODE |
| LOW⚡ | tests/test_v05310.py | 117 | presidio_hits = [{"kind": "email", "snippet": "user@example.com"}] | CODE |
| LOW⚡ | tests/test_v07116.py | 235 | "lorem ipsum dolor sit amet consectetur", | CODE |
| LOW⚡ | tests/test_v07116.py | 235 | "lorem ipsum dolor sit amet consectetur", | CODE |
| LOW⚡ | tests/test_v07122.py | 1201 | assert original_assistant["content"] == "placeholder" | CODE |
| LOW | tests/test_v07122.py | 1188 | original_assistant = {"role": "assistant", "content": "placeholder"} | CODE |
| LOW | tests/test_v0537.py | 2064 | target = self._write_arrow_dir(tmp_path, cache_key="placeholder") | CODE |
| LOW | tests/test_v0537.py | 2110 | target = self._write_arrow_dir(tmp_path, cache_key="placeholder") | CODE |
| LOW⚡ | tests/test_issue1231_expect_all_formats.py | 465 | row = {"instruction": "Summarise", "input": "lorem ipsum " * 5000, "output": PII_SAMPLE} | STRING |
| LOW⚡ | tests/test_v0712.py | 1523 | existing.write_text("placeholder", encoding="utf-8") | CODE |
| LOW⚡ | tests/test_issue687_ui_auth.py | 23 | "/api/runs/dummy_id", | CODE |
| LOW⚡ | tests/test_issue687_ui_auth.py | 24 | "/api/runs/dummy_id/metrics", | CODE |
| LOW⚡ | tests/test_issue687_ui_auth.py | 25 | "/api/runs/dummy_id/eval", | CODE |
| LOW⚡ | tests/test_v0430_part_d.py | 82 | target.write_text("placeholder", encoding="utf-8") | CODE |
| LOW⚡ | tests/test_v0430_part_d.py | 107 | outside.write_text("placeholder", encoding="utf-8") | CODE |
| LOW⚡ | tests/test_issue1214_migrators_pref_tasks.py | 232 | assert any("online_dpo_judge" in w and "placeholder" in w for w in result["_warnings"]) | CODE |
| LOW | docs/adapters-and-governance.md | 250 | --subject "The CEO of Acme is" --target "Jane Doe" | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/test_awq_gptq_export.py | 1084 | # Set cwd to a subdirectory so ../evil is outside | COMMENT |
| LOW | src/soup_cli/ui/app.py | 948 | # Check if training is still running | COMMENT |
| LOW | src/soup_cli/utils/data_clean.py | 139 | # Check if wrapped in markdown ```json | COMMENT |
| LOW | src/soup_cli/utils/data_clean.py | 145 | # Check if already valid JSON | COMMENT |
| LOW | src/soup_cli/commands/export.py | 699 | # Check if it's in PATH | COMMENT |
| LOW | src/soup_cli/commands/export.py | 723 | # Check if LoRA adapter | COMMENT |
| LOW | src/soup_cli/commands/export.py | 823 | # Check if LoRA adapter | COMMENT |
| LOW | src/soup_cli/commands/export.py | 1067 | # Check if LoRA adapter — merge first | COMMENT |
| LOW | src/soup_cli/commands/export.py | 1211 | # Check if LoRA adapter — merge first | COMMENT |
| LOW | src/soup_cli/commands/eval.py | 431 | # Display results | COMMENT |
| LOW | src/soup_cli/commands/data.py | 689 | # Check if this is a vision dataset | COMMENT |
| LOW | src/soup_cli/data/loader.py | 497 | # Check if it's a HuggingFace dataset | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | examples/configs/dpo_example.yaml | 5 | # Usage: | COMMENT |
| LOW | examples/configs/rlhf_step3_ppo.yaml | 9 | # Usage: | COMMENT |
| LOW | examples/configs/sft_basic.yaml | 9 | # Usage: | COMMENT |
| LOW | examples/configs/rlhf_step1_sft.yaml | 8 | # Usage: | COMMENT |
| LOW | examples/configs/grpo_reasoning.yaml | 10 | # Usage: | COMMENT |
| LOW | examples/configs/rlhf_step2_reward.yaml | 8 | # Usage: | COMMENT |
| LOW | examples/configs/dpo_chat.yaml | 9 | # Usage: | COMMENT |
| LOW⚡ | src/soup_cli/config/schema.py | 8809 | # Usage: | COMMENT |
| LOW⚡ | src/soup_cli/templates/rlhf.yaml | 4 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/test_issue1215_migrate_quantization_none.py | 515 | "def helper():\n", | CODE |