🐢 Open-Source Evaluation & Testing library for LLM Agents
This report summarizes scanner evidence for Giskard-AI/giskard-oss, a Python project with 5,887 GitHub stars. SynthScan examined 62,976 lines of code across 405 source files, recording 2203 pattern matches distributed across 19 syntactic categories. The legacy adjusted pattern signal is 36.0 (High pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 1658 finding(s).
Are imports declared and package/API names verified? 435 finding(s).
Are wrappers, guards, and repeated structures necessary? 1726 finding(s).
Which changes need focused human review? 2203 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 2203 distinct pattern matches across 19 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/test_check_extra_pins.py | 13 | def test_matching_lower_bound_is_ok() -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 61 | def test_stale_lib_pin_is_ignored_by_root_only_check(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 67 | def test_stale_giskard_scan_pin_is_a_package_mismatch(tmp_path: Path) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 85 | def test_package_pins_reject_non_ge_operators(tmp_path: Path, scan_req: str) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 92 | def test_package_pins_ok_when_root_and_lib_match(tmp_path: Path) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 97 | def test_package_pins_ok_when_no_requirement_names_package(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 110 | def test_current_tree_giskard_has_no_versioned_pins() -> None: | CODE |
| LOW | tools/test_lint_readme_snippets.py | 6 | def test_lint_markdown_checks_spaced_python_fences(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_lint_readme_snippets.py | 18 | def test_lint_markdown_checks_positional_f_strings(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 12 | def test_rewrite_preserves_extras_comments_and_other_packages() -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 27 | def test_rewrite_does_not_match_package_name_prefix() -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 42 | def test_bump_pins_updates_root_and_downstream_libs(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 72 | def test_simulated_checks_release_keeps_root_pins_aligned() -> None: | CODE |
| LOW | tools/check_extra_pins.py | 82 | def _iter_project_requirements( | CODE |
| LOW | tools/check_extra_pins.py | 153 | def collect_package_pin_mismatches( | CODE |
| LOW | libs/giskard-scan/tests/test_judge_prompts.py | 37 | async def test_scan_judge_template_preserves_criteria_with_optional_context( | CODE |
| LOW | libs/giskard-scan/tests/conftest.py | 7 | def isolated_vulnerability_registry(): | CODE |
| LOW | libs/giskard-scan/tests/conftest.py | 18 | def isolated_quality_registry(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 47 | def test_scan_src_imports_upstream_libs_only_from_package_roots(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 5 | def test_prompt_injection_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 10 | async def test_prompt_injection_scenarios_have_tags(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 28 | def _registry_generator_types(registry: SuiteGeneratorRegistry) -> set[type]: | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 32 | def test_default_registry_generators_are_public(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 43 | def test_all_public_symbols_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 56 | def test_vulnerability_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 64 | def test_vulnerability_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 75 | def test_quality_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 86 | def test_quality_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 96 | def test_list_scan_items_giskard(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 19 | def test_scenario_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 24 | async def test_scenario_generator_generate_scenario_accepts_target_mode(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 44 | def test_dataset_generator_has_dataset_name_field(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 28 | def test_resolve_scan_options_matches_for_identical_shared_kwargs(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 50 | def test_scan_entrypoints_expose_shared_explicit_kwargs(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 70 | def test_scan_entrypoints_require_keyword_only_options_after_languages(): | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 40 | async def test_vulnerability_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 93 | async def test_vulnerability_scan_forwards_run_options( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 147 | async def test_vulnerability_scan_emits_privacy_safe_product_telemetry( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 215 | async def test_vulnerability_scan_allowlists_caller_controlled_telemetry_values( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 250 | async def test_vulnerability_scan_finishes_telemetry_when_scan_raises( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 72 | async def test_quality_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 140 | async def test_quality_scan_forwards_run_options( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 193 | async def test_quality_scan_uses_empty_registry_by_default( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 223 | async def test_quality_scan_warns_and_skips_empty_raw_knowledge_base( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 274 | async def test_quality_scan_configures_knowledge_base_generator( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 335 | async def test_quality_scan_does_not_generate_recommendation_without_failures( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 360 | async def test_quality_scan_hides_recommendation_when_generation_fails( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 388 | async def test_quality_scan_emits_privacy_safe_product_telemetry( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 442 | async def test_quality_scan_allowlists_caller_controlled_telemetry_values( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 475 | async def test_quality_scan_finishes_telemetry_when_scan_raises( | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 30 | def test_knowledge_base_from_texts_does_not_compute_embeddings(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 43 | def test_normalize_knowledge_base_accepts_existing_instance_and_texts(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 53 | def test_normalize_knowledge_base_rejects_single_string(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 59 | def test_knowledge_base_drops_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 68 | def test_knowledge_base_rejects_all_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 73 | async def test_closest_documents_computes_missing_embeddings_lazily_in_batch(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 99 | async def test_closest_documents_initializes_embeddings_once_for_concurrent_callers(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 123 | async def test_closest_documents_recomputes_all_embeddings_when_any_is_missing(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 150 | async def test_closest_documents_to_text_embeds_query_against_cached_documents(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 172 | async def test_closest_documents_to_text_rejects_empty_query_without_embedding(): | CODE |
| 1554 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/lint_readme_snippets.py | 3 | CODE | |
| LOW | …iskard-scan/tests/integrations/garak/test_garak_e2e.py | 26 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 8 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 9 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 16 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 17 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 18 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 19 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 28 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 30 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 30 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 31 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 32 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 33 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 33 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 34 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 34 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 3 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 4 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 4 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 28 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 4 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 4 | CODE | |
| LOW | …giskard/scan/integrations/deepteam/_judge_generator.py | 17 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 3 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 5 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 6 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 8 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| 366 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 213 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 215 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 194 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 196 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 255 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 257 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 355 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 357 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 2 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 208 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 210 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 310 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 312 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 408 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 410 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 26 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 28 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 59 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 61 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 452 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 454 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 40 | __all__ = [ | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 6 | __all__ = ["ScanTool", "list_scan_items", "third_party_scan"] | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/_adapter.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …can/src/giskard/scan/integrations/deepteam/_adapter.py | 31 | logger = logging.getLogger(__name__) | CODE |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 6 | __all__ = [ | CODE |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/generators/base.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard/scan/generators/knowledge_base/multi_topic.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …giskard/scan/generators/knowledge_base/out_of_scope.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 40 | __all__ = [ | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/openai_chat.py | 28 | logger = logging.getLogger(__name__) | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/google_chat.py | 59 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/giskard-llm/src/giskard/llm/translators/anthropic.py | 57 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/openai_response.py | 27 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/types/__init__.py | 68 | __all__ = [ | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 96 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/openai.py | 84 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard-llm/src/giskard/llm/providers/azure_openai.py | 47 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/anthropic.py | 79 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/azure_ai.py | 58 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/utils/__init__.py | 7 | __all__ = [ | CODE |
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 46 | __all__ = [ | CODE |
| LOW | …bs/giskard-core/src/giskard/core/telemetry/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 100 | def _set_tags() -> None: | CODE |
| LOW | …giskard-core/src/giskard/core/rate_limiter/__init__.py | 9 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/__init__.py | 21 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/pipeline.py | 5 | __all__ = ["Pipeline"] | CODE |
| LOW | …bs/giskard-agents/src/giskard/agents/tools/__init__.py | 5 | __all__ = ["Tool", "tool", "Function", "ToolCall"] | CODE |
| LOW | …skard-agents/src/giskard/agents/embeddings/__init__.py | 7 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/som/__init__.py | 21 | __all__ = [ | CODE |
| LOW | …iskard-agents/src/giskard/agents/templates/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …skard-agents/src/giskard/agents/generators/__init__.py | 20 | __all__ = [ | CODE |
| LOW | …skard-agents/src/giskard/agents/errors/serializable.py | 5 | __all__ = ["Error"] | CODE |
| LOW | …s/giskard-agents/src/giskard/agents/errors/__init__.py | 4 | __all__ = ["ModelRefusalError", "WorkflowError", "Error"] | CODE |
| LOW | libs/giskard-checks/tests/generators/conftest.py | 3 | __all__ = ["LLMTrace", "MockGenerator"] | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/__init__.py | 97 | __all__ = [ | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/settings.py | 81 | def set_default_generator(generator: BaseGenerator | str) -> None: | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/settings.py | 173 | def set_default_embedding_model(model: BaseEmbeddingModel | str | None) -> None: | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/core/__init__.py | 23 | __all__ = [ | CODE |
| LOW | …checks/src/giskard/checks/core/interaction/__init__.py | 6 | __all__ = ["InteractionSpec", "Interact", "Interaction", "Trace"] | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/judges/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …/giskard-checks/src/giskard/checks/testing/__init__.py | 3 | __all__ = ["WithSpy"] | CODE |
| LOW | …iskard-checks/src/giskard/checks/scenarios/__init__.py | 4 | __all__ = [ | CODE |
| LOW | …skard-checks/src/giskard/checks/generators/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/export/__init__.py | 4 | __all__ = ["to_hub_format", "to_junit_xml"] | CODE |
| LOW | …/giskard-checks/src/giskard/checks/builtin/__init__.py | 22 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .claude/hooks/quality_gate.py | 132 | except Exception: # noqa: BLE001 — fail open on any unexpected hook error | CODE |
| MEDIUM | .claude/hooks/quality_gate.py | 39 | def main() -> int: | CODE |
| LOW | .claude/hooks/enforce_uv.py | 44 | except Exception: # noqa: BLE001 — fail open on any unexpected hook error | CODE |
| MEDIUM | .claude/hooks/enforce_uv.py | 18 | def main() -> int: | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/vulnerability.py | 141 | except Exception: | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 147 | except Exception: | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 155 | except Exception: | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 171 | except Exception as exc: # noqa: BLE001 — one bad plugin must not abort | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 199 | except Exception as exc: # noqa: BLE001 — one bad plugin must not abort the scan | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 276 | except Exception: # noqa: BLE001 — unknown module: fall back to instance path | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 369 | except Exception as exc: # noqa: BLE001 — one bad detector must not abort the scan | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 441 | except Exception as exc: # noqa: BLE001 — a broken detector skips, not aborts | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 514 | except Exception as exc: # noqa: BLE001 — one bad probe must not abort the scan | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 150 | except Exception as exc: | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 286 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 319 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 382 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| MEDIUM | libs/giskard-llm/src/giskard/llm/providers/google.py | 106 | def _import_genai() -> Any: | CODE |
| LOW | libs/giskard-core/tests/test_telemetry.py | 214 | except Exception: | CODE |
| LOW | libs/giskard-core/src/giskard/core/welcome.py | 31 | except Exception: | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 237 | except Exception as e: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/workflow.py | 474 | except Exception as err: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 192 | except Exception as error: | CODE |
| LOW | …rd-checks/src/giskard/checks/core/interaction/trace.py | 170 | except Exception as error: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 19 | except Exception: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 32 | except Exception: | CODE |
| LOW | …bs/giskard-checks/src/giskard/checks/testing/runner.py | 30 | except Exception as e: | CODE |
| LOW | …/giskard-checks/src/giskard/checks/scenarios/runner.py | 185 | except Exception as caught: | CODE |
| LOW | …skard-checks/src/giskard/checks/builtin/nlp_metrics.py | 147 | except Exception as exc: | CODE |
| LOW | …iskard-checks/src/giskard/checks/builtin/comparison.py | 99 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/README.md | 298 | ### Step 1: Define a custom check | COMMENT |
| LOW | libs/giskard-checks/README.md | 320 | ### Step 2: Define a custom interaction specification | COMMENT |
| LOW | libs/giskard-checks/README.md | 341 | ### Step 3: Verify registration | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 821 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 822 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 823 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 824 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 825 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 139 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 49 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 121 | CODE | |
| LOW | libs/giskard-llm/tests/conftest.py | 49 | CODE | |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 278 | CODE | |
| LOW | …s/giskard-llm/src/giskard/llm/translators/anthropic.py | 254 | CODE | |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 219 | CODE | |
| LOW | libs/giskard-agents/tests/conftest.py | 32 | CODE | |
| LOW | libs/giskard-checks/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/core/result.py | 753 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/core/judge.py | 170 | CODE | |
| LOW | …/giskard-checks/src/giskard/checks/scenarios/runner.py | 139 | CODE | |
| LOW | …s/giskard-checks/src/giskard/checks/generators/base.py | 72 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/export/junit.py | 66 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | libs/giskard-llm/tests/test_providers.py | 597 | except (ImportError, AttributeError): | CODE |
| HIGH | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 118 | except OSError: | CODE |
| HIGH | …s/giskard-checks/src/giskard/checks/utils/inference.py | 62 | except TypeError: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 821 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 822 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 823 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 824 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 825 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …ard-scan/src/giskard/scan/integrations/_entry_point.py | 63 | Run an external security scanner against a Giskard target. Requires the scanner's optional extra: ``pip install gis | STRING |
| HIGH | …giskard-scan/src/giskard/scan/integrations/_listing.py | 28 | List selectable item names for a scan tool. Args: tool: ``"giskard"`` returns scenario generator class name | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 33 | # Check if the legacy package is installed | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 81 | # Check if this parameter is a RunContext | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 273 | # Check if this is a class method by looking for 'self' as first parameter | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 296 | # Check if extraction failed | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 312 | # Check if keyword appears in text | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 454 | # Check if extraction failed | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .claude/hooks/enforce_uv.py | 5 | forms (`cd x && pytest`, `PYTHONPATH=. python f.py`) are NOT caught -- a | STRING |
| HIGH | …skard-llm/tests/translators/test_openai_chat_return.py | 158 | """Tool-only turn: `content` is null, `tool_calls` populated, `finish_reason` is `tool_calls`.""" | STRING |
| HIGH | …skard-checks/src/giskard/checks/builtin/rego_policy.py | 162 | undefined, error if evaluation fails or the rule value is not | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | README.md | 205 | ### [RAGET](https://legacy-docs.giskard.ai/en/stable/open_source/testset_generation/index.html) — generate evaluation da | COMMENT |
| MEDIUM | …skard-agents/src/giskard/agents/embeddings/__init__.py | 4 | # Default embedding model uses Litellm | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …can/tests/integrations/garak/test_resolve_detectors.py | 151 | # Create a minimal probe with only a skipped detector | COMMENT |
| MEDIUM | …can/tests/integrations/garak/test_resolve_detectors.py | 166 | # Create a minimal attempt with 2 conversations (_evaluate_attempt only | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/tests/integration/test_stateless.py | 123 | expected_value="test@test.com", | STRING |
| LOW | libs/giskard-checks/tests/integration/test_stateful.py | 149 | expected_value="test@test.com", | STRING |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 41 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 123 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 240 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .pre-commit-config.yaml | 39 | args: ["--baseline", ".secrets.baseline", "--exclude-secrets", "YOUR_API_KEY"] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-core/src/giskard/core/rate_limiter/base.py | 162 | CODE | |
| MEDIUM | libs/giskard-checks/tests/core/test_scenario.py | 142 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/vulnerability.py | 38 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 41 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …rd-scan/src/giskard/scan/integrations/lidar/_target.py | 41 | COMMENT | |
| LOW | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 1 | # THIS MODULE IS TAKEN FROM Pydantic AI, with only slight modifications. | COMMENT |