🐢 Open-Source Evaluation & Testing library for LLM Agents
This report presents the forensic synthetic code analysis of Giskard-AI/giskard-oss, a Python project with 5,782 GitHub stars. SynthScan v2.0 examined 58,114 lines of code across 391 source files, recording 2018 pattern matches distributed across 18 syntactic categories. The overall adjusted score of 35.7 places this repository in the Strong AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 2018 distinct pattern matches across 18 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/test_check_extra_pins.py | 13 | def test_matching_lower_bound_is_ok() -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 61 | def test_stale_lib_pin_is_ignored_by_root_only_check(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 67 | def test_stale_giskard_scan_pin_is_a_package_mismatch(tmp_path: Path) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 85 | def test_package_pins_reject_non_ge_operators(tmp_path: Path, scan_req: str) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 92 | def test_package_pins_ok_when_root_and_lib_match(tmp_path: Path) -> None: | CODE |
| LOW⚡ | tools/test_check_extra_pins.py | 97 | def test_package_pins_ok_when_no_requirement_names_package(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_check_extra_pins.py | 110 | def test_current_tree_giskard_has_no_versioned_pins() -> None: | CODE |
| LOW | tools/test_lint_readme_snippets.py | 6 | def test_lint_markdown_checks_spaced_python_fences(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_lint_readme_snippets.py | 18 | def test_lint_markdown_checks_positional_f_strings(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 12 | def test_rewrite_preserves_extras_comments_and_other_packages() -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 27 | def test_rewrite_does_not_match_package_name_prefix() -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 42 | def test_bump_pins_updates_root_and_downstream_libs(tmp_path: Path) -> None: | CODE |
| LOW | tools/test_bump_workspace_pins.py | 72 | def test_simulated_checks_release_keeps_root_pins_aligned() -> None: | CODE |
| LOW | tools/check_extra_pins.py | 82 | def _iter_project_requirements( | CODE |
| LOW | tools/check_extra_pins.py | 153 | def collect_package_pin_mismatches( | CODE |
| LOW | libs/giskard-scan/tests/conftest.py | 7 | def isolated_vulnerability_registry(): | CODE |
| LOW | libs/giskard-scan/tests/conftest.py | 18 | def isolated_quality_registry(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 47 | def test_scan_src_imports_upstream_libs_only_from_package_roots(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 5 | def test_prompt_injection_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 10 | async def test_prompt_injection_scenarios_have_tags(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 28 | def _registry_generator_types(registry: SuiteGeneratorRegistry) -> set[type]: | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 32 | def test_default_registry_generators_are_public(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 43 | def test_all_public_symbols_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 56 | def test_vulnerability_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 64 | def test_vulnerability_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 75 | def test_quality_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 86 | def test_quality_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 96 | def test_list_scan_items_giskard(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 19 | def test_scenario_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 24 | async def test_scenario_generator_generate_scenario_accepts_target_mode(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 44 | def test_dataset_generator_has_dataset_name_field(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 28 | def test_resolve_scan_options_matches_for_identical_shared_kwargs(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 50 | def test_scan_entrypoints_expose_shared_explicit_kwargs(): | CODE |
| LOW | libs/giskard-scan/tests/test_scan_options.py | 70 | def test_scan_entrypoints_require_keyword_only_options_after_languages(): | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 39 | async def test_vulnerability_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 92 | async def test_vulnerability_scan_forwards_run_options( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 71 | async def test_quality_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 139 | async def test_quality_scan_forwards_run_options( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 192 | async def test_quality_scan_uses_empty_registry_by_default( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 222 | async def test_quality_scan_warns_and_skips_empty_raw_knowledge_base( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 273 | async def test_quality_scan_configures_knowledge_base_generator( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 334 | async def test_quality_scan_does_not_generate_recommendation_without_failures( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 359 | async def test_quality_scan_hides_recommendation_when_generation_fails( | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 30 | def test_knowledge_base_from_texts_does_not_compute_embeddings(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 43 | def test_normalize_knowledge_base_accepts_existing_instance_and_texts(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 53 | def test_normalize_knowledge_base_rejects_single_string(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 59 | def test_knowledge_base_drops_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 68 | def test_knowledge_base_rejects_all_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 73 | async def test_closest_documents_computes_missing_embeddings_lazily_in_batch(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 99 | async def test_closest_documents_initializes_embeddings_once_for_concurrent_callers(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 123 | async def test_closest_documents_recomputes_all_embeddings_when_any_is_missing(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 150 | async def test_closest_documents_to_text_embeds_query_against_cached_documents(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 172 | async def test_closest_documents_to_text_rejects_empty_query_without_embedding(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 188 | async def test_closest_documents_rejects_non_finite_embeddings(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 201 | def test_knowledge_base_documents_are_immutable(): | CODE |
| LOW⚡ | …iskard-scan/tests/integrations/test_list_scan_items.py | 7 | def test_list_scan_items_rejects_unknown_tool() -> None: | CODE |
| LOW⚡ | …iskard-scan/tests/integrations/test_list_scan_items.py | 14 | def test_list_scan_items_garak() -> None: | CODE |
| LOW⚡ | …iskard-scan/tests/integrations/test_list_scan_items.py | 23 | def test_list_scan_items_deepteam() -> None: | CODE |
| LOW | …iskard-scan/tests/integrations/garak/test_garak_e2e.py | 40 | def test_resolve_probes_empty_list_returns_no_probes() -> None: | CODE |
| LOW | …iskard-scan/tests/integrations/garak/test_garak_e2e.py | 47 | async def test_run_actual_garak_probe() -> None: | CODE |
| 1401 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/lint_readme_snippets.py | 3 | CODE | |
| LOW | …iskard-scan/tests/integrations/garak/test_garak_e2e.py | 26 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 8 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 9 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 16 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 17 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 18 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 19 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 20 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 28 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 29 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 30 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 30 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 31 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 32 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 33 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 33 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 34 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 34 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 3 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 4 | CODE | |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 4 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 28 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 3 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 3 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 4 | CODE | |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 4 | CODE | |
| LOW | …giskard/scan/integrations/deepteam/_judge_generator.py | 17 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 3 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 5 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 6 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 8 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 9 | CODE | |
| 348 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 213 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 215 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 194 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 196 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 255 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 257 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 355 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 357 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 2 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 208 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 210 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 310 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 312 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 408 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 410 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 26 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 28 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 59 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 61 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 452 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 454 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 40 | __all__ = [ | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …giskard-scan/src/giskard/scan/integrations/__init__.py | 6 | __all__ = ["ScanTool", "list_scan_items", "third_party_scan"] | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 33 | logger = logging.getLogger(__name__) | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/_adapter.py | 23 | logger = logging.getLogger(__name__) | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/lidar/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …can/src/giskard/scan/integrations/deepteam/_adapter.py | 31 | logger = logging.getLogger(__name__) | CODE |
| LOW | …can/src/giskard/scan/integrations/deepteam/__init__.py | 6 | __all__ = [ | CODE |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/generators/base.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard/scan/generators/knowledge_base/multi_topic.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …giskard/scan/generators/knowledge_base/out_of_scope.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 40 | __all__ = [ | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/openai_chat.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/google_chat.py | 59 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | …s/giskard-llm/src/giskard/llm/translators/anthropic.py | 50 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/openai_response.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/types/__init__.py | 68 | __all__ = [ | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 96 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/openai.py | 84 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard-llm/src/giskard/llm/providers/azure_openai.py | 47 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/anthropic.py | 78 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/azure_ai.py | 58 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/utils/__init__.py | 7 | __all__ = [ | CODE |
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 46 | __all__ = [ | CODE |
| LOW | …bs/giskard-core/src/giskard/core/telemetry/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 71 | def _set_tags() -> None: | CODE |
| LOW | …giskard-core/src/giskard/core/rate_limiter/__init__.py | 9 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/__init__.py | 20 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/pipeline.py | 5 | __all__ = ["Pipeline"] | CODE |
| LOW | …bs/giskard-agents/src/giskard/agents/tools/__init__.py | 5 | __all__ = ["Tool", "tool", "Function", "ToolCall"] | CODE |
| LOW | …skard-agents/src/giskard/agents/embeddings/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …iskard-agents/src/giskard/agents/templates/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …skard-agents/src/giskard/agents/generators/__init__.py | 20 | __all__ = [ | CODE |
| LOW | …skard-agents/src/giskard/agents/errors/serializable.py | 5 | __all__ = ["Error"] | CODE |
| LOW | …s/giskard-agents/src/giskard/agents/errors/__init__.py | 4 | __all__ = ["ModelRefusalError", "WorkflowError", "Error"] | CODE |
| LOW | libs/giskard-checks/tests/generators/conftest.py | 3 | __all__ = ["LLMTrace", "MockGenerator"] | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/__init__.py | 86 | __all__ = [ | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/settings.py | 65 | def set_default_generator(generator: BaseGenerator | str) -> None: | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/core/__init__.py | 22 | __all__ = [ | CODE |
| LOW | …checks/src/giskard/checks/core/interaction/__init__.py | 6 | __all__ = ["InteractionSpec", "Interact", "Interaction", "Trace"] | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/judges/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …/giskard-checks/src/giskard/checks/testing/__init__.py | 3 | __all__ = ["WithSpy"] | CODE |
| LOW | …iskard-checks/src/giskard/checks/scenarios/__init__.py | 4 | __all__ = [ | CODE |
| LOW | …skard-checks/src/giskard/checks/generators/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/export/__init__.py | 4 | __all__ = ["to_hub_format", "to_junit_xml"] | CODE |
| LOW | …/giskard-checks/src/giskard/checks/builtin/__init__.py | 22 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 133 | except Exception: | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 171 | except Exception as exc: # noqa: BLE001 — one bad plugin must not abort | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 199 | except Exception as exc: # noqa: BLE001 — one bad plugin must not abort the scan | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 276 | except Exception: # noqa: BLE001 — unknown module: fall back to instance path | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 369 | except Exception as exc: # noqa: BLE001 — one bad detector must not abort the scan | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 441 | except Exception as exc: # noqa: BLE001 — a broken detector skips, not aborts | CODE |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 514 | except Exception as exc: # noqa: BLE001 — one bad probe must not abort the scan | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 150 | except Exception as exc: | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 286 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 319 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 382 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| MEDIUM | libs/giskard-llm/src/giskard/llm/providers/google.py | 106 | def _import_genai() -> Any: | CODE |
| LOW | libs/giskard-core/src/giskard/core/welcome.py | 31 | except Exception: | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 186 | except Exception as e: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/workflow.py | 474 | except Exception as err: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 192 | except Exception as error: | CODE |
| LOW | …rd-checks/src/giskard/checks/core/interaction/trace.py | 170 | except Exception as error: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 19 | except Exception: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 32 | except Exception: | CODE |
| LOW | …bs/giskard-checks/src/giskard/checks/testing/runner.py | 30 | except Exception as e: | CODE |
| LOW | …/giskard-checks/src/giskard/checks/scenarios/runner.py | 184 | except Exception as caught: | CODE |
| LOW | …skard-checks/src/giskard/checks/builtin/nlp_metrics.py | 147 | except Exception as exc: | CODE |
| LOW | …iskard-checks/src/giskard/checks/builtin/comparison.py | 99 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/README.md | 248 | ### Step 1: Define a custom check | COMMENT |
| LOW | libs/giskard-checks/README.md | 270 | ### Step 2: Define a custom interaction specification | COMMENT |
| LOW | libs/giskard-checks/README.md | 291 | ### Step 3: Verify registration | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 787 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 788 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 789 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 790 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 791 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | …d-scan/src/giskard/scan/integrations/garak/_adapter.py | 139 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 49 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 121 | CODE | |
| LOW | libs/giskard-llm/tests/conftest.py | 49 | CODE | |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 278 | CODE | |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 169 | CODE | |
| LOW | libs/giskard-agents/tests/conftest.py | 32 | CODE | |
| LOW | libs/giskard-checks/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/core/result.py | 747 | CODE | |
| LOW | …/giskard-checks/src/giskard/checks/scenarios/runner.py | 138 | CODE | |
| LOW | …s/giskard-checks/src/giskard/checks/generators/base.py | 72 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/export/junit.py | 58 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 787 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 788 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 789 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 790 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 791 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …ard-scan/src/giskard/scan/integrations/_entry_point.py | 63 | Run an external security scanner against a Giskard target. Requires the scanner's optional extra: ``pip install gis | STRING |
| HIGH | …giskard-scan/src/giskard/scan/integrations/_listing.py | 28 | List selectable item names for a scan tool. Args: tool: ``"giskard"`` returns scenario generator class name | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-core/src/giskard/core/discriminated.py | 100 | # Check if the class is directly registered | COMMENT |
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 33 | # Check if the legacy package is installed | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 81 | # Check if this parameter is a RunContext | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 273 | # Check if this is a class method by looking for 'self' as first parameter | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 296 | # Check if extraction failed | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 312 | # Check if keyword appears in text | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 454 | # Check if extraction failed | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | README.md | 205 | ### [RAGET](https://legacy-docs.giskard.ai/en/stable/open_source/testset_generation/index.html) — generate evaluation da | COMMENT |
| MEDIUM | …skard-agents/src/giskard/agents/embeddings/__init__.py | 4 | # Default embedding model uses Litellm | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | …can/tests/integrations/garak/test_resolve_detectors.py | 151 | # Create a minimal probe with only a skipped detector | COMMENT |
| MEDIUM | …can/tests/integrations/garak/test_resolve_detectors.py | 166 | # Create a minimal attempt with 2 conversations (_evaluate_attempt only | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/tests/integration/test_stateless.py | 123 | expected_value="test@test.com", | STRING |
| LOW | libs/giskard-checks/tests/integration/test_stateful.py | 149 | expected_value="test@test.com", | STRING |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 41 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 123 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 240 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .pre-commit-config.yaml | 39 | args: ["--baseline", ".secrets.baseline", "--exclude-secrets", "YOUR_API_KEY"] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …skard-llm/tests/translators/test_openai_chat_return.py | 158 | """Tool-only turn: `content` is null, `tool_calls` populated, `finish_reason` is `tool_calls`.""" | STRING |
| HIGH | …skard-checks/src/giskard/checks/builtin/rego_policy.py | 162 | undefined, error if evaluation fails or the rule value is not | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-core/src/giskard/core/rate_limiter/base.py | 162 | CODE | |
| MEDIUM | libs/giskard-checks/tests/core/test_scenario.py | 142 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/vulnerability.py | 35 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 38 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …rd-scan/src/giskard/scan/integrations/lidar/_target.py | 41 | COMMENT | |
| LOW | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 1 | # THIS MODULE IS TAKEN FROM Pydantic AI, with only slight modifications. | COMMENT |