🐢 Open-Source Evaluation & Testing library for LLM Agents
This report presents the forensic synthetic code analysis of Giskard-AI/giskard-oss, a Python project with 5,518 GitHub stars. SynthScan v2.0 examined 46,501 lines of code across 317 source files, recording 1639 pattern matches distributed across 15 syntactic categories. The overall adjusted score of 35.8 places this repository in the Strong AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 1639 distinct pattern matches across 15 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/tests/conftest.py | 7 | def isolated_vulnerability_registry(): | CODE |
| LOW | libs/giskard-scan/tests/conftest.py | 18 | def isolated_quality_registry(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 47 | def test_scan_src_imports_upstream_libs_only_from_package_roots(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 5 | def test_prompt_injection_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_concrete.py | 10 | async def test_prompt_injection_scenarios_have_tags(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_public_api.py | 23 | def test_all_public_symbols_importable(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_public_api.py | 33 | def test_vulnerability_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_public_api.py | 41 | def test_vulnerability_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 52 | def test_quality_scan_accepts_target_mode_param(): | CODE |
| LOW | libs/giskard-scan/tests/test_public_api.py | 63 | def test_quality_suite_generator_registry_contains_builtin_generators(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 19 | def test_scenario_generator_is_importable(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 24 | async def test_scenario_generator_generate_scenario_accepts_target_mode(): | CODE |
| LOW | libs/giskard-scan/tests/test_generators_base.py | 44 | def test_dataset_generator_has_dataset_name_field(): | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 39 | async def test_vulnerability_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_vulnerability.py | 92 | async def test_vulnerability_scan_forwards_parallel_and_max_concurrency( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 71 | async def test_quality_scan_runs_registry_scenarios_and_returns_suite_result( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 139 | async def test_quality_scan_forwards_parallel_and_max_concurrency( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 188 | async def test_quality_scan_uses_empty_registry_by_default( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 218 | async def test_quality_scan_warns_and_skips_empty_raw_knowledge_base( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 268 | async def test_quality_scan_configures_knowledge_base_generator( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 328 | async def test_quality_scan_does_not_generate_recommendation_without_failures( | CODE |
| LOW | libs/giskard-scan/tests/test_quality.py | 353 | async def test_quality_scan_hides_recommendation_when_generation_fails( | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 30 | def test_knowledge_base_from_texts_does_not_compute_embeddings(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 43 | def test_normalize_knowledge_base_accepts_existing_instance_and_texts(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 53 | def test_normalize_knowledge_base_rejects_single_string(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 59 | def test_knowledge_base_drops_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 68 | def test_knowledge_base_rejects_all_empty_documents(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/test_knowledge_base.py | 73 | async def test_closest_documents_computes_missing_embeddings_lazily_in_batch(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 99 | async def test_closest_documents_initializes_embeddings_once_for_concurrent_callers(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 123 | async def test_closest_documents_recomputes_all_embeddings_when_any_is_missing(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 150 | async def test_closest_documents_to_text_embeds_query_against_cached_documents(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 172 | async def test_closest_documents_to_text_rejects_empty_query_without_embedding(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 188 | async def test_closest_documents_rejects_non_finite_embeddings(): | CODE |
| LOW | libs/giskard-scan/tests/test_knowledge_base.py | 201 | def test_knowledge_base_documents_are_immutable(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 38 | def test_register_bare_type_adds_instance(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 45 | def test_register_instance_adds_instance(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 51 | def test_register_bare_type_equivalent_to_default_instance(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 57 | def test_register_duplicate_bare_type_raises(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 64 | def test_register_duplicate_instance_raises(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 71 | def test_register_different_parameterized_instances_both_succeed(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 78 | def test_register_bare_type_same_as_default_param_raises(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 86 | def test_register_non_generator_type_raises(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 95 | def test_unregister_bare_type_removes_instance(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 102 | def test_unregister_instance_removes_it(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 109 | def test_unregister_not_registered_raises(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 118 | def test_clear_empties_registry(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 126 | def test_clear_on_empty_registry_is_noop(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_registry.py | 135 | def test_generators_returns_copy(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_registry.py | 161 | def test_generators_commercial_use_false_returns_all(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_registry.py | 168 | def test_generators_commercial_use_true_excludes_non_commercial(): | CODE |
| LOW | …giskard-scan/tests/generators/test_prompt_injection.py | 7 | async def test_prompt_injection_generator_loads_scenarios(): | CODE |
| LOW | …giskard-scan/tests/generators/test_prompt_injection.py | 15 | async def test_prompt_injection_generator_injects_annotations(): | CODE |
| LOW | …giskard-scan/tests/generators/test_prompt_injection.py | 36 | async def test_prompt_injection_multiturn_keeps_dataset_max_steps(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_catalog.py | 48 | async def test_generate_suite_requires_explicit_generators(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_catalog.py | 57 | async def test_generate_suite_does_not_read_vulnerability_registry( | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_catalog.py | 69 | async def test_generate_suite_generators_bare_type_is_normalized(): | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_catalog.py | 79 | async def test_generate_suite_empty_generators_returns_empty_suite( | CODE |
| LOW⚡ | libs/giskard-scan/tests/generators/test_catalog.py | 87 | async def test_generate_suite_max_scenarios_limits_output(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_catalog.py | 100 | async def test_generate_suite_max_scenarios_distributed_across_generators(): | CODE |
| LOW | libs/giskard-scan/tests/generators/test_catalog.py | 130 | async def test_generate_suite_no_max_passes_none_to_generators(): | CODE |
| 1074 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 8 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 9 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 10 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 11 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 12 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 13 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 14 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 22 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 23 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 23 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 24 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 25 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 26 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 26 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 27 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 27 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 3 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 3 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 4 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 5 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 6 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 7 | CODE | |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 15 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 3 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 3 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 3 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 3 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 9 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 10 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 11 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 11 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 11 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 16 | CODE | |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 17 | CODE | |
| LOW | libs/giskard-llm/tests/test_providers.py | 29 | CODE | |
| LOW | libs/giskard-llm/tests/test_providers.py | 30 | CODE | |
| LOW | libs/giskard-llm/tests/test_smoke.py | 15 | CODE | |
| LOW | libs/giskard-llm/tests/test_smoke.py | 36 | CODE | |
| LOW | libs/giskard-llm/tests/test_smoke.py | 36 | CODE | |
| LOW | libs/giskard-llm/tests/test_smoke.py | 36 | CODE | |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 5 | CODE | |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 6 | CODE | |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 6 | CODE | |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 6 | CODE | |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 6 | CODE | |
| 323 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 213 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_generator.py | 215 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 194 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 196 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 255 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-agents/tests/test_tools.py | 257 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 355 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-agents/tests/test_tools.py | 357 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 2 | # ============================================================================== | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 164 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 166 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 247 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 249 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 311 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | libs/giskard-checks/tests/builtin/test_composition.py | 313 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 28 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 30 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 61 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 63 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 355 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | libs/giskard-checks/tests/builtin/test_composition.py | 357 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/__init__.py | 33 | __all__ = [ | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/utils/__init__.py | 5 | __all__ = [ | CODE |
| LOW | …s/giskard-scan/src/giskard/scan/generators/__init__.py | 17 | __all__ = [ | CODE |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 18 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-scan/src/giskard/scan/generators/base.py | 13 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard/scan/generators/knowledge_base/multi_topic.py | 20 | logger = logging.getLogger(__name__) | CODE |
| LOW | …src/giskard/scan/generators/knowledge_base/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …giskard/scan/generators/knowledge_base/out_of_scope.py | 25 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/__init__.py | 40 | __all__ = [ | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/openai_chat.py | 26 | logger = logging.getLogger(__name__) | CODE |
| LOW | …giskard-llm/src/giskard/llm/translators/google_chat.py | 59 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 56 | logger = logging.getLogger(__name__) | CODE |
| LOW | …ard-llm/src/giskard/llm/translators/openai_response.py | 22 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/types/__init__.py | 68 | __all__ = [ | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 94 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/openai.py | 84 | logger = logging.getLogger(__name__) | CODE |
| LOW | …/giskard-llm/src/giskard/llm/providers/azure_openai.py | 47 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/anthropic.py | 78 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/azure_ai.py | 58 | logger = logging.getLogger(__name__) | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/utils/__init__.py | 7 | __all__ = [ | CODE |
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 46 | __all__ = [ | CODE |
| LOW | …bs/giskard-core/src/giskard/core/telemetry/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 82 | def _set_tags() -> None: | CODE |
| LOW | …giskard-core/src/giskard/core/rate_limiter/__init__.py | 9 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/__init__.py | 21 | __all__ = [ | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/pipeline.py | 5 | __all__ = ["Pipeline"] | CODE |
| LOW | …bs/giskard-agents/src/giskard/agents/tools/__init__.py | 5 | __all__ = ["Tool", "tool", "Function", "ToolCall"] | CODE |
| LOW | …skard-agents/src/giskard/agents/embeddings/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …iskard-agents/src/giskard/agents/templates/__init__.py | 12 | __all__ = [ | CODE |
| LOW | …skard-agents/src/giskard/agents/generators/__init__.py | 20 | __all__ = [ | CODE |
| LOW | …s/giskard-agents/src/giskard/agents/errors/__init__.py | 4 | __all__ = ["ModelRefusalError", "WorkflowError", "Error"] | CODE |
| LOW | libs/giskard-checks/tests/generators/conftest.py | 3 | __all__ = ["LLMTrace", "MockGenerator"] | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/__init__.py | 82 | __all__ = [ | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/settings.py | 65 | def set_default_generator(generator: BaseGenerator) -> None: | CODE |
| LOW | libs/giskard-checks/src/giskard/checks/core/__init__.py | 19 | __all__ = [ | CODE |
| LOW | …checks/src/giskard/checks/core/interaction/__init__.py | 6 | __all__ = ["InteractionSpec", "Interact", "Interaction", "Trace"] | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/judges/__init__.py | 11 | __all__ = [ | CODE |
| LOW | …/giskard-checks/src/giskard/checks/testing/__init__.py | 3 | __all__ = ["WithSpy"] | CODE |
| LOW | …iskard-checks/src/giskard/checks/scenarios/__init__.py | 4 | __all__ = [ | CODE |
| LOW | …skard-checks/src/giskard/checks/generators/__init__.py | 7 | __all__ = [ | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/export/__init__.py | 4 | __all__ = ["to_hub_format", "to_junit_xml"] | CODE |
| LOW | …/giskard-checks/src/giskard/checks/builtin/__init__.py | 35 | __all__ = [ | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/README.md | 243 | ### Step 1: Define a custom check | COMMENT |
| LOW | libs/giskard-checks/README.md | 263 | ### Step 2: Define a custom interaction specification | COMMENT |
| LOW | libs/giskard-checks/README.md | 284 | ### Step 3: Verify registration | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 681 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 682 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 683 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 684 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 685 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/src/giskard/scan/quality.py | 98 | except Exception: | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 148 | except Exception as exc: | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 281 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 314 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| LOW | libs/giskard-llm/src/giskard/llm/providers/google.py | 377 | except Exception as e: # Broad catch: _map_error checks SDK types first, then applies timeout heuristic, then r | CODE |
| MEDIUM | libs/giskard-llm/src/giskard/llm/providers/google.py | 104 | def _import_genai() -> Any: | CODE |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 193 | except Exception as e: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/workflow.py | 474 | except Exception as err: | CODE |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 192 | except Exception as error: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 18 | except Exception: | CODE |
| LOW | …s/giskard-checks/src/giskard/checks/utils/inference.py | 31 | except Exception: | CODE |
| LOW | …bs/giskard-checks/src/giskard/checks/testing/runner.py | 30 | except Exception as e: | CODE |
| LOW | …skard-checks/src/giskard/checks/builtin/nlp_metrics.py | 147 | except Exception as exc: | CODE |
| LOW | …iskard-checks/src/giskard/checks/builtin/comparison.py | 95 | except Exception: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-scan/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 49 | CODE | |
| LOW | …iskard-scan/src/giskard/scan/generators/huggingface.py | 121 | CODE | |
| LOW | libs/giskard-llm/tests/conftest.py | 49 | CODE | |
| LOW | …ard-llm/src/giskard/llm/translators/google_response.py | 273 | CODE | |
| LOW | …s/giskard-core/src/giskard/core/telemetry/telemetry.py | 176 | CODE | |
| LOW | libs/giskard-agents/tests/conftest.py | 32 | CODE | |
| LOW | libs/giskard-checks/tests/test_public_import_surface.py | 29 | CODE | |
| LOW | …giskard-checks/tests/core/test_jsonpath_enforcement.py | 55 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/core/result.py | 678 | CODE | |
| LOW | …s/giskard-checks/src/giskard/checks/generators/base.py | 72 | CODE | |
| LOW | libs/giskard-checks/src/giskard/checks/export/junit.py | 58 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 681 | # Step 1: interaction0, check1 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 682 | # Step 2: interaction2, check3 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 683 | # Step 3: interaction4, check5 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 684 | # Step 4: interaction6, check7 | COMMENT |
| LOW⚡ | libs/giskard-checks/tests/core/test_scenario.py | 685 | # Step 5: interaction8, check9 | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-core/src/giskard/core/discriminated.py | 77 | # Check if the class is directly registered | COMMENT |
| LOW | libs/giskard-core/src/giskard/core/__init__.py | 33 | # Check if the legacy package is installed | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 81 | # Check if this parameter is a RunContext | COMMENT |
| LOW | libs/giskard-agents/src/giskard/agents/tools/tool.py | 273 | # Check if this is a class method by looking for 'self' as first parameter | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 292 | # Check if extraction failed | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 308 | # Check if keyword appears in text | COMMENT |
| LOW | …ard-checks/src/giskard/checks/builtin/text_matching.py | 450 | # Check if extraction failed | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | README.md | 171 | ### [RAGET](https://legacy-docs.giskard.ai/en/stable/open_source/testset_generation/index.html) — generate evaluation da | COMMENT |
| MEDIUM | …skard-agents/src/giskard/agents/embeddings/__init__.py | 4 | # Default embedding model uses Litellm | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | libs/giskard-checks/tests/integration/test_stateless.py | 123 | expected_value="test@test.com", | STRING |
| LOW | libs/giskard-checks/tests/integration/test_stateful.py | 149 | expected_value="test@test.com", | STRING |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 41 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 123 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| LOW⚡ | libs/giskard-checks/tests/generators/test_dataset.py | 240 | with pytest.raises(ValueError, match="placeholder"): | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | .pre-commit-config.yaml | 39 | args: ["--baseline", ".secrets.baseline", "--exclude-secrets", "YOUR_API_KEY"] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …skard-llm/tests/translators/test_openai_chat_return.py | 158 | """Tool-only turn: `content` is null, `tool_calls` populated, `finish_reason` is `tool_calls`.""" | STRING |
| HIGH | …skard-checks/src/giskard/checks/builtin/rego_policy.py | 162 | undefined, error if evaluation fails or the rule value is not | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | libs/giskard-core/src/giskard/core/rate_limiter/base.py | 145 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …d-agents/src/giskard/agents/tools/_docstring_parser.py | 1 | # THIS MODULE IS TAKEN FROM Pydantic AI, with only slight modifications. | COMMENT |