data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
This report presents the forensic synthetic code analysis of dlt-hub/dlt, a Python project with 5,723 GitHub stars. SynthScan v2.0 examined 478,847 lines of code across 1931 source files, recording 7383 pattern matches distributed across 26 syntactic categories. The overall adjusted score of 16.7 places this repository in the Moderate AI signal band.
The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).
Longitudinal tracking requires multiple scan runs. Once this repository is re-scanned after new commits land, this chart will visualise how the synthetic code signal evolves over time — enabling you to detect whether AI authorship is growing, stabilising, or being actively corrected by human engineers.
Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.
This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.
The scanner identified 7383 distinct pattern matches across 26 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/install_airflow_ci.py | 22 | def is_supported_airflow_version(version: Version, specifier: SpecifierSet) -> bool: | CODE |
| LOW | tools/check_hub_extras.py | 56 | def dlt_version_to_plugin_version(dlt_ver: str) -> str: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 19 | def test_breakage_source_file_direct() -> None: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 26 | def test_breakage_source_file_alias_uses_parent() -> None: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 36 | def test_breakage_source_file_without_filepath_returns_none(exc: Type[Exception]) -> None: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 41 | def relative_package_filepath(self) -> Path: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 47 | def test_public_api_reports_roots_and_sources() -> None: | CODE |
| LOW⚡ | tools/tests/test_check_api_breaking.py | 55 | def test_cmd_check_propagates_griffe_failure(monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | tools/tests/test_check_dependency_changes.py | 161 | def test_multi_section_changes(monkeypatch: pytest.MonkeyPatch) -> None: | CODE |
| LOW | dlt/version.py | 13 | def get_installed_requirement_string( | CODE |
| LOW | dlt/version.py | 37 | def get_dependency_requirement( | CODE |
| LOW | dlt/pipeline/configuration.py | 75 | def ensure_correct_pipeline_kwargs(f: AnyFun, **kwargs: Any) -> None: | CODE |
| LOW | dlt/pipeline/configuration.py | 81 | def get_default_pipeline_name(entry_point_file: str) -> str: | CODE |
| LOW | dlt/pipeline/track.py | 30 | def slack_notify_load_success(incoming_hook: str, load_info: LoadInfo, trace: PipelineTrace) -> int: | CODE |
| LOW | dlt/pipeline/warnings.py | 7 | def full_refresh_argument_deprecated(caller_name: str, full_refresh: t.Optional[bool]) -> None: | CODE |
| LOW | dlt/pipeline/platform.py | 80 | def _sync_schemas_to_platform(trace: PipelineTrace, pipeline: SupportsPipeline) -> None: | CODE |
| LOW⚡ | dlt/pipeline/pipeline.py | 1148 | def list_extracted_load_packages(self) -> Sequence[str]: | CODE |
| LOW⚡ | dlt/pipeline/pipeline.py | 1155 | def list_normalized_load_packages(self) -> Sequence[str]: | CODE |
| LOW⚡ | dlt/pipeline/pipeline.py | 1162 | def list_completed_load_packages(self) -> Sequence[str]: | CODE |
| LOW | dlt/pipeline/pipeline.py | 951 | def _install_state_and_schemas( | CODE |
| LOW | dlt/pipeline/pipeline.py | 1189 | def list_failed_jobs_in_package(self, load_id: str) -> Sequence[LoadJobInfo]: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1196 | def list_pending_retry_jobs_in_package( | CODE |
| LOW | dlt/pipeline/pipeline.py | 1466 | def _verify_destination_capabilities( | CODE |
| LOW | dlt/pipeline/pipeline.py | 1521 | def _normalize_storage_config(self) -> NormalizeStorageConfiguration: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1716 | def _get_destination_capabilities(self) -> DestinationCapabilitiesContext: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1737 | def _get_staging_capabilities(self) -> Optional[DestinationCapabilitiesContext]: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1831 | def _maybe_destination_capabilities( | CODE |
| LOW | dlt/pipeline/pipeline.py | 1899 | def _create_pipeline_instance_id(self) -> str: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1928 | def _restore_state_from_destination(self) -> Optional[TPipelineState]: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1962 | def _get_schemas_from_destination( | CODE |
| LOW | dlt/pipeline/pipeline.py | 2084 | def _save_and_extract_state_and_schema( | CODE |
| LOW | dlt/pipeline/pipeline.py | 2108 | def _bump_version_and_extract_state( | CODE |
| LOW | dlt/pipeline/state_sync.py | 31 | def generate_pipeline_state_version_hash(state: TPipelineState) -> str: | CODE |
| LOW | dlt/pipeline/state_sync.py | 35 | def bump_pipeline_state_version_if_modified(state: TPipelineState) -> Tuple[int, str, str]: | CODE |
| LOW | dlt/pipeline/state_sync.py | 133 | def load_pipeline_state_from_destination( | CODE |
| LOW | dlt/destinations/type_mapping.py | 96 | def precision_tuple_or_default( | CODE |
| LOW | dlt/destinations/sql_client.py | 233 | def fully_qualified_dataset_name( | CODE |
| LOW | dlt/destinations/sql_client.py | 244 | def make_qualified_table_name( | CODE |
| LOW | dlt/destinations/sql_client.py | 252 | def make_qualified_table_name_path( | CODE |
| LOW | dlt/destinations/sql_client.py | 291 | def get_qualified_table_names( | CODE |
| LOW | dlt/destinations/sql_client.py | 314 | def with_alternative_dataset_name( | CODE |
| LOW | dlt/destinations/sql_client.py | 331 | def is_staging_dataset_active(self) -> bool: | CODE |
| LOW | dlt/destinations/sql_client.py | 354 | def _qualify_info_schema_table_name(self, table_name: str) -> str: | CODE |
| LOW | dlt/destinations/sql_client.py | 357 | def _get_information_schema_components(self, *tables: str) -> Tuple[str, str, List[str]]: | CODE |
| LOW | dlt/destinations/sql_client.py | 465 | def _set_default_schema_columns(self) -> None: | CODE |
| LOW | dlt/destinations/sql_client.py | 556 | def raise_open_connection_error(f: TFun) -> TFun: | CODE |
| LOW | dlt/destinations/utils.py | 68 | def parse_db_data_type_str_with_precision(db_type: str) -> Tuple[str, Optional[int], Optional[int]]: | CODE |
| LOW | dlt/destinations/utils.py | 84 | def get_pipeline_state_query_columns() -> TTableSchema: | CODE |
| LOW | dlt/destinations/utils.py | 92 | def verify_schema_replace_disposition( | CODE |
| LOW | dlt/destinations/utils.py | 130 | def verify_schema_merge_disposition( | CODE |
| LOW | dlt/destinations/utils.py | 302 | def get_deterministic_temp_table_name(table_name: str, op: str) -> str: | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 378 | def should_truncate_table_before_load(self, table_name: str) -> bool: | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 385 | def _create_append_followup_jobs( | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 390 | def _create_merge_followup_jobs( | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 395 | def _create_replace_followup_jobs( | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 404 | def create_table_chain_completed_followup_jobs( | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 870 | def _delete_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 879 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/job_client_impl.py | 127 | def _string_contains_ddl_statements(self, sql: str) -> bool: | CODE |
| LOW | dlt/destinations/job_client_impl.py | 133 | def _has_out_of_transaction_commands(self, sql: str) -> bool: | CODE |
| 4680 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/check_dependency_changes.py | 36 | CODE | |
| LOW | tools/tests/test_check_api_breaking.py | 3 | CODE | |
| LOW | tools/tests/test_check_api_breaking.py | 3 | CODE | |
| LOW | dlt/__init__.py | 23 | CODE | |
| LOW | dlt/__init__.py | 24 | CODE | |
| LOW | dlt/__init__.py | 24 | CODE | |
| LOW | dlt/__init__.py | 27 | CODE | |
| LOW | dlt/__init__.py | 29 | CODE | |
| LOW | dlt/__init__.py | 30 | CODE | |
| LOW | dlt/__init__.py | 30 | CODE | |
| LOW | dlt/__init__.py | 30 | CODE | |
| LOW | dlt/__init__.py | 30 | CODE | |
| LOW | dlt/__init__.py | 31 | CODE | |
| LOW | dlt/__init__.py | 32 | CODE | |
| LOW | dlt/__init__.py | 32 | CODE | |
| LOW | dlt/__init__.py | 32 | CODE | |
| LOW | dlt/__init__.py | 34 | CODE | |
| LOW | dlt/__init__.py | 34 | CODE | |
| LOW | dlt/__init__.py | 34 | CODE | |
| LOW | dlt/__init__.py | 34 | CODE | |
| LOW | dlt/__init__.py | 43 | CODE | |
| LOW | dlt/__init__.py | 44 | CODE | |
| LOW | dlt/__init__.py | 45 | CODE | |
| LOW | dlt/__plugins__.py | 9 | CODE | |
| LOW | dlt/__plugins__.py | 10 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/mark.py | 2 | CODE | |
| LOW | dlt/pipeline/track.py | 2 | CODE | |
| LOW | dlt/pipeline/track.py | 18 | CODE | |
| LOW | dlt/pipeline/current.py | 1 | CODE | |
| LOW | dlt/pipeline/current.py | 7 | CODE | |
| LOW | dlt/pipeline/current.py | 7 | CODE | |
| LOW | dlt/pipeline/current.py | 19 | CODE | |
| LOW | dlt/pipeline/platform.py | 14 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 7 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 115 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 124 | CODE | |
| LOW | dlt/pipeline/helpers.py | 28 | CODE | |
| LOW | dlt/pipeline/drop.py | 1 | CODE | |
| LOW | dlt/pipeline/drop.py | 1 | CODE | |
| LOW | dlt/pipeline/drop.py | 17 | CODE | |
| LOW | dlt/destinations/sql_client.py | 6 | CODE | |
| LOW | dlt/destinations/sql_client.py | 6 | CODE | |
| LOW | dlt/destinations/sql_client.py | 6 | CODE | |
| LOW | dlt/destinations/sql_client.py | 6 | CODE | |
| LOW | dlt/destinations/sql_client.py | 35 | CODE | |
| LOW | dlt/destinations/sql_client.py | 35 | CODE | |
| LOW | dlt/destinations/sql_client.py | 35 | CODE | |
| LOW | dlt/destinations/sql_client.py | 42 | CODE | |
| LOW | dlt/destinations/sql_client.py | 56 | CODE | |
| LOW | dlt/destinations/sql_client.py | 57 | CODE | |
| LOW | dlt/destinations/sql_client.py | 500 | CODE | |
| LOW | dlt/destinations/configuration.py | 2 | CODE | |
| LOW | dlt/destinations/job_impl.py | 1 | CODE | |
| LOW | dlt/destinations/job_impl.py | 33 | CODE | |
| LOW | dlt/destinations/__init__.py | 1 | CODE | |
| 1230 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/check_api_breaking.py | 60 | except Exception: | CODE |
| LOW | tools/check_api_breaking.py | 72 | except Exception: | CODE |
| LOW | tools/check_dependency_changes.py | 101 | except Exception: | CODE |
| MEDIUM | tools/check_dependency_changes.py | 246 | print(f"Error: no merge-base between {base_ref} and {head_ref}") | CODE |
| LOW | tools/check_hub_extras.py | 112 | except Exception as e: | CODE |
| LOW | dlt/pipeline/trace.py | 356 | except Exception as ex: | CODE |
| LOW | dlt/pipeline/track.py | 54 | except Exception as ex: | CODE |
| LOW | dlt/pipeline/platform.py | 70 | except Exception as e: | CODE |
| LOW | dlt/pipeline/platform.py | 115 | except Exception as e: | CODE |
| MEDIUM | dlt/pipeline/platform.py | 61 | def _future_send() -> None: | CODE |
| MEDIUM | dlt/pipeline/platform.py | 107 | def _future_send() -> None: | CODE |
| MEDIUM | dlt/pipeline/pipeline.py | 1109 | def __repr__(self) -> str: | CODE |
| MEDIUM | dlt/pipeline/pipeline.py | 1750 | def _validate_pipeline_name(self) -> None: | CODE |
| LOW | dlt/pipeline/pipeline.py | 191 | except Exception: | CODE |
| LOW | dlt/pipeline/pipeline.py | 239 | except Exception as ex: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1132 | except Exception: | CODE |
| LOW | dlt/pipeline/pipeline.py | 2017 | except Exception: | CODE |
| LOW | dlt/pipeline/pipeline.py | 2244 | except Exception: | CODE |
| LOW | dlt/pipeline/helpers.py | 168 | except Exception: | CODE |
| LOW | dlt/destinations/sql_client.py | 533 | except Exception as ex: | CODE |
| LOW | dlt/destinations/sql_client.py | 544 | except Exception as ex: | CODE |
| LOW | dlt/destinations/sql_client.py | 561 | except Exception as ex: | CODE |
| LOW | dlt/destinations/sql_jobs.py | 70 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/clickhouse/utils.py | 38 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/dremio/pydremio.py | 132 | except Exception: | CODE |
| LOW | dlt/destinations/impl/dremio/pydremio.py | 258 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 60 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 74 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 80 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 101 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 115 | except Exception: | CODE |
| LOW | dlt/destinations/impl/databricks/configuration.py | 135 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/databricks/configuration.py | 55 | def on_resolved(self) -> None: | CODE |
| LOW | dlt/destinations/impl/lance/lance_client.py | 191 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/lance/lance_client.py | 216 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/lance/lance_client.py | 467 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/lance/exceptions.py | 80 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/bigquery/sql_client.py | 161 | except Exception: | CODE |
| LOW | dlt/destinations/impl/bigquery/sql_client.py | 167 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/bigquery/sql_client.py | 137 | def begin_transaction(self) -> Iterator[DBTransaction]: | CODE |
| LOW | dlt/destinations/impl/snowflake/sql_client.py | 102 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/snowflake/sql_client.py | 97 | def begin_transaction(self) -> Iterator[DBTransaction]: | CODE |
| LOW | dlt/destinations/impl/snowflake/configuration.py | 68 | except Exception: | CODE |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 836 | except Exception as e: | CODE |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 850 | except Exception as rm_e: | CODE |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 1590 | except Exception as ex: | CODE |
| LOW | dlt/destinations/impl/sqlalchemy/configuration.py | 68 | except Exception: | CODE |
| LOW | dlt/destinations/impl/sqlalchemy/db_api_client.py | 138 | except Exception: | CODE |
| LOW | dlt/destinations/impl/sqlalchemy/db_api_client.py | 147 | except Exception: | CODE |
| LOW | dlt/destinations/impl/sqlalchemy/db_api_client.py | 173 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/sqlalchemy/db_api_client.py | 68 | def _get_columns(self) -> List[str]: | CODE |
| LOW | dlt/destinations/impl/redshift/factory.py | 196 | except Exception: | CODE |
| LOW | dlt/destinations/impl/postgres/sql_client.py | 69 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/postgres/sql_client.py | 64 | def begin_transaction(self) -> Iterator[DBTransaction]: | CODE |
| LOW | dlt/destinations/impl/postgres/factory.py | 144 | except Exception: | CODE |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 212 | except Exception: | CODE |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 221 | except Exception: | CODE |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 265 | except Exception: | CODE |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 285 | except Exception: | CODE |
| MEDIUM | dlt/destinations/impl/duckdb/sql_client.py | 208 | def begin_transaction(self) -> Iterator[DBTransaction]: | CODE |
| 248 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | dlt/common/data_writers/writers.py | 663 | elif promote_options != "none" and not table.schema.equals( | CODE |
| HIGH | dlt/common/data_writers/writers.py | 677 | if promote_options == "none" and not table.schema.equals( | CODE |
| HIGH | dlt/common/data_writers/writers.py | 779 | if not item.schema.equals(self._first_schema): | CODE |
| HIGH | dlt/sources/helpers/rest_client/detector.py | 86 | # when a list was returned (or in rare case a simple type or null) | COMMENT |
| HIGH | dlt/helpers/mermaid.py | 72 | # (columns whose first observed values were all null). if that propagation is removed so that | COMMENT |
| HIGH | tests/pipeline/test_arrow_sources.py | 95 | assert tbl_expected.schema.equals(tbl.schema) | CODE |
| HIGH | tests/pipeline/test_arrow_sources.py | 100 | assert df_result.equals(df_tbl) | CODE |
| HIGH | tests/pipeline/test_arrow_sources.py | 407 | assert actual_tbl_no_binary.schema.equals(shuffled_removed_column.schema) | CODE |
| HIGH | tests/pipeline/test_arrow_sources.py | 428 | assert actual_tbl.schema.equals(actual_tbl_shuffled.schema) | CODE |
| HIGH | tests/pipeline/test_import_export_schema.py | 280 | # 1. Column 'b' is null, should NOT appear in schema at all | COMMENT |
| HIGH | tests/pipeline/test_import_export_schema.py | 341 | # 1. Column 'a' is null, should NOT appear in schema at all | COMMENT |
| HIGH | tests/pipeline/test_pipeline_extra.py | 404 | "child": '{"child_attribute":"any string","optional_child_attribute":null}', | CODE |
| HIGH | tests/libs/test_ibis.py | 82 | assert expected_schema.equals(ibis_schema) | CODE |
| HIGH | tests/libs/test_deltalake.py | 144 | assert not dt_arrow_table.equals(arrow_table) | CODE |
| HIGH⚡ | tests/libs/pyarrow/test_pyarrow.py | 511 | assert new_table.equals(expected_table) | CODE |
| HIGH | tests/libs/pyarrow/test_row_tuples_to_arrow.py | 164 | assert arrow_array.equals(expected_arrow_array) | CODE |
| HIGH | tests/libs/pyarrow/test_row_tuples_to_arrow.py | 774 | assert tbl.equals(expected) | CODE |
| HIGH | tests/libs/pyarrow/test_row_tuples_to_arrow.py | 842 | assert tbl.equals(expected) | CODE |
| HIGH | tests/common/configuration/test_toml_provider.py | 491 | # yaml fragment with a null, a list and a table replacing what was a plain value | COMMENT |
| HIGH | tests/load/test_job_client.py | 1207 | # use third schema where one of the fields is non null, but the field exists so it is ignored | COMMENT |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 614 | assert actual.equals(expected) | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 637 | assert actual.schema.equals(expected.schema) | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 639 | assert actual.schema.equals(expected.schema) | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 689 | assert actual.schema.equals(expected.schema) | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 767 | assert empty_arrow_table.schema.equals(arrow_table.schema) # it has a schema | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 780 | assert actual.schema.equals(expected.schema) | CODE |
| HIGH | tests/load/pipeline/test_open_table_pipeline.py | 791 | assert actual.schema.equals(expected.schema) | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 300 | "id" bigint [pk, not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 303 | "_dlt_load_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 304 | "_dlt_id" text [unique, not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 308 | "id" bigint [pk, not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 313 | "_dlt_load_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 314 | "_dlt_id" text [unique, not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 318 | "purchase_id" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 319 | "name" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 320 | "price" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 321 | "_dlt_root_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 322 | "_dlt_parent_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 323 | "_dlt_list_idx" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 324 | "_dlt_id" text [unique, not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 328 | "version" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 329 | "engine_version" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 330 | "inserted_at" timestamp [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 331 | "schema_name" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 332 | "version_hash" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 333 | "schema" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 340 | "load_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 342 | "status" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 343 | "inserted_at" timestamp [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 351 | "version" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 352 | "engine_version" bigint [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 353 | "pipeline_name" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 354 | "state" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 355 | "created_at" timestamp [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 357 | "_dlt_load_id" text [not null] | CODE |
| HIGH⚡ | tests/helpers/test_dbml.py | 358 | "_dlt_id" text [unique, not null] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tools/check_api_breaking.py | 46 | CODE | |
| LOW | tools/check_dependency_changes.py | 110 | CODE | |
| LOW | tools/check_dependency_changes.py | 142 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 442 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 855 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 1773 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 1928 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 1962 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 2030 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 2184 | CODE | |
| LOW | dlt/pipeline/exceptions.py | 51 | CODE | |
| LOW | dlt/destinations/type_mapping.py | 73 | CODE | |
| LOW | dlt/destinations/utils.py | 92 | CODE | |
| LOW | dlt/destinations/utils.py | 130 | CODE | |
| LOW | dlt/destinations/job_client_impl.py | 777 | CODE | |
| LOW | dlt/destinations/sql_jobs.py | 168 | CODE | |
| LOW | dlt/destinations/sql_jobs.py | 800 | CODE | |
| LOW | dlt/destinations/insert_job_client.py | 31 | CODE | |
| LOW | dlt/destinations/file_batching.py | 62 | CODE | |
| LOW | dlt/destinations/path_utils.py | 133 | CODE | |
| LOW | dlt/destinations/impl/clickhouse/sql_client.py | 260 | CODE | |
| LOW | dlt/destinations/impl/clickhouse/sql_client.py | 330 | CODE | |
| LOW | dlt/destinations/impl/clickhouse/clickhouse.py | 84 | CODE | |
| LOW | dlt/destinations/impl/clickhouse/clickhouse.py | 480 | CODE | |
| LOW | dlt/destinations/impl/dremio/sql_client.py | 142 | CODE | |
| LOW | dlt/destinations/impl/dremio/pydremio.py | 215 | CODE | |
| LOW | dlt/destinations/impl/databricks/sql_client.py | 161 | CODE | |
| LOW | dlt/destinations/impl/databricks/configuration.py | 55 | CODE | |
| LOW | dlt/destinations/impl/databricks/databricks_adapter.py | 23 | CODE | |
| LOW | dlt/destinations/impl/databricks/factory.py | 74 | CODE | |
| LOW | dlt/destinations/impl/databricks/databricks.py | 205 | CODE | |
| LOW | dlt/destinations/impl/databricks/databricks.py | 288 | CODE | |
| LOW | dlt/destinations/impl/databricks/databricks.py | 682 | CODE | |
| LOW | dlt/destinations/impl/lance/lance_client.py | 367 | CODE | |
| LOW | dlt/destinations/impl/bigquery/bigquery.py | 89 | CODE | |
| LOW | dlt/destinations/impl/bigquery/bigquery.py | 324 | CODE | |
| LOW | dlt/destinations/impl/bigquery/bigquery.py | 391 | CODE | |
| LOW | dlt/destinations/impl/bigquery/bigquery.py | 676 | CODE | |
| LOW | dlt/destinations/impl/snowflake/sql_client.py | 172 | CODE | |
| LOW | dlt/destinations/impl/snowflake/utils.py | 53 | CODE | |
| LOW | dlt/destinations/impl/filesystem/sql_client.py | 73 | CODE | |
| LOW | dlt/destinations/impl/filesystem/sql_client.py | 151 | CODE | |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 811 | CODE | |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 829 | CODE | |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 1260 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/configuration.py | 247 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/merge_job.py | 36 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/load_jobs.py | 89 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/load_jobs.py | 143 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/db_api_client.py | 458 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/type_mapper.py | 35 | CODE | |
| LOW | dlt/destinations/impl/sqlalchemy/type_mapper.py | 184 | CODE | |
| LOW | dlt/destinations/impl/postgres/sql_client.py | 128 | CODE | |
| LOW | dlt/destinations/impl/qdrant/qdrant_job_client.py | 264 | CODE | |
| LOW | dlt/destinations/impl/qdrant/qdrant_job_client.py | 311 | CODE | |
| LOW | dlt/destinations/impl/qdrant/qdrant_job_client.py | 494 | CODE | |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 377 | CODE | |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 535 | CODE | |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 711 | CODE | |
| LOW | dlt/destinations/impl/duckdb/configuration.py | 167 | CODE | |
| 285 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | dlt/pipeline/__init__.py | 302 | Loads the data in `data` argument into the destination specified in `destination` and dataset specified in `dataset_name | STRING |
| HIGH | dlt/pipeline/pipeline.py | 715 | Loads the data from `data` argument into the destination specified in `destination` and dataset specified in `dataset_na | STRING |
| HIGH | dlt/destinations/utils.py | 261 | Converts a query string from the new pyformat style to the old pyformat style. The new pyformat style uses placehol | STRING |
| HIGH | dlt/destinations/impl/clickhouse/clickhouse.py | 92 | Creates the ClickHouse table function for loading from cloud storage. Args: bucket_scheme: The sche | STRING |
| HIGH | dlt/destinations/impl/clickhouse/clickhouse_adapter.py | 35 | Adapts the given data by applying Clickhouse-specific hints. Args: data (Any): The data to be transformed. | STRING |
| HIGH | dlt/destinations/impl/databricks/databricks_adapter.py | 34 | Prepares data for loading into Databricks. This function takes data, which can be raw or already wrapped in a | STRING |
| HIGH | dlt/destinations/impl/lance/lance_adapter.py | 21 | Prepares data for the Lance destination by specifying which columns should be embedded. Args: data (Any): T | STRING |
| HIGH | dlt/destinations/impl/bigquery/bigquery_adapter.py | 86 | Prepares data for loading into BigQuery. This function takes data, which can be raw or already wrapped in a Dl | STRING |
| HIGH | …destinations/impl/filesystem/iceberg_partition_spec.py | 49 | Get the PyIceberg Transform object for a partition spec. Args: spec: The PartitionSpec to get the transform | STRING |
| HIGH | dlt/destinations/impl/filesystem/iceberg_adapter.py | 155 | Prepares data or a DltResource for loading into Apache Iceberg table. Takes raw data or an existing DltResource and | STRING |
| HIGH | dlt/destinations/impl/postgres/postgres_adapter.py | 16 | Prepares data for the postgres destination by specifying which columns should be cast to PostGIS geometry types. | STRING |
| HIGH | dlt/destinations/impl/qdrant/qdrant_adapter.py | 15 | Prepares data for the Qdrant destination by specifying which columns should be embedded. Args: data (An | STRING |
| HIGH | dlt/destinations/impl/weaviate/weaviate_adapter.py | 23 | Prepares data for the Weaviate destination by specifying which columns should be vectorized and which tokenization m | STRING |
| HIGH | dlt/destinations/impl/lancedb/lancedb_adapter.py | 20 | Prepares data for the LanceDB destination by specifying which columns should be embedded. Args: data (Any): | STRING |
| HIGH | dlt/destinations/impl/synapse/synapse_adapter.py | 21 | Prepares data for the Synapse destination by specifying which table index type should be used. Args: da | STRING |
| HIGH | dlt/destinations/impl/athena/athena_adapter.py | 66 | Prepares data for loading into Athena Args: data: The data to be transformed. This can be | STRING |
| HIGH | dlt/dataset/relation.py | 473 | Join this relation to another table. Without `on`, join conditions are discovered automatically from the | STRING |
| HIGH | dlt/dataset/_incremental.py | 121 | Build the WHERE condition for an Incremental cursor on `column_ref`. Args: incremental (Incremental): The i | STRING |
| HIGH | dlt/common/validation.py | 36 | Validate the `doc` dictionary based on the given typed dictionary specification `spec`. Args: spec (Type[_T | STRING |
| HIGH | dlt/common/configuration/container.py | 164 | Insert ``config`` into the container and restore the previous value on exit. Args: config (TInjecta | STRING |
| HIGH | dlt/common/runtime/telemetry.py | 75 | Decorator factory that attaches telemetry to a callable. The returned decorator wraps a function so that an anonymo | STRING |
| HIGH | dlt/common/destination/client.py | 973 | Returns a copy of this model. The copy carries the new `query` and `dialect`. The new model keeps the attach in | STRING |
| HIGH | dlt/common/destination/client.py | 1014 | Creates a `SqlModel` from a raw SQL query string. The method raises when the query is not a SELECT. Arg | STRING |
| HIGH | dlt/common/destination/reference.py | 210 | Create an ibis backend for this destination bound to `client`'s dataset. Args: client (TDestination | STRING |
| HIGH | dlt/common/libs/pyiceberg.py | 214 | Load Iceberg catalog through pyiceberg load_catalog mechanism. See https://py.iceberg.apache.org/configuration/#setting- | STRING |
| HIGH | dlt/common/libs/pyiceberg.py | 288 | Load Iceberg catalog from configuration dictionary Args: catalog_name: Name of the catalog config_d | STRING |
| HIGH | dlt/common/libs/pyiceberg.py | 332 | Get an Iceberg catalog using multiple configuration methods. This function tries to load a catalog in the following | STRING |
| HIGH | dlt/common/schema/utils.py | 600 | Merges `partial_table` into `table` in place. Merge rules: - New columns from `partial_table` are added to `tab | STRING |
| HIGH | dlt/common/schema/schema.py | 357 | Adds or merges `partial_table` into the schema. Args: partial_table: Table schema to add or merge | STRING |
| HIGH | dlt/common/runners/runnable.py | 66 | Decorator for a process/thread pool worker function facilitates passing bound configuration type across the process boun | STRING |
| HIGH | dlt/common/reflection/inspect.py | 16 | Return True if obj (or an inner layer) has given code flag. The test mimics inspect.isgeneratorfunction but is more | STRING |
| HIGH | dlt/_workspace/mcp/tools/_ai_context_api_client.py | 17 | Search for available sources by name and description. Args: query: Search string. Empty string returns all | STRING |
| HIGH | dlt/_workspace/cli/dlthub/utils.py | 146 | Display paths to be deleted and ask for confirmation. Args: run_context: current run context. skip_ | STRING |
| HIGH | dlt/_workspace/cli/dlthub/ai/utils.py | 345 | Return install-order list of dependencies for a toolkit. Args: name: Toolkit to resolve dependencies for. | STRING |
| HIGH | dlt/_workspace/cli/dlthub/ai/utils.py | 402 | Scan workbench directory and return mapping of toolkit name -> TToolkitInfo. Args: base: Root directory of | STRING |
| HIGH | dlt/_workspace/cli/dlthub/ai/utils.py | 438 | Build a TToolkitInfo from a raw metadata dict. Args: meta: Raw metadata dictionary with toolkit fields. | STRING |
| HIGH | dlt/_workspace/deployment/interval.py | 98 | Half-open `[start, end)` interval for a non-interval job run. `schedule:` and `every:` triggers carry continuity: ` | STRING |
| HIGH | dlt/_workspace/deployment/manifest.py | 605 | Import a module, generate a manifest, and validate it. Args: name_or_path: Python module name or file path. | STRING |
| HIGH | dlt/_workspace/deployment/requirements.py | 302 | Export a workspace's dependencies as a `TWorkspaceRequirementsManifest`. If no default group names dlt, the install | STRING |
| HIGH | dlt/_workspace/deployment/_run_helpers.py | 365 | Resolve a run/serve request to a launchable `TRunJobInfo`. Args: selector: User-supplied positional — a sel | STRING |
| HIGH | dlt/sources/helpers/rest_client/client.py | 209 | Iterates over paginated API responses, yielding pages of data. Args: path (str): Endpoint path for | STRING |
| HIGH | dlt/extract/resource.py | 241 | Records a data location this resource reads from, to be emitted in the pipeline trace. Call on the resource ins | STRING |
| HIGH | dlt/extract/resource.py | 459 | Limit the number of items that will be processed by the resource: by count and by time. By default, dlt counts n | STRING |
| HIGH | dlt/extract/source.py | 532 | Limits the items processed in all selected resources in the source that are not transformers: by count or time. | STRING |
| HIGH | dlt/extract/state.py | 80 | Returns a dictionary with the resource-scoped state. Resource-scoped state is visible only to resource requesting the ac | STRING |
| HIGH | dlt/extract/decorators.py | 600 | When used as a decorator, transforms any generator (yielding) function into a `dlt resource`. When used as a function, i | STRING |
| HIGH | dlt/extract/decorators.py | 901 | A form of `dlt resource` that takes input from other resources via `data_from` argument in order to enrich or transform | STRING |
| HIGH | dlt/helpers/dbt_cloud/client.py | 60 | Trigger a job run in dbt Cloud. Uses `https://cloud.getdbt.com/api/v2/accounts/{account_id}/jobs/{job | STRING |
| HIGH | dlt/helpers/dbt_cloud/client.py | 116 | Get the status of a dbt Cloud job run by run_id. Uses `https://cloud.getdbt.com/api/v2/accounts/{accou | STRING |
| HIGH | dlt/helpers/dbt_cloud/__init__.py | 21 | Trigger a dbt Cloud job run and retrieve its status. Args: credentials (DBTCloudConfiguration): Config | STRING |
| HIGH | dlt/helpers/dbt_cloud/__init__.py | 105 | Retrieve the status of a dbt Cloud job run. Args: credentials (DBTCloudConfiguration): Configuration p | STRING |
| HIGH | tests/pipeline/utils.py | 257 | Load full table contents from the SFTP filesystem destination. Args: p (dlt.Pipeline): Pipeline whose desti | STRING |
| HIGH | .claude/rules/docstrings.md | 25 | Iterates over paginated API responses, yielding pages of data. Args: path (str): Endpoint path for the requ | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | dlt/destinations/impl/sqlalchemy/merge_job.py | 146 | # Create the insert "temporary" table (but use a concrete table) | COMMENT |
| MEDIUM | dlt/destinations/impl/weaviate/weaviate_client.py | 483 | # Create the collection | COMMENT |
| MEDIUM | dlt/common/libs/pyarrow.py | 877 | # Create a regular array filled with the repeated value | COMMENT |
| MEDIUM | dlt/common/libs/sql_alchemy_shims.py | 73 | # This module is part of SQLAlchemy and is released under | STRING |
| MEDIUM | dlt/common/schema/utils.py | 995 | # Create a copy of the table schema, preserving all fields except columns | COMMENT |
| MEDIUM | …plates/_core_source_templates/sql_database_pipeline.py | 20 | # Create a pipeline | COMMENT |
| MEDIUM | …plates/_core_source_templates/sql_database_pipeline.py | 147 | # Create a pipeline | COMMENT |
| MEDIUM | …_templates/_core_source_templates/rest_api_pipeline.py | 15 | # Create a REST API configuration for the GitHub API | COMMENT |
| MEDIUM | …emplates/_single_file_templates/github_api_pipeline.py | 27 | # Define a resource which fetches repos data | COMMENT |
| MEDIUM | …emplates/_single_file_templates/github_api_pipeline.py | 33 | # Define a resource which fetches issues data (incremental by updated_at timestamp) | COMMENT |
| MEDIUM | …/_templates/_single_file_templates/default_pipeline.py | 16 | # Create a dlt pipeline that will load | COMMENT |
| MEDIUM | dlt/sources/helpers/requests/retry.py | 258 | # Create a new session if config has changed | COMMENT |
| MEDIUM | dlt/extract/utils.py | 344 | # Create a new str instance | COMMENT |
| MEDIUM | tests/pipeline/test_arrow_sources.py | 577 | # Define the schema based on the CSV input | COMMENT |
| MEDIUM | tests/pipeline/test_arrow_sources.py | 597 | # Create an empty table with the defined schema | COMMENT |
| MEDIUM | tests/pipeline/test_schema_contracts.py | 871 | # Create a database schema with table | COMMENT |
| MEDIUM | tests/pipeline/test_pipeline_trace.py | 845 | # Create a pipeline | COMMENT |
| MEDIUM | tests/libs/pyarrow/test_pyarrow.py | 158 | # Create a complex nested type | COMMENT |
| MEDIUM | tests/normalize/test_model_item_normalizer.py | 271 | # Define a query with illegal characters in column names to test normalization | COMMENT |
| MEDIUM⚡ | tests/normalize/test_model_item_normalizer.py | 348 | # Define a query using the randomly ordered columns | STRING |
| MEDIUM⚡ | tests/normalize/test_model_item_normalizer.py | 353 | # Create a SQL model from the query | STRING |
| MEDIUM | tests/normalize/test_model_item_normalizer.py | 282 | # Create a SQL model from the query | STRING |
| MEDIUM | tests/normalize/test_model_item_normalizer.py | 386 | # Create a SQL model from the query | STRING |
| MEDIUM | tests/normalize/test_model_item_normalizer.py | 425 | # Create a SQL model from the query | STRING |
| MEDIUM | tests/normalize/test_model_item_normalizer.py | 679 | # Create a query that selects variant columns | STRING |
| MEDIUM | tests/normalize/test_json_item_inference.py | 319 | # Create an incomplete column first | COMMENT |
| MEDIUM | tests/common/libs/test_pyiceberg.py | 136 | # Create a namespace | COMMENT |
| MEDIUM | tests/common/schema/test_schema_contract.py | 495 | # Create a table update that tries to add a variant column | COMMENT |
| MEDIUM | tests/load/test_job_client.py | 441 | # Create a second schema with 2 hashes | COMMENT |
| MEDIUM | tests/load/test_model_item_format.py | 372 | # Define a resource that aliases column "a" as "b" | COMMENT |
| MEDIUM | tests/load/test_model_item_format.py | 444 | # Define a resource for a SQL model that excludes column "b" and "_dlt_id" from the query | COMMENT |
| MEDIUM | tests/load/test_model_item_format.py | 458 | # Define a resource for a SQL model that reverses the column order in the query | COMMENT |
| MEDIUM | tests/load/test_read_interfaces.py | 923 | # Create a sqlglot expression: id = 42 | COMMENT |
| MEDIUM | tests/load/dremio/bootstrap/bootstrap_dremio.sh | 39 | # Create a NAS source. This will contain final ICEBERG tables. | COMMENT |
| MEDIUM | tests/load/dremio/bootstrap/bootstrap_dremio.sh | 49 | # Create an S3 source using minio. This will be used for staging data. | COMMENT |
| MEDIUM | tests/load/athena_iceberg/test_lakeformation.py | 316 | # Create a table without LakeFormation | COMMENT |
| MEDIUM | tests/load/athena_iceberg/test_lakeformation.py | 363 | # Create a table with LakeFormation enabled | COMMENT |
| MEDIUM | tests/load/bigquery/test_bigquery_client.py | 495 | # Create a wrapper that uses base GcpCredentials type | COMMENT |
| MEDIUM | tests/load/sources/sql_database/postgres_source.py | 216 | # Create a view | COMMENT |
| MEDIUM⚡ | tests/load/fabric/test_fabric_table_builder.py | 126 | # Create a table schema with a unique text column | COMMENT |
| MEDIUM | tests/load/fabric/test_fabric_configuration.py | 126 | # Create a mock table for testing | COMMENT |
| MEDIUM⚡ | tests/sources/helpers/rest_client/test_client.py | 667 | # Create a mock response with sensitive URL | COMMENT |
| MEDIUM | docs/website/docs/intro-snippets.py | 10 | # Create a dlt pipeline that will load | COMMENT |
| MEDIUM⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 471 | # Create a relation with a limit of 1 million rows | STRING |
| MEDIUM⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 474 | # Create a new pipeline | STRING |
| MEDIUM | …pipeline/deploy_snippets/deploy-with-modal-snippets.py | 8 | # Define the Modal Image | COMMENT |
| MEDIUM | …pipeline/deploy_snippets/deploy-with-modal-snippets.py | 32 | # Define the source database credentials; in production, you would save this as a Modal Secret which can be refe | COMMENT |
| MEDIUM | …on/dlt-advanced-course/lesson_8_logging_and_tracing.py | 38 | mo.md(r"""## Create the pipeline we will inspect""") | STRING |
| MEDIUM | …on/dlt-advanced-course/lesson_8_logging_and_tracing.py | 342 | # Create a logger | STRING |
| MEDIUM | …on/dlt-advanced-course/lesson_8_logging_and_tracing.py | 348 | # Create a file handler | STRING |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 212 | # Create a dlt pipeline | STRING |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 236 | # Define a dlt resource that allows the creation of new tables | COMMENT |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 261 | # Define a dlt resource that prevents any changes to the schema at the table level (no new tables can be added) | COMMENT |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 370 | # Define a dlt resource that skips rows that have new columns but loads those that follow the existing schema | STRING |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 405 | # Define a dlt resource that only skips the values of new columns, loading the rest of the row data | STRING |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 441 | # Define a dlt resource that does not allow new columns in the data | STRING |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 590 | # Define a dlt resource that replaces unverifiable values with None, but retains the rest of the row data | COMMENT |
| MEDIUM | …dlt_sources_and_resources_create_first_dlt_pipeline.py | 112 | # Create a dlt resource from the data | COMMENT |
| MEDIUM | …dlt_sources_and_resources_create_first_dlt_pipeline.py | 190 | # Define a resource to load data from a CSV | STRING |
| MEDIUM | …dlt_sources_and_resources_create_first_dlt_pipeline.py | 260 | # Define a resource to fetch pokemons from PokeAPI | STRING |
| 4 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/__init__.py | 60 | __all__ = [ | CODE |
| LOW | dlt/pipeline/mark.py | 11 | __all__ = [ | CODE |
| LOW | dlt/pipeline/current.py | 62 | __all__ = [ | CODE |
| LOW | dlt/pipeline/pipeline.py | 1357 | def set_local_state_val(self, key: str, value: Any) -> None: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1377 | def _update_last_run_context(self) -> None: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1756 | def _set_context(self, is_active: bool) -> None: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1856 | def _set_dataset_name(self, new_dataset_name: Optional[str]) -> None: | CODE |
| LOW | dlt/pipeline/pipeline.py | 1895 | def _set_default_schema_name(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/sql_client.py | 335 | def set_query_tags(self, tags: Optional[TQueryTags]) -> None: | CODE |
| LOW | dlt/destinations/sql_client.py | 401 | def set_schemas(self, schemas: Sequence[Schema]) -> None: ... | CODE |
| LOW | dlt/destinations/sql_client.py | 465 | def _set_default_schema_columns(self) -> None: | CODE |
| LOW | dlt/destinations/configuration.py | 4 | __all__ = ["WithLocalFiles"] | CODE |
| LOW | dlt/destinations/job_impl.py | 93 | def set_final_state(self, state: TLoadJobState, failed_message: Optional[str] = None) -> None: | CODE |
| LOW | dlt/destinations/__init__.py | 25 | __all__ = [ | CODE |
| LOW⚡ | dlt/destinations/job_client_impl.py | 879 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/adapters.py | 15 | __all__ = [ | CODE |
| LOW | dlt/destinations/impl/clickhouse/clickhouse.py | 360 | def _set_internal_table_sort_hints(self, table: PreparedTableSchema) -> None: | CODE |
| LOW | dlt/destinations/impl/dremio/pydremio.py | 333 | def set_call_credential(self, call_credential: Tuple[bytes, bytes]) -> None: | CODE |
| LOW⚡ | dlt/destinations/impl/lance/configuration.py | 279 | def _set_provider_api_key_env_var(self, api_key: str) -> None: | CODE |
| LOW | dlt/destinations/impl/lance/lance_client.py | 641 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/impl/snowflake/sql_client.py | 157 | def set_query_tags(self, tags: Optional[TQueryTags]) -> None: | CODE |
| LOW⚡ | dlt/destinations/impl/filesystem/filesystem.py | 1653 | def update_dataset_card_metadata(self, load_id: str) -> None: | CODE |
| LOW | dlt/destinations/impl/filesystem/filesystem.py | 1291 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW⚡ | …/destinations/impl/sqlalchemy/sqlalchemy_job_client.py | 270 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/impl/sqlalchemy/load_jobs.py | 265 | __all__ = [ | CODE |
| LOW | dlt/destinations/impl/qdrant/qdrant_job_client.py | 479 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/impl/duckdb/sql_client.py | 689 | def set_schemas(self, schemas: Sequence[Schema]) -> None: | CODE |
| LOW | dlt/destinations/impl/fabric/configuration.py | 186 | __all__ = ["FabricCredentials", "FabricClientConfiguration"] | CODE |
| LOW | dlt/destinations/impl/fabric/__init__.py | 7 | __all__ = [ | CODE |
| LOW | dlt/destinations/impl/weaviate/weaviate_client.py | 998 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/destinations/impl/lancedb/lancedb_client.py | 462 | def _update_schema_in_storage(self, schema: Schema) -> None: | CODE |
| LOW | dlt/normalize/__init__.py | 3 | __all__ = ["Normalize"] | CODE |
| LOW | dlt/normalize/items_normalizers/__init__.py | 7 | __all__ = [ | CODE |
| LOW | dlt/common/__init__.py | 8 | __all__ = ["Decimal", "Wei", "pendulum", "json", "sleep", "logger"] | CODE |
| LOW | dlt/common/utils.py | 316 | def update_dict_with_prune(dest: DictStrAny, update: StrAny) -> None: | CODE |
| LOW | dlt/common/pipeline.py | 653 | def _set_context(self, is_active: bool) -> None: | CODE |
| LOW | dlt/common/jsonpath.py | 113 | def set_value_at_path(obj: dict[str, Any], path: str, value: Any) -> None: | CODE |
| LOW | dlt/common/normalizers/__init__.py | 5 | __all__ = [ | CODE |
| LOW | dlt/common/normalizers/json/__init__.py | 68 | def update_normalizer_config(cls, schema: Schema, config: TNormalizerConfig) -> None: | CODE |
| LOW | dlt/common/normalizers/json/__init__.py | 89 | __all__ = [ | CODE |
| LOW | dlt/common/normalizers/json/relational.py | 448 | def update_normalizer_config(cls, schema: Schema, config: RelationalNormalizerConfig) -> None: | CODE |
| LOW | dlt/common/normalizers/naming/__init__.py | 3 | __all__ = ["NamingConvention"] | CODE |
| LOW | dlt/common/configuration/resolve.py | 244 | def _set_field(is_resolvable: bool = True) -> None: | CODE |
| LOW | dlt/common/configuration/__init__.py | 20 | __all__ = [ | CODE |
| LOW | dlt/common/configuration/inject.py | 35 | def set_fun_spec(f: AnyFun, spec: Type[BaseConfiguration]) -> None: | CODE |
| LOW | dlt/common/configuration/inject.py | 41 | def set_fun_last_config(f: AnyFun, config: BaseConfiguration) -> None: | CODE |
| LOW | dlt/common/configuration/specs/__init__.py | 43 | __all__ = [ | CODE |
| LOW | dlt/common/configuration/specs/base_configuration.py | 544 | def _set_default_credentials(self, credentials: Any) -> None: | CODE |
| LOW | dlt/common/configuration/providers/provider.py | 26 | def set_value(self, key: str, value: Any, pipeline_name: Optional[str], *sections: str) -> None: | CODE |
| LOW | dlt/common/configuration/providers/__init__.py | 18 | __all__ = [ | CODE |
| LOW | dlt/common/configuration/providers/doc.py | 46 | def set_value(self, key: str, value: Any, pipeline_name: Optional[str], *sections: str) -> None: | CODE |
| LOW | dlt/common/configuration/providers/toml.py | 133 | def set_value(self, key: str, value: Any, pipeline_name: Optional[str], *sections: str) -> None: | CODE |
| LOW | dlt/common/data_types/__init__.py | 4 | __all__ = ["coerce_value", "py_type_to_sc_type", "TDataType", "DATA_TYPES"] | CODE |
| LOW | dlt/common/storages/load_package.py | 905 | def set_abort_flag(self, load_id: str) -> None: | CODE |
| LOW | dlt/common/storages/__init__.py | 28 | __all__ = [ | CODE |
| LOW | dlt/common/data_writers/configuration.py | 15 | __all__ = ["CsvQuoting", "CsvFormatConfiguration", "ParquetFormatConfiguration"] | CODE |
| LOW | dlt/common/data_writers/__init__.py | 17 | __all__ = [ | CODE |
| LOW | dlt/common/runtime/signals.py | 80 | def set_received_signal(sig: int) -> None: | CODE |
| LOW | dlt/common/destination/client.py | 462 | def set_final_state(self, state: TLoadJobState, failed_message: Optional[str] = None) -> None: | CODE |
| LOW | dlt/common/destination/client.py | 527 | def set_final_state(self, state: TLoadJobState, failed_message: Optional[str] = None) -> None: | CODE |
| 85 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | dlt/destinations/job_client_impl.py | 0 | starts sqlloadjob for files ending with .sql or returns none to let derived classes to handle their specific jobs | STRING |
| HIGH | dlt/destinations/impl/redshift/redshift.py | 0 | starts sqlloadjob for files ending with .sql or returns none to let derived classes to handle their specific jobs | STRING |
| HIGH | dlt/destinations/impl/athena/athena.py | 0 | starts sqlloadjob for files ending with .sql or returns none to let derived classes to handle their specific jobs | STRING |
| HIGH | dlt/destinations/impl/lance/factory.py | 0 | create an ibis duckdb backend that maps the lancedb tables as in-memory views. | STRING |
| HIGH | dlt/destinations/impl/filesystem/factory.py | 0 | create an ibis duckdb backend that maps the lancedb tables as in-memory views. | STRING |
| HIGH | dlt/destinations/impl/lancedb/factory.py | 0 | create an ibis duckdb backend that maps the lancedb tables as in-memory views. | STRING |
| HIGH | dlt/destinations/impl/sqlalchemy/type_mapper.py | 0 | a custom sqlalchemy type that stores json data as a string in the database. automatically serializes python objects to j | STRING |
| HIGH | tests/load/pipeline/test_sqlalchemy_pipeline.py | 0 | a custom sqlalchemy type that stores json data as a string in the database. automatically serializes python objects to j | STRING |
| HIGH | …/website/docs/dlt-ecosystem/destinations/sqlalchemy.md | 0 | a custom sqlalchemy type that stores json data as a string in the database. automatically serializes python objects to j | STRING |
| HIGH | …_templates/_single_file_templates/requests_pipeline.py | 0 | a source function groups all resources into one schema. | STRING |
| HIGH | …templates/_single_file_templates/dataframe_pipeline.py | 0 | a source function groups all resources into one schema. | STRING |
| HIGH | …ce/_templates/_single_file_templates/arrow_pipeline.py | 0 | a source function groups all resources into one schema. | STRING |
| HIGH | …ce/_templates/_single_file_templates/debug_pipeline.py | 0 | a source function groups all resources into one schema. | STRING |
| HIGH | …templates/_single_file_templates/fruitshop_pipeline.py | 0 | a source function groups all resources into one schema. | STRING |
| HIGH | tests/extract/test_polars.py | 0 | most of the following tests rely on the old default fifo next item mode | STRING |
| HIGH | tests/extract/test_sources.py | 0 | most of the following tests rely on the old default fifo next item mode | STRING |
| HIGH | tests/extract/test_incremental.py | 0 | most of the following tests rely on the old default fifo next item mode | STRING |
| HIGH | tests/helpers/test_mermaid.py | 0 | tables with only incomplete columns are excluded, and incomplete columns within complete tables are skipped. | STRING |
| HIGH | tests/helpers/test_dbml.py | 0 | tables with only incomplete columns are excluded, and incomplete columns within complete tables are skipped. | STRING |
| HIGH | tests/helpers/test_graphviz.py | 0 | tables with only incomplete columns are excluded, and incomplete columns within complete tables are skipped. | STRING |
| HIGH | …-usage/customising-pipelines/pseudonymizing_columns.md | 0 | pseudonymization is a deterministic type of pii-obscuring. its role is to allow identifying users by their hash, without | STRING |
| HIGH | …s/dlt-ecosystem/verified-sources/sql_database/usage.md | 0 | pseudonymization is a deterministic type of pii-obscuring. its role is to allow identifying users by their hash, without | STRING |
| HIGH | …se/lesson_5_transform_data_before_and_after_loading.py | 0 | pseudonymization is a deterministic type of pii-obscuring. its role is to allow identifying users by their hash, without | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM⚡ | …sts/workspace/cli/dlthub/ai/test_ai_command_helpers.py | 716 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM⚡ | …sts/workspace/cli/dlthub/ai/test_ai_command_helpers.py | 718 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …sts/workspace/cli/dlthub/ai/test_ai_command_helpers.py | 827 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …sts/workspace/cli/dlthub/ai/test_ai_command_helpers.py | 829 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 351 | "body": "### What happens?\n\nI found that the juliapkg doesn't contains a TimeZone config, which means when using | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 2274 | "body": "### What happens?\n\nCasting from string to hugeint behaves incorrect.\n\n### To Reproduce\n\n```sql\r\nD | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 3489 | "body": "See #4308\r\n\r\nWe should support the `UNION BY NAME` operator to union together tables with re-ordered | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 3745 | "body": "### What happens?\n\nThe CSV auto import functionality does not seem to work correctly when there is a re | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 3745 | "body": "### What happens?\n\nThe CSV auto import functionality does not seem to work correctly when there is a re | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 5336 | "body": "As reported here https://github.com/facebookincubator/velox/issues/895\r\nParquet reader seems to have a | CODE |
| MEDIUM | …ts/normalize/cases/github.issues.load_page_5_duck.json | 6731 | "body": "Hello,\r\n\r\nI've exported a large PostgreSQL 12 table to a CSV file using the [PostgreSQL COPY](https:/ | CODE |
| MEDIUM⚡ | tests/common/libs/docker-compose-iceberg.yml | 17 | # ========================================================================== | COMMENT |
| MEDIUM⚡ | tests/common/libs/docker-compose-iceberg.yml | 19 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 44 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 46 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 66 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 68 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 90 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 92 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 113 | # ========================================================================== | COMMENT |
| MEDIUM | tests/common/libs/docker-compose-iceberg.yml | 116 | # ========================================================================== | COMMENT |
| MEDIUM | .claude/rules/heresy.md | 22 | #------------------------------ | COMMENT |
| MEDIUM | .claude/rules/heresy.md | 24 | #------------------------------ | COMMENT |
| MEDIUM⚡ | docs/website/docs/dlt-ecosystem/destinations/duckdb.md | 200 | # ┌──────────┬───────────────┬─────────────────────┬──────────────────────┬───────────────────────┬───────────┐ | COMMENT |
| MEDIUM⚡ | docs/website/docs/dlt-ecosystem/destinations/duckdb.md | 203 | # ├──────────┼───────────────┼─────────────────────┼──────────────────────┼───────────────────────┼───────────┤ | COMMENT |
| MEDIUM⚡ | docs/website/docs/dlt-ecosystem/destinations/duckdb.md | 208 | # └──────────┴───────────────┴─────────────────────┴──────────────────────┴───────────────────────┴───────────┘ | COMMENT |
| MEDIUM | .github/workflows/agentic-docs.yml | 12 | # --------------------------------------------------------------------------- | COMMENT |
| MEDIUM | .github/workflows/agentic-docs.yml | 33 | # --------------------------------------------------------------------------- | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | dlt/_workspace/configuration.py | 35 | ai_context_api_url: Optional[str] = "https://scaffold.apps.dlthub.com" | CODE |
| MEDIUM | dlt/_workspace/cli/dlthub/_init_command.py | 24 | # default keeps user files; --force overwrites the dlthub-managed scaffold but | COMMENT |
| MEDIUM | dlt/_workspace/cli/dlthub/utils.py | 496 | # scaffold via uv.sources / `-e <path>` so a fresh `uv sync` matches the dev env | COMMENT |
| MEDIUM | dlt/_workspace/deployment/requirements.py | 237 | # the scaffold's transitive resolution pulls dlthub / dlthub-client. The other two | COMMENT |
| MEDIUM⚡ | tests/workspace/cli/dlthub/test_init_command.py | 400 | # scaffold actually written | COMMENT |
| MEDIUM⚡ | tests/load/pipeline/test_open_table_pipeline.py | 205 | # job orchestration is same across table formats—no need to test all formats | COMMENT |
| MEDIUM⚡ | tests/load/pipeline/test_open_table_pipeline.py | 207 | # job orchestration is particular to filesystem open tables implementation | COMMENT |
| MEDIUM⚡ | tests/load/pipeline/test_open_table_pipeline.py | 252 | # job orchestration is same across table formats—no need to test all formats | COMMENT |
| MEDIUM⚡ | tests/load/pipeline/test_open_table_pipeline.py | 254 | # job orchestration is particular to filesystem open tables implementation | COMMENT |
| MEDIUM | tests/load/pipeline/test_open_table_pipeline.py | 824 | # job orchestration is same across table formats—no need to test all formats | COMMENT |
| MEDIUM | tests/load/pipeline/test_open_table_pipeline.py | 873 | # job orchestration is same across table formats—no need to test all formats | COMMENT |
| MEDIUM | …/website/docs/dlt-ecosystem/destinations/filesystem.md | 406 | #### 2. SSH agent-based authentication | COMMENT |
| MEDIUM | docs/website/docs/hub/ingestion/init.md | 77 | Start with the [`/find-source` skill](./rest-api-source.md#find-source--discover-your-data-source) to describe your data | CODE |
| MEDIUM | docs/website/docs/hub/ingestion/rest-api-source.md | 25 | Install `uv` ([instructions](https://docs.astral.sh/uv/getting-started/installation/)), then scaffold a dltHub workspace | CODE |
| MEDIUM | docs/website/docs/hub/ingestion/rest-api-source.md | 136 | ### `/create-rest-api-pipeline` — scaffold the pipeline | COMMENT |
| MEDIUM | docs/website/docs/hub/getting-started/introduction.md | 60 | For an end-to-end walkthrough, watch the [dltHub demo](https://youtu.be/rmpiFSCV8aA), take the [dltHub agentic data engi | CODE |
| MEDIUM | docs/website/docs/hub/getting-started/oss-and-dlthub.md | 61 | | **dltHub Context** | Per-source agent contexts (specs, endpoint documentation, prompts) that prime your coding assista | CODE |
| MEDIUM | docs/website/docs/hub/getting-started/oss-and-dlthub.md | 61 | | **dltHub Context** | Per-source agent contexts (specs, endpoint documentation, prompts) that prime your coding assista | CODE |
| MEDIUM | docs/website/docs/reference/explainers/how-dlt-works.md | 15 | The main building block of dlt is the [pipeline](../../general-usage/glossary.md#pipeline), which orchestrates the loadi | CODE |
| MEDIUM | …kthroughs/deploy-a-pipeline/orchestrate-with-dlthub.md | 13 | ## 1. Install and scaffold a workspace | COMMENT |
| MEDIUM | …kthroughs/deploy-a-pipeline/orchestrate-with-dlthub.md | 15 | If you don't have `uv` yet, follow the [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/). | CODE |
| MEDIUM | …s/walkthroughs/deploy-a-pipeline/deploy-with-kestra.md | 40 | ### How does `dlt` integrate with Kestra for pipeline orchestration? | COMMENT |
| MEDIUM | …/walkthroughs/deploy-a-pipeline/deploy-with-prefect.md | 28 | ### How does **`dlt`** integrate with Prefect for pipeline orchestration? | COMMENT |
| MEDIUM | docs/docs_tools/grammar/fix_grammar_gpt.py | 22 | MAX_CHUNK_SIZE = 4000 # make sure that this is below the context window size of the model to not have cut off files | CODE |
| MEDIUM | .github/workflows/agentic-docs.yml | 1 | # Copy this file to dlt-hub/dlt as .github/workflows/agentic-docs.yml. | COMMENT |
| MEDIUM | .github/workflows/agentic-docs.yml | 32 | # of the agentic-docs README. | COMMENT |
| MEDIUM | .github/workflows/agentic-docs.yml | 81 | # A comment carrying `<!-- agentic-docs -->` is the bot's own and starts | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/pipeline/__init__.py | 32 | CODE | |
| LOW | dlt/pipeline/__init__.py | 113 | CODE | |
| LOW | dlt/pipeline/__init__.py | 287 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 442 | CODE | |
| LOW | dlt/pipeline/pipeline.py | 697 | CODE | |
| LOW | dlt/destinations/decorators.py | 25 | CODE | |
| LOW | dlt/destinations/decorators.py | 100 | CODE | |
| LOW | dlt/destinations/decorators.py | 201 | CODE | |
| LOW | dlt/destinations/impl/databricks/databricks_adapter.py | 23 | CODE | |
| LOW | dlt/destinations/impl/databricks/factory.py | 208 | CODE | |
| LOW | dlt/destinations/impl/bigquery/bigquery_adapter.py | 74 | CODE | |
| LOW | dlt/destinations/impl/snowflake/factory.py | 234 | CODE | |
| LOW | dlt/destinations/impl/filesystem/factory.py | 156 | CODE | |
| LOW | dlt/common/pipeline.py | 619 | CODE | |
| LOW | dlt/common/pipeline.py | 658 | CODE | |
| LOW | dlt/common/configuration/inject.py | 47 | CODE | |
| LOW | dlt/common/configuration/inject.py | 63 | CODE | |
| LOW | dlt/common/configuration/inject.py | 78 | CODE | |
| LOW | dlt/common/storages/live_schema_storage.py | 36 | CODE | |
| LOW | dlt/common/data_writers/writers.py | 317 | CODE | |
| LOW | dlt/common/data_writers/writers.py | 494 | CODE | |
| LOW | dlt/common/schema/utils.py | 1415 | CODE | |
| LOW | dlt/_workspace/cli/_init_command.py | 123 | CODE | |
| LOW | dlt/_workspace/deployment/_run_helpers.py | 349 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 259 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 309 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 330 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 350 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 435 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 450 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 464 | CODE | |
| LOW | dlt/_workspace/deployment/decorators.py | 524 | CODE | |
| LOW | dlt/sources/sql_database/__init__.py | 39 | CODE | |
| LOW | dlt/sources/sql_database/__init__.py | 188 | CODE | |
| LOW | dlt/sources/filesystem/__init__.py | 97 | CODE | |
| LOW | dlt/sources/helpers/requests/retry.py | 66 | CODE | |
| LOW | dlt/sources/helpers/requests/retry.py | 175 | CODE | |
| LOW | dlt/sources/helpers/rest_client/client.py | 194 | CODE | |
| LOW | dlt/sources/helpers/rest_client/paginators.py | 333 | CODE | |
| LOW | dlt/sources/helpers/rest_client/paginators.py | 489 | CODE | |
| LOW | dlt/sources/helpers/rest_client/__init__.py | 12 | CODE | |
| LOW | dlt/extract/reference.py | 79 | CODE | |
| LOW | dlt/extract/reference.py | 233 | CODE | |
| LOW | dlt/extract/hints.py | 128 | CODE | |
| LOW | dlt/extract/hints.py | 441 | CODE | |
| LOW | dlt/extract/extract.py | 85 | CODE | |
| LOW | dlt/extract/decorators.py | 365 | CODE | |
| LOW | dlt/extract/decorators.py | 381 | CODE | |
| LOW | dlt/extract/decorators.py | 398 | CODE | |
| LOW | dlt/extract/decorators.py | 493 | CODE | |
| LOW | dlt/extract/decorators.py | 520 | CODE | |
| LOW | dlt/extract/decorators.py | 549 | CODE | |
| LOW | dlt/extract/decorators.py | 575 | CODE | |
| LOW | dlt/extract/decorators.py | 826 | CODE | |
| LOW | dlt/extract/decorators.py | 854 | CODE | |
| LOW | dlt/extract/decorators.py | 878 | CODE | |
| LOW | dlt/extract/decorators.py | 159 | CODE | |
| LOW | dlt/extract/incremental/__init__.py | 151 | CODE | |
| LOW | dlt/helpers/airflow_helper.py | 72 | CODE | |
| LOW | dlt/helpers/graphviz.py | 452 | CODE | |
| 7 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .claude/skills/release-notes/SKILL.md | 13 | ## Step 1: Validate the tag | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 23 | ## Step 2: Determine the previous tag | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 37 | ## Step 3: Gather commits | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 51 | ## Step 4: Fetch PR metadata | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 80 | ## Step 5: Categorize PRs | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 113 | ## Step 6: Identify new contributors | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 123 | ## Step 7: Format the release notes | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 166 | ## Step 8: Present for approval | COMMENT |
| LOW | .claude/skills/release-notes/SKILL.md | 176 | ## Step 9: Write to file | COMMENT |
| LOW | …/skills/review-vocabulary/references/simple-english.md | 45 | ## Step 1: Classify the Text | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 278 | // Step 1: Discover pages from HTML build output | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 310 | // Step 2: Map HTML path to source .md file | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 372 | // Step 3: Copy source .md files with cleanup | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 387 | // Step 4: Generate llms.txt (master version only) | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 46 | # Step 1: Get the dataset from the pipeline | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 49 | # Step 2: Access a table as a Relation | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 52 | # Step 3: Read the entire table as a Pandas DataFrame | COMMENT |
| LOW | docs/website/docs/hub/ingestion/init.md | 25 | ## Step 0: Install dlt with workspace support | COMMENT |
| LOW | docs/website/docs/hub/ingestion/init.md | 40 | ## Step 1: Initialize a custom pipeline | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 31 | ### Step 1: Initialize source | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 37 | ### Step 2: Define Modal Image | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 41 | ### Step 3: Define Modal Function | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 50 | ### Step 4: Set up credentials | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 60 | ### Step 5: Run pipeline | COMMENT |
| LOW⚡ | …cs/walkthroughs/deploy-a-pipeline/deploy-with-modal.md | 67 | ### Step 6: Deploy | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 50 | mo.md(r"""## Step 1: Load data from SQL Databases""") | STRING |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 98 | ## Step 2: Customize SQL queries with `query_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 187 | ## Step 3: Modify table schema with `table_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 248 | ## Step 4: Adapt column data types with `type_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 352 | ## Step 5: Incremental loads with `sql_database` | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 71 | ## Step 1: Load Parquet file from Local Filesystem | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 117 | ## Step 2: Enrich records with file metadata | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 149 | mo.md(r"""## Step 3: Filter files by metadata""") | STRING |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 172 | ## Step 4: Load files incrementally | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 192 | ## Step 5: Create a custom transformer | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 240 | ## Step 6: Copy files before loading | COMMENT |
| LOW | …e/lesson_4_using_pre_build_sources_and_destinations.py | 246 | ## Step 4: Explore your data | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/destinations/impl/lance/lance_client.py | 398 | # Check if this table has orphan removal enabled (either explicitly or via merge strategy) | COMMENT |
| LOW | dlt/destinations/impl/fabric/fabric.py | 54 | # Check if this is OneLake storage | COMMENT |
| LOW | dlt/destinations/impl/fabric/fabric.py | 106 | # Check if we've already initialized the token for this client | STRING |
| LOW | dlt/destinations/impl/fabric/fabric.py | 148 | # Check if this is a OneLake path (abfss:// scheme or onelake in the hostname) | STRING |
| LOW | dlt/destinations/impl/lancedb/lancedb_client.py | 298 | # Check if this table has orphan removal enabled (either explicitly or via merge strategy) | COMMENT |
| LOW | dlt/destinations/impl/lancedb/lancedb_client.py | 310 | # Check if _dlt_load_id column is required but not present | COMMENT |
| LOW | dlt/common/libs/pyarrow.py | 713 | # Check if destination supports dictionary encoding (default True if not specified) | COMMENT |
| LOW | dlt/common/libs/pyiceberg.py | 269 | # Check if any PYICEBERG_CATALOG_* environment variable is set | COMMENT |
| LOW | dlt/_workspace/_workspace_context.py | 216 | # Check if profile has any toml config files | COMMENT |
| LOW | dlt/_workspace/_workspace_context.py | 221 | # Check if profile has any pipelines | COMMENT |
| LOW | dlt/sources/sql_database/schema_types.py | 121 | # Check if sqla type class rather than instance is returned | COMMENT |
| LOW | tests/pipeline/test_arrow_sources.py | 720 | # Check if the _dlt_load_id column has been replaced with the correct load_id | COMMENT |
| LOW | tests/pipeline/test_arrow_sources.py | 728 | # Check if the _dlt_load_id column has not been replaced | COMMENT |
| LOW | tests/pipeline/test_pipeline_extra.py | 335 | # Check if child dictionary is flattened and added to schema | COMMENT |
| LOW | tests/pipeline/test_pipeline_extra.py | 401 | # Check if nested fields preserved | COMMENT |
| LOW⚡ | tests/pipeline/test_pipeline.py | 161 | # Set dataset_name_layout to "bobby_%s" | COMMENT |
| LOW⚡ | tests/pipeline/test_pipeline.py | 693 | # Set dataset_name_prefix to "bobby" | COMMENT |
| LOW | tests/pipeline/test_pipeline.py | 331 | # Set dataset_name_layout to "bobby_%s" | COMMENT |
| LOW⚡ | tests/common/test_utils.py | 487 | # Check if the sizes of the decomposed dicts are decreasing | COMMENT |
| LOW | tests/load/pipeline/test_open_table_pipeline.py | 1421 | # Check if all data is present | COMMENT |
| LOW | tests/load/pipeline/test_open_table_pipeline.py | 1425 | # Check if partition pruning works by scanning with filter | COMMENT |
| LOW⚡ | tests/load/athena_iceberg/test_lakeformation.py | 58 | # Check if data lake settings exist and are properly configured | COMMENT |
| LOW⚡ | tests/load/filesystem_sftp/test_filesystem_sftp.py | 50 | # Check if SSH agent is running | COMMENT |
| LOW⚡ | tests/load/filesystem_sftp/test_filesystem_sftp.py | 55 | # Check if the key is present and matches | COMMENT |
| LOW | tests/load/qdrant/utils.py | 25 | # Check if collection exists | COMMENT |
| LOW⚡ | tests/load/duckdb/test_duckdb_client.py | 528 | # Check if passing external duckdb connection works fine | STRING |
| LOW⚡ | tests/load/duckdb/test_duckdb_client.py | 532 | # Check if passing :memory: to factory fails | STRING |
| LOW | tests/load/duckdb/test_duckdb_client.py | 884 | # Check if dataset_name does not collide with pipeline_name | STRING |
| LOW⚡ | tests/load/fabric/test_fabric_table_builder.py | 30 | # Set driver to skip check | COMMENT |
| LOW⚡ | tests/load/fabric/test_fabric_configuration.py | 41 | # Set driver to skip ODBC check | COMMENT |
| LOW | tests/load/weaviate/utils.py | 29 | # Check if class exists | COMMENT |
| LOW | tests/load/lancedb/test_pipeline.py | 166 | # Check if we can get a stored schema and state. | COMMENT |
| LOW | docs/docs_tools/examples/prepare_examples_tests.py | 69 | # Check if CI is running on a fork pull request | STRING |
| LOW | docs/examples/partial_loading/partial_loading.py | 117 | # Check if the table directory exists | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/pipeline/pipeline.py | 1621 | load_package_state_update = copy(load_package_state_update or {}) | COMMENT |
| LOW | dlt/destinations/impl/databricks/sql_client.py | 121 | f = curr.fetchall() | COMMENT |
| LOW | dlt/destinations/impl/duckdb/duck.py | 81 | super().initialize_storage(truncate_tables) | COMMENT |
| LOW | dlt/common/utils.py | 181 | return False | COMMENT |
| LOW | dlt/common/storages/configuration.py | 421 | # if pipeline had already run | COMMENT |
| LOW | dlt/common/storages/fsspecs/google_drive.py | 541 | self.location = head["location"] | COMMENT |
| LOW | dlt/common/runners/runnable.py | 61 | COMMENT | |
| LOW | dlt/common/runners/runnable.py | 81 | # @wraps(f) | COMMENT |
| LOW | …emplates/_core_source_templates/filesystem_pipeline.py | 141 | # create dest folder | COMMENT |
| LOW | …plates/_core_source_templates/sql_database_pipeline.py | 321 | COMMENT | |
| LOW | …_templates/_core_source_templates/rest_api_pipeline.py | 41 | # This is a simple resource definition, | COMMENT |
| LOW | tests/tools/early_dependencies.py | 1 | # import requests | COMMENT |
| LOW | tests/tools/early_dependencies.py | 21 | # upload_time = datetime.fromisoformat(release_data[0]["upload_time_iso_8601"][:-1]) # Strip 'Z' | COMMENT |
| LOW | tests/tools/early_dependencies.py | 41 | # # For git dependencies, you might return the branch, commit, or tag | COMMENT |
| LOW | tests/tools/early_dependencies.py | 61 | # # if old_versions: | COMMENT |
| LOW | tests/workspace/cli/test_init_command.py | 661 | # add pyproject.toml to trigger dependency system | COMMENT |
| LOW | tests/common/normalizers/test_json_relational.py | 261 | # row = { | COMMENT |
| LOW | tests/common/normalizers/test_json_relational.py | 301 | # f_lo must be ordered - list of objects | COMMENT |
| LOW | tests/common/configuration/test_sections.py | 301 | resolve.resolve_configuration(SingleValConfiguration()) | COMMENT |
| LOW | tests/common/libs/docker-compose-iceberg.yml | 1 | # Docker Compose for Iceberg Catalog Integration Tests | COMMENT |
| LOW | tests/common/schema/test_merges.py | 121 | # full = add_column_defaults(copy(COL_1_HINTS)) | COMMENT |
| LOW | tests/common/schema/test_merges.py | 981 | # "cluster": False, | COMMENT |
| LOW | tests/common/runners/test_std_pipes.py | 21 | # NOTE: do not remove commented out code | COMMENT |
| LOW | tests/load/utils.py | 561 | ), | COMMENT |
| LOW | tests/load/test_read_interfaces.py | 541 | # qualified query has aliases | COMMENT |
| LOW | tests/load/pipeline/test_postgres.py | 141 | # ids=lambda x: x.name, | COMMENT |
| LOW | tests/load/motherduck/test_motherduck_client.py | 141 | # "credentials": { | COMMENT |
| LOW | tests/load/weaviate/test_naming.py | 81 | # print(pipeline.dataset_name) | COMMENT |
| LOW | tests/load/weaviate/test_naming.py | 101 | # print(eth_V8_schema.data_tables()) | COMMENT |
| LOW | docs/website/tools/check_orphan_docs.js | 1 | // Detects orphan docs pages: pages not reachable from any sidebar (directly, via a | COMMENT |
| LOW | …ughs/deploy-a-pipeline/deploy-with-airflow-composer.md | 181 | # pipeline, | COMMENT |
| LOW | …t-advanced-course/lesson_4_destinations_reverse_etl.py | 1 | # /// script | COMMENT |
| LOW | docs/examples/archive/google_drive_csv.py | 1 | # from apiclient.discovery import build | COMMENT |
| LOW | docs/examples/archive/google_drive_csv.py | 21 | # Returns: | COMMENT |
| LOW | docs/examples/archive/google_drive_csv.py | 41 | # downloader = MediaIoBaseDownload(fh, request) | COMMENT |
| LOW | docs/examples/archive/google_drive_csv.py | 61 | # gcp_credential_json_file_path = KEY_FILE_LOCATION | COMMENT |
| LOW | docs/examples/archive/google_drive_csv.py | 81 | # # and extract it | COMMENT |
| LOW | docs/examples/archive/discord_iterator.py | 1 | # from dlt.common import json | COMMENT |
| LOW | docs/examples/archive/discord_iterator.py | 21 | # # LOG_LEVEL=INFO PG__PASSWORD=.... python examples/discord_iterator.py | COMMENT |
| LOW | docs/examples/archive/discord_iterator.py | 41 | # messages = json.load(f) | COMMENT |
| LOW | docs/examples/archive/discord_iterator.py | 61 | # new_loads = pipeline.list_normalized_loads() | COMMENT |
| LOW | docs/examples/archive/restore_pipeline.py | 1 | # from dlt.pipeline import Pipeline, GCPPipelineCredentials | COMMENT |
| LOW | docs/examples/archive/sync_schema_example.py | 1 | # from dlt.pipeline import Pipeline, GCPPipelineCredentials | COMMENT |
| LOW | .github/workflows/agentic-docs.yml | 1 | # Copy this file to dlt-hub/dlt as .github/workflows/agentic-docs.yml. | COMMENT |
| LOW | .github/workflows/agentic-docs.yml | 21 | # So before you apply the labels or write /revise: | COMMENT |
| LOW | .github/workflows/agentic-docs.yml | 61 | # command, and may be on either an issue (`/position`, before any PR exists) | COMMENT |
| LOW | .github/workflows/agentic-docs.yml | 141 | service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }} | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/workspace/helpers/dashboard/example_pipelines.py | 283 | def dummy_data(): | CODE |
| LOW⚡ | tests/workspace/helpers/dashboard/example_pipelines.py | 290 | pipeline.run(dummy_data()) | CODE |
| LOW⚡ | tests/libs/pyarrow/test_pyarrow.py | 500 | placeholder = "placeholder" | CODE |
| LOW⚡ | tests/libs/pyarrow/test_pyarrow.py | 505 | pa.array(["placeholder", "hello", "placeholder"]), | CODE |
| LOW⚡ | tests/libs/pyarrow/test_pyarrow.py | 506 | pa.array(["hello", "placeholder", "placeholder"]), | CODE |
| LOW | tests/common/cases/schemas/ev1/event.schema.json | 3224 | "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et | CODE |
| LOW | tests/common/cases/schemas/ev1/event.schema.json | 3224 | "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et | CODE |
| LOW⚡ | tests/common/destination/test_reference.py | 270 | destination_name="fake_name", environment="prod", naming_convention="duck_case" | CODE |
| LOW⚡ | tests/common/destination/test_reference.py | 274 | >= {"destination_name": "fake_name", "environment": "prod"}.items() | CODE |
| LOW⚡ | tests/common/destination/test_reference.py | 277 | assert mod_dest.destination_name == "fake_name" | CODE |
| LOW | …oad/clickhouse/test_clickhouse_gcs_s3_compatibility.py | 17 | def dummy_data() -> Generator[Dict[str, int], None, None]: | CODE |
| LOW | …oad/clickhouse/test_clickhouse_gcs_s3_compatibility.py | 31 | pack = pipe.run([dummy_data]) | CODE |
| LOW⚡ | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 32 | aws_access_key_id="fake_key", | CODE |
| LOW⚡ | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 44 | aws_access_key_id="fake_key", aws_secret_access_key=TSecretValue("fake_secret") | CODE |
| LOW⚡ | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 51 | == "fake_key:dlt/" | CODE |
| LOW | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 113 | assert provider.get_value_location("my_source", None, "sources") == "fake_key@eu-central-1:dlt/" | CODE |
| LOW⚡ | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 176 | aws_access_key_id="fake_key", aws_secret_access_key=TSecretValue("fake_secret") | CODE |
| LOW⚡ | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 188 | environment["PROVIDERS__AWS_SECRETS__CREDENTIALS__AWS_ACCESS_KEY_ID"] = "fake_key" | CODE |
| LOW | …ts/helpers/providers/test_aws_secrets_provider_stub.py | 202 | assert provider.locations == ["fake_key@eu-central-1:team/dlt/"] | CODE |
| LOW | …eneral-usage/customising-pipelines/removing_columns.md | 22 | def dummy_data(): | CODE |
| LOW | …eneral-usage/customising-pipelines/removing_columns.md | 26 | return dummy_data() | CODE |
| LOW | …eneral-usage/customising-pipelines/removing_columns.md | 62 | source_instance.dummy_data.add_map( | CODE |
| LOW | …-usage/customising-pipelines/pseudonymizing_columns.md | 18 | def dummy_data(): | CODE |
| LOW | …-usage/customising-pipelines/pseudonymizing_columns.md | 21 | return dummy_data(), | CODE |
| LOW | …-usage/customising-pipelines/pseudonymizing_columns.md | 39 | for row in dummy_source().dummy_data.add_map(pseudonymize_name): | CODE |
| LOW | …-usage/customising-pipelines/pseudonymizing_columns.md | 51 | data_resource = source_instance.dummy_data.add_map(pseudonymize_name) | CODE |
| LOW⚡ | …eneral-usage/customising-pipelines/renaming_columns.md | 20 | def dummy_data(): | CODE |
| LOW⚡ | …eneral-usage/customising-pipelines/renaming_columns.md | 23 | return dummy_data(), | CODE |
| LOW | …eneral-usage/customising-pipelines/renaming_columns.md | 45 | source_instance.dummy_data().add_map(replace_umlauts_in_dict_keys) | CODE |
| LOW | …/website/docs/dlt-ecosystem/transformations/add-map.md | 106 | {'id': 1, 'first_name': 'John', 'last_name': 'Doe', 'email': '<hashed_value>', 'full_name': 'John Doe'} | CODE |
| LOW | …ughs/deploy-a-pipeline/deploy-with-airflow-composer.md | 133 | 'email': 'test@test.com', | CODE |
| LOW | …ughs/deploy-a-pipeline/deploy-with-airflow-composer.md | 318 | 'email': 'test@test.com', | CODE |
| LOW | docs/examples/data_masking/data_masking.py | 164 | {"id": 1, "name": "Dana", "phone": "555-0001"}, | CODE |
| LOW | docs/examples/data_masking/data_masking.py | 165 | {"id": 2, "name": "Eve", "phone": "555-0002"}, | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/destinations/impl/clickhouse/sql_client.py | 310 | # without dataset just use the table name | COMMENT |
| LOW | dlt/destinations/impl/filesystem/sql_client.py | 220 | # NOTE: since we must list all the files anyway we just pass them to duckdb without further globbing | COMMENT |
| MEDIUM | dlt/destinations/impl/synapse/configuration.py | 38 | # HEAP tables (no indexing) by default. HEAP is a more robust choice, because | COMMENT |
| LOW | dlt/common/typing.py | 550 | # 1. Module: just return its dict | COMMENT |
| MEDIUM | dlt/common/configuration/providers/vault.py | 156 | """Lists keys in the vault in order to skip lookups for non existing keys | STRING |
| MEDIUM | dlt/common/destination/reference.py | 361 | """Finds or auto-imports destination factory that can be further called in order to instantiate it | STRING |
| LOW | dlt/common/libs/git.py | 230 | # repo is a directory so just return storage | COMMENT |
| MEDIUM | dlt/common/schema/schema.py | 1045 | """Computes schema hash in order to check if schema content was modified. In such case the schema ``stored_versi | STRING |
| MEDIUM | dlt/extract/decorators.py | 411 | """A decorator that transforms a function returning one or more `dlt resources` into a `dlt source` in order to load | STRING |
| MEDIUM | dlt/extract/decorators.py | 901 | """A form of `dlt resource` that takes input from other resources via `data_from` argument in order to enrich or tra | STRING |
| MEDIUM⚡ | tests/pipeline/test_pipeline.py | 167 | # this is a name of executing test harness or blank pipeline on windows | COMMENT |
| MEDIUM | tests/pipeline/test_pipeline.py | 115 | # this is a name of executing test harness or blank pipeline on windows | COMMENT |
| MEDIUM | tests/load/pipeline/test_pipelines.py | 67 | # this is a name of executing test harness or blank pipeline on windows | COMMENT |
| LOW | tests/load/sources/filesystem/test_filesystem_source.py | 45 | # we just pass the glob parameter to the resource if it is not None | COMMENT |
| LOW | tests/sources/helpers/rest_client/test_paginators.py | 226 | # The next request should just use the "next" URL without any duplicate parameters. | COMMENT |
| LOW | tests/extract/test_decorators.py | 1646 | # we just use it to test wrapping and changing signatures | COMMENT |
| MEDIUM | docs/website/package-lock.json | 20293 | "resolved": "https://registry.npmjs.org/robust-predicates/-/robust-predicates-3.0.2.tgz", | CODE |
| LOW | …reference/performance_snippets/performance-snippets.py | 113 | # just return the results, if you yield, generator will be evaluated in main thread | COMMENT |
| MEDIUM | …_6_write_disposition_strategies_and_advanced_tricks.py | 1324 | r"""Specifically for the `sql_database` source you can utilize another possible approach - load data in fixed ch | CODE |
| LOW | docs/examples/custom_naming/custom_naming.py | 75 | # tags are deterministic so we can just use the naming convention to get table names to select | COMMENT |
| LOW | …mples/custom_config_provider/custom_config_provider.py | 67 | # just return the injected config and secret | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/libs/test_pydantic.py | 278 | # We need to check if pydantic_to_table_schema_columns is idempotent | COMMENT |
| LOW | tests/libs/test_pydantic.py | 438 | # We need to check if pydantic_to_table_schema_columns is idempotent | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 278 | // Step 1: Discover pages from HTML build output | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 310 | // Step 2: Map HTML path to source .md file | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 372 | // Step 3: Copy source .md files with cleanup | COMMENT |
| LOW | docs/website/plugins/llms-txt.js | 387 | // Step 4: Generate llms.txt (master version only) | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 46 | # Step 1: Get the dataset from the pipeline | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 49 | # Step 2: Access a table as a Relation | COMMENT |
| LOW⚡ | …ge/dataset-access/dataset_snippets/dataset_snippets.py | 52 | # Step 3: Read the entire table as a Pandas DataFrame | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 50 | mo.md(r"""## Step 1: Load data from SQL Databases""") | STRING |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 98 | ## Step 2: Customize SQL queries with `query_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 187 | ## Step 3: Modify table schema with `table_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 248 | ## Step 4: Adapt column data types with `type_adapter_callback` | COMMENT |
| LOW | …anced-course/lesson_2_custom_sources_sql_databases_.py | 352 | ## Step 5: Incremental loads with `sql_database` | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 71 | ## Step 1: Load Parquet file from Local Filesystem | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 117 | ## Step 2: Enrich records with file metadata | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 149 | mo.md(r"""## Step 3: Filter files by metadata""") | STRING |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 172 | ## Step 4: Load files incrementally | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 192 | ## Step 5: Create a custom transformer | COMMENT |
| LOW | …esson_3_custom_sources_filesystem_and_cloud_storage.py | 240 | ## Step 6: Copy files before loading | COMMENT |
| LOW | …e/lesson_4_using_pre_build_sources_and_destinations.py | 246 | ## Step 4: Explore your data | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …ts/normalize/cases/github.issues.load_page_5_duck.json | 2202 | "body": "### What happens?\n\nWe've encountered a strange bug on a specific query over a large sample dataset. Wit | CODE |
| HIGH | …ts/normalize/cases/github.issues.load_page_5_duck.json | 2354 | "body": "This pr addresses issue #5174 and offers an alternative solution. \r\n\r\n@Mause suggested to output an | CODE |
| HIGH | …ts/normalize/cases/github.issues.load_page_5_duck.json | 2354 | "body": "This pr addresses issue #5174 and offers an alternative solution. \r\n\r\n@Mause suggested to output an | CODE |
| HIGH | …/docs/dlt-ecosystem/verified-sources/rest_api/basic.md | 199 | 2. `resource_defaults`: Contains default settings for all [resources](#resource-configuration). In this example, we defi | CODE |
| HIGH | docs/examples/nested_data/nested_data.py | 41 | # In this example, we specify that we only want to generate nested tables up to level 2, | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/destinations/impl/bigquery/bigquery.py | 320 | # if any table has schema autodetect, we need to make sure to only truncate tables that exist | COMMENT |
| LOW | tests/common/test_time.py | 316 | # when converting from datetimes make sure to shift to UTC before doing date | COMMENT |
| LOW | tests/load/lancedb/test_pipeline.py | 971 | # We make sure to reset it as well to make sure no globals are propagated to the spawned process. | COMMENT |
| LOW | tests/extract/test_decorators.py | 948 | # make sure to import resource first | COMMENT |
| MEDIUM | …te/docs/hub/transformations/transformation-snippets.py | 195 | # you can use a different dialect than the destination with the query_dialect parameter. | COMMENT |
| MEDIUM | …ucation/dlt-advanced-course/lesson_7_data_contracts.py | 254 | r"""The `freeze` mode at the table level, as mentioned earlier, won't allow any changes to the schema, so the pi | CODE |
| LOW | …sson_1_custom_sources_restapi_source_and_restclient.py | 55 | > Don't forget to use your [GitHub API token](https://docs.github.com/en/rest/authentication/authenticating-to-the-r | CODE |
| MEDIUM | …t-advanced-course/lesson_9_performance_optimisation.py | 1318 | As mentioned earlier, each file created during the extract stage is sent to the process pool of the normalization st | CODE |
| MEDIUM | …pagination_and_authentication_and_dlt_configuration.py | 509 | To define the `access_token` secret value, we can use (as mentioned earlier): | CODE |
| MEDIUM | docs/examples/archive/quickstart.py | 85 | # now you can use your data | COMMENT |
| MEDIUM | docs/examples/chess_production/chess_production.py | 135 | # warning if there is no data, you can use the `execute_query` method | COMMENT |
| MEDIUM | docs/examples/chess_production/chess_production.py | 146 | # warning if there is no data, you can use the `row_counts` attribute. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | dlt/destinations/impl/clickhouse/sql_client.py | 67 | # TODO: implement arrow reading | COMMENT |
| LOW | dlt/destinations/impl/motherduck/configuration.py | 113 | # TODO: implement default connection interface | COMMENT |
| LOW | dlt/destinations/impl/synapse/factory.py | 48 | # TODO: implement as property everywhere and makes sure not accessed as class property | COMMENT |
| LOW | dlt/common/runtime/collector.py | 378 | # TODO: implement once total change is supported | COMMENT |
| LOW | dlt/_workspace/configuration.py | 11 | # TODO: implement default profile switch. it requires reading the configuration, discovering | COMMENT |
| LOW | dlt/sources/rest_api/__init__.py | 295 | **resource_kwargs, # TODO: implement typing.Unpack | CODE |
| LOW | dlt/sources/rest_api/__init__.py | 325 | **resource_kwargs, # TODO: implement typing.Unpack | CODE |
| LOW | dlt/sources/rest_api/typing.py | 233 | # TODO: implement param type to bind incremental to | COMMENT |
| LOW | dlt/sources/rest_api/config_setup.py | 257 | # TODO: implement param type to bind incremental to | COMMENT |
| LOW | tests/load/pipeline/test_open_table_pipeline.py | 802 | # TODO: implement nested types so this test will actually pass | COMMENT |
| LOW | docs/docs_tools/snippets/lint_embedded_snippets.py | 189 | # TODO: implement hcl parsers | STRING |
| LOW | docs/examples/archive/google_drive_csv.py | 33 | # # TODO: consider using https://pythonhosted.org/PyDrive/index.html or https://github.com/wkentaro/gdown | COMMENT |
| LOW | docs/examples/archive/sources/google_sheets.py | 19 | # TODO: consider using https://github.com/burnash/gspread for spreadsheet discovery | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | dlt/_workspace/cli/_plugins.py | 3 | 1. create a new command class in like ie. `dlt._workspace.cli.dlthub.commands.ProfileCommand(SupportsCliCommand):` | STRING |
| CRITICAL | …ts/normalize/cases/github.issues.load_page_5_duck.json | 7742 | "body": "I have tried to write a SELECT query which mentions a BLOB column, among others. However, I receive the f | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | tests/pipeline/test_pipeline_trace.py | 573 | CODE | |
| MEDIUM | tests/pipeline/test_pipeline_trace.py | 734 | CODE | |
| MEDIUM | tests/load/test_read_interfaces.py | 1477 | CODE | |
| MEDIUM | tests/extract/test_extract.py | 560 | CODE | |
| MEDIUM | tests/extract/test_extract.py | 585 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …bsite/docs/dlt-ecosystem/verified-sources/freshdesk.md | 44 | The full guide is available at [this link.](https://support.freshdesk.com/en/support/solutions/articles/215517-how-to-fi | CODE |
| HIGH | …sson_1_custom_sources_restapi_source_and_restclient.py | 297 | GET /v2/everything?q=python&page=1&apiKey=YOUR_API_KEY | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | tests/workspace/cases/runtime_workspace/plain_module.py | 6 | def helper(): | CODE |
| LOW | …pace/cases/runtime_workspace/deployment_with_extras.py | 17 | def helper() -> str: | CODE |
| LOW | tests/helpers/airflow_tests/utils.py | 103 | def run_task( | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | tests/common/libs/docker-compose-iceberg.yml | 11 | # Usage: | COMMENT |