An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
This report summarizes scanner evidence for delta-io/delta, a Scala project with 9,040 GitHub stars. SynthScan examined 689,900 lines of code across 2779 source files, recording 893 pattern matches distributed across 21 syntactic categories. The legacy adjusted pattern signal is 5.2 (Moderate pattern signal); it is not an authorship estimate.
Use the findings to ask whether maintainers can explain intent, verify dependencies, simplify unnecessary scaffolding, and focus review effort. Risk Payload v1 adds ownership, supply-chain, over-engineering, and review-effort vectors when available. These are prompts for human verification, not probabilities, security findings, or measured maintenance costs.
Use each vector to guide manual verification of the underlying findings.
Can maintainers explain intent, assumptions, and trade-offs? 542 finding(s).
Are imports declared and package/API names verified? 280 finding(s).
Are wrappers, guards, and repeated structures necessary? 205 finding(s).
Which changes need focused human review? 893 finding(s).
This chart tracks the legacy adjusted pattern score across scan runs. Changes can reflect source changes, scan coverage, or detector/scoring versions; they do not measure AI adoption or prove that a team refactored code. Compare runs only when their scoring versions are compatible. The dashed line tracks the raw pattern-hit count.
Groups findings by configured severity weight. Severity is not a confidence estimate, security rating, or proof that a package is missing. Verify imports against the project's manifests and inspect each finding in its local design context.
This horizontal bar chart shows where configured pattern scores accumulate across directories. Treat the bars as locations to inspect, not as evidence of authorship or code quality. Use the file-level findings to decide whether a directory needs a focused ownership, dependency, complexity, or review check.
The scanner identified 893 distinct pattern matches across 21 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a detector match to inspect in context. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.
Reading the findings table: The Severity column indicates the configured finding weight (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — legacy comment-context matches receive a ×1.5 score weight. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier under the legacy scoring model.
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| CRITICAL | …elta/sharing/spark/DeltaFormatSharingSourceSuite.scala | 2868 | val batch2Json = batch2OffsetOpt.get.offsets.head.get.json() | CODE |
| CRITICAL | …est/scala/io/delta/connect/tables/DeltaQueryTest.scala | 159 | |${org.apache.spark.util.SparkErrorUtils.stackTraceToString(e)} | CODE |
| CRITICAL | python/delta/tables.py | 347 | jdt = jvm.io.delta.tables.DeltaTable.convertToDelta( | CODE |
| CRITICAL | python/delta/tables.py | 353 | jdt = jvm.io.delta.tables.DeltaTable.convertToDelta( | CODE |
| CRITICAL | python/delta/tables.py | 399 | jdt = jvm.io.delta.tables.DeltaTable.forPath(jsparkSession, path, hadoopConf) | CODE |
| CRITICAL | python/delta/tables.py | 436 | jdt = jvm.io.delta.tables.DeltaTable.forName(jsparkSession, tableOrViewName) | CODE |
| CRITICAL | python/delta/tables.py | 471 | jdt = jvm.io.delta.tables.DeltaTable.create(jsparkSession) | CODE |
| CRITICAL | python/delta/tables.py | 506 | jdt = jvm.io.delta.tables.DeltaTable.createIfNotExists(jsparkSession) | CODE |
| CRITICAL | python/delta/tables.py | 541 | jdt = jvm.io.delta.tables.DeltaTable.replace(jsparkSession) | CODE |
| CRITICAL | python/delta/tables.py | 576 | jdt = jvm.io.delta.tables.DeltaTable.createOrReplace(jsparkSession) | CODE |
| CRITICAL | python/delta/tables.py | 605 | return jvm.io.delta.tables.DeltaTable.isDeltaTable(jsparkSession, identifier) | CODE |
| CRITICAL | python/delta/tables.py | 1437 | _col_jbuilder = jvm.io.delta.tables.DeltaTable.columnBuilder(jsparkSession, colName) | CODE |
| CRITICAL | python/delta/exceptions/captured.py | 135 | stacktrace = jvm.org.apache.spark.util.Utils.exceptionString(e) | CODE |
| CRITICAL | python/delta/connect/exceptions.py | 172 | pyspark.errors.exceptions.connect.EXCEPTION_CLASS_MAPPING.update( | CODE |
| CRITICAL | python/delta/connect/tests/test_exceptions.py | 41 | return pyspark.sql.connect.client.core.convert_exception(info, truncated_message, resp) | CODE |
| CRITICAL | …k/src/test/java/io/delta/flink/sink/DeltaSinkTest.java | 335 | new org.apache.flink.table.types.logical.ArrayType( | CODE |
| CRITICAL | …st/java/io/delta/flink/sink/DeltaWriterResultTest.java | 47 | Map.of("part", io.delta.kernel.expressions.Literal.ofInt(value)))) | CODE |
| CRITICAL | …test/java/io/delta/flink/sink/DeltaWriterTaskTest.java | 61 | Map.of("part", io.delta.kernel.expressions.Literal.ofString("p0")); | CODE |
| CRITICAL | …test/java/io/delta/flink/sink/DeltaWriterTaskTest.java | 122 | Map.of("part", io.delta.kernel.expressions.Literal.ofString("p0")); | CODE |
| CRITICAL | …est/java/io/delta/flink/sink/DeltaCommittableTest.java | 47 | Map.of("part", io.delta.kernel.expressions.Literal.ofInt(value)))) | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 79 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 143 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 209 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 269 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 314 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 369 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …/test/java/io/delta/flink/sink/DeltaCommitterTest.java | 417 | io.delta.kernel.expressions.Literal.ofString( | CODE |
| CRITICAL | …a/io/delta/flink/sink/mergestrategy/MoRUpsertTest.java | 388 | return new io.delta.kernel.internal.data.GenericRow( | CODE |
| CRITICAL⚡ | …/test/java/io/delta/flink/table/VersionHelperTest.java | 31 | Package kernelPkg = io.delta.kernel.Table.class.getPackage(); | CODE |
| CRITICAL⚡ | …/test/java/io/delta/flink/table/VersionHelperTest.java | 33 | Package ucPkg = io.unitycatalog.client.ApiClientBuilder.class.getPackage(); | CODE |
| CRITICAL⚡ | …/test/java/io/delta/flink/table/VersionHelperTest.java | 35 | Package flinkPkg = org.apache.flink.core.execution.JobClient.class.getPackage(); | CODE |
| CRITICAL⚡ | …/test/java/io/delta/flink/table/VersionHelperTest.java | 37 | Package flinkSinkPkg = io.delta.flink.table.DeltaTable.class.getPackage(); | CODE |
| CRITICAL | …ain/java/io/delta/flink/table/AbstractKernelTable.java | 462 | new io.delta.kernel.internal.fs.Path(tablePath), | CODE |
| CRITICAL⚡ | …/src/main/java/io/delta/flink/table/VersionHelper.java | 26 | Package flinkPkg = org.apache.flink.core.execution.JobClient.class.getPackage(); | CODE |
| CRITICAL⚡ | …/src/main/java/io/delta/flink/table/VersionHelper.java | 28 | Package kernelPkg = io.delta.kernel.Table.class.getPackage(); | CODE |
| CRITICAL⚡ | …/src/main/java/io/delta/flink/table/VersionHelper.java | 30 | Package flinkSinkPkg = io.delta.flink.table.DeltaTable.class.getPackage(); | CODE |
| CRITICAL | …s/src/main/scala/io/delta/storage/IBMCOSLogStore.scala | 48 | extends org.apache.spark.sql.delta.storage.HadoopFileSystemLogStore(sparkConf, initHadoopConf) { | CODE |
| CRITICAL | …/main/scala/io/delta/storage/OracleCloudLogStore.scala | 45 | extends org.apache.spark.sql.delta.storage.HadoopFileSystemLogStore(sparkConf, initHadoopConf) { | CODE |
| CRITICAL | …pache/spark/sql/delta/ConvertIcebergToDeltaSuite.scala | 615 | org.apache.commons.io.IOUtils.toByteArray(inputStream) | CODE |
| CRITICAL | …ala/org/apache/spark/sql/delta/CloneIcebergSuite.scala | 378 | .addField(org.apache.iceberg.expressions.Expressions.month("date")) | CODE |
| CRITICAL | …dePlanning/IcebergRESTCatalogPlanningClientSuite.scala | 673 | val oauthServer = com.sun.net.httpserver.HttpServer.create( | CODE |
| CRITICAL | …dePlanning/IcebergRESTCatalogPlanningClientSuite.scala | 697 | val tokenProvider = io.unitycatalog.client.auth.TokenProvider.create( | CODE |
| CRITICAL | …verSidePlanning/IcebergRESTCatalogPlanningClient.scala | 551 | java.util.concurrent.TimeUnit.SECONDS.toMillis(1L << (lastExecutionCount.get() - 1)) | CODE |
| CRITICAL | …t/java/io/delta/spark/internal/v2/DeltaV2TestBase.java | 156 | org.apache.commons.io.FileUtils.deleteDirectory(new java.io.File(tablePath)); | CODE |
| CRITICAL | …a/io/delta/kernel/benchmarks/WorkloadOutputFormat.java | 329 | org.openjdk.jmh.Main.class.getPackage().getImplementationVersion(), | CODE |
| CRITICAL | …kernel/unitycatalog/UcLoadSnapshotTelemetrySuite.scala | 156 | val engineWithReporter = new io.delta.kernel.defaults.engine.DefaultEngine( | CODE |
| CRITICAL | …kernel/unitycatalog/UcLoadSnapshotTelemetrySuite.scala | 157 | new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO( | CODE |
| CRITICAL | …delta/kernel/unitycatalog/UcCommitTelemetrySuite.scala | 114 | new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO(new Configuration())) { | CODE |
| CRITICAL | …ta/kernel/unitycatalog/UCCatalogManagedTestUtils.scala | 79 | new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO(hadoopConf)) { | CODE |
| CRITICAL | …/delta/kernel/unitycatalog/UCCatalogManagedClient.java | 560 | return io.delta.kernel.utils.FileStatus.of( | CODE |
| CRITICAL | …lta/kernel/unitycatalog/UCCatalogManagedCommitter.java | 532 | org.apache.hadoop.fs.permission.FsPermission.getFileDefault() /* permission */, | CODE |
| CRITICAL | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 370 | val logPath = new io.delta.kernel.internal.fs.Path(path + "/_delta_log") | CODE |
| CRITICAL | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 437 | val logPath = new io.delta.kernel.internal.fs.Path(path + "/_delta_log") | CODE |
| CRITICAL | …scala/io/delta/kernel/defaults/TableChangesSuite.scala | 814 | val log = org.apache.spark.sql.delta.DeltaLog.forTable( | CODE |
| CRITICAL | …scala/io/delta/kernel/defaults/TableChangesSuite.scala | 817 | val deltaCommitFileProvider = org.apache.spark.sql.delta.util.DeltaCommitFileProvider( | CODE |
| CRITICAL | …la/io/delta/kernel/defaults/DeltaTableReadsSuite.scala | 1056 | val log = org.apache.spark.sql.delta.DeltaLog.forTable( | CODE |
| CRITICAL | …la/io/delta/kernel/defaults/DeltaTableReadsSuite.scala | 1059 | val deltaCommitFileProvider = org.apache.spark.sql.delta.util.DeltaCommitFileProvider( | CODE |
| CRITICAL | …faults/catalogManaged/CatalogManagedE2EReadSuite.scala | 356 | org.apache.commons.io.FileUtils.copyDirectory( | CODE |
| CRITICAL | …faults/catalogManaged/CatalogManagedE2EReadSuite.scala | 369 | org.apache.commons.io.FileUtils.copyFile(stagedCommitFile, publishedCommitFile) | CODE |
| CRITICAL | …faults/catalogManaged/CatalogManagedE2EReadSuite.scala | 440 | org.apache.commons.io.FileUtils.copyDirectory( | CODE |
| 172 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …ain/resources/golden/spark-variant-checkpoint/info.txt | 0 | with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_js | STRING |
| HIGH | …-variant-stable-feature-checkpoint/_delta_log/info.txt | 0 | with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_js | STRING |
| HIGH | …esources/spark-shredded-variant-preview-delta/info.txt | 0 | with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_js | STRING |
| HIGH | …s/src/test/resources/spark-variant-checkpoint/info.txt | 0 | with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_js | STRING |
| HIGH | …ain/resources/golden/spark-variant-checkpoint/info.txt | 0 | create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2) | STRING |
| HIGH | …-variant-stable-feature-checkpoint/_delta_log/info.txt | 0 | create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2) | STRING |
| HIGH | …s/src/test/resources/spark-variant-checkpoint/info.txt | 0 | create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2) | STRING |
| HIGH | …o/delta/sharing/spark/DeltaSharingTestSparkUtils.scala | 0 | { | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |} | STRING |
| HIGH | …a/sharing/spark/DeltaSharingDeltaLogBuilderSuite.scala | 0 | { | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |} | STRING |
| HIGH | …io/delta/sharing/spark/DeltaSharingCDFUtilsSuite.scala | 0 | { | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |} | STRING |
| HIGH | …o/delta/sharing/spark/DeltaSharingFileIndexSuite.scala | 0 | { | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |} | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolving | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolving | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolving | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4. | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4. | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4. | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionadded | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionadded | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionadded | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. n | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. n | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. n | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolvi | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolvi | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolvi | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. | STRING |
| HIGH | python/delta/exceptions/base.py | 0 | thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. note | STRING |
| HIGH | python/delta/exceptions/captured.py | 0 | thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. note | STRING |
| HIGH | python/delta/connect/exceptions.py | 0 | thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. note | STRING |
| HIGH | …spark/sql/delta/v2/kernel/KernelActionUtilsSuite.scala | 0 | {"protocol":{"minreaderversion":1,"minwriterversion":2}} | STRING |
| HIGH | …delta/kernel/unitycatalog/UnityCatalogUtilsSuite.scala | 0 | {"protocol":{"minreaderversion":1,"minwriterversion":2}} | STRING |
| HIGH | …org/apache/spark/sql/delta/ActionSerializerSuite.scala | 0 | {"protocol":{"minreaderversion":1,"minwriterversion":2}} | STRING |
| HIGH | …/scala/org/apache/spark/sql/delta/DeleteSQLSuite.scala | 0 | create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = fa | STRING |
| HIGH | …/scala/org/apache/spark/sql/delta/UpdateSQLSuite.scala | 0 | create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = fa | STRING |
| HIGH | …ala/org/apache/spark/sql/delta/MergeIntoSQLSuite.scala | 0 | create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = fa | STRING |
| HIGH | …test/scala/org/apache/spark/sql/delta/DeltaSuite.scala | 0 | create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = fa | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | run-integration-tests.py | 1 | #!/usr/bin/env python3 | COMMENT |
| LOW | run-integration-tests.py | 21 | # Scala, Python, and pip tests against them. | COMMENT |
| LOW | run-tests.py | 1 | #!/usr/bin/env python3 | COMMENT |
| LOW | …elta/sharing/spark/DeltaFormatSharingSourceSuite.scala | 521 | } | COMMENT |
| LOW | …aring/spark/DeltaSharingDataSourceDeltaTestUtils.scala | 621 | // than startingVersion for real actions and possible metadata changes. When endingVersion | COMMENT |
| LOW | …/io/delta/sharing/spark/DeltaFormatSharingSource.scala | 281 | s"already set to $frozenServerVersionForAvailableNow." + | COMMENT |
| LOW | …/io/delta/sharing/spark/DeltaFormatSharingSource.scala | 981 | ) | COMMENT |
| LOW | spark-connect/common/src/main/buf.work.yaml | 1 | # | COMMENT |
| LOW | spark-connect/common/src/main/buf.gen.yaml | 1 | # | COMMENT |
| LOW | python/run-tests.py | 1 | #!/usr/bin/env python3 | COMMENT |
| LOW | python/delta/version.py | 1 | # | COMMENT |
| LOW | python/delta/_typing.py | 1 | # | COMMENT |
| LOW | python/delta/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/pip_utils.py | 1 | # | COMMENT |
| LOW | python/delta/tables.py | 1 | # | COMMENT |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_pip_utils.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_version.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_deltatable.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_exceptions.py | 1 | # | COMMENT |
| LOW | python/delta/tests/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_sql.py | 1 | # | COMMENT |
| LOW | python/delta/tests/test_exceptions_init.py | 1 | # | COMMENT |
| LOW | python/delta/exceptions/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/exceptions/base.py | 1 | # | COMMENT |
| LOW | python/delta/exceptions/captured.py | 1 | # | COMMENT |
| LOW | python/delta/testing/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/testing/utils.py | 1 | # | COMMENT |
| LOW | python/delta/connect/plan.py | 1 | # | COMMENT |
| LOW | python/delta/connect/_typing.py | 1 | # | COMMENT |
| LOW | python/delta/connect/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/connect/exceptions.py | 1 | # | COMMENT |
| LOW | python/delta/connect/tables.py | 1 | # | COMMENT |
| LOW | python/delta/connect/tests/test_deltatable.py | 1 | # | COMMENT |
| LOW | python/delta/connect/tests/test_exceptions.py | 1 | # | COMMENT |
| LOW | python/delta/connect/tests/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/connect/tests/test_exceptions_init.py | 1 | # | COMMENT |
| LOW | python/delta/connect/testing/__init__.py | 1 | # | COMMENT |
| LOW | python/delta/connect/testing/utils.py | 1 | # | COMMENT |
| LOW | …src/main/java/io/delta/flink/sink/DeltaSinkWriter.java | 121 | // - INSERT is taken at face value: the source claims this PK is new, so we just append. | COMMENT |
| LOW | …/main/java/io/delta/flink/kernel/CheckpointWriter.java | 261 | snapshot.getCreateCheckpointIterator(engine).map(filterAddFiles())) { | COMMENT |
| LOW | …age-s3-dynamodb/integration_tests/dynamodb_logstore.py | 1 | # | COMMENT |
| LOW | docs/apis/generate_api_docs.py | 1 | # !/usr/bin/env python3 | COMMENT |
| LOW | docs/apis/python/conf.py | 1 | # | COMMENT |
| LOW | docs/apis/python/conf.py | 61 | # List of patterns, relative to source directory, that match files and | COMMENT |
| LOW | iceberg/integration_tests/iceberg_converter.py | 1 | # | COMMENT |
| LOW | …e/spark/sql/delta/uniform/UniFormE2EIcebergSuite.scala | 341 | // Now v1 snapshot and v1 Iceberg catalog becomes stale so txn's commit would hit | COMMENT |
| LOW | …apache/spark/sql/delta/IcebergPartitionConverter.scala | 41 | object IcebergPartitionConverter { | COMMENT |
| LOW | …delta/icebergShaded/IcebergConversionTransaction.scala | 521 | .getOrElse(RowId.MISSING_HIGH_WATER_MARK).toString | COMMENT |
| LOW | …e/spark/sql/delta/icebergShaded/IcebergConverter.scala | 881 | // A rebased OPTIMIZE can violate this: when a concurrent DELETE fully removes one of | COMMENT |
| LOW | project/scripts/build_spark.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | project/scripts/install-buf.sh | 1 | #!/bin/sh | COMMENT |
| LOW | project/scripts/setup_unitycatalog_main.sh | 1 | #!/usr/bin/env bash | COMMENT |
| LOW | project/scripts/setup_unitycatalog_main.sh | 21 | # | COMMENT |
| LOW | project/scripts/setup_unitycatalog_main.sh | 41 | # script sanity-checks the two match when the slow path runs, so you'll get a loud error | COMMENT |
| LOW | project/scripts/setup_unitycatalog_main.sh | 61 | # Forwarded as -DsparkVersion to UC's sbt; also determines the published artifact | COMMENT |
| LOW | project/scripts/setup_unitycatalog_main.sh | 201 | # Force publishLocal / publishM2 to overwrite existing artifacts. UC artifacts at the same | COMMENT |
| LOW | examples/python/image_storage.py | 1 | # | COMMENT |
| LOW | examples/python/quickstart_sql.py | 1 | # | COMMENT |
| LOW | examples/python/missing_delta_storage_jar.py | 1 | # | COMMENT |
| 119 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | run-integration-tests.py | 109 | # Step 1: unsuffixed (backward compat) | COMMENT |
| LOW⚡ | run-integration-tests.py | 113 | # Step 2: suffixed for each non-master Spark version | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 193 | // Step 0: Fail if N.json already exists in FileSystem and overwrite=false. | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 197 | // Step 1: Ensure that N-1.json exists | COMMENT |
| LOW | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 238 | // Step 2: PREPARE the commit | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 255 | // Step 3: COMMIT the commit to the delta log. | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 259 | // Step 4: ACKNOWLEDGE the commit | COMMENT |
| LOW | project/CrossSparkVersions.scala | 563 | // Step 1: Publish non-Flink modules WITHOUT Spark suffix (backward compatibility). | COMMENT |
| LOW⚡ | project/tests/test_published_artifacts.py | 353 | # Step 1: All modules without suffix (uses default Spark version's iceberg support) | COMMENT |
| LOW⚡ | project/tests/test_published_artifacts.py | 358 | # Step 2: Spark-dependent modules WITH suffix for each non-master version | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 316 | # Step 1: Publish non-Flink modules WITHOUT a Spark suffix. | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 327 | # Step 2: Publish Spark-dependent modules WITH suffix for each non-master version | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 339 | # Step 3: Publish every supported Flink version through the release implementation. | COMMENT |
| LOW | kernel/USER_GUIDE.md | 74 | ### Step 1: Full scan on a Delta table | COMMENT |
| LOW | kernel/USER_GUIDE.md | 210 | ### Step 2: Improve scan performance with file skipping | COMMENT |
| LOW | kernel/USER_GUIDE.md | 597 | ### Step 0: Validate the prerequisites | COMMENT |
| LOW | kernel/USER_GUIDE.md | 610 | ### Step 1: Set up Delta Kernel in your connector project | COMMENT |
| LOW | kernel/USER_GUIDE.md | 635 | ### Step 2: Build your own Engine | COMMENT |
| LOW | kernel/USER_GUIDE.md | 784 | ### Step 3: Build read support in your connector | COMMENT |
| LOW | kernel/USER_GUIDE.md | 991 | ### Step 4: Build append support in your connector | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 64 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 72 | // Step 2: WRITE -- v1.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 80 | // Step 3: WRITE -- v2.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 129 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 137 | // Step 2: WRITE -- v1.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 145 | // Step 3: WRITE -- v2.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 183 | // Step 5: Use postPublish snapshot to write -- v3.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 191 | // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 309 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 316 | // Step 2: WRITE and commit data up to version 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 331 | // Step 3: PUBLISH v1.json and v2.json to the filesystem | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 334 | // Step 4: Configure the UC client to limit maxRatifiedVersion to 1 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 337 | // Step 5: Load snapshot with UC client that limits maxRatifiedVersion to 1 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 340 | // Step 6: Verify that snapshot is at version 1, not version 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 358 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 365 | // Step 2: WRITE multiple versions | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 388 | // Step 3: Publish all versions | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 391 | // Step 4: Load CommitRange with end boundary (should go from 0 to 3) | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 405 | // Step 5: Configure UC client to limit maxRatifiedVersion to 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 408 | // Step 6: Load CommitRange again (should now be limited to version 2) | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 97 | // Step 5: WRITE -- v3.uuid.json | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 114 | // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.json | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 163 | // Step 5: Read the latest snapshot from disk. Post-publish snapshot should be similar to it | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 422 | // Step 8: Load CommitRange with start version at maxCatalogVersion (should work) | COMMENT |
| LOW | …est/scala/io/delta/kernel/defaults/SnapshotSuite.scala | 43 | // Step 1: Create a table with the given partition columns | COMMENT |
| LOW | …est/scala/io/delta/kernel/defaults/SnapshotSuite.scala | 62 | // Step 2: Check the partition columns | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 275 | // Step 1: Create a table (v0.json) and get the post-commit snapshot | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 280 | // Step 2: Load a new snapshot to latest (v0) | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 285 | // Step 3: Use the fresh snapshot to write the checksum | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 290 | // Step 4: Use the first (post-commit) snapshot to write the checksum -- should NOT fail | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1003 | // Step 1: Verify initial checksum | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1006 | // Step 2: Delete and regenerate the checksum | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1010 | // Step 3: Verify regenerated checksum | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 187 | // Step 1: If row tracking is enabled, check for physical row tracking columns in the data | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 195 | // Step 2: Get the selectionVector if DV is present | COMMENT |
| LOW | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 217 | // Step 3: If a column was only requested to compute other columns, we remove it | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 232 | // Step 4: Add partition columns back to the data batch | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 240 | // Step 5: Transform column names back to logical names if column mapping is enabled | COMMENT |
| LOW⚡ | …/delta/kernel/internal/TransactionMetadataFactory.java | 157 | // Step 1: We put the required catalog table properties *first* so that we persist the intent, | COMMENT |
| LOW⚡ | …/delta/kernel/internal/TransactionMetadataFactory.java | 162 | // Step 2: Preserve a few important delta properties | COMMENT |
| 36 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW⚡ | run-integration-tests.py | 109 | # Step 1: unsuffixed (backward compat) | COMMENT |
| LOW⚡ | run-integration-tests.py | 113 | # Step 2: suffixed for each non-master Spark version | COMMENT |
| LOW | …/main/java/io/delta/flink/kernel/CheckpointWriter.java | 208 | // Step 1: | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 193 | // Step 0: Fail if N.json already exists in FileSystem and overwrite=false. | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 197 | // Step 1: Ensure that N-1.json exists | COMMENT |
| LOW | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 238 | // Step 2: PREPARE the commit | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 255 | // Step 3: COMMIT the commit to the delta log. | COMMENT |
| LOW⚡ | …c/main/java/io/delta/storage/BaseExternalLogStore.java | 259 | // Step 4: ACKNOWLEDGE the commit | COMMENT |
| LOW | project/CrossSparkVersions.scala | 563 | // Step 1: Publish non-Flink modules WITHOUT Spark suffix (backward compatibility). | COMMENT |
| LOW⚡ | project/tests/test_published_artifacts.py | 353 | # Step 1: All modules without suffix (uses default Spark version's iceberg support) | COMMENT |
| LOW⚡ | project/tests/test_published_artifacts.py | 358 | # Step 2: Spark-dependent modules WITH suffix for each non-master version | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 316 | # Step 1: Publish non-Flink modules WITHOUT a Spark suffix. | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 327 | # Step 2: Publish Spark-dependent modules WITH suffix for each non-master version | COMMENT |
| LOW | project/tests/test_published_artifacts.py | 339 | # Step 3: Publish every supported Flink version through the release implementation. | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 64 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 72 | // Step 2: WRITE -- v1.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 80 | // Step 3: WRITE -- v2.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 129 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 137 | // Step 2: WRITE -- v1.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 145 | // Step 3: WRITE -- v2.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 183 | // Step 5: Use postPublish snapshot to write -- v3.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 191 | // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 309 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 316 | // Step 2: WRITE and commit data up to version 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 331 | // Step 3: PUBLISH v1.json and v2.json to the filesystem | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 334 | // Step 4: Configure the UC client to limit maxRatifiedVersion to 1 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 337 | // Step 5: Load snapshot with UC client that limits maxRatifiedVersion to 1 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 340 | // Step 6: Verify that snapshot is at version 1, not version 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 358 | // Step 1: CREATE -- v0.json | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 365 | // Step 2: WRITE multiple versions | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 388 | // Step 3: Publish all versions | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 391 | // Step 4: Load CommitRange with end boundary (should go from 0 to 3) | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 405 | // Step 5: Configure UC client to limit maxRatifiedVersion to 2 | COMMENT |
| LOW⚡ | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 408 | // Step 6: Load CommitRange again (should now be limited to version 2) | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 97 | // Step 5: WRITE -- v3.uuid.json | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 114 | // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.json | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 163 | // Step 5: Read the latest snapshot from disk. Post-publish snapshot should be similar to it | COMMENT |
| LOW | …st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala | 422 | // Step 8: Load CommitRange with start version at maxCatalogVersion (should work) | COMMENT |
| LOW | …est/scala/io/delta/kernel/defaults/SnapshotSuite.scala | 43 | // Step 1: Create a table with the given partition columns | COMMENT |
| LOW | …est/scala/io/delta/kernel/defaults/SnapshotSuite.scala | 62 | // Step 2: Check the partition columns | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 275 | // Step 1: Create a table (v0.json) and get the post-commit snapshot | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 280 | // Step 2: Load a new snapshot to latest (v0) | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 285 | // Step 3: Use the fresh snapshot to write the checksum | COMMENT |
| LOW⚡ | …defaults/SnapshotChecksumStatisticsAndWriteSuite.scala | 290 | // Step 4: Use the first (post-commit) snapshot to write the checksum -- should NOT fail | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1003 | // Step 1: Verify initial checksum | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1006 | // Step 2: Delete and regenerate the checksum | COMMENT |
| LOW⚡ | …t/scala/io/delta/kernel/defaults/utils/TestUtils.scala | 1010 | // Step 3: Verify regenerated checksum | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 187 | // Step 1: If row tracking is enabled, check for physical row tracking columns in the data | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 195 | // Step 2: Get the selectionVector if DV is present | COMMENT |
| LOW | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 217 | // Step 3: If a column was only requested to compute other columns, we remove it | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 232 | // Step 4: Add partition columns back to the data batch | COMMENT |
| LOW⚡ | …nel/kernel-api/src/main/java/io/delta/kernel/Scan.java | 240 | // Step 5: Transform column names back to logical names if column mapping is enabled | COMMENT |
| LOW⚡ | …/delta/kernel/internal/TransactionMetadataFactory.java | 157 | // Step 1: We put the required catalog table properties *first* so that we persist the intent, | COMMENT |
| LOW⚡ | …/delta/kernel/internal/TransactionMetadataFactory.java | 162 | // Step 2: Preserve a few important delta properties | COMMENT |
| LOW⚡ | …/delta/kernel/internal/TransactionMetadataFactory.java | 168 | // Step 3: Insert the new user-provided table properties | COMMENT |
| LOW | …io/delta/kernel/internal/snapshot/MetadataCleanup.java | 100 | // Step 1: Check if the `lastSeenCheckpointFiles` contains a complete checkpoint | COMMENT |
| LOW | …io/delta/kernel/internal/snapshot/MetadataCleanup.java | 134 | // Step 2: If the timestamp is earlier than the retention period, stop | COMMENT |
| LOW⚡ | …io/delta/kernel/internal/snapshot/MetadataCleanup.java | 148 | // Step 3: If the file is a delta log file, add it to the `potentialFilesToDelete` list | COMMENT |
| LOW⚡ | …io/delta/kernel/internal/snapshot/MetadataCleanup.java | 153 | // Step 4: If the file is a checkpoint file, add it to the `lastSeenCheckpointFiles` | COMMENT |
| LOW⚡ | …io/delta/kernel/internal/snapshot/SnapshotManager.java | 367 | // Step 3: Perform some basic validations on the listed file statuses // | COMMENT |
| 30 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | run-integration-tests.py | 182 | def run_scala_integration_tests(root_dir, version, test_name, extra_maven_repo, scala_version, | CODE |
| LOW | run-integration-tests.py | 246 | def run_python_integration_tests(root_dir, version, test_name, extra_maven_repo, variant): | CODE |
| LOW | run-integration-tests.py | 293 | def test_missing_delta_storage_jar(root_dir, version, use_local): | CODE |
| LOW | run-integration-tests.py | 332 | def run_dynamodb_logstore_integration_tests(root_dir, version, test_name, extra_maven_repo, | CODE |
| LOW | run-integration-tests.py | 385 | def run_dynamodb_commit_coordinator_integration_tests(root_dir, version, test_name, extra_maven_repo, | CODE |
| LOW | run-integration-tests.py | 454 | def run_flink_integration_tests(): | CODE |
| LOW | run-integration-tests.py | 466 | def run_iceberg_integration_tests(root_dir, version, iceberg_version, extra_maven_repo, variant): | CODE |
| LOW | run-integration-tests.py | 519 | def run_uniform_hudi_integration_tests(root_dir, version, hudi_version, extra_maven_repo, variant): | CODE |
| LOW | run-integration-tests.py | 575 | def run_pip_installation_tests(root_dir, version, use_testpypi, use_localpypi, extra_maven_repo): | CODE |
| LOW | run-integration-tests.py | 614 | def run_unity_catalog_commit_coordinator_integration_tests(root_dir, version, test_name, | CODE |
| LOW | run-tests.py | 157 | def pull_or_build_docker_image(root_dir): | CODE |
| LOW | …/io/delta/sharing/spark/DeltaFormatSharingSource.scala | 171 | private def getOrCreateLegacySource(snapshotVersion: Long): DeltaSharingSource = { | CODE |
| LOW | …/io/delta/sharing/spark/DeltaFormatSharingSource.scala | 309 | private def getOrUpdateLatestTableVersion: Long = { | CODE |
| LOW | python/run-tests.py | 227 | def run_delta_connect_codegen_python(root_dir): | CODE |
| LOW | python/delta/pip_utils.py | 21 | def configure_spark_with_delta_pip( | CODE |
| LOW | python/delta/tables.py | 1338 | def _check_identity_column_spec(self, identityGenerator: IdentityGenerator) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 114 | def test_read_from_managed_table_without_catalog_owned(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 117 | def test_write_to_managed_catalog_owned_table(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 123 | def test_read_from_managed_catalog_owned_table(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 129 | def test_write_to_managed_table_without_catalog_owned(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 135 | def test_unset_catalog_owned_feature(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 142 | def test_drop_catalog_owned_property(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 238 | def test_sql_insert_overwrite(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 245 | def test_sql_insert_replace_where(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 253 | def test_sql_insert_dynamic_partition_overwrite(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 271 | def test_insert_into_overwrite(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 278 | def test_insert_into_overwrite_replace_where(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 285 | def test_insert_into_overwrite_partition_overwrite(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 293 | def test_save_as_table_append_existing_table(self) -> None: | CODE |
| LOW⚡ | …/unity-catalog-commit-coordinator-integration-tests.py | 303 | def test_save_append_using_path(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 199 | def test_merge_schema_evolution(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 216 | def test_insert_schema_evolution(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 330 | def test_overwrite_partitions(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 375 | def test_create_non_catalog_owned(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 386 | def test_clone_into_catalog_owned(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 394 | def test_clone_into_non_catalog_owned(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 410 | def test_alter_table_add_column(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 416 | def test_alter_table_set_tbl_properties(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 517 | def test_change_data_feed_with_timestamp(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 526 | def test_change_data_feed_with_version(self) -> None: | CODE |
| LOW | …/unity-catalog-commit-coordinator-integration-tests.py | 537 | def test_delta_table_for_path(self) -> None: | CODE |
| LOW⚡ | python/delta/tests/test_version.py | 35 | def test_version_import_from_module(self) -> None: | CODE |
| LOW⚡ | python/delta/tests/test_version.py | 40 | def test_version_import_from_package(self) -> None: | CODE |
| LOW⚡ | python/delta/tests/test_version.py | 45 | def test_version_consistency_across_imports(self) -> None: | CODE |
| LOW | python/delta/tests/test_version.py | 59 | def test_version_sbt_and_version_py_consistency(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 499 | def test_merge_with_inconsistent_sessions(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 684 | def __verify_generated_column(self, tableName: str, deltaTable: DeltaTable) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 730 | def test_create_table_with_existing_schema(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 759 | def test_create_replace_table_with_cluster_by(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 789 | def test_create_replace_table_with_no_spark_session_passed(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 828 | def test_create_table_with_name_only(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 845 | def test_create_table_with_location_only(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 861 | def test_create_table_with_name_and_location(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 880 | def test_create_table_behavior(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 899 | def test_replace_table_with_name_only(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 917 | def test_replace_table_with_location_only(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 935 | def test_replace_table_with_name_and_location(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 956 | def test_replace_table_behavior(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 1001 | def test_create_table_with_identity_column(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 1035 | def test_delta_table_builder_with_bad_args(self) -> None: | CODE |
| 74 more matches not shown… | ||||
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | python/delta/version.py | 17 | # This file is auto-generated by the build.sbt generatePythonVersion task. | COMMENT |
| MEDIUM | python/delta/tests/test_sql.py | 35 | # Create a simple Delta table inside the temp directory to test SQL commands. | COMMENT |
| MEDIUM | project/tests/test_published_artifacts.py | 286 | # Create a spec without suffix for backward compatibility | COMMENT |
| MEDIUM | examples/python/quickstart_sql.py | 36 | # Create a table | COMMENT |
| MEDIUM | examples/python/quickstart_sql.py | 37 | print("############# Creating a table ###############") | CODE |
| MEDIUM | examples/python/table_exists.py | 47 | # Create a delta table at filepath | COMMENT |
| MEDIUM | examples/python/using_with_pip.py | 48 | print("########### Create a Parquet table ##############") | CODE |
| MEDIUM | examples/python/quickstart_sql_on_paths.py | 23 | # Create a table | COMMENT |
| MEDIUM | examples/python/quickstart_sql_on_paths.py | 24 | print("############# Creating a table ###############") | CODE |
| MEDIUM | examples/python/utilities.py | 32 | # Create a table | COMMENT |
| MEDIUM | examples/python/utilities.py | 33 | print("########### Create a Parquet table ##############") | CODE |
| MEDIUM | examples/python/streaming.py | 35 | # Create a table(key, value) of some data | COMMENT |
| MEDIUM | examples/python/quickstart.py | 34 | # Create a table | COMMENT |
| MEDIUM | examples/python/quickstart.py | 35 | print("############# Creating a table ###############") | CODE |
| MEDIUM | …marks/infrastructure/gcp/terraform/.terraform.lock.hcl | 1 | # This file is maintained automatically by "terraform init". | COMMENT |
| MEDIUM | …marks/infrastructure/aws/terraform/.terraform.lock.hcl | 1 | # This file is maintained automatically by "terraform init". | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | hudi/integration_tests/write_uniform_hudi.py | 2 | CODE | |
| LOW | hudi/integration_tests/write_uniform_hudi.py | 2 | CODE | |
| LOW | hudi/integration_tests/write_uniform_hudi.py | 4 | CODE | |
| LOW | hudi/integration_tests/write_uniform_hudi.py | 6 | CODE | |
| LOW | python/delta/__init__.py | 17 | CODE | |
| LOW | python/delta/__init__.py | 18 | CODE | |
| LOW | python/delta/__init__.py | 19 | CODE | |
| LOW | python/delta/tables.py | 35 | CODE | |
| LOW | python/delta/tables.py | 35 | CODE | |
| LOW | python/delta/tables.py | 36 | CODE | |
| LOW | python/delta/tests/test_deltatable.py | 24 | CODE | |
| LOW | python/delta/tests/test_deltatable.py | 25 | CODE | |
| LOW | python/delta/tests/test_deltatable.py | 32 | CODE | |
| LOW | python/delta/tests/test_exceptions.py | 26 | CODE | |
| LOW | python/delta/tests/test_exceptions_init.py | 25 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 24 | CODE | |
| LOW | python/delta/exceptions/__init__.py | 22 | CODE | |
| LOW | python/delta/exceptions/captured.py | 35 | CODE | |
| LOW | python/delta/exceptions/captured.py | 35 | CODE | |
| LOW | python/delta/testing/utils.py | 19 | CODE | |
| LOW | python/delta/testing/utils.py | 21 | CODE | |
| LOW | python/delta/connect/__init__.py | 18 | CODE | |
| LOW | python/delta/connect/__init__.py | 19 | CODE | |
| LOW | python/delta/connect/tests/test_exceptions_init.py | 23 | CODE | |
| LOW | docs/generate_docs.py | 6 | CODE | |
| LOW | docs/generate_docs.py | 8 | CODE | |
| LOW | docs/generate_docs.py | 9 | CODE | |
| LOW | iceberg/integration_tests/iceberg_converter.py | 18 | CODE | |
| LOW | iceberg/integration_tests/iceberg_converter.py | 19 | CODE | |
| LOW | iceberg/integration_tests/iceberg_converter.py | 21 | CODE | |
| LOW | examples/python/change_data_feed.py | 18 | CODE | |
| LOW | examples/python/using_with_pip.py | 22 | CODE | |
| LOW | examples/python/using_with_pip.py | 65 | CODE | |
| LOW | benchmarks/run-benchmark.py | 20 | CODE | |
| LOW | benchmarks/scripts/benchmarks.py | 17 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | project/scripts/setup_unitycatalog_main.sh | 82 | # --------------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | project/scripts/setup_unitycatalog_main.sh | 88 | # --------------------------------------------------------------------------------------------- | COMMENT |
| MEDIUM | …/spark/internal/v2/ddl/CreateTableIntegrationTest.java | 140 | // ── helpers ──────────────────────────────────────────────────────── | COMMENT |
| MEDIUM | …elta/spark/internal/v2/ddl/CreateTableBuilderTest.java | 40 | // ── filterProperties ────────────────────────────────────────────── | COMMENT |
| MEDIUM | …elta/spark/internal/v2/ddl/CreateTableBuilderTest.java | 84 | // ── toDataLayoutSpec ────────────────────────────────────────────── | COMMENT |
| MEDIUM | …elta/spark/internal/v2/ddl/CreateTableBuilderTest.java | 132 | // ── resolveTablePath ────────────────────────────────────────────── | COMMENT |
| MEDIUM | …elta/spark/internal/v2/ddl/CreateTableBuilderTest.java | 158 | // ── extractComment ──────────────────────────────────────────────── | COMMENT |
| MEDIUM | …elta/spark/internal/v2/ddl/CreateTableBuilderTest.java | 169 | // ── validateClusteringColumns ───────────────────────────────────── | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | run-tests.py | 192 | except Exception as e: | CODE |
| LOW | run-tests.py | 203 | except Exception as e: | CODE |
| MEDIUM | run-tests.py | 185 | def pull_image(registry_image_tag): | CODE |
| MEDIUM | run-tests.py | 196 | def push_image(registry_image_tag): | CODE |
| LOW | python/run-tests.py | 122 | except Exception as e: | CODE |
| LOW | python/delta/pip_utils.py | 70 | except Exception as e: | STRING |
| MEDIUM | python/delta/tests/test_deltatable.py | 974 | def test_verify_paritionedBy_compatibility(self) -> None: | CODE |
| LOW | python/delta/tests/test_deltatable.py | 1697 | except Exception as e: | STRING |
| MEDIUM | project/tests/test_published_artifacts.py | 910 | print("Error: build.sbt not found. Run from Delta repository root.") | CODE |
| LOW | project/tests/test_published_artifacts.py | 497 | except Exception as e: | CODE |
| LOW | project/tests/test_published_artifacts.py | 770 | except Exception as e: | CODE |
| LOW | project/tests/test_published_artifacts.py | 812 | except Exception as e: | CODE |
| LOW | project/tests/test_published_artifacts.py | 852 | except Exception as e: | CODE |
| LOW | project/tests/test_published_artifacts.py | 980 | except Exception as e: | CODE |
| LOW | project/scripts/get_spark_version_info.py | 392 | except Exception as e: | CODE |
| MEDIUM | project/scripts/collect_test_durations.py | 48 | print(f"Error running: {' '.join(cmd)}", file=sys.stderr) | CODE |
| MEDIUM | project/scripts/collect_test_durations.py | 107 | print(f"Error downloading artifact {artifact_id}", file=sys.stderr) | CODE |
| LOW | examples/python/missing_delta_storage_jar.py | 35 | except Exception as e: | CODE |
| LOW | examples/python/delta_connect.py | 72 | except Exception as e: | CODE |
| LOW | examples/python/delta_connect.py | 80 | except Exception as e: | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | .github/workflows/spark_python_test.yaml | 78 | # just use the cache. The cache is immutable so we need to use a new key when trying to | COMMENT |
| LOW | .github/workflows/iceberg_test.yaml | 40 | # just use the cache. The cache is immutable so we need to use a new key when trying to | COMMENT |
| LOW | …/java/io/delta/kernel/internal/actions/CommitInfo.java | 128 | // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables. | COMMENT |
| LOW | …/java/io/delta/kernel/internal/actions/CommitInfo.java | 152 | // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables. | COMMENT |
| LOW | …/java/io/delta/kernel/internal/actions/CommitInfo.java | 167 | // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables. | COMMENT |
| MEDIUM | …che/spark/sql/delta/IdentityColumnIngestionSuite.scala | 506 | // have a sensible behavior and be robust to these bad watermark. | COMMENT |
| MEDIUM | …umnmapping/RemoveColumnMappingStreamingReadSuite.scala | 69 | // Essentially, prohibit reading across the downgrade. | COMMENT |
| MEDIUM | …k/sql/delta/hooks/metrics/UpdateMetricsHookSuite.scala | 234 | // suffix so we stay robust to any ApiClient base-path that future SDK changes might | COMMENT |
| MEDIUM | …ache/spark/sql/delta/streaming/SchemaTrackingLog.scala | 122 | // Subclass can leverage this to compare the differences. | COMMENT |
| MEDIUM | …apache/spark/sql/delta/sources/DeltaSourceOffset.scala | 271 | // index right away when we need to utilize this offset in memory. | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …e/spark/sql/delta/hudi/HudiConversionTransaction.scala | 252 | // create a clean instant and mark it as requested with the clean plan | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | …internal/actions/GenerateIcebergCompatActionUtils.java | 402 | // I've added these APIs here since they rely on the assumptions validated within | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| HIGH | python/delta/exceptions/captured.py | 130 | s: str = e.toString() | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | project/tests/test_published_artifacts.py | 371 | CODE | |
| LOW | project/tests/test_published_artifacts.py | 456 | CODE | |
| LOW | project/scripts/get_spark_version_info.py | 101 | CODE | |
| LOW | project/scripts/get_spark_version_info.py | 239 | CODE | |
| LOW | project/scripts/collect_test_durations.py | 117 | CODE | |
| LOW | dev/spark_structured_logging_style.py | 42 | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | run-integration-tests.py | 23 | # Usage: | COMMENT |
| LOW | project/scripts/install-uv.sh | 5 | # Usage: | COMMENT |
| LOW | project/scripts/install-buf.sh | 8 | # Usage: | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …a/io/delta/kernel/defaults/CreateCheckpointSuite.scala | 413 | spark.sql(s"INSERT INTO delta.`$tablePath` VALUES ('John Doe', 30), ('Bob Johnson', 35)") | CODE |
| LOW | …ain/java/io/delta/kernel/examples/BaseTableWriter.java | 88 | "123 Main St", | CODE |
| LOW | …internal/v2/snapshot/unitycatalog/UCTableInfoTest.java | 59 | authConfig.put("token", "fake-token"); | CODE |
| LOW | …delta/spark/internal/v2/ddl/DDLRequestContextTest.java | 63 | Map.of("token", "fake-token")); | CODE |
| LOW | …test/scala/org/apache/spark/sql/delta/DeltaSuite.scala | 3117 | "fs.unitycatalog.auth.fake-token" -> "dapi_secret_token" | STRING |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | python/delta/__init__.py | 21 | __all__ = ['DeltaTable', 'configure_spark_with_delta_pip', '__version__'] | CODE |
| LOW | python/delta/exceptions/__init__.py | 35 | __all__ = [ | CODE |
| LOW | python/delta/testing/__init__.py | 17 | __all__ = ['utils'] | CODE |
| LOW | python/delta/connect/__init__.py | 21 | __all__ = ['DeltaTable'] | CODE |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | project/TestParallelization.scala | 241 | * Here's a simple example using 3 shards and 2 groups per shard: | STRING |
| LOW | .github/workflows/spark_python_test.yaml | 95 | # when changing TEST_PARALLELISM_COUNT make sure to also change it in spark_test.yaml | COMMENT |
| LOW | .github/workflows/iceberg_test.yaml | 52 | # when changing TEST_PARALLELISM_COUNT make sure to also change it in spark_master_test.yaml | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| LOW | …dePlanning/IcebergRESTCatalogPlanningClientSuite.scala | 132 | // TODO: Add test for partitioned table rejection | COMMENT |
| LOW | …k/internal/v2/read/changelog/DeltaV2ChangelogScan.java | 61 | // TODO: implement toMicroBatchStream() so spark.readStream...loadChangelog(...) can drive a | COMMENT |
| Severity | File | Line | Snippet | Context |
|---|---|---|---|---|
| MEDIUM | project/scripts/setup_unitycatalog_main.sh | 10 | # scaffolding to rip out when Delta can use a released UC version again (flip | COMMENT |