Repository Analysis

delta-io/delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

5.7 Low AI signal View on GitHub

Analysis Overview

This report presents the forensic synthetic code analysis of delta-io/delta, a Scala project with 8,905 GitHub stars. SynthScan v2.0 examined 634,060 lines of code across 2662 source files, recording 869 pattern matches distributed across 21 syntactic categories. The overall adjusted score of 5.7 places this repository in the Low AI signal band.

The scanner applied 160+ deterministic lexical heuristics, multi-line block detectors, abstract syntax tree depth profilers, and a cross-file Jaccard similarity matrix to construct a statistically normalised synthetic code estimate. All matches are individually weighted by severity coefficient and contextual multiplier before summation, and the resulting headline score is temporally discounted to account for the repository's development history relative to the commercial emergence of large language model coding tooling (November 2022 onward).

5.7
Adjusted Score
5.7
Raw Score
100%
Time Factor
2026-07-14
Last Push
8.9K
Stars
Scala
Language
634.1K
Lines of Code
2.7K
Files
869
Pattern Hits
2026-07-14
Scan Date
0.11
HC Hit Rate

What These Metrics Mean

Adjusted Score
Primary synthetic code indicator. Raw score normalised per 1,000 lines of code and multiplied by the temporal discount factor. This is the definitive comparative metric — use it to rank repositories by AI authorship density.
Raw Score
The unmodified sum of all severity-weighted, context-multiplied pattern match scores before temporal discounting. Reflects the absolute signal strength independent of when the repository was last active.
Time Factor
The temporal discount multiplier (0–100%) applied to the raw score. Repositories last updated before ChatGPT's launch (Nov 2022) receive a 5% factor. Full signal is only assigned to repositories active in the post-adoption era (Jan 2024+).
Pattern Hits
Total count of individual pattern matches across all files and categories. A high hit count with a low score may indicate a very large codebase with isolated AI snippets; a low count with a high score indicates dense, concentrated AI signatures.
HC Hit Rate
High+Critical pattern hits per file, averaged across the repository. This orthogonal signal catches repositories where a few files are densely packed with high-severity AI tells — a strong indicator even when the normalised score appears moderate due to codebase size.
Lines of Code / Files
Total lines and files analysed. The scanner examines 94 file extensions. These denominators are used to normalise the score, enabling fair comparison between repositories of vastly different sizes.

Score History

This chart maps the temporal evolution of the adjusted synthetic code score across successive scan runs. An upward trajectory indicates ongoing incorporation of AI-generated code or expanding LLM-assisted scaffolding; a stable or declining trajectory may reflect active human refactoring, code removal, or the adoption of stricter authorship policies. The dashed secondary line (right axis) independently tracks total raw pattern hit count, which can diverge from the normalised score when codebase size changes significantly between scans.

Severity Breakdown

Classifies detected patterns by their diagnostic confidence and structural impact. CRITICAL patterns (coefficient 10) represent definitive synthetic signatures — hallucinated imports, explicit LLM attribution metadata — virtually never produced by human authors. HIGH (5) indicates strong structural tells such as cross-file repetition or cross-linguistic idioms. MEDIUM (2) covers recognisable conversational padding and AI-specific vocabulary. LOW (1) captures subtle indicators like tautological comments and generic boilerplate that require density to carry independent signal.

CRITICAL 233HIGH 54MEDIUM 37LOW 545

Directory Score Breakdown

This horizontal bar chart decomposes the repository's raw synthetic code score by top-level directory, allowing you to pinpoint precisely which modules or components carry the highest AI authorship density. Directories with disproportionately high scores relative to their size warrant targeted manual review: concentrated AI signatures often trace back to mass-generated configuration layers, auto-ported test suites, LLM-scaffolded boilerplate classes, or entire subsystems authored under heavy copilot assistance. Use this view to prioritise your human code-review effort.

Pattern Findings

The scanner identified 869 distinct pattern matches across 21 syntactic categories. Each entry below represents a discrete location in the source code where the engine recorded a statistically significant AI authorship indicator. Expand any category row to inspect the individual file paths, line numbers, code snippets, and the lexical context (CODE, COMMENT, or STRING) in which each match was detected.

Reading the findings table: The Severity column indicates the diagnostic confidence level (CRITICAL / HIGH / MEDIUM / LOW). The Context column identifies whether the match occurred inside executable code, an inline comment, or a string literal — comment-context matches receive a ×1.5 weight because LLMs systematically over-annotate. The ⚡ bolt icon marks clustered matches: three or more patterns within a 10-line window, each receiving an additional ×1.5 density multiplier as dense clusters constitute far stronger evidence of synthetic authorship than isolated hits.

Hallucination Indicators233 hits · 2520 pts
SeverityFileLineSnippetContext
CRITICAL…elta/sharing/spark/DeltaFormatSharingSourceSuite.scala2716 val batch2Json = batch2OffsetOpt.get.offsets.head.get.json()CODE
CRITICAL…est/scala/io/delta/connect/tables/DeltaQueryTest.scala159 |${org.apache.spark.util.SparkErrorUtils.stackTraceToString(e)}CODE
CRITICALpython/delta/tables.py347 jdt = jvm.io.delta.tables.DeltaTable.convertToDelta(CODE
CRITICALpython/delta/tables.py353 jdt = jvm.io.delta.tables.DeltaTable.convertToDelta(CODE
CRITICALpython/delta/tables.py399 jdt = jvm.io.delta.tables.DeltaTable.forPath(jsparkSession, path, hadoopConf)CODE
CRITICALpython/delta/tables.py436 jdt = jvm.io.delta.tables.DeltaTable.forName(jsparkSession, tableOrViewName)CODE
CRITICALpython/delta/tables.py471 jdt = jvm.io.delta.tables.DeltaTable.create(jsparkSession)CODE
CRITICALpython/delta/tables.py506 jdt = jvm.io.delta.tables.DeltaTable.createIfNotExists(jsparkSession)CODE
CRITICALpython/delta/tables.py541 jdt = jvm.io.delta.tables.DeltaTable.replace(jsparkSession)CODE
CRITICALpython/delta/tables.py576 jdt = jvm.io.delta.tables.DeltaTable.createOrReplace(jsparkSession)CODE
CRITICALpython/delta/tables.py605 return jvm.io.delta.tables.DeltaTable.isDeltaTable(jsparkSession, identifier)CODE
CRITICALpython/delta/tables.py1437 _col_jbuilder = jvm.io.delta.tables.DeltaTable.columnBuilder(jsparkSession, colName)CODE
CRITICALpython/delta/exceptions/captured.py135 stacktrace = jvm.org.apache.spark.util.Utils.exceptionString(e)CODE
CRITICALpython/delta/connect/exceptions.py172 pyspark.errors.exceptions.connect.EXCEPTION_CLASS_MAPPING.update(CODE
CRITICALpython/delta/connect/tests/test_exceptions.py41 return pyspark.sql.connect.client.core.convert_exception(info, truncated_message, resp)CODE
CRITICAL…k/src/test/java/io/delta/flink/sink/DeltaSinkTest.java286 new org.apache.flink.table.types.logical.ArrayType(CODE
CRITICAL…st/java/io/delta/flink/sink/DeltaWriterResultTest.java47 Map.of("part", io.delta.kernel.expressions.Literal.ofInt(value))))CODE
CRITICAL…test/java/io/delta/flink/sink/DeltaWriterTaskTest.java61 Map.of("part", io.delta.kernel.expressions.Literal.ofString("p0"));CODE
CRITICAL…test/java/io/delta/flink/sink/DeltaWriterTaskTest.java122 Map.of("part", io.delta.kernel.expressions.Literal.ofString("p0"));CODE
CRITICAL…est/java/io/delta/flink/sink/DeltaCommittableTest.java47 Map.of("part", io.delta.kernel.expressions.Literal.ofInt(value))))CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java79 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java143 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java209 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java269 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java314 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java369 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/sink/DeltaCommitterTest.java417 io.delta.kernel.expressions.Literal.ofString(CODE
CRITICAL…/test/java/io/delta/flink/table/VersionHelperTest.java31 Package kernelPkg = io.delta.kernel.Table.class.getPackage();CODE
CRITICAL…/test/java/io/delta/flink/table/VersionHelperTest.java33 Package ucPkg = io.unitycatalog.client.ApiClientBuilder.class.getPackage();CODE
CRITICAL…/test/java/io/delta/flink/table/VersionHelperTest.java35 Package flinkPkg = org.apache.flink.core.execution.JobClient.class.getPackage();CODE
CRITICAL…/test/java/io/delta/flink/table/VersionHelperTest.java37 Package flinkSinkPkg = io.delta.flink.table.DeltaTable.class.getPackage();CODE
CRITICAL…ain/java/io/delta/flink/table/AbstractKernelTable.java382 new io.delta.kernel.internal.fs.Path(tablePath),CODE
CRITICAL…/src/main/java/io/delta/flink/table/VersionHelper.java26 Package flinkPkg = org.apache.flink.core.execution.JobClient.class.getPackage();CODE
CRITICAL…/src/main/java/io/delta/flink/table/VersionHelper.java28 Package kernelPkg = io.delta.kernel.Table.class.getPackage();CODE
CRITICAL…/src/main/java/io/delta/flink/table/VersionHelper.java30 Package flinkSinkPkg = io.delta.flink.table.DeltaTable.class.getPackage();CODE
CRITICAL…s/src/main/scala/io/delta/storage/IBMCOSLogStore.scala48 extends org.apache.spark.sql.delta.storage.HadoopFileSystemLogStore(sparkConf, initHadoopConf) {CODE
CRITICAL…/main/scala/io/delta/storage/OracleCloudLogStore.scala45 extends org.apache.spark.sql.delta.storage.HadoopFileSystemLogStore(sparkConf, initHadoopConf) {CODE
CRITICAL…pache/spark/sql/delta/ConvertIcebergToDeltaSuite.scala615 org.apache.commons.io.IOUtils.toByteArray(inputStream)CODE
CRITICAL…ala/org/apache/spark/sql/delta/CloneIcebergSuite.scala378 .addField(org.apache.iceberg.expressions.Expressions.month("date"))CODE
CRITICAL…dePlanning/IcebergRESTCatalogPlanningClientSuite.scala673 val oauthServer = com.sun.net.httpserver.HttpServer.create(CODE
CRITICAL…dePlanning/IcebergRESTCatalogPlanningClientSuite.scala697 val tokenProvider = io.unitycatalog.client.auth.TokenProvider.create(CODE
CRITICAL…verSidePlanning/IcebergRESTCatalogPlanningClient.scala558 java.util.concurrent.TimeUnit.SECONDS.toMillis(1L << (lastExecutionCount.get() - 1))CODE
CRITICAL…t/java/io/delta/spark/internal/v2/DeltaV2TestBase.java139 org.apache.commons.io.FileUtils.deleteDirectory(new java.io.File(tablePath));CODE
CRITICAL…a/io/delta/kernel/benchmarks/WorkloadOutputFormat.java329 org.openjdk.jmh.Main.class.getPackage().getImplementationVersion(),CODE
CRITICAL…kernel/unitycatalog/UcLoadSnapshotTelemetrySuite.scala156 val engineWithReporter = new io.delta.kernel.defaults.engine.DefaultEngine(CODE
CRITICAL…kernel/unitycatalog/UcLoadSnapshotTelemetrySuite.scala157 new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO(CODE
CRITICAL…delta/kernel/unitycatalog/UcCommitTelemetrySuite.scala114 new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO(new Configuration())) {CODE
CRITICAL…ta/kernel/unitycatalog/UCCatalogManagedTestUtils.scala79 new io.delta.kernel.defaults.engine.hadoopio.HadoopFileIO(hadoopConf)) {CODE
CRITICAL…/delta/kernel/unitycatalog/UCCatalogManagedClient.java544 return io.delta.kernel.utils.FileStatus.of(CODE
CRITICAL…lta/kernel/unitycatalog/UCCatalogManagedCommitter.java505 org.apache.hadoop.fs.permission.FsPermission.getFileDefault() /* permission */,CODE
CRITICAL…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala370 val logPath = new io.delta.kernel.internal.fs.Path(path + "/_delta_log")CODE
CRITICAL…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala437 val logPath = new io.delta.kernel.internal.fs.Path(path + "/_delta_log")CODE
CRITICAL…scala/io/delta/kernel/defaults/TableChangesSuite.scala814 val log = org.apache.spark.sql.delta.DeltaLog.forTable(CODE
CRITICAL…scala/io/delta/kernel/defaults/TableChangesSuite.scala817 val deltaCommitFileProvider = org.apache.spark.sql.delta.util.DeltaCommitFileProvider(CODE
CRITICAL…la/io/delta/kernel/defaults/DeltaTableReadsSuite.scala1056 val log = org.apache.spark.sql.delta.DeltaLog.forTable(CODE
CRITICAL…la/io/delta/kernel/defaults/DeltaTableReadsSuite.scala1059 val deltaCommitFileProvider = org.apache.spark.sql.delta.util.DeltaCommitFileProvider(CODE
CRITICAL…faults/catalogManaged/CatalogManagedE2EReadSuite.scala352 org.apache.commons.io.FileUtils.copyDirectory(CODE
CRITICAL…faults/catalogManaged/CatalogManagedE2EReadSuite.scala365 org.apache.commons.io.FileUtils.copyFile(stagedCommitFile, publishedCommitFile)CODE
CRITICAL…faults/catalogManaged/CatalogManagedE2EReadSuite.scala436 org.apache.commons.io.FileUtils.copyDirectory(CODE
CRITICAL…faults/catalogManaged/CatalogManagedE2EReadSuite.scala449 org.apache.commons.io.FileUtils.copyFile(stagedCommitFile, publishedCommitFile)CODE
173 more matches not shown…
Cross-File Repetition51 hits · 255 pts
SeverityFileLineSnippetContext
HIGH…ain/resources/golden/spark-variant-checkpoint/info.txt0with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_jsSTRING
HIGH…-variant-stable-feature-checkpoint/_delta_log/info.txt0with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_jsSTRING
HIGH…esources/spark-shredded-variant-preview-delta/info.txt0with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_jsSTRING
HIGH…s/src/test/resources/spark-variant-checkpoint/info.txt0with jsonstrings as ( select id, format_string('{"key": %s}', id) as jsonstring from range(0, 100) ) select id, parse_jsSTRING
HIGH…ain/resources/golden/spark-variant-checkpoint/info.txt0create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2)STRING
HIGH…-variant-stable-feature-checkpoint/_delta_log/info.txt0create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2)STRING
HIGH…s/src/test/resources/spark-variant-checkpoint/info.txt0create or replace table $tablename using delta tblproperties (delta.checkpointinterval = 2)STRING
HIGH…o/delta/sharing/spark/DeltaSharingTestSparkUtils.scala0{ | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |}STRING
HIGH…a/sharing/spark/DeltaSharingDeltaLogBuilderSuite.scala0{ | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |}STRING
HIGH…io/delta/sharing/spark/DeltaSharingCDFUtilsSuite.scala0{ | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |}STRING
HIGH…o/delta/sharing/spark/DeltaSharingFileIndexSuite.scala0{ | "sharecredentialsversion": 1, | "endpoint": "https://localhost:12345/not-used-endpoint", | "bearertoken": "mock" |}STRING
HIGHpython/delta/exceptions/base.py0the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolvingSTRING
HIGHpython/delta/exceptions/captured.py0the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolvingSTRING
HIGHpython/delta/connect/exceptions.py0the basic class for all delta commit conflict exceptions. .. versionadded:: 4.0 .. note:: evolvingSTRING
HIGHpython/delta/exceptions/base.py0thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4.STRING
HIGHpython/delta/exceptions/captured.py0thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4.STRING
HIGHpython/delta/connect/exceptions.py0thrown when a concurrent transaction has written data after the current transaction read the table. .. versionadded:: 4.STRING
HIGHpython/delta/exceptions/base.py0thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionaddedSTRING
HIGHpython/delta/exceptions/captured.py0thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionaddedSTRING
HIGHpython/delta/connect/exceptions.py0thrown when the metadata of the delta table has changed between the time of read and the time of commit. .. versionaddedSTRING
HIGHpython/delta/exceptions/base.py0thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. nSTRING
HIGHpython/delta/exceptions/captured.py0thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. nSTRING
HIGHpython/delta/connect/exceptions.py0thrown when the protocol version has changed between the time of read and the time of commit. .. versionadded:: 4.0 .. nSTRING
HIGHpython/delta/exceptions/base.py0thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolviSTRING
HIGHpython/delta/exceptions/captured.py0thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolviSTRING
HIGHpython/delta/connect/exceptions.py0thrown when files are added that would have been read by the current transaction. .. versionadded:: 4.0 .. note:: evolviSTRING
HIGHpython/delta/exceptions/base.py0thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. STRING
HIGHpython/delta/exceptions/captured.py0thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. STRING
HIGHpython/delta/connect/exceptions.py0thrown when the current transaction deletes data that was deleted by a concurrent transaction. .. versionadded:: 4.0 .. STRING
HIGHpython/delta/exceptions/base.py0thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. noteSTRING
HIGHpython/delta/exceptions/captured.py0thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. noteSTRING
HIGHpython/delta/connect/exceptions.py0thrown when concurrent transaction both attempt to update the same idempotent transaction. .. versionadded:: 4.0 .. noteSTRING
HIGH…park-4.1/org/apache/spark/sql/RowComparisonUtils.scala0|== results == |${ sidebyside( s"== correct answer - ${expectedanswer.size} ==" +: getrowtype(expectedanswer.headoption)STRING
HIGH…park-4.0/org/apache/spark/sql/RowComparisonUtils.scala0|== results == |${ sidebyside( s"== correct answer - ${expectedanswer.size} ==" +: getrowtype(expectedanswer.headoption)STRING
HIGH…park-4.2/org/apache/spark/sql/RowComparisonUtils.scala0|== results == |${ sidebyside( s"== correct answer - ${expectedanswer.size} ==" +: getrowtype(expectedanswer.headoption)STRING
HIGH…spark-4.1/org/apache/spark/sql/CheckAnswerHelper.scala0|failed to analyze query: $ae |${ae.plan.get} | |${sparkerrorutils.stacktracetostring(ae)} |STRING
HIGH…spark-4.0/org/apache/spark/sql/CheckAnswerHelper.scala0|failed to analyze query: $ae |${ae.plan.get} | |${sparkerrorutils.stacktracetostring(ae)} |STRING
HIGH…spark-4.2/org/apache/spark/sql/CheckAnswerHelper.scala0|failed to analyze query: $ae |${ae.plan.get} | |${sparkerrorutils.stacktracetostring(ae)} |STRING
HIGH…spark-4.1/org/apache/spark/sql/CheckAnswerHelper.scala0cannot determine whether df is sorted: $df. |maybe the suite is missing the connect.sessionquerytest mixin?STRING
HIGH…spark-4.0/org/apache/spark/sql/CheckAnswerHelper.scala0cannot determine whether df is sorted: $df. |maybe the suite is missing the connect.sessionquerytest mixin?STRING
HIGH…spark-4.2/org/apache/spark/sql/CheckAnswerHelper.scala0cannot determine whether df is sorted: $df. |maybe the suite is missing the connect.sessionquerytest mixin?STRING
HIGH…spark-4.1/org/apache/spark/sql/CheckAnswerHelper.scala0|exception thrown while executing query: |${if (df.isinstanceof[classic.dataframe]) { df.queryexecution } else df.tostriSTRING
HIGH…spark-4.0/org/apache/spark/sql/CheckAnswerHelper.scala0|exception thrown while executing query: |${if (df.isinstanceof[classic.dataframe]) { df.queryexecution } else df.tostriSTRING
HIGH…spark-4.2/org/apache/spark/sql/CheckAnswerHelper.scala0|exception thrown while executing query: |${if (df.isinstanceof[classic.dataframe]) { df.queryexecution } else df.tostriSTRING
HIGH…spark-4.1/org/apache/spark/sql/CheckAnswerHelper.scala0|results do not match for query: |timezone: ${timezone.getdefault} |timezone env: ${sys.env.getorelse("tz", "")} | |${ifSTRING
HIGH…spark-4.0/org/apache/spark/sql/CheckAnswerHelper.scala0|results do not match for query: |timezone: ${timezone.getdefault} |timezone env: ${sys.env.getorelse("tz", "")} | |${ifSTRING
HIGH…spark-4.2/org/apache/spark/sql/CheckAnswerHelper.scala0|results do not match for query: |timezone: ${timezone.getdefault} |timezone env: ${sys.env.getorelse("tz", "")} | |${ifSTRING
HIGH…/scala/org/apache/spark/sql/delta/DeleteSQLSuite.scala0create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = faSTRING
HIGH…/scala/org/apache/spark/sql/delta/UpdateSQLSuite.scala0create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = faSTRING
HIGH…ala/org/apache/spark/sql/delta/MergeIntoSQLSuite.scala0create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = faSTRING
HIGH…test/scala/org/apache/spark/sql/delta/DeltaSuite.scala0create table tab using delta |${if (p) "partitioned by (part) " else ""} |tblproperties (delta.enablechangedatafeed = faSTRING
Structural Annotation Overuse95 hits · 174 pts
SeverityFileLineSnippetContext
LOWrun-integration-tests.py109 # Step 1: unsuffixed (backward compat)COMMENT
LOWrun-integration-tests.py113 # Step 2: suffixed for each non-master Spark versionCOMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java193 // Step 0: Fail if N.json already exists in FileSystem and overwrite=false.COMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java197 // Step 1: Ensure that N-1.json existsCOMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java238 // Step 2: PREPARE the commitCOMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java255 // Step 3: COMMIT the commit to the delta log.COMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java259 // Step 4: ACKNOWLEDGE the commitCOMMENT
LOWproject/CrossSparkVersions.scala501 // Step 1: Publish ALL modules WITHOUT Spark suffix (backward compatibility)COMMENT
LOWproject/tests/test_cross_spark_publish.py265 # Step 1: Publish all modules WITHOUT suffix (backward compatibility)COMMENT
LOWproject/tests/test_cross_spark_publish.py272 # Step 2: Publish Spark-dependent modules WITH suffix for each non-master versionCOMMENT
LOWproject/tests/test_cross_spark_publish.py290 # Step 1: All modules without suffix (uses default Spark version's iceberg support)COMMENT
LOWproject/tests/test_cross_spark_publish.py295 # Step 2: Spark-dependent modules WITH suffix for each non-master versionCOMMENT
LOWkernel/USER_GUIDE.md73### Step 1: Full scan on a Delta tableCOMMENT
LOWkernel/USER_GUIDE.md209### Step 2: Improve scan performance with file skippingCOMMENT
LOWkernel/USER_GUIDE.md596### Step 0: Validate the prerequisitesCOMMENT
LOWkernel/USER_GUIDE.md609### Step 1: Set up Delta Kernel in your connector projectCOMMENT
LOWkernel/USER_GUIDE.md634### Step 2: Build your own EngineCOMMENT
LOWkernel/USER_GUIDE.md783### Step 3: Build read support in your connectorCOMMENT
LOWkernel/USER_GUIDE.md990### Step 4: Build append support in your connectorCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala64 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala72 // Step 2: WRITE -- v1.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala80 // Step 3: WRITE -- v2.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala129 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala137 // Step 2: WRITE -- v1.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala145 // Step 3: WRITE -- v2.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala183 // Step 5: Use postPublish snapshot to write -- v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala191 // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala309 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala316 // Step 2: WRITE and commit data up to version 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala331 // Step 3: PUBLISH v1.json and v2.json to the filesystemCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala334 // Step 4: Configure the UC client to limit maxRatifiedVersion to 1COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala337 // Step 5: Load snapshot with UC client that limits maxRatifiedVersion to 1COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala340 // Step 6: Verify that snapshot is at version 1, not version 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala358 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala365 // Step 2: WRITE multiple versionsCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala388 // Step 3: Publish all versionsCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala391 // Step 4: Load CommitRange with end boundary (should go from 0 to 3)COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala405 // Step 5: Configure UC client to limit maxRatifiedVersion to 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala408 // Step 6: Load CommitRange again (should now be limited to version 2)COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala97 // Step 5: WRITE -- v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala114 // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala163 // Step 5: Read the latest snapshot from disk. Post-publish snapshot should be similar to itCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala422 // Step 8: Load CommitRange with start version at maxCatalogVersion (should work)COMMENT
LOW…est/scala/io/delta/kernel/defaults/SnapshotSuite.scala43 // Step 1: Create a table with the given partition columnsCOMMENT
LOW…est/scala/io/delta/kernel/defaults/SnapshotSuite.scala62 // Step 2: Check the partition columnsCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala275 // Step 1: Create a table (v0.json) and get the post-commit snapshotCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala280 // Step 2: Load a new snapshot to latest (v0)COMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala285 // Step 3: Use the fresh snapshot to write the checksumCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala290 // Step 4: Use the first (post-commit) snapshot to write the checksum -- should NOT failCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala970 // Step 1: Verify initial checksumCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala973 // Step 2: Delete and regenerate the checksumCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala977 // Step 3: Verify regenerated checksumCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java187 // Step 1: If row tracking is enabled, check for physical row tracking columns in the dataCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java195 // Step 2: Get the selectionVector if DV is presentCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java217 // Step 3: If a column was only requested to compute other columns, we remove itCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java232 // Step 4: Add partition columns back to the data batchCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java240 // Step 5: Transform column names back to logical names if column mapping is enabledCOMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java157 // Step 1: We put the required catalog table properties *first* so that we persist the intent,COMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java162 // Step 2: Preserve a few important delta propertiesCOMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java168 // Step 3: Insert the new user-provided table propertiesCOMMENT
35 more matches not shown…
Verbosity Indicators89 hits · 165 pts
SeverityFileLineSnippetContext
LOWrun-integration-tests.py109 # Step 1: unsuffixed (backward compat)COMMENT
LOWrun-integration-tests.py113 # Step 2: suffixed for each non-master Spark versionCOMMENT
LOW…/main/java/io/delta/flink/kernel/CheckpointWriter.java208 // Step 1:COMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java193 // Step 0: Fail if N.json already exists in FileSystem and overwrite=false.COMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java197 // Step 1: Ensure that N-1.json existsCOMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java238 // Step 2: PREPARE the commitCOMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java255 // Step 3: COMMIT the commit to the delta log.COMMENT
LOW…c/main/java/io/delta/storage/BaseExternalLogStore.java259 // Step 4: ACKNOWLEDGE the commitCOMMENT
LOWproject/CrossSparkVersions.scala501 // Step 1: Publish ALL modules WITHOUT Spark suffix (backward compatibility)COMMENT
LOWproject/tests/test_cross_spark_publish.py265 # Step 1: Publish all modules WITHOUT suffix (backward compatibility)COMMENT
LOWproject/tests/test_cross_spark_publish.py272 # Step 2: Publish Spark-dependent modules WITH suffix for each non-master versionCOMMENT
LOWproject/tests/test_cross_spark_publish.py290 # Step 1: All modules without suffix (uses default Spark version's iceberg support)COMMENT
LOWproject/tests/test_cross_spark_publish.py295 # Step 2: Spark-dependent modules WITH suffix for each non-master versionCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala64 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala72 // Step 2: WRITE -- v1.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala80 // Step 3: WRITE -- v2.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala129 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala137 // Step 2: WRITE -- v1.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala145 // Step 3: WRITE -- v2.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala183 // Step 5: Use postPublish snapshot to write -- v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala191 // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala309 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala316 // Step 2: WRITE and commit data up to version 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala331 // Step 3: PUBLISH v1.json and v2.json to the filesystemCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala334 // Step 4: Configure the UC client to limit maxRatifiedVersion to 1COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala337 // Step 5: Load snapshot with UC client that limits maxRatifiedVersion to 1COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala340 // Step 6: Verify that snapshot is at version 1, not version 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala358 // Step 1: CREATE -- v0.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala365 // Step 2: WRITE multiple versionsCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala388 // Step 3: Publish all versionsCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala391 // Step 4: Load CommitRange with end boundary (should go from 0 to 3)COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala405 // Step 5: Configure UC client to limit maxRatifiedVersion to 2COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala408 // Step 6: Load CommitRange again (should now be limited to version 2)COMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala97 // Step 5: WRITE -- v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala114 // Step 6: LOAD -- should read v0.json, v1.json, v2.json, and v3.uuid.jsonCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala163 // Step 5: Read the latest snapshot from disk. Post-publish snapshot should be similar to itCOMMENT
LOW…st/scala/io/delta/kernel/unitycatalog/UCE2ESuite.scala422 // Step 8: Load CommitRange with start version at maxCatalogVersion (should work)COMMENT
LOW…est/scala/io/delta/kernel/defaults/SnapshotSuite.scala43 // Step 1: Create a table with the given partition columnsCOMMENT
LOW…est/scala/io/delta/kernel/defaults/SnapshotSuite.scala62 // Step 2: Check the partition columnsCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala275 // Step 1: Create a table (v0.json) and get the post-commit snapshotCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala280 // Step 2: Load a new snapshot to latest (v0)COMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala285 // Step 3: Use the fresh snapshot to write the checksumCOMMENT
LOW…defaults/SnapshotChecksumStatisticsAndWriteSuite.scala290 // Step 4: Use the first (post-commit) snapshot to write the checksum -- should NOT failCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala970 // Step 1: Verify initial checksumCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala973 // Step 2: Delete and regenerate the checksumCOMMENT
LOW…t/scala/io/delta/kernel/defaults/utils/TestUtils.scala977 // Step 3: Verify regenerated checksumCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java187 // Step 1: If row tracking is enabled, check for physical row tracking columns in the dataCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java195 // Step 2: Get the selectionVector if DV is presentCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java217 // Step 3: If a column was only requested to compute other columns, we remove itCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java232 // Step 4: Add partition columns back to the data batchCOMMENT
LOW…nel/kernel-api/src/main/java/io/delta/kernel/Scan.java240 // Step 5: Transform column names back to logical names if column mapping is enabledCOMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java157 // Step 1: We put the required catalog table properties *first* so that we persist the intent,COMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java162 // Step 2: Preserve a few important delta propertiesCOMMENT
LOW…/delta/kernel/internal/TransactionMetadataFactory.java168 // Step 3: Insert the new user-provided table propertiesCOMMENT
LOW…io/delta/kernel/internal/snapshot/MetadataCleanup.java100 // Step 1: Check if the `lastSeenCheckpointFiles` contains a complete checkpointCOMMENT
LOW…io/delta/kernel/internal/snapshot/MetadataCleanup.java134 // Step 2: If the timestamp is earlier than the retention period, stopCOMMENT
LOW…io/delta/kernel/internal/snapshot/MetadataCleanup.java148 // Step 3: If the file is a delta log file, add it to the `potentialFilesToDelete` listCOMMENT
LOW…io/delta/kernel/internal/snapshot/MetadataCleanup.java153 // Step 4: If the file is a checkpoint file, add it to the `lastSeenCheckpointFiles`COMMENT
LOW…io/delta/kernel/internal/snapshot/SnapshotManager.java367 // Step 3: Perform some basic validations on the listed file statuses //COMMENT
LOW…io/delta/kernel/internal/snapshot/SnapshotManager.java377 // Step 4: Partition $listedFileStatuses into the checkpoints, deltas, and compactions. //COMMENT
29 more matches not shown…
Over-Commented Block155 hits · 154 pts
SeverityFileLineSnippetContext
LOWrun-integration-tests.py1#!/usr/bin/env python3COMMENT
LOWrun-integration-tests.py21# Scala, Python, and pip tests against them.COMMENT
LOWrun-tests.py1#!/usr/bin/env python3COMMENT
LOW…elta/sharing/spark/DeltaFormatSharingSourceSuite.scala481 }COMMENT
LOW…/io/delta/sharing/spark/DeltaFormatSharingSource.scala281 s"already set to $frozenServerVersionForAvailableNow." +COMMENT
LOW…/io/delta/sharing/spark/DeltaFormatSharingSource.scala961 deltaLogMetadata.minUrlExpirationTimestamp.getCOMMENT
LOW…/io/delta/sharing/spark/DeltaFormatSharingSource.scala981 // - Start 1: latestOffset(None) -> getBatch(None, offset1)COMMENT
LOWspark-connect/common/src/main/buf.work.yaml1#COMMENT
LOWspark-connect/common/src/main/buf.gen.yaml1#COMMENT
LOWpython/run-tests.py1#!/usr/bin/env python3COMMENT
LOWpython/delta/version.py1#COMMENT
LOWpython/delta/_typing.py1#COMMENT
LOWpython/delta/__init__.py1#COMMENT
LOWpython/delta/pip_utils.py1#COMMENT
LOWpython/delta/tables.py1#COMMENT
LOW…/unity-catalog-commit-coordinator-integration-tests.py1#COMMENT
LOWpython/delta/tests/test_pip_utils.py1#COMMENT
LOWpython/delta/tests/test_version.py1#COMMENT
LOWpython/delta/tests/test_deltatable.py1#COMMENT
LOWpython/delta/tests/test_exceptions.py1#COMMENT
LOWpython/delta/tests/__init__.py1#COMMENT
LOWpython/delta/tests/test_sql.py1#COMMENT
LOWpython/delta/tests/test_exceptions_init.py1#COMMENT
LOWpython/delta/exceptions/__init__.py1#COMMENT
LOWpython/delta/exceptions/base.py1#COMMENT
LOWpython/delta/exceptions/captured.py1#COMMENT
LOWpython/delta/testing/__init__.py1#COMMENT
LOWpython/delta/testing/utils.py1#COMMENT
LOWpython/delta/connect/plan.py1#COMMENT
LOWpython/delta/connect/_typing.py1#COMMENT
LOWpython/delta/connect/__init__.py1#COMMENT
LOWpython/delta/connect/exceptions.py1#COMMENT
LOWpython/delta/connect/tables.py1#COMMENT
LOWpython/delta/connect/tests/test_deltatable.py1#COMMENT
LOWpython/delta/connect/tests/test_exceptions.py1#COMMENT
LOWpython/delta/connect/tests/__init__.py1#COMMENT
LOWpython/delta/connect/tests/test_exceptions_init.py1#COMMENT
LOWpython/delta/connect/testing/__init__.py1#COMMENT
LOWpython/delta/connect/testing/utils.py1#COMMENT
LOW…/main/java/io/delta/flink/kernel/CheckpointWriter.java261 snapshot.getCreateCheckpointIterator(engine).map(filterAddFiles())) {COMMENT
LOW…age-s3-dynamodb/integration_tests/dynamodb_logstore.py1#COMMENT
LOWdocs/apis/generate_api_docs.py1# !/usr/bin/env python3COMMENT
LOWdocs/apis/python/conf.py1#COMMENT
LOWdocs/apis/python/conf.py61# List of patterns, relative to source directory, that match files andCOMMENT
LOWiceberg/integration_tests/iceberg_converter.py1#COMMENT
LOW…e/spark/sql/delta/uniform/UniFormE2EIcebergSuite.scala341 // Now v1 snapshot and v1 Iceberg catalog becomes stale so txn's commit would hitCOMMENT
LOW…apache/spark/sql/delta/IcebergPartitionConverter.scala41object IcebergPartitionConverter {COMMENT
LOW…delta/icebergShaded/IcebergConversionTransaction.scala521 .getOrElse(RowId.MISSING_HIGH_WATER_MARK).toStringCOMMENT
LOWproject/scripts/install-buf.sh1#!/bin/shCOMMENT
LOWproject/scripts/setup_unitycatalog_main.sh1#!/usr/bin/env bashCOMMENT
LOWproject/scripts/setup_unitycatalog_main.sh21#COMMENT
LOWproject/scripts/setup_unitycatalog_main.sh41# script sanity-checks the two match when the slow path runs, so you'll get a loud errorCOMMENT
LOWproject/scripts/setup_unitycatalog_main.sh61# Forwarded as -DsparkVersion to UC's sbt; also determines the published artifactCOMMENT
LOWproject/scripts/setup_unitycatalog_main.sh201# Force publishLocal / publishM2 to overwrite existing artifacts. UC artifacts at the sameCOMMENT
LOW…pache/spark/sql/delta/v2/interop/DeltaV2Snapshot.scala81 // scalastyle:off deltahadoopconfigurationCOMMENT
LOW…pache/spark/sql/delta/v2/interop/DeltaV2Snapshot.scala101COMMENT
LOWexamples/python/image_storage.py1#COMMENT
LOWexamples/python/quickstart_sql.py1#COMMENT
LOWexamples/python/missing_delta_storage_jar.py1#COMMENT
LOWexamples/python/change_data_feed.py1#COMMENT
95 more matches not shown…
Hyper-Verbose Identifiers125 hits · 129 pts
SeverityFileLineSnippetContext
LOWrun-integration-tests.py181def run_scala_integration_tests(root_dir, version, test_name, extra_maven_repo, scala_version,CODE
LOWrun-integration-tests.py245def run_python_integration_tests(root_dir, version, test_name, extra_maven_repo, variant):CODE
LOWrun-integration-tests.py292def test_missing_delta_storage_jar(root_dir, version, use_local):CODE
LOWrun-integration-tests.py331def run_dynamodb_logstore_integration_tests(root_dir, version, test_name, extra_maven_repo,CODE
LOWrun-integration-tests.py384def run_dynamodb_commit_coordinator_integration_tests(root_dir, version, test_name, extra_maven_repo,CODE
LOWrun-integration-tests.py453def run_flink_integration_tests():CODE
LOWrun-integration-tests.py465def run_iceberg_integration_tests(root_dir, version, iceberg_version, extra_maven_repo, variant):CODE
LOWrun-integration-tests.py518def run_uniform_hudi_integration_tests(root_dir, version, hudi_version, extra_maven_repo, variant):CODE
LOWrun-integration-tests.py574def run_pip_installation_tests(root_dir, version, use_testpypi, use_localpypi, extra_maven_repo):CODE
LOWrun-integration-tests.py613def run_unity_catalog_commit_coordinator_integration_tests(root_dir, version, test_name,CODE
LOWrun-tests.py157def pull_or_build_docker_image(root_dir):CODE
LOW…/io/delta/sharing/spark/DeltaFormatSharingSource.scala171 private def getOrCreateLegacySource(snapshotVersion: Long): DeltaSharingSource = {CODE
LOW…/io/delta/sharing/spark/DeltaFormatSharingSource.scala309 private def getOrUpdateLatestTableVersion: Long = {CODE
LOWpython/run-tests.py227def run_delta_connect_codegen_python(root_dir):CODE
LOWpython/delta/pip_utils.py21def configure_spark_with_delta_pip(CODE
LOWpython/delta/tables.py1338 def _check_identity_column_spec(self, identityGenerator: IdentityGenerator) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py114 def test_read_from_managed_table_without_catalog_owned(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py117 def test_write_to_managed_catalog_owned_table(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py123 def test_read_from_managed_catalog_owned_table(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py129 def test_write_to_managed_table_without_catalog_owned(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py135 def test_unset_catalog_owned_feature(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py142 def test_drop_catalog_owned_property(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py238 def test_sql_insert_overwrite(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py245 def test_sql_insert_replace_where(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py253 def test_sql_insert_dynamic_partition_overwrite(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py271 def test_insert_into_overwrite(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py278 def test_insert_into_overwrite_replace_where(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py285 def test_insert_into_overwrite_partition_overwrite(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py293 def test_save_as_table_append_existing_table(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py303 def test_save_append_using_path(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py199 def test_merge_schema_evolution(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py216 def test_insert_schema_evolution(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py330 def test_overwrite_partitions(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py375 def test_create_non_catalog_owned(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py386 def test_clone_into_catalog_owned(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py394 def test_clone_into_non_catalog_owned(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py410 def test_alter_table_add_column(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py416 def test_alter_table_set_tbl_properties(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py517 def test_change_data_feed_with_timestamp(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py526 def test_change_data_feed_with_version(self) -> None:CODE
LOW…/unity-catalog-commit-coordinator-integration-tests.py537 def test_delta_table_for_path(self) -> None:CODE
LOWpython/delta/tests/test_version.py35 def test_version_import_from_module(self) -> None:CODE
LOWpython/delta/tests/test_version.py40 def test_version_import_from_package(self) -> None:CODE
LOWpython/delta/tests/test_version.py45 def test_version_consistency_across_imports(self) -> None:CODE
LOWpython/delta/tests/test_version.py59 def test_version_sbt_and_version_py_consistency(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py499 def test_merge_with_inconsistent_sessions(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py684 def __verify_generated_column(self, tableName: str, deltaTable: DeltaTable) -> None:CODE
LOWpython/delta/tests/test_deltatable.py730 def test_create_table_with_existing_schema(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py759 def test_create_replace_table_with_cluster_by(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py789 def test_create_replace_table_with_no_spark_session_passed(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py828 def test_create_table_with_name_only(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py845 def test_create_table_with_location_only(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py861 def test_create_table_with_name_and_location(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py880 def test_create_table_behavior(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py899 def test_replace_table_with_name_only(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py917 def test_replace_table_with_location_only(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py935 def test_replace_table_with_name_and_location(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py956 def test_replace_table_behavior(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py1001 def test_create_table_with_identity_column(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py1035 def test_delta_table_builder_with_bad_args(self) -> None:CODE
65 more matches not shown…
Self-Referential Comments16 hits · 43 pts
SeverityFileLineSnippetContext
MEDIUMpython/delta/version.py17# This file is auto-generated by the build.sbt generatePythonVersion task.COMMENT
MEDIUMpython/delta/tests/test_sql.py35 # Create a simple Delta table inside the temp directory to test SQL commands.COMMENT
MEDIUMproject/tests/test_cross_spark_publish.py236 # Create a spec without suffix for backward compatibilityCOMMENT
MEDIUMexamples/python/quickstart_sql.py36 # Create a tableCOMMENT
MEDIUMexamples/python/quickstart_sql.py37 print("############# Creating a table ###############")CODE
MEDIUMexamples/python/table_exists.py47# Create a delta table at filepathCOMMENT
MEDIUMexamples/python/using_with_pip.py48print("########### Create a Parquet table ##############")CODE
MEDIUMexamples/python/quickstart_sql_on_paths.py23 # Create a tableCOMMENT
MEDIUMexamples/python/quickstart_sql_on_paths.py24 print("############# Creating a table ###############")CODE
MEDIUMexamples/python/utilities.py32# Create a tableCOMMENT
MEDIUMexamples/python/utilities.py33print("########### Create a Parquet table ##############")CODE
MEDIUMexamples/python/streaming.py35# Create a table(key, value) of some dataCOMMENT
MEDIUMexamples/python/quickstart.py34# Create a tableCOMMENT
MEDIUMexamples/python/quickstart.py35print("############# Creating a table ###############")CODE
MEDIUM…marks/infrastructure/gcp/terraform/.terraform.lock.hcl1# This file is maintained automatically by "terraform init".COMMENT
MEDIUM…marks/infrastructure/aws/terraform/.terraform.lock.hcl1# This file is maintained automatically by "terraform init".COMMENT
Unused Imports42 hits · 40 pts
SeverityFileLineSnippetContext
LOWhudi/integration_tests/write_uniform_hudi.py2CODE
LOWhudi/integration_tests/write_uniform_hudi.py2CODE
LOWhudi/integration_tests/write_uniform_hudi.py4CODE
LOWhudi/integration_tests/write_uniform_hudi.py6CODE
LOWpython/delta/__init__.py17CODE
LOWpython/delta/__init__.py18CODE
LOWpython/delta/__init__.py19CODE
LOWpython/delta/tables.py35CODE
LOWpython/delta/tables.py35CODE
LOWpython/delta/tables.py36CODE
LOWpython/delta/tests/test_deltatable.py24CODE
LOWpython/delta/tests/test_deltatable.py25CODE
LOWpython/delta/tests/test_deltatable.py32CODE
LOWpython/delta/tests/test_exceptions.py26CODE
LOWpython/delta/tests/test_exceptions_init.py25CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py24CODE
LOWpython/delta/exceptions/__init__.py22CODE
LOWpython/delta/exceptions/captured.py35CODE
LOWpython/delta/exceptions/captured.py35CODE
LOWpython/delta/testing/utils.py19CODE
LOWpython/delta/testing/utils.py21CODE
LOWpython/delta/connect/__init__.py18CODE
LOWpython/delta/connect/__init__.py19CODE
LOWpython/delta/connect/tests/test_exceptions_init.py23CODE
LOWdocs/generate_docs.py6CODE
LOWdocs/generate_docs.py8CODE
LOWdocs/generate_docs.py9CODE
LOWiceberg/integration_tests/iceberg_converter.py18CODE
LOWiceberg/integration_tests/iceberg_converter.py19CODE
LOWiceberg/integration_tests/iceberg_converter.py21CODE
LOWexamples/python/change_data_feed.py18CODE
LOWexamples/python/using_with_pip.py22CODE
LOWexamples/python/using_with_pip.py65CODE
LOWbenchmarks/run-benchmark.py20CODE
LOWbenchmarks/scripts/benchmarks.py17CODE
Decorative Section Separators8 hits · 24 pts
SeverityFileLineSnippetContext
MEDIUMproject/scripts/setup_unitycatalog_main.sh82# ---------------------------------------------------------------------------------------------COMMENT
MEDIUMproject/scripts/setup_unitycatalog_main.sh88# ---------------------------------------------------------------------------------------------COMMENT
MEDIUM…/spark/internal/v2/ddl/CreateTableIntegrationTest.java140 // ── helpers ────────────────────────────────────────────────────────COMMENT
MEDIUM…elta/spark/internal/v2/ddl/CreateTableBuilderTest.java40 // ── filterProperties ──────────────────────────────────────────────COMMENT
MEDIUM…elta/spark/internal/v2/ddl/CreateTableBuilderTest.java84 // ── toDataLayoutSpec ──────────────────────────────────────────────COMMENT
MEDIUM…elta/spark/internal/v2/ddl/CreateTableBuilderTest.java132 // ── resolveTablePath ──────────────────────────────────────────────COMMENT
MEDIUM…elta/spark/internal/v2/ddl/CreateTableBuilderTest.java158 // ── extractComment ────────────────────────────────────────────────COMMENT
MEDIUM…elta/spark/internal/v2/ddl/CreateTableBuilderTest.java169 // ── validateClusteringColumns ─────────────────────────────────────COMMENT
AI Slop Vocabulary10 hits · 22 pts
SeverityFileLineSnippetContext
LOW.github/workflows/spark_python_test.yaml78 # just use the cache. The cache is immutable so we need to use a new key when trying toCOMMENT
LOW.github/workflows/iceberg_test.yaml40 # just use the cache. The cache is immutable so we need to use a new key when trying toCOMMENT
LOW…/java/io/delta/kernel/internal/actions/CommitInfo.java128 // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables.COMMENT
LOW…/java/io/delta/kernel/internal/actions/CommitInfo.java152 // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables.COMMENT
LOW…/java/io/delta/kernel/internal/actions/CommitInfo.java167 // TODO: [delta-io/delta#5147] Can't just use the logPath & version on catalogManaged tables.COMMENT
MEDIUM…che/spark/sql/delta/IdentityColumnIngestionSuite.scala506 // have a sensible behavior and be robust to these bad watermark.COMMENT
MEDIUM…umnmapping/RemoveColumnMappingStreamingReadSuite.scala69 // Essentially, prohibit reading across the downgrade.COMMENT
MEDIUM…k/sql/delta/hooks/metrics/UpdateMetricsHookSuite.scala233 // suffix so we stay robust to any ApiClient base-path that future SDK changes mightCOMMENT
MEDIUM…ache/spark/sql/delta/streaming/SchemaTrackingLog.scala122 // Subclass can leverage this to compare the differences.COMMENT
MEDIUM…apache/spark/sql/delta/sources/DeltaSourceOffset.scala271 // index right away when we need to utilize this offset in memory.COMMENT
Excessive Try-Catch Wrapping17 hits · 22 pts
SeverityFileLineSnippetContext
LOWrun-tests.py192 except Exception as e:CODE
LOWrun-tests.py203 except Exception as e:CODE
MEDIUMrun-tests.py185def pull_image(registry_image_tag):CODE
MEDIUMrun-tests.py196def push_image(registry_image_tag):CODE
LOWpython/run-tests.py122 except Exception as e:CODE
LOWpython/delta/pip_utils.py70 except Exception as e:STRING
MEDIUMpython/delta/tests/test_deltatable.py974def test_verify_paritionedBy_compatibility(self) -> None:CODE
LOWpython/delta/tests/test_deltatable.py1697 except Exception as e:STRING
LOWproject/tests/test_cross_spark_publish.py427 except Exception as e:CODE
MEDIUMproject/tests/test_cross_spark_publish.py575 print("Error: build.sbt not found. Run from Delta repository root.")CODE
LOWproject/tests/test_cross_spark_publish.py631 except Exception as e:CODE
LOWproject/scripts/get_spark_version_info.py162 except Exception as e:CODE
MEDIUMproject/scripts/collect_test_durations.py48 print(f"Error running: {' '.join(cmd)}", file=sys.stderr)CODE
MEDIUMproject/scripts/collect_test_durations.py107 print(f"Error downloading artifact {artifact_id}", file=sys.stderr)CODE
LOWexamples/python/missing_delta_storage_jar.py35except Exception as e:CODE
LOWexamples/python/delta_connect.py72except Exception as e:CODE
LOWexamples/python/delta_connect.py80except Exception as e:CODE
Synthetic Comment Markers1 hit · 8 pts
SeverityFileLineSnippetContext
HIGH…e/spark/sql/delta/hudi/HudiConversionTransaction.scala252 // create a clean instant and mark it as requested with the clean planCOMMENT
AI Response Leakage1 hit · 8 pts
SeverityFileLineSnippetContext
HIGH…internal/actions/GenerateIcebergCompatActionUtils.java402 // I've added these APIs here since they rely on the assumptions validated withinCOMMENT
Fake / Example Data7 hits · 6 pts
SeverityFileLineSnippetContext
LOWflink/src/test/java/io/delta/flink/MockHttp.java65 String.format("{\"storage_location\": \"%s\", \"table_id\": \"dummy_id\"}", tablePath));CODE
LOWflink/src/test/java/io/delta/flink/DummyHttp.java56 String.format("{\"storage_location\": \"%s\", \"table_id\": \"dummy_id\"}", tablePath));CODE
LOW…a/io/delta/kernel/defaults/CreateCheckpointSuite.scala406 spark.sql(s"INSERT INTO delta.`$tablePath` VALUES ('John Doe', 30), ('Bob Johnson', 35)")CODE
LOW…ain/java/io/delta/kernel/examples/BaseTableWriter.java88 "123 Main St",CODE
LOW…internal/v2/snapshot/unitycatalog/UCTableInfoTest.java58 authConfig.put("token", "fake-token");CODE
LOW…delta/spark/internal/v2/ddl/DDLRequestContextTest.java63 Map.of("token", "fake-token"));CODE
LOW…test/scala/org/apache/spark/sql/delta/DeltaSuite.scala3056 "fs.unitycatalog.auth.fake-token" -> "dapi_secret_token"STRING
Cross-Language Confusion1 hit · 5 pts
SeverityFileLineSnippetContext
HIGHpython/delta/exceptions/captured.py130 s: str = e.toString()CODE
Deep Nesting5 hits · 5 pts
SeverityFileLineSnippetContext
LOWproject/tests/test_cross_spark_publish.py306CODE
LOWproject/tests/test_cross_spark_publish.py391CODE
LOWproject/scripts/get_spark_version_info.py72CODE
LOWproject/scripts/collect_test_durations.py117CODE
LOWdev/spark_structured_logging_style.py42CODE
Example Usage Blocks3 hits · 4 pts
SeverityFileLineSnippetContext
LOWrun-integration-tests.py23# Usage:COMMENT
LOWproject/scripts/install-uv.sh5# Usage:COMMENT
LOWproject/scripts/install-buf.sh8# Usage:COMMENT
Modern Structural Boilerplate4 hits · 4 pts
SeverityFileLineSnippetContext
LOWpython/delta/__init__.py21__all__ = ['DeltaTable', 'configure_spark_with_delta_pip', '__version__']CODE
LOWpython/delta/exceptions/__init__.py35__all__ = [CODE
LOWpython/delta/testing/__init__.py17__all__ = ['utils']CODE
LOWpython/delta/connect/__init__.py21__all__ = ['DeltaTable']CODE
Slop Phrases3 hits · 4 pts
SeverityFileLineSnippetContext
MEDIUMproject/TestParallelization.scala241 * Here's a simple example using 3 shards and 2 groups per shard:STRING
LOW.github/workflows/spark_python_test.yaml98 # when changing TEST_PARALLELISM_COUNT make sure to also change it in spark_test.yamlCOMMENT
LOW.github/workflows/iceberg_test.yaml52 # when changing TEST_PARALLELISM_COUNT make sure to also change it in spark_master_test.yamlCOMMENT
TODO Padding2 hits · 3 pts
SeverityFileLineSnippetContext
LOW…dePlanning/IcebergRESTCatalogPlanningClientSuite.scala132 // TODO: Add test for partitioned table rejectionCOMMENT
LOW…ark/internal/v2/read/changelog/DeltaChangelogScan.java56 // TODO: implement toMicroBatchStream() so spark.readStream...loadChangelog(...) can drive aCOMMENT
Modern AI Meta-Vocabulary1 hit · 3 pts
SeverityFileLineSnippetContext
MEDIUMproject/scripts/setup_unitycatalog_main.sh10# scaffolding to rip out when Delta can use a released UC version again (flipCOMMENT