Fix failure when reading deep or shallow cloned Delta Lake tables #27098

yangshangqing95 · 2025-10-25T03:44:09Z

Description

For cloned Delta Lake tables (either deep or shallow clones), the checkpoint version may start at 0.
The previous validation in the CheckpointMetadataEntry constructor required the version to be positive,
which caused the following exception:

Query xxxx failed: Unable to parse transaction log entry: {"checkpointMetadata":{"version":0, ....}}

Root cause is:

com.fasterxml.jackson.databind.exc.ValueInstantiationException: 
Cannot construct instance of `io.trino.plugin.deltalake.transactionlog.CheckpointMetadataEntry`,
problem: version is not positive: 0

Additional context and related issues

Fixes #27097

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## Delta Lake
* Fix potential failure when reading [cloned tables](https://docs.databricks.com/aws/en/delta/clone). ({issue}`27098`)

sourcery-ai · 2025-10-25T03:44:15Z

Reviewer's Guide

This PR adjusts the checkpoint metadata validation to accept version 0 on Delta Lake table clones and supplements it with comprehensive unit tests covering valid, zero, and negative version scenarios as well as JSON serialization.

Class diagram for updated CheckpointMetadataEntry validation

classDiagram
    class CheckpointMetadataEntry {
        +long version
        +Optional<Map<String, String>> tags
        +CheckpointMetadataEntry(long version, Optional<Map<String, String>> tags)
    }
    CheckpointMetadataEntry : version >= 0 validation
    CheckpointMetadataEntry : tags are copied as ImmutableMap

Class diagram for new TestCheckpointMetadataEntry unit tests

classDiagram
    class TestCheckpointMetadataEntry {
        +testValidVersion()
        +testZeroVersion()
        +testNegativeVersionThrows()
        +testJsonSerialization()
    }
    TestCheckpointMetadataEntry --> CheckpointMetadataEntry

File-Level Changes

Change	Details	Files
Relax version validation in CheckpointMetadataEntry to accept version 0	Change precondition from version > 0 to version >= 0 Update validation error message to reflect only negative versions	`plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/transactionlog/CheckpointMetadataEntry.java`
Add unit tests covering boundary cases for checkpoint metadata	Test deserialization with version 0 alongside existing positive cases Verify failure on negative version and missing tags Test JSON serialization output formatting	`plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java`

Possibly linked issues

Failure when reading deep or shallow cloned Delta Lake tables #27097: The PR fixes the issue by allowing checkpoint version 0 in Delta Lake tables, resolving the transaction log parsing exception.

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey there - I've reviewed your changes and they look great!

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java:55-64` </location>
<code_context>
+    }
+
+    @Test
+    void testInvalidCheckpointMetadataEntry()
+    {
+        @Language("JSON")
+        String jsonWithNegativeVersion = "{\"version\":-1,\"tags\":{\"sidecarNumActions\":\"1\",\"sidecarSizeInBytes\":\"20965\",\"numOfAddFiles\":\"1\",\"sidecarFileSchema\":\"\"}}";
+        assertThatThrownBy(() -> codec.fromJson(jsonWithNegativeVersion))
+                .isInstanceOf(IllegalArgumentException.class)
+                .hasMessageContaining("Invalid JSON string for");
+
+        @Language("JSON")
+        String jsonWithoutTags = "{\"version\":-1}";
+        assertThatThrownBy(() -> codec.fromJson(jsonWithoutTags))
+                .isInstanceOf(IllegalArgumentException.class)
</code_context>

<issue_to_address>
**suggestion (testing):** Missing test for valid CheckpointMetadataEntry with absent 'tags' field.

Please add a test for deserializing a valid CheckpointMetadataEntry with a non-negative version and no 'tags' field to confirm correct handling of this case.
</issue_to_address>

### Comment 2
<location> `plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java:70-89` </location>
<code_context>
+    }
+
+    @Test
+    void testCheckpointMetadataEntryToJson()
+    {
+        assertThat(codec.toJson(new CheckpointMetadataEntry(
+                100,
+                Optional.of(ImmutableMap.of(
+                        "sidecarNumActions", "1",
+                        "sidecarSizeInBytes", "20965",
+                        "numOfAddFiles", "1",
+                        "sidecarFileSchema", "")))))
+                .isEqualTo("{\n" +
+                        "  \"version\" : 100,\n" +
+                        "  \"tags\" : {\n" +
+                        "    \"sidecarNumActions\" : \"1\",\n" +
+                        "    \"sidecarSizeInBytes\" : \"20965\",\n" +
+                        "    \"numOfAddFiles\" : \"1\",\n" +
+                        "    \"sidecarFileSchema\" : \"\"\n" +
+                        "  }\n" +
+                        "}");
+    }
</code_context>

<issue_to_address>
**suggestion (testing):** Consider adding a test for serialization with version 0 and absent 'tags'.

Please add a test case for serializing a CheckpointMetadataEntry with version 0 and no 'tags', and verify the resulting JSON structure.

```suggestion
    @Test
    void testCheckpointMetadataEntryToJson()
    {
        assertThat(codec.toJson(new CheckpointMetadataEntry(
                100,
                Optional.of(ImmutableMap.of(
                        "sidecarNumActions", "1",
                        "sidecarSizeInBytes", "20965",
                        "numOfAddFiles", "1",
                        "sidecarFileSchema", "")))))
                .isEqualTo("{\n" +
                        "  \"version\" : 100,\n" +
                        "  \"tags\" : {\n" +
                        "    \"sidecarNumActions\" : \"1\",\n" +
                        "    \"sidecarSizeInBytes\" : \"20965\",\n" +
                        "    \"numOfAddFiles\" : \"1\",\n" +
                        "    \"sidecarFileSchema\" : \"\"\n" +
                        "  }\n" +
                        "}");
    }

    @Test
    void testCheckpointMetadataEntryToJsonWithVersionZeroAndNoTags()
    {
        assertThat(codec.toJson(new CheckpointMetadataEntry(
                0,
                Optional.empty())))
                .isEqualTo("{\n" +
                        "  \"version\" : 0\n" +
                        "}");
    }
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java

ebyhr · 2025-10-25T03:55:49Z

Fix failure when reading deep or shallow cloned Delta Lake tables.

https://trino.io/development/process#pull-request-and-commit-guidelines

Do not end the subject line with a period.

yangshangqing95 · 2025-10-26T00:32:41Z

Fix failure when reading deep or shallow cloned Delta Lake tables.

https://trino.io/development/process#pull-request-and-commit-guidelines

Do not end the subject line with a period.

Fixed

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java

chenjian2664 · 2025-10-29T00:23:25Z

Fix failure when reading deep or shallow cloned Delta Lake tables.

https://trino.io/development/process#pull-request-and-commit-guidelines

Do not end the subject line with a period.

reminder

chenjian2664

Looks good

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java

...ources/databricks154/clone_checkpoint_version_zero/checkpoint_v2/deep_cloned_table/README.md

github-actions · 2025-11-21T17:03:22Z

This pull request has gone a while without any activity. Ask for help on #core-dev on Trino slack.

yangshangqing95 · 2025-11-24T06:35:42Z

Hi @chenjian2664 , got above alert, any update on this? thx!

chenjian2664 · 2025-11-24T09:22:38Z

@yangshangqing95 Hi, I think the pr looks good overall. my only concern is the test coverage, we should still cover both deep and shallow clone cases, since they are different scenarios, and adding dedicated tests won't cause much overhead/hurt

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java

yangshangqing95 · 2025-11-25T02:53:38Z

@yangshangqing95 Hi, I think the pr looks good overall. my only concern is the test coverage, we should still cover both deep and shallow clone cases, since they are different scenarios, and adding dedicated tests won't cause much overhead/hurt

sure, I'll update

yangshangqing95 · 2025-11-25T09:47:01Z

@yangshangqing95 Hi, I think the pr looks good overall. my only concern is the test coverage, we should still cover both deep and shallow clone cases, since they are different scenarios, and adding dedicated tests won't cause much overhead/hurt

Hi @chenjian2664, shallow cloned table test case added

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java

ebyhr · 2025-11-26T23:51:36Z

/test-with-secrets sha=4e3f2a99fdbbe8882aa654626b05c2569e3b5b24

github-actions · 2025-11-26T23:52:47Z

The CI workflow run with tests that require additional secrets has been started: https://github.com/trinodb/trino/actions/runs/19720679824

cla-bot bot added the cla-signed label Oct 25, 2025

github-actions bot added the delta-lake Delta Lake connector label Oct 25, 2025

sourcery-ai bot reviewed Oct 25, 2025

View reviewed changes

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java Show resolved Hide resolved

ebyhr reviewed Oct 25, 2025

View reviewed changes

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java Show resolved Hide resolved

yangshangqing95 changed the title ~~Fix failure when reading deep or shallow cloned Delta Lake tables.~~ Fix failure when reading deep or shallow cloned Delta Lake tables Oct 25, 2025

yangshangqing95 force-pushed the fix-read-cloned-delta-lake-table-error branch from 14dd843 to d1593a9 Compare October 26, 2025 00:31

chenjian2664 reviewed Oct 26, 2025

View reviewed changes

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java Outdated Show resolved Hide resolved

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java Outdated Show resolved Hide resolved

yangshangqing95 force-pushed the fix-read-cloned-delta-lake-table-error branch 2 times, most recently from 199baac to 3be6244 Compare October 28, 2025 15:49

chenjian2664 approved these changes Oct 29, 2025

View reviewed changes

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java Outdated Show resolved Hide resolved

plugin/trino-delta-lake/src/test/java/io/trino/plugin/deltalake/TestDeltaLakeBasic.java Outdated Show resolved Hide resolved

yangshangqing95 force-pushed the fix-read-cloned-delta-lake-table-error branch from 3be6244 to 78997c3 Compare October 29, 2025 14:25

chenjian2664 reviewed Oct 31, 2025

View reviewed changes

...ources/databricks154/clone_checkpoint_version_zero/checkpoint_v2/deep_cloned_table/README.md Outdated Show resolved Hide resolved

github-actions bot added the stale label Nov 21, 2025

chenjian2664 reviewed Nov 24, 2025

View reviewed changes

...lake/src/test/java/io/trino/plugin/deltalake/transactionlog/TestCheckpointMetadataEntry.java Outdated Show resolved Hide resolved

github-actions bot removed the stale label Nov 24, 2025

yangshangqing95 force-pushed the fix-read-cloned-delta-lake-table-error branch 2 times, most recently from 32be4a9 to 9027663 Compare November 25, 2025 09:14

chenjian2664 approved these changes Nov 26, 2025

View reviewed changes

ebyhr reviewed Nov 26, 2025

View reviewed changes

Fix failure when reading deep or shallow cloned Delta Lake tables.

4e3f2a9

ebyhr force-pushed the fix-read-cloned-delta-lake-table-error branch from 9027663 to 4e3f2a9 Compare November 26, 2025 23:49

ebyhr merged commit e6b50a0 into trinodb:master Nov 27, 2025
25 checks passed

github-actions bot added this to the 479 milestone Nov 27, 2025

chenjian2664 mentioned this pull request Nov 28, 2025

Add Trino 479 release notes #27153

Open

Fix failure when reading deep or shallow cloned Delta Lake tables #27098

Fix failure when reading deep or shallow cloned Delta Lake tables #27098

Uh oh!

Conversation

yangshangqing95 commented Oct 25, 2025 • edited by ebyhr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Additional context and related issues

Release notes

Uh oh!

sourcery-ai bot commented Oct 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Class diagram for updated CheckpointMetadataEntry validation

Class diagram for new TestCheckpointMetadataEntry unit tests

File-Level Changes

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ebyhr commented Oct 25, 2025

Uh oh!

yangshangqing95 commented Oct 26, 2025

Uh oh!

Uh oh!

Uh oh!

chenjian2664 commented Oct 29, 2025

Uh oh!

chenjian2664 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Nov 21, 2025

Uh oh!

yangshangqing95 commented Nov 24, 2025

Uh oh!

chenjian2664 commented Nov 24, 2025

Uh oh!

Uh oh!

yangshangqing95 commented Nov 25, 2025

Uh oh!

yangshangqing95 commented Nov 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ebyhr commented Nov 26, 2025

Uh oh!

github-actions bot commented Nov 26, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

yangshangqing95 commented Oct 25, 2025 •

edited by ebyhr

Loading

sourcery-ai bot commented Oct 25, 2025 •

edited

Loading