Allow time travel reads to Hudi Tables #27140

jordepic · 2025-10-29T00:01:35Z

This commit handles simple "FOR VERSION AS OF X" syntax in
the Hudi connector order to ensure that we can read previous
table state.

Previously, all reads to Hudi tables would occur at the latest
commit timestamp. While a user could technically filter down
the data using a predicate on the _hoodie_commit_time column,
there is no functionality that pushes this down into the Hudi
API to minimize file reads.

Timestamps are provided as strings..

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

Added time travel reads in Hudi connector.

cla-bot · 2025-10-29T00:01:38Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

sourcery-ai · 2025-10-29T00:01:41Z

Reviewer's Guide

This PR introduces a new session variable hudi.time_travel_read_timestamp and threads it from the connector session through HudiSplitSource into HudiReadOptimizedDirectoryLister, where listStatus conditionally uses Hudi’s time-travel API (getLatestBaseFilesBeforeOrOn) when a timestamp is supplied. It also consolidates duplicated helper methods and updates the documentation to reflect the new property.

Sequence diagram for time travel read in Hudi connector

sequenceDiagram
  actor User
  participant TrinoSession
  participant HudiSplitSource
  participant HudiReadOptimizedDirectoryLister
  participant HudiFileSystemView

  User->>TrinoSession: Set hudi.time_travel_read_timestamp
  TrinoSession->>HudiSplitSource: getTimeTravelReadTimestamp(session)
  HudiSplitSource->>HudiReadOptimizedDirectoryLister: Pass timeTravelReadTimestamp
  HudiReadOptimizedDirectoryLister->>HudiFileSystemView: getLatestBaseFilesBeforeOrOn(partitionPath, timeTravelReadTimestamp)
  HudiFileSystemView-->>HudiReadOptimizedDirectoryLister: Return base files as of timestamp
  HudiReadOptimizedDirectoryLister-->>HudiSplitSource: Return filtered file statuses
  HudiSplitSource-->>TrinoSession: Return data as of timestamp

Class diagram for updated HudiReadOptimizedDirectoryLister and HudiSessionProperties

classDiagram
  class HudiReadOptimizedDirectoryLister {
    - HoodieTableFileSystemView fileSystemView
    - List<Column> partitionColumns
    - Map<String, HudiPartitionInfo> allPartitionInfoMap
    - String timeTravelReadTimestamp
    + HudiReadOptimizedDirectoryLister(..., String timeTravelReadTimestamp)
    + List<HudiFileStatus> listStatus(HudiPartitionInfo partitionInfo)
    - static StoragePathInfo getStoragePathInfo(HoodieBaseFile baseFile)
    + void close()
  }

  class HudiSessionProperties {
    - static final String TIME_TRAVEL_READ_TIMESTAMP
    + static String getTimeTravelReadTimestamp(ConnectorSession session)
    + List<PropertyMetadata<?>> getSessionProperties()
  }

  HudiSplitSource --> HudiReadOptimizedDirectoryLister : passes timeTravelReadTimestamp
  HudiSessionProperties <.. HudiSplitSource : static method getTimeTravelReadTimestamp
  HudiSessionProperties <.. TrinoSession : session property

Class diagram for HudiSplitSource changes

classDiagram
  class HudiSplitSource {
    + HudiSplitSource(..., String timeTravelReadTimestamp)
    + CompletableFuture<ConnectorSplitBatch> getNextBatch(int maxSize)
    + boolean isFinished()
    - static HudiSplitWeightProvider createSplitWeightProvider(ConnectorSession session)
  }

  HudiSplitSource --> HudiReadOptimizedDirectoryLister : instantiates with timeTravelReadTimestamp

File-Level Changes

Change	Details	Files
Propagate time travel timestamp from session into the Hudi reader	Define TIME_TRAVEL_READ_TIMESTAMP constant and stringProperty in HudiSessionProperties Implement getTimeTravelReadTimestamp accessor in HudiSessionProperties Extend HudiSplitSource to fetch and pass the timestamp into the reader Add a new field and constructor parameter in HudiReadOptimizedDirectoryLister Conditionally call getLatestBaseFilesBeforeOrOn vs getLatestBaseFiles based on timestamp	`plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSessionProperties.java` `plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSplitSource.java` `plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/query/HudiReadOptimizedDirectoryLister.java`
Consolidate duplicated helper methods	Remove duplicate getStoragePathInfo implementation and move a single copy to the top of HudiReadOptimizedDirectoryLister Eliminate redundant createSplitWeightProvider method in HudiSplitSource	`plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/query/HudiReadOptimizedDirectoryLister.java` `plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSplitSource.java`
Update connector documentation with time travel property	Add `hudi.time_travel_read_timestamp` entry and description in hudi.md	`docs/src/main/sphinx/connector/hudi.md`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey there - I've reviewed your changes and they look great!

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSessionProperties.java:150` </location>
<code_context>
-    {
-        return sessionProperties;
+                        false),
+                stringProperty(TIME_TRAVEL_READ_TIMESTAMP, "Read data as of provided timestamp - if empty Trino will read from current snapshot", "", false));
     }

</code_context>

<issue_to_address>
**suggestion:** Clarify expected format for time_travel_read_timestamp property.

Consider adding the required timestamp format to the property description to help users avoid mistakes.

```suggestion
                stringProperty(
                    TIME_TRAVEL_READ_TIMESTAMP,
                    "Read data as of provided timestamp in format 'yyyy-MM-dd HH:mm:ss' - if empty Trino will read from current snapshot",
                    "",
                    false));
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

sourcery-ai · 2025-10-29T00:02:23Z

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSessionProperties.java

-    {
-        return sessionProperties;
+                        false),
+                stringProperty(TIME_TRAVEL_READ_TIMESTAMP, "Read data as of provided timestamp - if empty Trino will read from current snapshot", "", false));


suggestion: Clarify expected format for time_travel_read_timestamp property.

Consider adding the required timestamp format to the property description to help users avoid mistakes.

Suggested change

stringProperty(TIME_TRAVEL_READ_TIMESTAMP, "Read data as of provided timestamp - if empty Trino will read from current snapshot", "", false));

stringProperty(

TIME_TRAVEL_READ_TIMESTAMP,

"Read data as of provided timestamp in format 'yyyy-MM-dd HH:mm:ss' - if empty Trino will read from current snapshot",

"",

false));

docs/src/main/sphinx/connector/hudi.md

cla-bot · 2025-10-30T00:54:14Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2025-10-30T00:57:31Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

docs/src/main/sphinx/connector/hudi.md

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiSessionProperties.java

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiMetadata.java

cla-bot · 2025-10-30T23:50:36Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

jordepic · 2025-10-30T23:51:40Z

@ebyhr I signed my CLA but nobody has reviewed it in a week. Would it be possible to get anybody to take a look?

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java

cla-bot · 2025-10-31T00:59:17Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2025-10-31T14:45:54Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

ebyhr · 2025-11-06T03:42:09Z

@cla-bot check

cla-bot · 2025-11-06T03:42:12Z

The cla-bot has been summoned, and re-checked this pull request!

docs/src/main/sphinx/connector/hudi.md

...in/trino-hudi/src/main/java/io/trino/plugin/hudi/query/HudiReadOptimizedDirectoryLister.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiTableHandle.java

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/testing/TpchHudiTablesInitializer.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

...in/trino-hudi/src/main/java/io/trino/plugin/hudi/query/HudiReadOptimizedDirectoryLister.java

voonhous

Logic looks good to me, no logic changes, just some stylistic suggestions.

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiTableHandle.java

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java

This commit handles simple "FOR VERSION AS OF X" syntax in the Hudi connector order to ensure that we can read previous table state. Previously, all reads to Hudi tables would occur at the latest commit timestamp. While a user could technically filter down the data using a predicate on the _hoodie_commit_time column, there is no functionality that pushes this down into the Hudi API to minimize file reads. Timestamps are provided as strings. Allow time travel reads to Hudi Tables This commit handles simple "FOR VERSION AS OF X" syntax in the Hudi connector order to ensure that we can read previous table state. Previously, all reads to Hudi tables would occur at the latest commit timestamp. While a user could technically filter down the data using a predicate on the _hoodie_commit_time column, there is no functionality that pushes this down into the Hudi API to minimize file reads. Timestamps are provided as strings.

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

chenjian2664 · 2025-11-19T10:24:31Z

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

+
+        ConnectorTableVersion version = endVersion.get();
+        if (version.getPointerType() == PointerType.TEMPORAL) {
+            throw new TrinoException(NOT_SUPPORTED, "Cannot read 'TIMESTAMP' of Hudi table, use 'VERSION' instead");


Suggested change

throw new TrinoException(NOT_SUPPORTED, "Cannot read 'TIMESTAMP' of Hudi table, use 'VERSION' instead");

throw new TrinoException(NOT_SUPPORTED, "This connector does not support reading tables with TIMESTAMP AS OF");

chenjian2664 · 2025-11-19T10:25:19Z

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java

+
+        ConnectorTableVersion version = endVersion.get();
+        if (version.getPointerType() == PointerType.TEMPORAL) {
+            throw new TrinoException(NOT_SUPPORTED, "Cannot read 'TIMESTAMP' of Hudi table, use 'VERSION' instead");


Also, is this limitation(reading with temporal) coming from Hudi, or is it something that could be supported in the future?
If it's the latter, could you file an issue and add a TODO here with a link to it?

chenjian2664 · 2025-11-19T10:31:52Z

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java

+        assertThat(query("SELECT CAST(nationkey AS INT) FROM hudi.tests.nation")).matches(expectedValues);
+        assertThat(query("SELECT CAST(nationkey AS INT) FROM hudi.tests.nation FOR VERSION AS OF '" + COMMIT_TIMESTAMP + "'")).matches(expectedValues);
+        assertThat(query("SELECT CAST(nationkey AS INT) FROM hudi.tests.nation FOR VERSION AS OF '" + (COMMIT_TIMESTAMP - 1) + "'")).returnsEmptyResult();
+        assertQueryFails("SELECT CAST(nationkey AS INT) FROM hudi.tests.nation FOR VERSION AS OF 0", "Provided read version must be a string");


Could you also add a case that is negative number/string?

chenjian2664 · 2025-11-19T10:34:56Z

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java

 public class TestHudiConnectorTest
        extends BaseConnectorTest
 {
+    private static final long COMMIT_TIMESTAMP = 20251027183851494L;


Is the naming aligned with the Hudi glossary? using "commit timestamp" sounds more like a temporal read, it actually more like a "commit version"

github-actions bot added docs hudi Hudi connector labels Oct 29, 2025

sourcery-ai bot reviewed Oct 29, 2025

View reviewed changes

ebyhr reviewed Oct 29, 2025

View reviewed changes

docs/src/main/sphinx/connector/hudi.md Outdated Show resolved Hide resolved

jordepic force-pushed the trino-hudi-snapshot branch from ef12d42 to 6746772 Compare October 30, 2025 00:54

jordepic force-pushed the trino-hudi-snapshot branch from 6746772 to 5102fa4 Compare October 30, 2025 00:57

ebyhr reviewed Oct 30, 2025

View reviewed changes

jordepic force-pushed the trino-hudi-snapshot branch from 5102fa4 to 3471a5a Compare October 30, 2025 23:50

ebyhr reviewed Oct 30, 2025

View reviewed changes

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java Show resolved Hide resolved

jordepic force-pushed the trino-hudi-snapshot branch from 3471a5a to 63c448a Compare October 31, 2025 00:59

jordepic force-pushed the trino-hudi-snapshot branch from 63c448a to 7c6ca6c Compare October 31, 2025 14:45

cla-bot bot added the cla-signed label Nov 6, 2025

ebyhr reviewed Nov 6, 2025

View reviewed changes

chenjian2664 reviewed Nov 6, 2025

View reviewed changes

jordepic force-pushed the trino-hudi-snapshot branch 3 times, most recently from e6c3331 to 7c9fd62 Compare November 7, 2025 03:11

voonhous reviewed Nov 10, 2025

View reviewed changes

jordepic force-pushed the trino-hudi-snapshot branch from 7c9fd62 to 22db806 Compare November 11, 2025 00:37

ebyhr reviewed Nov 11, 2025

View reviewed changes

plugin/trino-hudi/src/main/java/io/trino/plugin/hudi/HudiMetadata.java Outdated Show resolved Hide resolved

jordepic force-pushed the trino-hudi-snapshot branch from 22db806 to b62d294 Compare November 11, 2025 00:43

ebyhr reviewed Nov 11, 2025

View reviewed changes

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java Outdated Show resolved Hide resolved

plugin/trino-hudi/src/test/java/io/trino/plugin/hudi/TestHudiConnectorTest.java Outdated Show resolved Hide resolved

jordepic force-pushed the trino-hudi-snapshot branch from 0e3d9a0 to c558142 Compare November 12, 2025 00:08

ebyhr force-pushed the trino-hudi-snapshot branch from c558142 to a51096a Compare November 13, 2025 02:12

jordepic requested a review from ebyhr November 15, 2025 01:43

ebyhr force-pushed the trino-hudi-snapshot branch from f095231 to 7ad8eb0 Compare November 15, 2025 23:28

chenjian2664 reviewed Nov 19, 2025

View reviewed changes

	throw new TrinoException(NOT_SUPPORTED, "Cannot read 'TIMESTAMP' of Hudi table, use 'VERSION' instead");
	throw new TrinoException(NOT_SUPPORTED, "This connector does not support reading tables with TIMESTAMP AS OF");

Allow time travel reads to Hudi Tables #27140

Are you sure you want to change the base?

Allow time travel reads to Hudi Tables #27140

Uh oh!

Conversation

jordepic commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release notes

Uh oh!

cla-bot bot commented Oct 29, 2025

Uh oh!

sourcery-ai bot commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for time travel read in Hudi connector

Class diagram for updated HudiReadOptimizedDirectoryLister and HudiSessionProperties

Class diagram for HudiSplitSource changes

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

sourcery-ai bot Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cla-bot bot commented Oct 30, 2025

Uh oh!

cla-bot bot commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cla-bot bot commented Oct 30, 2025

Uh oh!

jordepic commented Oct 30, 2025

Uh oh!

Uh oh!

cla-bot bot commented Oct 31, 2025

Uh oh!

cla-bot bot commented Oct 31, 2025

Uh oh!

ebyhr commented Nov 6, 2025

Uh oh!

cla-bot bot commented Nov 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

voonhous left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chenjian2664 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

chenjian2664 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

chenjian2664 Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

chenjian2664 Nov 19, 2025

jordepic commented Oct 29, 2025 •

edited

Loading

sourcery-ai bot commented Oct 29, 2025 •

edited

Loading