Skip to content

Conversation

@clairemcginty
Copy link
Contributor

@clairemcginty clairemcginty commented Nov 11, 2025

This PR contains a workaround for element timestamp validation that has been added to beam and which affects AsyncDoFn that was suggested by beam authors.

  // Set to arbitrarily high value; required to preserve timestamp of original element
  // See: https://github.com/apache/beam/pull/34902#discussion_r2527777237
  // See: https://github.com/apache/beam/pull/36838#issuecomment-3558736022
  @Override
  public @UnknownKeyFor @NonNull @Initialized Duration getAllowedTimestampSkew() {
    return Duration.standardDays(30);
  }

About this PR

📦 Updates

  • org.apache.beam:beam-examples-java
  • org.apache.beam:beam-examples-java-cdap
  • org.apache.beam:beam-examples-java-cdap-hubspot
  • org.apache.beam:beam-examples-java-cdap-salesforce
  • org.apache.beam:beam-examples-java-cdap-servicenow
  • org.apache.beam:beam-examples-java-cdap-zendesk
  • org.apache.beam:beam-examples-java-iceberg
  • org.apache.beam:beam-examples-java-sql
  • org.apache.beam:beam-examples-java-twitter
  • org.apache.beam:beam-examples-java-webapis
  • org.apache.beam:beam-examples-kotlin
  • org.apache.beam:beam-examples-multi-language
  • org.apache.beam:beam-it
  • org.apache.beam:beam-it-cassandra
  • org.apache.beam:beam-it-clickhouse
  • org.apache.beam:beam-it-common
  • org.apache.beam:beam-it-conditions
  • org.apache.beam:beam-it-elasticsearch
  • org.apache.beam:beam-it-google-cloud-platform
  • org.apache.beam:beam-it-jdbc
  • org.apache.beam:beam-it-kafka
  • org.apache.beam:beam-it-mongodb
  • org.apache.beam:beam-it-neo4j
  • org.apache.beam:beam-it-splunk
  • org.apache.beam:beam-it-testcontainers
  • org.apache.beam:beam-it-truthmatchers
  • org.apache.beam:beam-model-fn-execution
  • org.apache.beam:beam-model-job-management
  • org.apache.beam:beam-model-pipeline
  • org.apache.beam:beam-runners-core-java
  • org.apache.beam:beam-runners-direct-java
  • org.apache.beam:beam-runners-extensions-java-metrics
  • org.apache.beam:beam-runners-flink-1.17
  • org.apache.beam:beam-runners-flink-1.17-job-server
  • org.apache.beam:beam-runners-flink-1.18
  • org.apache.beam:beam-runners-flink-1.18-job-server
  • org.apache.beam:beam-runners-flink-1.19
  • org.apache.beam:beam-runners-flink-1.19-job-server
  • org.apache.beam:beam-runners-google-cloud-dataflow-java
  • org.apache.beam:beam-runners-google-cloud-dataflow-java-legacy-worker
  • org.apache.beam:beam-runners-java-fn-execution
  • org.apache.beam:beam-runners-java-job-service
  • org.apache.beam:beam-runners-jet
  • org.apache.beam:beam-runners-local-java-core
  • org.apache.beam:beam-runners-portability-java
  • org.apache.beam:beam-runners-prism-java
  • org.apache.beam:beam-runners-samza
  • org.apache.beam:beam-runners-samza-job-server
  • org.apache.beam:beam-runners-spark-3
  • org.apache.beam:beam-runners-spark-3-job-server
  • org.apache.beam:beam-runners-twister2
  • org.apache.beam:beam-sdks-java-core
  • org.apache.beam:beam-sdks-java-expansion-service
  • org.apache.beam:beam-sdks-java-expansion-service-app
  • org.apache.beam:beam-sdks-java-extensions-arrow
  • org.apache.beam:beam-sdks-java-extensions-avro
  • org.apache.beam:beam-sdks-java-extensions-avro-vendored-test
  • org.apache.beam:beam-sdks-java-extensions-combiners
  • org.apache.beam:beam-sdks-java-extensions-euphoria
  • org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core
  • org.apache.beam:beam-sdks-java-extensions-join-library
  • org.apache.beam:beam-sdks-java-extensions-json-jackson
  • org.apache.beam:beam-sdks-java-extensions-kryo
  • org.apache.beam:beam-sdks-java-extensions-ml
  • org.apache.beam:beam-sdks-java-extensions-ordered
  • org.apache.beam:beam-sdks-java-extensions-protobuf
  • org.apache.beam:beam-sdks-java-extensions-python
  • org.apache.beam:beam-sdks-java-extensions-sbe
  • org.apache.beam:beam-sdks-java-extensions-schemaio-expansion-service
  • org.apache.beam:beam-sdks-java-extensions-sketching
  • org.apache.beam:beam-sdks-java-extensions-sorter
  • org.apache.beam:beam-sdks-java-extensions-sql
  • org.apache.beam:beam-sdks-java-extensions-sql-datacatalog
  • org.apache.beam:beam-sdks-java-extensions-sql-expansion-service
  • org.apache.beam:beam-sdks-java-extensions-sql-hcatalog
  • org.apache.beam:beam-sdks-java-extensions-sql-iceberg
  • org.apache.beam:beam-sdks-java-extensions-sql-jdbc
  • org.apache.beam:beam-sdks-java-extensions-sql-udf
  • org.apache.beam:beam-sdks-java-extensions-timeseries
  • org.apache.beam:beam-sdks-java-extensions-yaml
  • org.apache.beam:beam-sdks-java-extensions-zetasketch
  • org.apache.beam:beam-sdks-java-harness
  • org.apache.beam:beam-sdks-java-io-amazon-web-services2
  • org.apache.beam:beam-sdks-java-io-amazon-web-services2-expansion-service
  • org.apache.beam:beam-sdks-java-io-amqp
  • org.apache.beam:beam-sdks-java-io-azure
  • org.apache.beam:beam-sdks-java-io-azure-cosmos
  • org.apache.beam:beam-sdks-java-io-cassandra
  • org.apache.beam:beam-sdks-java-io-cdap
  • org.apache.beam:beam-sdks-java-io-clickhouse
  • org.apache.beam:beam-sdks-java-io-common
  • org.apache.beam:beam-sdks-java-io-contextualtextio
  • org.apache.beam:beam-sdks-java-io-csv
  • org.apache.beam:beam-sdks-java-io-debezium
  • org.apache.beam:beam-sdks-java-io-debezium-expansion-service
  • org.apache.beam:beam-sdks-java-io-elasticsearch
  • org.apache.beam:beam-sdks-java-io-expansion-service
  • org.apache.beam:beam-sdks-java-io-file-schema-transform
  • org.apache.beam:beam-sdks-java-io-google-ads
  • org.apache.beam:beam-sdks-java-io-google-cloud-platform
  • org.apache.beam:beam-sdks-java-io-google-cloud-platform-expansion-service
  • org.apache.beam:beam-sdks-java-io-hadoop-common
  • org.apache.beam:beam-sdks-java-io-hadoop-file-system
  • org.apache.beam:beam-sdks-java-io-hadoop-format
  • org.apache.beam:beam-sdks-java-io-hbase
  • org.apache.beam:beam-sdks-java-io-hcatalog
  • org.apache.beam:beam-sdks-java-io-iceberg
  • org.apache.beam:beam-sdks-java-io-influxdb
  • org.apache.beam:beam-sdks-java-io-jdbc
  • org.apache.beam:beam-sdks-java-io-jms
  • org.apache.beam:beam-sdks-java-io-json
  • org.apache.beam:beam-sdks-java-io-kafka
  • org.apache.beam:beam-sdks-java-io-kafka-upgrade
  • org.apache.beam:beam-sdks-java-io-kudu
  • org.apache.beam:beam-sdks-java-io-mongodb
  • org.apache.beam:beam-sdks-java-io-mqtt
  • org.apache.beam:beam-sdks-java-io-neo4j
  • org.apache.beam:beam-sdks-java-io-parquet
  • org.apache.beam:beam-sdks-java-io-pulsar
  • org.apache.beam:beam-sdks-java-io-rabbitmq
  • org.apache.beam:beam-sdks-java-io-redis
  • org.apache.beam:beam-sdks-java-io-rrio
  • org.apache.beam:beam-sdks-java-io-singlestore
  • org.apache.beam:beam-sdks-java-io-snowflake
  • org.apache.beam:beam-sdks-java-io-snowflake-expansion-service
  • org.apache.beam:beam-sdks-java-io-solace
  • org.apache.beam:beam-sdks-java-io-solr
  • org.apache.beam:beam-sdks-java-io-sparkreceiver-3
  • org.apache.beam:beam-sdks-java-io-splunk
  • org.apache.beam:beam-sdks-java-io-synthetic
  • org.apache.beam:beam-sdks-java-io-thrift
  • org.apache.beam:beam-sdks-java-io-tika
  • org.apache.beam:beam-sdks-java-io-xml
  • org.apache.beam:beam-sdks-java-managed
  • org.apache.beam:beam-sdks-java-maven-archetypes-examples
  • org.apache.beam:beam-sdks-java-maven-archetypes-gcp-bom-examples
  • org.apache.beam:beam-sdks-java-maven-archetypes-starter
  • org.apache.beam:beam-sdks-java-nexmark
  • org.apache.beam:beam-sdks-java-test-utils
  • org.apache.beam:beam-sdks-java-testing-junit
  • org.apache.beam:beam-sdks-java-tpcds
  • org.apache.beam:beam-sdks-java-transform-service
  • org.apache.beam:beam-sdks-java-transform-service-app
  • org.apache.beam:beam-sdks-java-transform-service-launcher

from 2.68.0 to 2.69.0

Usage

Please merge!

I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.

If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.

Configure Scala Steward for your repository with a .scala-steward.conf file.

Have a fantastic day writing Scala!

🔍 Files still referring to the old version number

The following files still refer to the old version number (2.68.0).
You might want to review and update them manually.

build.sbt
⚙ Adjust future updates

Add this to your .scala-steward.conf file to ignore future updates of this dependency:

updates.ignore = [ { groupId = "org.apache.beam" } ]

Or, add this to slow down future updates of this dependency:

dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.apache.beam" }
}]
labels: library-update, early-semver-minor, semver-spec-minor, old-version-remains, commit-count:1

@clairemcginty clairemcginty force-pushed the update/beam-examples-java-2.69.0 branch from 62a6830 to 761f4d4 Compare November 12, 2025 16:06
@clairemcginty clairemcginty requested a review from kellen November 12, 2025 16:06
@codecov
Copy link

codecov bot commented Nov 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.52%. Comparing base (88294b9) to head (c583a48).
⚠️ Report is 24 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5806      +/-   ##
==========================================
+ Coverage   61.49%   61.52%   +0.03%     
==========================================
  Files         317      317              
  Lines       11650    11662      +12     
  Branches      818      850      +32     
==========================================
+ Hits         7164     7175      +11     
- Misses       4486     4487       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@clairemcginty clairemcginty force-pushed the update/beam-examples-java-2.69.0 branch 2 times, most recently from 25eb6a8 to 79d68e2 Compare November 24, 2025 20:04
@shnapz
Copy link
Contributor

shnapz commented Nov 26, 2025

Do we want to merge it into 0.14 or 0.15?

@clairemcginty clairemcginty force-pushed the update/beam-examples-java-2.69.0 branch from 79d68e2 to 8b2868c Compare December 11, 2025 20:21
@clairemcginty clairemcginty force-pushed the update/beam-examples-java-2.69.0 branch from 8b2868c to 7bf10da Compare December 11, 2025 21:09
@kellen
Copy link
Contributor

kellen commented Jan 7, 2026

Not super happy about this workaround but we don't really have another option at present

@clairemcginty clairemcginty merged commit 883cc9e into main Jan 7, 2026
12 checks passed
@clairemcginty clairemcginty deleted the update/beam-examples-java-2.69.0 branch January 7, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants