Skip to content

feat: catch-up for read-only ProviderFactorys#23357

Open
klkvr wants to merge 5 commits intomainfrom
klkvr/catch-up
Open

feat: catch-up for read-only ProviderFactorys#23357
klkvr wants to merge 5 commits intomainfrom
klkvr/catch-up

Conversation

@klkvr
Copy link
Copy Markdown
Member

@klkvr klkvr commented Apr 3, 2026

Implements catch-up logic for read-only ProviderFactory.

It is implemented like the following:

/// For read-only factories, checks whether the MDBX committed txn ID has advanced since the
/// last sync and, if so, catches up the RocksDB secondary instance and re-initializes the
/// static file index.
///
/// The fast path (txnid unchanged) is lock-free. Only the actual sync I/O is serialized.
///
/// No-op for read-write factories.
pub fn sync_providers_if_needed(&self) -> ProviderResult<()> {
let Some(sync_state) = &self.read_only_sync else { return Ok(()) };
let current_txnid = self.db.last_txnid().unwrap_or(0);
// Fast path: no contention when nothing changed.
if current_txnid == sync_state.last_synced_txnid.load(Ordering::Relaxed) {
return Ok(());
}
// Slow path: serialize the actual catch-up I/O.
let _guard = sync_state.sync_lock.lock().unwrap_or_else(|e| e.into_inner());
// Double-check after acquiring the lock — another thread may have already synced.
if current_txnid == sync_state.last_synced_txnid.load(Ordering::Relaxed) {
return Ok(());
}
self.rocksdb_provider.try_catch_up_with_primary()?;
self.static_file_provider.initialize_index()?;
sync_state.last_synced_txnid.store(current_txnid, Ordering::Relaxed);
Ok(())
}

This function is invoked in 2 cases:

  1. For every created DatabaseProvider or before every read directly from static files (those bypass mdbx entirely)
  2. Every time when mdbx dir changes — this part is identical to static files watcher we have right now and is kept to make sure that read-only factories don't fall too far behind when unused, causing potentially long catch up to happen on next access

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

⚠️ Changelog not found.

A changelog entry is required before merging. We've generated a suggested changelog based on your changes:

Preview
---
reth-provider: minor
reth-chain-state: minor
reth-engine-tree: patch
reth-db: patch
reth-cli-commands: patch
reth-e2e-test-utils: patch
---

Moved read-only sync (watch) behavior from `StaticFileProvider` into `ProviderFactory` via a new `with_read_only_sync` method, simplifying `StaticFileProvider::read_only` to drop its `watch` bool parameter and renaming `ReadOnlyConfig::watch_static_files` to `watch`. Added `with_state` support to `TestBlockBuilder` for generating proper `BundleState` with account/storage changesets and reverts, and added a comprehensive `test_read_only_consistency_across_reorg` test validating snapshot isolation across chain reorgs.

Add changelog to commit this to your branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant