feat(workflow): add late join test workflow with 3000 inserts and verification

[rtb-12]

[Core] Add late-join test workflow for KV store

Description

Adds a workflow that tests late-join sync: Node 1 performs 3000 inserts, then Node 2 joins and verifies all entries are synced and accessible.

The workflow (apps/kv-store/workflows/late-join-test.yml) covers:

• 3000 inserts on Node 1 before Node 2 joins
• Late join of Node 2 after all inserts complete
• Verification that all 3000 entries sync to Node 2
• Assertions that specific keys (first, middle, near-end, last) are accessible on Node 2
• Cross-verification that Node 1 still has all entries after sync

This tests the sync protocol's ability to catch up a late-joining node with a large state history.

On local testing it took around 30 sec for the context hash to match for both nodes.

Test plan

Documentation update

No documentation changes required .

---

Note

Strengthens late-join sync coverage and scalability.

• Adds apps/kv-store/workflows/late-join-test.yml to simulate 5000 pre-join inserts on Node 1, then Node 2 joins and verifies state via len, entries, and specific get checks (with a 100s wait)
• Increases MAX_DELTA_QUERY_LIMIT in crates/dag/src/lib.rs and MAX_DELTA_FETCH_LIMIT in crates/node/src/sync/delta_request.rs from 3000 to 1_000_000 to support large history syncs

^{Written by Cursor Bugbot for commit d1b8022. This will update automatically on new commits. Configure here.}

[github-actions]

This pull request has been automatically marked as stale. If this pull request is still relevant, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize reviewing it yet. Your contribution is very much appreciated.

[cursor]

Excessive limit increase removes OOM protection

High Severity

MAX_DELTA_QUERY_LIMIT and MAX_DELTA_FETCH_LIMIT increased from 3000 to 1,000,000 (333x) when the test only needs ~5000. The original limits were explicitly designed to "prevent resource exhaustion" and "prevent OOM where a peer sends a delta with a deep chain." This dramatic increase removes important memory safety guardrails that protect against malicious or corrupted peers causing out-of-memory conditions during sync operations.

Additional Locations (1)

• crates/node/src/sync/delta_request.rs#L20-L21

 

[github-actions]

SDK JS Workflows Failed

The following SDK JS workflow(s) failed:

• examples/kv-store/workflows/simple-store-js.yml
• examples/private-data/workflows/private-data-js.yml

Please check the workflow logs for more details.

[github-actions]

Merobox Workflows Failed

The following workflow(s) failed after retries:

• kv-store/workflows/late-join-test.yml

Please check the workflow logs for more details.