Skip to content

test: add seeded message delivery shuffling to QBFT tests#906

Open
diegomrsantos wants to merge 1 commit intosigp:unstablefrom
diegomrsantos:feat/seeded-qbft-tests
Open

test: add seeded message delivery shuffling to QBFT tests#906
diegomrsantos wants to merge 1 commit intosigp:unstablefrom
diegomrsantos:feat/seeded-qbft-tests

Conversation

@diegomrsantos
Copy link
Member

Problem, Evidence, and Context (Required)

  • QBFT integration tests deliver messages to operators in a fixed order (1, 2, 3, ..., N), meaning they never explore what happens when messages arrive at different operators in different orders.
  • Real networks don't deliver messages simultaneously or in a fixed order — message timing and arrival order vary per topology and load.
  • Inspired by commonware-runtime's deterministic simulation approach: same seed = same interleaving = reproducible failures.

Change Overview (Required)

  • QbftTester gains an optional seeded RNG that shuffles operator delivery order in process_network_message. When a seed is provided, each message broadcast delivers to operators in a different (but reproducible) shuffled order.
  • TestContext::new_seeded() threads the seed through for ergonomic test construction.
  • Three new multi-seed tests exercise basic consensus, f-faulty scenarios across committee sizes, and concurrent instances — each across multiple seeds.
  • Existing tests are unchanged (pass None for seed, preserving fixed-order delivery).

Risks, Trade-offs, and Mitigations (Required)

  • Test-only change, zero production code modified.
  • Seeded shuffling is deterministic: if a seed fails in CI, re-running with that seed reproduces the exact same behavior locally.
  • Existing tests preserve their original behavior.

Validation (Required)

  • All 19 tests pass (16 existing + 3 new seeded tests).
  • make cargo-fmt-check, make lint clean.
  • New seeded tests cover: 10 seeds × basic consensus, 5 seeds × 4 committee sizes with faults, 10 seeds × concurrent instances.

Rollback (Required for behavior or runtime changes; optional otherwise)

N/A — test-only change, safe to revert.

Additional Info / Next Steps (Optional)

  • This is the first step toward commonware-style deterministic testing. Future work could add clock abstraction to the QBFT orchestration layer, enabling full deterministic simulation of timing and scheduling interleavings.

Explore different network interleavings by shuffling operator delivery
order with a seeded RNG. Same seed = same ordering = reproducible.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant