Feature/protocol workflow by hperezrodal · Pull Request #267 · FairgateLabs/rust-bitvmx-client

hperezrodal · 2026-02-03T14:05:05Z

Protocol Workflow Feature

Overview

This PR introduces a major refactoring of the protocol setup and execution workflow in BitVMX, introducing a new SetupEngine orchestration system and ProgramV2 implementation that provides better separation of concerns, improved testability, and more robust protocol initialization.

Key Features

1. SetupEngine - Protocol Setup Orchestration

Introduces a new SetupEngine that orchestrates multi-step protocol setup processes:

Step-based setup: Protocols define their setup steps via ProtocolHandler::setup_steps()
State management: Tracks step completion state across participants
Leader broadcast pattern: Non-leaders send data to leader, leader broadcasts to all
Automatic progression: Advances to next step when all participants complete current step
Persistent state: Setup state is saved and restored across program loads

2. ProgramV2 - Next Generation Program Implementation

New ProgramV2 implementation that:

Uses SetupEngine for orchestrating setup steps
Delegates aggregation responsibility to protocols themselves
Provides cleaner separation of concerns
Implements a cleaner state machine (New → SettingUpV2 → Monitoring → Ready)
Removes protocol-specific setup logic from Program layer

Key differences from Program (Legacy):

No prepare_aggregated_keys() - protocols handle their own aggregation
Uses SetupEngine to orchestrate setup steps
Cleaner state transitions
Protocol-specific setup logic lives in the protocol itself

3. AggregatedKeyProtocol

Replaces the old Collaboration flow with a new AggregatedKeyProtocol:

Unified approach: Single protocol for aggregated key generation (MuSig2)
Single participant support: Handles both single and multi-participant scenarios naturally
Storage in globals: Final aggregated key stored in globals under final_aggregated_key
No special cases: Removed single-participant bypass logic

Breaking Changes

Removed Features

Collaboration flow: The old Collaboration class and related methods have been removed
- process_collaboration_message()
- process_collaboration()
- get_collaboration()
- save_collaboration()
- mark_collaboration_as_complete()
src/collaborate.rs: File removed

Migration Path

Existing programs using the legacy Program implementation continue to work
New programs should use setup_v2() API to leverage ProgramV2
Aggregated key generation now uses AggregatedKeyProtocol instead of Collaboration

Technical Details

SetupEngine Architecture

SetupEngine
├── Steps: Defined by ProtocolHandler::setup_steps()
├── State: Pending → ReadyToSend → WaitingForParticipants → Complete
├── Data Storage: Stored in globals with step-specific keys
└── Progression: Automatic when all participants complete current step

ProgramV2 State Machine

New
  ↓
SettingUpV2 (SetupEngine orchestrates setup steps)
  ↓
Monitoring (Blockchain monitoring setup)
  ↓
Ready (Protocol ready for execution)

Leader Broadcast Flow

Participant 1 (Non-leader) ──┐
                             ├──> Leader ──> Broadcast ──> All Participants
Participant 2 (Non-leader) ──┘

Testing

New Tests

test_aggregated_key: Multi-participant aggregated key generation test
test_aggregated_key_single_participant: Single-participant scenario test

Test Improvements

Improved Docker container cleanup to prevent conflicts between sequential test runs
Better error handling in test infrastructure
Tests validate that all participants compute the same aggregated key

Files Changed

New Files

src/program/setup/setup_engine.rs - SetupEngine implementation
src/program/setup/setup_step.rs - Setup step definitions
src/program/program_v2.rs - ProgramV2 implementation
src/program/protocols/aggregated_key.rs - AggregatedKeyProtocol
tests/aggregated_key.rs - Aggregated key tests

Modified Files

src/bitvmx.rs - Updated to support ProgramV2, removed Collaboration flow
src/types.rs - Updated message types, added ProgramVersion enum
src/api.rs - Added setup_v2() method
src/client.rs - Updated API calls to match new signatures
src/main.rs - Updated tick() return type handling
tests/common/mod.rs - Improved Docker container cleanup

Removed Files

src/collaborate.rs - Replaced by AggregatedKeyProtocol

Performance Considerations

Reduced message overhead: Leader broadcast pattern reduces total messages
Better state management: SetupEngine prevents duplicate state saves
Cleaner code paths: Removed special-case handling improves maintainability

Backward Compatibility

Legacy Program implementation remains available
Existing programs continue to work unchanged
New programs can opt into ProgramV2 via setup_v2() API
Both implementations can coexist in the same BitVMX instance

Future Work

Migrate more protocols to use SetupEngine
Deprecate legacy Program implementation
Add more comprehensive tests for edge cases

Checklist

Code compiles without errors
All tests pass (when run sequentially)
Documentation updated where necessary
Breaking changes documented
Migration path provided
Backward compatibility maintained

Related Issues

This PR addresses the need for:

Better protocol setup orchestration
Cleaner separation of concerns
More robust multi-participant coordination
Improved testability of protocol setup flows

Add a new setup module providing a framework for managing multi-party protocol setup steps. Includes SetupStep trait for step lifecycle management, StepPhase enum for state tracking, ExchangeConfig for exchange configuration, and template implementations for keys, nonces, and signatures steps

Add a new setup orchestration system with SetupEngine that provides a state machine-based approach to managing multi-party protocol setup steps. This enables cleaner separation of concerns and allows protocols to opt-in gradually to the new system.

This is the next generation Program implementation that: - Uses SetupEngine for orchestrating setup steps - Delegates aggregation responsibility to protocols - Provides cleaner separation of concerns - Allows protocols to opt-in gradually via UsesSetupSteps trait

…-leaders -> leader -> all non-leaders)

… leader broadcast

…ndler Remove the UsesSetupSteps trait and integrate setup steps functionality directly into the ProtocolHandler trait. This simplifies the architecture by eliminating an extra trait layer

- Removed use_broadcasting from ExchangeConfig struct and Default impl - Updated exchange_config() in KeysStep, NoncesStep, and SignaturesStep

Rename the collaboration protocol to aggregated key protocol to better reflect its purpose of generating aggregated MuSig2 keys.

Rename the template_steps directory to steps for better clarity and consistency.

Add default implementations for optional protocol methods in the ProtocolHandler trait to reduce boilerplate code in simple protocols like AggregatedKeyProtocol.

Replace manual impl Default implementations with #[derive(Default)] for KeysStep, NoncesStep, and SignaturesStep. These are unit structs where the manual implementation was redundant, as it only called new().

… implementation.

…on.rs structure - Remove unused UTXO funding section that was not being used in the test - Remove unused wallet funding initialization - Replace manual bitcoin/bitcoind setup with prepare_bitcoin() helper - Add bitcoind cleanup at test end (matching integration.rs pattern) - Clean up unused imports (Bitcoind, BitcoinClient, Wallet, etc.) - Remove unused MIN_TX_FEE constant

…_steps() - Replace Option<Vec<Box<dyn SetupStep>>> with Option<Vec<SetupStepName>> in ProtocolHandler::setup_steps() - Add SetupStepName enum (Keys, Nonces, Signatures) for type-safe step names - Implement factory function create_setup_step() to create steps from enum - Update SetupEngine::new() to accept Vec<SetupStepName> instead of Vec<Box<dyn SetupStep>> - Remove trait object allocations at protocol definition time - Update all protocol implementations (AggregatedKeyProtocol, default implementation) - Update tests to use enum variants instead of string literals

Move all setup-related logic from ProgramV2 to SetupEngine to improve separation of concerns. ProgramV2 should only care about whether setup is complete or not, not the internal details of setup steps.

…control - Remove duplicate save() call in BitVMX after receive_setup_data() to prevent state overwrites that caused tick() to re-enter build() and send duplicate messages. ProgramV2 now handles saving internally. - Write state to legacy key in ProgramV2::save() so is_active_program() can correctly identify Ready programs and skip them in process_programs(). - Add send_setup_completed() trait method to ProtocolHandler to allow protocols to suppress SetupCompleted messages. AggregatedKeyProtocol returns false to maintain backward compatibility with SetupKey callers that only expect AggregatedPubkey responses. - Clean up aggregated_key test by removing unused helper struct and setup completion assertions that are no longer relevant.

- Remove special-case handling for single-participant protocols in ProgramV2::setup() This allows SetupEngine to handle all cases uniformly through the normal setup flow - Add single-participant handling in AggregatedKeyProtocol::build_protocol() When only one participant exists, use their own key directly instead of attempting MuSig2 aggregation (which requires at least 2 participants) - Add test_aggregated_key_single_participant() test Verifies that single-participant protocols work correctly through SetupEngine without any special-case bypass logic

- Remove deprecated `collaborate` module and all collaboration processing/storage paths - Read aggregated pubkey/keypair derivation from globals (`final_aggregated_key`) - Update aggregated-key single participant test to use `SetupKey` and assert globals match API response

- Remove confirmation_threshold parameter from SubscribeToTransaction and SubscribeToRskPegin API messages - Simplify tick() return type from Result<bool> to Result<()> - Remove unused GracefulShutdown trait implementation - Update imports: use PubKeyHash from bitvmx_broker instead of bitvmx_operator_comms - Add None parameters to TypesToMonitor calls for consistency - Improve test infrastructure: - Add existence checks before removing directories - Add Docker container cleanup logic to prevent conflicts - Update shutdown policy tests to match new tick() signature - Code formatting and cleanup (remove unused imports, improve formatting)

Resolved conflict in tests/common/mod.rs by combining imports: - Added Path from std::path (needed for check_bitvmx_cpu_built) - Kept Command from std::process (needed for Docker cleanup) - Added warn to tracing imports (used in helper functions)

jonasmartin

comments on aggregated_key

src/program/protocols/aggregated_key.rs

jonasmartin · 2026-02-03T19:27:02Z

src/program/protocols/aggregated_key.rs

+        let aggregated_key = if aggregated_pub_keys.len() == 1 {
+            tracing::info!("AggregatedKeyProtocol: Single participant, using own key directly");
+            *my_key
+        } else {


this case is not valid. just resturn error if there is not enough participants

Fixed: Removed the invalid single-participant case and added validation to require at least 2 participants. The protocol now returns an error if there are not enough participants for MuSig2 aggregation.

src/program/protocols/aggregated_key.rs

jonasmartin · 2026-02-03T19:58:42Z

src/program/setup/exchange_config.rs

Not sure if exchange_config is needed, but if it's, remove the custom implementations from keys_steps, nonces_step and signatures_step so the base impl in the steupStep trait is used

jonasmartin · 2026-02-03T20:23:27Z

src/program/setup/steps/nonces_step.rs

At some point on this step something similar to this needs to be done:
https://github.com/FairgateLabs/rust-bitvmx-client/blob/main/src/program/program.rs#L520-L525

Replace manual match-based trait delegation with enum_dispatch macro, consistent with the existing ProtocolHandler/ProtocolType pattern.

jonasmartin · 2026-02-21T22:50:14Z

src/bitvmx.rs

-        participants_keys: Option<Vec<PublicKey>>,
+        _participants_keys: Option<Vec<PublicKey>>,


There was a usecase used by union where the public keys where provided externally instead of generated, we need to support that case

partially solved

…ol-workflow

…ust-bitvmx-client into feature/protocol-workflow

jonasmartin

LGTM

hperezrodal added 24 commits February 3, 2026 10:11

feat: add bitvmx api setup_v2

7c7b2e3

feat: add collaboration_v2.rs test for aggregated MuSig2 key generation

e28d6bc

refactor: replace direct broadcast with leader broadcast pattern (non…

77c7289

…-leaders -> leader -> all non-leaders)

test(collaboration_v2): update test to use 3 participants to validate…

221a60e

… leader broadcast

refactor: remove UsesSetupSteps trait, move setup_steps to ProtocolHa…

63e9fca

…ndler Remove the UsesSetupSteps trait and integrate setup steps functionality directly into the ProtocolHandler trait. This simplifies the architecture by eliminating an extra trait layer

refactor: remove unused use_broadcasting from ExchangeConfig

e64e3c8

- Removed use_broadcasting from ExchangeConfig struct and Default impl - Updated exchange_config() in KeysStep, NoncesStep, and SignaturesStep

refactor: rename collaboration protocol to aggregated key protocol

8ea0f72

Rename the collaboration protocol to aggregated key protocol to better reflect its purpose of generating aggregated MuSig2 keys.

refactor: rename template_steps module to steps

0b9dbae

Rename the template_steps directory to steps for better clarity and consistency.

feat: add default implementations to ProtocolHandler trait

916f2ae

Add default implementations for optional protocol methods in the ProtocolHandler trait to reduce boilerplate code in simple protocols like AggregatedKeyProtocol.

refactor: remove unnecessary Send + Sync bounds from SetupStep trait

f46b79c

refactor: replace manual Default impl with derive for step structs

3665d98

Replace manual impl Default implementations with #[derive(Default)] for KeysStep, NoncesStep, and SignaturesStep. These are unit structs where the manual implementation was redundant, as it only called new().

fix: update the save() method documentation to accurately reflect its…

c7541ee

… implementation.

refactor: encapsulate setup logic in SetupEngine

b07b6a3

Move all setup-related logic from ProgramV2 to SetupEngine to improve separation of concerns. ProgramV2 should only care about whether setup is complete or not, not the internal details of setup steps.

Remove COMMIT_MSG.txt that was accidentally committed

1e27abc

hperezrodal marked this pull request as ready for review February 3, 2026 17:30

hperezrodal requested a review from jonasmartin February 3, 2026 17:45

hperezrodal force-pushed the feature/protocol-workflow branch from 99aa681 to 1e27abc Compare February 3, 2026 17:58

jonasmartin reviewed Feb 3, 2026

View reviewed changes

hperezrodal and others added 2 commits February 20, 2026 16:53

refactor: use #[enum_dispatch] for SetupStep trait on SetupStepEnum

f300b25

Replace manual match-based trait delegation with enum_dispatch macro, consistent with the existing ProtocolHandler/ProtocolType pattern.

remove dev test protocol

eaf5676

jonasmartin reviewed Feb 21, 2026

View reviewed changes

jonasmartin added 25 commits February 21, 2026 19:51

remove unnecesary use of from

b4652d9

unused

a77b6d6

cargo fmt

0ebdac1

rename

4b36a95

remove no longer needed states

3ab1afe

remove usage of keys

28ebaf2

remove participant data

91e9552

attempt to restore optional key for aggregation

f0b70d1

simplify

6595016

remove dyn

096ed46

simplify steps managment

8d5b906

remove redundant state

f7740f9

simplify engine states

6c45012

simplify state engine

81edba1

Merge remote-tracking branch 'remotes/origin/dev' into feature/protoc…

bc8ee8a

…ol-workflow

fix

c19451b

improve efficiency

57f87d1

attempt to avoid wasted ticks

5885474

Merge branch 'feature/protocol-workflow' of github.com:FairgateLabs/r…

66bd536

…ust-bitvmx-client into feature/protocol-workflow

improve debug

d14cab1

keystep support for only pregenerated keys

4dc4e0d

edge case tick

2b0e5cf

fix repeated message

8e4e1ea

use defaults

c992138

avoid sending back to origin messages in broadcast

46df167

jonasmartin approved these changes Feb 26, 2026

View reviewed changes

jonasmartin merged commit ebc46e6 into dev Feb 26, 2026
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/protocol workflow#267

Feature/protocol workflow#267
jonasmartin merged 75 commits intodevfrom
feature/protocol-workflow

hperezrodal commented Feb 3, 2026

Uh oh!

jonasmartin left a comment

Uh oh!

Uh oh!

Uh oh!

jonasmartin Feb 3, 2026

Uh oh!

hperezrodal Feb 5, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jonasmartin Feb 3, 2026

Uh oh!

jonasmartin Feb 3, 2026

Uh oh!

jonasmartin Feb 21, 2026

Uh oh!

jonasmartin Feb 23, 2026

Uh oh!

jonasmartin left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		participants_keys: Option<Vec<PublicKey>>,
		_participants_keys: Option<Vec<PublicKey>>,

Conversation

hperezrodal commented Feb 3, 2026

Protocol Workflow Feature

Overview

Key Features

1. SetupEngine - Protocol Setup Orchestration

2. ProgramV2 - Next Generation Program Implementation

3. AggregatedKeyProtocol

Breaking Changes

Removed Features

Migration Path

Technical Details

SetupEngine Architecture

ProgramV2 State Machine

Leader Broadcast Flow

Testing

New Tests

Test Improvements

Files Changed

New Files

Modified Files

Removed Files

Performance Considerations

Backward Compatibility

Future Work

Checklist

Related Issues

Uh oh!

jonasmartin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jonasmartin Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

hperezrodal Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jonasmartin Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

jonasmartin Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

jonasmartin Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

jonasmartin Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

jonasmartin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants