|
| 1 | +# Integration Tests for Reconnection Behavior |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +This PR implements a comprehensive integration test infrastructure for testing ComfyUI client reconnection behavior by spawning **real mock server processes**, then killing and restarting them to simulate actual server crashes and recoveries. |
| 6 | + |
| 7 | +## Problem |
| 8 | + |
| 9 | +The previous reconnection tests used mocked WebSocket connections, which cannot accurately simulate: |
| 10 | +- Real server process crashes and restarts |
| 11 | +- Actual network disconnections and timing issues |
| 12 | +- Connection state transitions during real network operations |
| 13 | +- Multiple reconnection cycles with real retry logic |
| 14 | + |
| 15 | +## Solution |
| 16 | + |
| 17 | +Created a complete integration test infrastructure with: |
| 18 | + |
| 19 | +### 1. Mock Server (`test/integration/mock-server.ts`) |
| 20 | +- Standalone ComfyUI-compatible server running in separate process |
| 21 | +- HTTP endpoints: `/queue`, `/prompt`, `/system_stats`, `/history` |
| 22 | +- WebSocket server at `/ws` with periodic status messages |
| 23 | +- Graceful shutdown via SIGTERM/SIGINT |
| 24 | +- IPC messaging to signal readiness |
| 25 | +- Can be run standalone: `bun test/integration/mock-server.ts [port]` |
| 26 | + |
| 27 | +### 2. Server Manager (`test/integration/server-manager.ts`) |
| 28 | +- Process lifecycle management (spawn, kill, restart) |
| 29 | +- `startServer(port)` - Spawns server and waits for ready signal |
| 30 | +- `killServer(port)` - Gracefully terminates server |
| 31 | +- `restartServer(port, delay)` - Kill and restart with optional delay |
| 32 | +- `killAll()` - Cleanup all servers |
| 33 | +- Configurable timeouts and error handling |
| 34 | + |
| 35 | +### 3. Test Helpers (`test/integration/test-helpers.ts`) |
| 36 | +- `initializeClient(api)` - Initialize and wait for ready |
| 37 | +- `waitForConnection(api)` - Poll until connected |
| 38 | +- `waitForDisconnection(api)` - Poll until disconnected |
| 39 | +- `pollUntil(condition)` - Generic polling utility |
| 40 | +- `trackEvent(api, event)` - Track event firing |
| 41 | +- `sleep(ms)` - Simple delay |
| 42 | + |
| 43 | +### 4. Integration Tests (`reconnection.integration.spec.ts`) |
| 44 | +10 comprehensive test cases covering: |
| 45 | +- ✅ Manual reconnection after server restart |
| 46 | +- ✅ Multiple reconnection attempts |
| 47 | +- ✅ Automatic reconnection with `autoReconnect: true` |
| 48 | +- ✅ `reconnected` event emission |
| 49 | +- ✅ Connection state transitions (connecting → connected → disconnected → reconnecting → connected) |
| 50 | +- ✅ Connection validation (`validateConnection()`) |
| 51 | +- ✅ Reconnection failure handling |
| 52 | +- ✅ Multiple server restart cycles |
| 53 | +- ✅ WebSocket message handling after reconnection |
| 54 | + |
| 55 | +### 5. Simple Examples (`reconnection-simple.example.spec.ts`) |
| 56 | +3 well-documented, step-by-step examples: |
| 57 | +- Basic reconnection flow |
| 58 | +- Auto-reconnection example |
| 59 | +- Multiple restart cycles |
| 60 | + |
| 61 | +### 6. Validation Script (`validate-mock-server.ts`) |
| 62 | +Standalone script verifying entire infrastructure works correctly |
| 63 | + |
| 64 | +### 7. Documentation |
| 65 | +- `README.md` - Comprehensive guide (276 lines) |
| 66 | +- `SUMMARY.md` - Architecture overview (253 lines) |
| 67 | +- `QUICKSTART.md` - Developer quick-start guide (405 lines) |
| 68 | + |
| 69 | +## How It Works |
| 70 | + |
| 71 | +```typescript |
| 72 | +// 1. Start mock server in separate process |
| 73 | +await serverManager.startServer(8189); |
| 74 | + |
| 75 | +// 2. Create and initialize client |
| 76 | +const api = new ComfyApi("http://localhost:8189"); |
| 77 | +await initializeClient(api); |
| 78 | + |
| 79 | +// 3. Kill server (simulates crash) |
| 80 | +await serverManager.killServer(8189); |
| 81 | +await sleep(500); |
| 82 | + |
| 83 | +// 4. Restart server (simulates recovery) |
| 84 | +await serverManager.startServer(8189); |
| 85 | + |
| 86 | +// 5. Verify reconnection |
| 87 | +await api.reconnectWs(true); |
| 88 | +await waitForConnection(api); |
| 89 | + |
| 90 | +expect(api.isConnected()).toBe(true); |
| 91 | + |
| 92 | +// 6. Cleanup |
| 93 | +api.destroy(); |
| 94 | +await serverManager.killAll(); |
| 95 | +``` |
| 96 | + |
| 97 | +## Test Results |
| 98 | + |
| 99 | +All 13 integration tests passing: |
| 100 | + |
| 101 | +``` |
| 102 | +✓ Manual Reconnection > successfully reconnects after server restart |
| 103 | +✓ Manual Reconnection > handles multiple reconnection attempts |
| 104 | +✓ Automatic Reconnection > automatically reconnects when autoReconnect is enabled |
| 105 | +✓ Automatic Reconnection > emits reconnected event on successful auto-reconnection |
| 106 | +✓ Connection State Transitions > tracks connection state through server lifecycle |
| 107 | +✓ Connection Validation > validateConnection returns true when server is up |
| 108 | +✓ Connection Validation > validateConnection returns false when server is down |
| 109 | +✓ Reconnection Failure Handling > invokes onReconnectionFailed callback |
| 110 | +✓ Multiple Server Restarts > handles multiple server restarts gracefully |
| 111 | +✓ WebSocket Message Handling > receives messages correctly after reconnection |
| 112 | +✓ Simple Example > basic reconnection flow |
| 113 | +✓ Simple Example > auto-reconnection with server restart |
| 114 | +✓ Simple Example > multiple server restarts |
| 115 | +
|
| 116 | +13 pass | 0 fail | 37 expect() calls |
| 117 | +``` |
| 118 | + |
| 119 | +## Files Added |
| 120 | + |
| 121 | +``` |
| 122 | +test/integration/ |
| 123 | +├── README.md # Full documentation (276 lines) |
| 124 | +├── SUMMARY.md # Architecture summary (253 lines) |
| 125 | +├── QUICKSTART.md # Developer quick-start (405 lines) |
| 126 | +├── mock-server.ts # Standalone server (262 lines) |
| 127 | +├── server-manager.ts # Process management (289 lines) |
| 128 | +├── test-helpers.ts # Test utilities (177 lines) |
| 129 | +├── validate-mock-server.ts # Validation script (154 lines) |
| 130 | +├── reconnection.integration.spec.ts # Comprehensive tests (375 lines) |
| 131 | +└── reconnection-simple.example.spec.ts # Simple examples (158 lines) |
| 132 | +
|
| 133 | +Total: 9 files, ~2,350 lines |
| 134 | +``` |
| 135 | + |
| 136 | +## NPM Scripts Added |
| 137 | + |
| 138 | +```json |
| 139 | +{ |
| 140 | + "test:integration": "bun test ./test/integration/", |
| 141 | + "test:integration:simple": "bun test ./test/integration/reconnection-simple.example.spec.ts", |
| 142 | + "test:integration:reconnection": "bun test ./test/integration/reconnection.integration.spec.ts" |
| 143 | +} |
| 144 | +``` |
| 145 | + |
| 146 | +## Running Tests |
| 147 | + |
| 148 | +```bash |
| 149 | +# All integration tests |
| 150 | +bun test test/integration/ |
| 151 | + |
| 152 | +# Simple examples (recommended first) |
| 153 | +bun run test:integration:simple |
| 154 | + |
| 155 | +# Comprehensive tests |
| 156 | +bun run test:integration:reconnection |
| 157 | + |
| 158 | +# Validate infrastructure |
| 159 | +bun test/integration/validate-mock-server.ts |
| 160 | +``` |
| 161 | + |
| 162 | +## Key Features |
| 163 | + |
| 164 | +### Real Process Isolation |
| 165 | +- Mock servers run in separate OS processes |
| 166 | +- Can be killed/restarted without affecting test process |
| 167 | +- Simulates production scenarios accurately |
| 168 | + |
| 169 | +### Comprehensive Testing |
| 170 | +- Tests manual and automatic reconnection |
| 171 | +- Verifies connection state machine |
| 172 | +- Confirms event emission |
| 173 | +- Validates message handling across reconnections |
| 174 | +- Tests multiple restart cycles |
| 175 | + |
| 176 | +### Developer-Friendly |
| 177 | +- Well-documented with 3 comprehensive guides |
| 178 | +- Helper functions reduce code duplication |
| 179 | +- Simple examples as templates |
| 180 | +- Standalone validation script |
| 181 | + |
| 182 | +### Production-Ready |
| 183 | +- CI/CD compatible |
| 184 | +- Proper resource cleanup (no orphaned processes) |
| 185 | +- Timeout handling for reliability |
| 186 | +- Detailed logging for debugging |
| 187 | +- Uses unique ports to avoid conflicts |
| 188 | + |
| 189 | +## Benefits Over Unit Tests |
| 190 | + |
| 191 | +1. **Real Process Lifecycle** - Tests actual process spawning, killing, restarting |
| 192 | +2. **True Network Behavior** - Real WebSocket connections, not mocks |
| 193 | +3. **Timing Accuracy** - Tests actual connection timing and retry logic |
| 194 | +4. **State Management** - Verifies state transitions match real-world scenarios |
| 195 | +5. **Error Handling** - Tests actual error conditions and recovery |
| 196 | + |
| 197 | +## Breaking Changes |
| 198 | + |
| 199 | +None - this is purely additive. |
| 200 | + |
| 201 | +## Future Enhancements |
| 202 | + |
| 203 | +- Test connection pooling under server restarts |
| 204 | +- Test workflow execution across reconnections |
| 205 | +- Add network latency/packet loss simulation |
| 206 | +- Multi-client concurrent scenarios |
| 207 | +- Load testing with many clients |
| 208 | + |
| 209 | +## Checklist |
| 210 | + |
| 211 | +- [x] Tests implemented and passing (13/13) |
| 212 | +- [x] Documentation complete (3 comprehensive guides) |
| 213 | +- [x] Validation script working |
| 214 | +- [x] NPM scripts added |
| 215 | +- [x] No breaking changes |
| 216 | +- [x] CI/CD compatible |
| 217 | +- [x] Proper cleanup implemented |
| 218 | + |
| 219 | +## Related Issues |
| 220 | + |
| 221 | +Addresses the requirement for proper reconnection testing with real server process lifecycle simulation. |
0 commit comments