Message streaming over WebSocket for large payloads

[vrknetha]

Problem

Currently, messages are delivered as single frames over the WebSocket connection. This works for small payloads but breaks down with:

• Large file transfers (documents, images, model weights)
• Embedding vectors
• Batch data payloads
• Any payload approaching the frame size limit (relayMaxFrameBytes)

Messages that exceed the frame limit are rejected. There's no way to send large data between agents.

Proposal

Implement chunked streaming over the existing WebSocket connection. No new infrastructure needed — the WebSocket is already there, bidirectional, and persistent.

Frame Protocol

Add a streaming frame type alongside existing deliver and heartbeat:

stream_start  → { streamId, requestId, totalBytes, totalChunks, metadata }
stream_chunk  → { streamId, chunkIndex, data (base64 or binary), checksum }
stream_end    → { streamId, finalChecksum }
stream_abort  → { streamId, reason }
stream_ack    → { streamId, chunkIndex }  (flow control)

Flow

Sender's connector → Sender's proxy DO → WebSocket → Recipient's connector

1. Sender sends stream_start (metadata: size, type, sender DID)
2. Proxy DO stores stream metadata, forwards to recipient's WebSocket
3. Sender sends stream_chunk (N chunks, flow-controlled by stream_ack)
4. Recipient's connector reassembles chunks into the inbox
5. Sender sends stream_end with final checksum
6. Recipient verifies checksum → enqueues complete message for replay to OpenClaw
7. Receipt sent back on completion

Flow Control

• Recipient sends stream_ack per chunk (or per N chunks as a window)
• Sender pauses if no ack received within timeout
• Prevents fast sender from overwhelming slow recipient
• DO can enforce per-stream memory limits

Failure Handling

• stream_abort from either side cancels the stream
• If WebSocket drops mid-stream: recipient has partial chunks in temp storage
• On reconnect: sender can resume from last ack'd chunk (if stream hasn't expired)
• Stream TTL: incomplete streams are cleaned up after configurable timeout
• Checksum mismatch on stream_end: recipient sends stream_abort, sender can retry

DO Considerations

• DO doesn't need to buffer the entire payload — it forwards chunks as they arrive
• If recipient is offline: chunks queue in DO storage (same pattern as regular messages)
• Per-stream memory limit on DO to prevent abuse
• Concurrent stream limit per agent pair

Why Not HTTP Multipart?

The WebSocket is already established, authenticated, and bidirectional. HTTP would mean:

• New auth flow per upload
• No flow control (stream_ack)
• Can't resume on disconnect
• Separate infrastructure from the message path

Backward Compatibility

• Old connectors that don't understand stream_* frames ignore them (existing frame parsing skips unknown types)
• Protocol version negotiation (#future) would handle this cleanly
• Fallback: reject large messages with clear error suggesting connector upgrade

Implementation Phases

1. Phase 1: Basic chunking — split large messages into chunks, reassemble on receipt. No flow control, no resume.
2. Phase 2: Flow control — stream_ack window, backpressure
3. Phase 3: Resume on disconnect — checkpoint from last ack'd chunk

Related

• Frame size limit: relayMaxFrameBytes in proxy config
• DO queue: RELAY_QUEUE_MAX_MESSAGES_PER_AGENT — streaming chunks should count as one logical message, not N

[vrknetha]

Offline Connector Handling

If the recipient's connector is offline, storing all stream chunks in the DO is problematic:

• DO storage is expensive for large payloads
• Memory limits on DOs
• 100MB file as queued chunks = bad

Recommended approach (phased):

Phase 1 (v1): Reject streams when offline

• stream_start arrives → DO checks active WebSocket connections
• No connection → reject with RECIPIENT_OFFLINE error
• Sender decides: retry later, send a URL reference instead, or alert human
• Small regular messages (<frame limit) still queue as they do today
• Clear contract: streaming = real-time only

Phase 2 (future): R2 spillover for offline recipients

• stream_start arrives → no WebSocket → DO creates R2 upload
• Chunks written to R2 object storage instead of DO
• DO stores only a lightweight stream_reference (R2 key, metadata, checksum)
• On reconnect: DO sends reference frame → connector downloads from R2
• R2 free tier: 10GB storage, 10M reads/month
• TTL on R2 objects (auto-cleanup after 24h)

Phase 3 (future): Hybrid threshold

• Below 1MB total: queue in DO (treat as regular message)
• Above 1MB: R2 spillover or reject

This keeps the DO thin and avoids turning it into a file storage system.

[vrknetha]

Correction: Connector never touches R2

Same principle as Queues — connector only talks to its own proxy. R2 is internal to the proxy/DO layer.

Updated R2 spillover flow:

Sender → stream chunks → Recipient's proxy DO
  → No WebSocket connected
  → DO writes chunks to R2 (internal, connector doesn't know)
  → DO stores lightweight reference (R2 key, metadata, total size, checksum)

...later...

Connector reconnects → WebSocket established
  → DO reads chunks from R2
  → DO streams them to connector via WebSocket (normal stream_chunk frames)
  → Connector sees regular stream frames — no idea data came from R2
  → After successful delivery, DO deletes R2 object

Connector's interface is unchanged. It only sees WebSocket frames. R2 is the DO's internal spillover — an implementation detail of the proxy layer.