Skip to content

Streaming output and webhook-based completion when run in workflow#118

Open
pranaygp wants to merge 29 commits intomainfrom
pranay/serde-review
Open

Streaming output and webhook-based completion when run in workflow#118
pranaygp wants to merge 29 commits intomainfrom
pranay/serde-review

Conversation

@pranaygp
Copy link
Copy Markdown
Contributor

@pranaygp pranaygp commented Mar 27, 2026

Summary

Example app and SDK improvements that showcase @vercel/sandbox + Workflow DevKit integration:

  • Webhook-based command completion: Detached runCommand automatically creates a workflow webhook and wraps the command in a shell script that POSTs the exit code when done. cmd.wait() suspends the workflow until the webhook fires — no step is blocked polling, and we're not bound by serverless function timeouts while waiting for long-running sandbox processes.

  • Live streaming stdout/stderr: Sandbox command output is piped to workflow named streams (getWritable), which the client reads via SSE in real-time.

  • writeFiles accepts strings: Buffer isn't available in the workflow VM, so writeFiles now accepts string | Buffer for content, converting inside the step where Node.js APIs are available.

  • runCommand accepts Web WritableStream: In addition to Node.js Writable, enabling direct use of workflow's getWritable() streams.

How it works

export async function runCode(prompt: string) {
  "use workflow";

  const sandbox = await Sandbox.create({ runtime: "node24" });

  // Workflow named streams — client reads via run.getReadable()
  const stdout = getWritable<string>({ namespace: "stdout" });
  const stderr = getWritable<string>({ namespace: "stderr" });

  await sandbox.writeFiles([{ path: "script.js", content: code }]);

  // runCommand automatically creates a workflow webhook in workflow context.
  // The command is wrapped: `node script.js; curl -X POST webhook_url -d '{"exitCode":$?}'`
  const cmd = await sandbox.runCommand({
    cmd: "node",
    args: ["script.js"],
    stdout, // Web WritableStream from workflow — piped inside the step
    stderr,
    detached: true,
  });

  // Workflow suspends here. No step is blocked polling.
  // When the sandbox command finishes, it POSTs to the webhook,
  // and the workflow resumes with the exit code.
  const finished = await cmd.wait();

  await sandbox.stop();
}

SDK changes

Change File
writeFiles accepts string | Buffer content sandbox.ts
runCommand stdout/stderr accept Writable | WritableStream sandbox.ts
runCommand with detached: true auto-creates workflow webhook sandbox.ts
cmd.wait() uses webhook in workflow context, falls back to polling command.ts
Webhook included in WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE command.ts
ensureClient() called before logs() in getCachedOutput command.ts
"workflow" marked as external in tsdown config tsdown.config.ts

Example app

Multi-runtime (Node.js 24/22, Python 3.13, Bash) code runner with:

  • AI code generation + auto-retry on failure (up to 3 attempts)
  • Live streaming output via SSE
  • Split-pane UI: syntax-highlighted code (shiki) + terminal output
  • Real-time status updates (creating sandbox, generating, running, etc.)

Test plan

  • All 116 SDK unit tests pass
  • Build and typecheck pass
  • Run the example app locally and verify workflow completes with streaming output
  • Test with each runtime (Node.js, Python, Bash)
  • Verify webhook-based completion works on Vercel deployment (check workflow observability — should show webhook suspension, not _waitStep)

🤖 Generated with Claude Code

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
sandbox Error Error Apr 2, 2026 6:47pm
sandbox-cli Ready Ready Preview, Comment Apr 2, 2026 6:47pm
sandbox-sdk Ready Ready Preview, Comment Apr 2, 2026 6:47pm
sandbox-sdk-ai-example Ready Ready Preview, Comment Apr 2, 2026 6:47pm
workflow-code-runner Error Error Apr 2, 2026 6:47pm

Request Review

pranaygp and others added 2 commits March 27, 2026 15:50
Sandbox.create, writeFiles, runCommand, and stop all have "use step"
internally, so they don't need to be wrapped in separate step functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Use detached runCommand with stdout/stderr piped to workflow
  writable streams (via getWritable with namespaces)
- Add SSE endpoint to stream stdout/stderr to the browser
  via run.getReadable
- Update UI to render live streaming output as it arrives

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
runCommand now accepts both Node.js Writable and Web WritableStream
for stdout/stderr options. The conversion happens inside the step
(where Node.js APIs are available), so workflow code can pass
Web streams from getWritable() directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a CommandFinished is deserialized across a step boundary, it has
no client. getCachedOutput() now calls ensureClient() before logs()
so that stderr()/stdout() work on deserialized instances.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The transparent webhook approach failed because _webhookWait (a Promise)
can't survive workflow checkpointing between await expressions. The
Command gets serialized/deserialized and the promise is lost.

The webhook must be created and awaited at the workflow level where
the workflow runtime manages its lifecycle across checkpoints. The SDK
provides onCompleteUrl on runCommand to wrap the command in a shell
script that POSTs the exit code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
runCommand (no "use step") runs in workflow context and:
1. Creates a webhook via dynamic import("workflow").createWebhook()
2. Passes the webhook URL to _runCommandStep (the actual step)
3. Injects the webhook object onto the returned Command._webhook

The webhook is a workflow primitive that survives checkpointing via
WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE on Command.

Command.wait() checks _webhook first — if present, awaits the webhook
(suspending the workflow). If absent (non-workflow context), falls
back to _waitStep which polls via a step.

DX is completely transparent:
  const cmd = await sandbox.runCommand({ detached: true, ... });
  const finished = await cmd.wait(); // suspends workflow, no step blocked

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pranaygp and others added 11 commits March 27, 2026 18:41
The Function('return import("workflow")') trick hid the import from
the workflow bundler, so it was never resolved and createWebhook was
undefined at runtime.

Use a regular dynamic import with @ts-expect-error instead. Mark
"workflow" as external in tsdown config to prevent bundling it into
the dist.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The SSE endpoint only handled stdout and stderr, not status.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Show creating-sandbox, writing, and stopping phases in the UI
alongside generating/fixing/running.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- CodeBlock: shiki-powered syntax highlighting with github-dark theme
- Terminal: styled terminal output with ANSI color support, auto-scroll,
  window chrome dots, and title bar

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Code pane (left): persists generated code across status updates,
  shows script.js with attempt count and pass/fail badge
- Terminal pane (right): stdout on top, stderr below, with
  "Waiting for output..." placeholder while running
- Code no longer disappears between phases — stored in dedicated state
- Top bar with prompt, status indicator, and error messages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Pure black backgrounds matching Vercel/geistdocs design
- Line numbers via CSS counters on shiki output
- github-dark-default shiki theme
- Consistent border/muted/foreground CSS variables
- Terminal dots and text use matching muted color

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Runtime dropdown in the UI next to the prompt
- Workflow accepts runtime param, configures sandbox, filename,
  command, and AI prompts accordingly
- Code block uses correct syntax highlighting language
- File tab shows script.js or script.py based on runtime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bash uses a node24 sandbox (bash is available in all runtimes),
writes script.sh, and runs with bash. AI generates shell scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines +606 to +620
// When _onCompleteUrl is set (workflow webhook), wrap the command in a
// shell script that runs the original command then POSTs the exit code.
let cmd = params.cmd;
let cmdArgs = params.args ?? [];
if (params._onCompleteUrl) {
const escaped = [params.cmd, ...cmdArgs]
.map((a) => `'${a.replace(/'/g, "'\\''")}'`)
.join(" ");
cmd = "sh";
cmdArgs = [
"-c",
`${escaped}; EXIT_CODE=$?; curl -s -X POST -H 'Content-Type: application/json' -d "{\\"exitCode\\":$EXIT_CODE}" '${params._onCompleteUrl}'; exit $EXIT_CODE`,
];
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part seem hacky and needs review - not sure how safe this is in production where we're wrapping all sorts of arbitrary user code/programs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
"react-dom": "19.2.3",
"workflow": "4.2.0-beta.73",
"shiki": "^4.0.2",
"workflow": "https://workflow-docs-3fkphgcis.vercel.sh/workflow.tgz",
Copy link
Copy Markdown
Contributor Author

@pranaygp pranaygp Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for demo only - need to switch back to main before publishing. It's not required to work but just testing a perf improvement in workflow

): { write: (data: string) => void } | undefined => {
if (!stream) return undefined;
if ("getWriter" in stream) {
const writer = stream.getWriter();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WritableStream writer is never released after pipeLogs completes, causing TypeError when the same stream is reused across multiple runCommand calls.

Fix on Vercel

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances @vercel/sandbox’s Workflow DevKit integration by enabling webhook-based completion for detached commands and live stdout/stderr streaming into workflow named streams, plus updates the workflow code-runner example to demonstrate multi-runtime execution with real-time UI output.

Changes:

  • SDK: runCommand({ detached: true }) attempts to create a workflow webhook and Command.wait() can suspend on webhook completion; runCommand stdout/stderr now accept Node Writable or Web WritableStream; writeFiles now accepts string | Buffer.
  • SDK: Command serialization/deserialization includes the webhook primitive; getCachedOutput() ensures the client is initialized before streaming logs.
  • Example app: adds SSE streaming for stdout/stderr/status, multi-runtime runner (Node 24/22, Python 3.13, Bash), and a split-pane UI with syntax highlighting + ANSI terminal rendering.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
pnpm-lock.yaml Adds example dependencies (shiki, ansi-to-react) and switches workflow to a tarball URL.
packages/vercel-sandbox/tsdown.config.ts Marks workflow as external to avoid bundling an optional runtime import.
packages/vercel-sandbox/src/sandbox.ts Adds webhook wiring for detached commands, Web WritableStream support for log piping, and writeFiles string support.
packages/vercel-sandbox/src/command.ts Adds webhook field to serialization and webhook-based wait() path; ensures client before logs() usage in cached output.
examples/workflow-code-runner/workflows/code-runner.ts Reworks workflow to use Sandbox directly, add multi-runtime support, named stream piping, and webhook-based detached execution.
examples/workflow-code-runner/steps/status.ts Adds a workflow step to emit structured status updates to a named stream.
examples/workflow-code-runner/steps/sandbox.ts Removes old sandbox helper steps (now handled inline in the workflow).
examples/workflow-code-runner/steps/ai.ts Updates AI steps to generate/fix code for a selected language/runtime.
examples/workflow-code-runner/package.json Adds shiki, ansi-to-react, and switches workflow dependency to a tarball URL.
examples/workflow-code-runner/app/page.tsx Updates UI to multi-runtime input + split panes and consumes SSE streams for live output/status.
examples/workflow-code-runner/app/globals.css Adds theme variables and Shiki line-number styling.
examples/workflow-code-runner/app/components/terminal.tsx Adds ANSI-rendering terminal component with autoscroll.
examples/workflow-code-runner/app/components/code-block.tsx Adds Shiki-based syntax-highlighted code block with fallback rendering.
examples/workflow-code-runner/app/api/run/route.ts Adds SSE streaming endpoints for stdout/stderr/status and keeps polling endpoint for completion.
examples/workflow-code-runner/NOTES.md Documents known limitations for local webhook resumption and preview protection.
examples/workflow-code-runner/.gitignore Expands ignores for local Vercel and env files (one path needs correction).
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +83 to +87
const stdoutSource = new EventSource(
`/api/run?runId=${runId}&stream=stdout`,
);
const stderrSource = new EventSource(
`/api/run?runId=${runId}&stream=stderr`,
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These EventSource connections are only closed on the “completed/failed” poll paths. If polling throws (network/JSON error) or the handler exits early, the SSE connections can remain open and leak. Consider storing the sources in refs and closing them in the catch path and/or via an effect cleanup on unmount.

Copilot uses AI. Check for mistakes.
Comment on lines +549 to +553
if (!stream) return undefined;
if ("getWriter" in stream) {
const writer = stream.getWriter();
return { write: (data: string) => writer.write(data) };
}
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Web WritableStreams, writer.write() is async and can apply backpressure / reject; currently pipeLogs() doesn’t await writes, so failures/backpressure are ignored and may surface as unhandled rejections or memory growth. Consider awaiting Web writes and releasing the writer lock in finally (and/or aborting the stream on error).

Copilot uses AI. Check for mistakes.
/next-env.d.ts
.vercel
.env*.local
examples/workflow-code-runner/public/.well-known/
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gitignore entry likely won’t match because it’s already inside examples/workflow-code-runner/; examples/workflow-code-runner/public/.well-known/ is an extra path prefix. Use a path relative to this directory (e.g. public/.well-known/ or /public/.well-known/) so the well-known folder is actually ignored.

Suggested change
examples/workflow-code-runner/public/.well-known/
/public/.well-known/

Copilot uses AI. Check for mistakes.
Comment on lines 133 to 135
} else {
setTimeout(() => pollResult(runId), 1000);
setTimeout(pollResult, 1000);
}
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setTimeout(pollResult, 1000) isn’t tracked/cleared. If the component unmounts or a new run starts, this can still fire and set state on an unmounted component / race with the next run. Consider storing the timeout id and clearing it in cleanup/unmount.

Copilot uses AI. Check for mistakes.
@@ -1,48 +1,90 @@
import { FatalError } from "workflow";
import { createSandbox, execute, stopSandbox } from "@/steps/sandbox";
import { createWebhook, getWritable } from "workflow";
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import: createWebhook is imported but never referenced in this workflow. If it’s no longer needed now that the SDK handles webhook creation internally, remove it to avoid confusion and keep the example minimal.

Suggested change
import { createWebhook, getWritable } from "workflow";
import { getWritable } from "workflow";

Copilot uses AI. Check for mistakes.
Comment on lines +243 to +247
async wait(params?: { signal?: AbortSignal }): Promise<CommandFinished> {
// In workflow context, the webhook is a workflow primitive that was
// set by runCommand. Awaiting it suspends the workflow without blocking
// a step — the sandbox POSTs the exit code when the command finishes.
if (this._webhook) {
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This introduces a webhook-based completion branch in wait(), but there’s no unit coverage for the webhook path (payload parsing/validation, abort behavior). Adding a small unit test for the _webhook branch would help prevent regressions between workflow and non-workflow contexts.

Copilot uses AI. Check for mistakes.
Comment on lines +610 to +614
if (params._onCompleteUrl) {
const escaped = [params.cmd, ...cmdArgs]
.map((a) => `'${a.replace(/'/g, "'\\''")}'`)
.join(" ");
cmd = "sh";
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _onCompleteUrl path changes the executed command to sh -c ... and relies on the webhook URL being wired through correctly, but there’s no unit test asserting the wrapped command/args. Consider adding a unit test around this branch to lock in the behavior and avoid regressions.

Copilot uses AI. Check for mistakes.
Comment on lines +642 to +643
if (errStream && "emit" in errStream) {
errStream.emit("error", err);
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Detached log piping errors are only forwarded via .emit('error') when the provided stream is a Node Writable. If callers pass a Web WritableStream, errors are silently dropped. Consider detecting Web streams and signaling errors via writer.abort(err) (or another explicit error channel).

Suggested change
if (errStream && "emit" in errStream) {
errStream.emit("error", err);
if (errStream && typeof (errStream as any).emit === "function") {
(errStream as any).emit("error", err);
} else if (errStream && typeof (errStream as any).getWriter === "function") {
// Support Web WritableStream: forward error via writer.abort(err)
try {
const writer = (errStream as any).getWriter();
void writer.abort(err).catch(() => {
// Ignore abort errors to avoid unhandled rejections
});
} catch {
// Ignore failures obtaining writer or aborting
}

Copilot uses AI. Check for mistakes.
Comment on lines +614 to +617
cmd = "sh";
cmdArgs = [
"-c",
`${escaped}; EXIT_CODE=$?; curl -s -X POST -H 'Content-Type: application/json' -d "{\\"exitCode\\":$EXIT_CODE}" '${params._onCompleteUrl}'; exit $EXIT_CODE`,
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Webhook completion wraps the command with sh -c and uses curl to POST the exit code. The sandbox README’s system section doesn’t document curl as available, so this may be brittle across images/runtimes and could leave workflows hanging if curl is missing. Consider using a more guaranteed mechanism (e.g. Node runtime fetch) or documenting/ensuring curl availability.

Suggested change
cmd = "sh";
cmdArgs = [
"-c",
`${escaped}; EXIT_CODE=$?; curl -s -X POST -H 'Content-Type: application/json' -d "{\\"exitCode\\":$EXIT_CODE}" '${params._onCompleteUrl}'; exit $EXIT_CODE`,
const onCompleteUrlEscaped = params._onCompleteUrl.replace(/'/g, "'\\''");
cmd = "sh";
cmdArgs = [
"-c",
`${escaped}; EXIT_CODE=$?; HOOK_URL='${onCompleteUrlEscaped}'; PAYLOAD="{\\"exitCode\\":$EXIT_CODE}"; if command -v curl >/dev/null 2>&1; then curl -s -X POST -H 'Content-Type: application/json' -d "$PAYLOAD" "$HOOK_URL" >/dev/null 2>&1 || true; elif command -v node >/dev/null 2>&1; then node -e "const url=process.argv[2];const body=process.argv[3];(async()=>{try{await fetch(url,{method:'POST',headers:{'Content-Type':'application/json'},body});}catch(e){}})();" "$HOOK_URL" "$PAYLOAD" >/dev/null 2>&1 || true; fi; exit $EXIT_CODE`,

Copilot uses AI. Check for mistakes.
Comment on lines +9 to +10
await writer.write(JSON.stringify({ phase, attempt, code }));
writer.releaseLock();
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateStatus() should release the writer lock even if writer.write() throws/rejects; currently releaseLock() won’t run on errors and the stream can remain permanently locked. Wrap the write in try/finally (and consider reusing a writer if this is called frequently).

Suggested change
await writer.write(JSON.stringify({ phase, attempt, code }));
writer.releaseLock();
try {
await writer.write(JSON.stringify({ phase, attempt, code }));
} finally {
writer.releaseLock();
}

Copilot uses AI. Check for mistakes.
@socket-security
Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​shiki@​4.0.21001007796100
Addednpm/​ansi-to-react@​6.2.61001009989100

View full report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants