[Claimed #1913] Feat: Add Zoom tool to AnthropicCuaClient

[github-actions]

Mirrored from external contributor PR #1913 after approval by @miguelg719.

Original author: @chromiebot
Original PR: #1913
Approved source head SHA: fac3ca1f7204243475a47fce084d15e2b7083241

@chromiebot, please continue any follow-up discussion on this mirrored PR. When the external PR gets new commits, this same internal PR will be marked stale until the latest external commit is approved and refreshed here.

Original description

why

what changed

test plan

---

Summary by cubic

Adds Zoom tool support to AnthropicCUAClient and wires it into V3CuaAgentHandler so Claude can request high‑res crops of specific screen regions. Improves inspection accuracy without extra navigation.

• New Features
  • Enables enable_zoom for computer_20251124 models; converts zoom tool_use into an action with [x1,y1,x2,y2].
  • Adds setZoomedScreenshotProvider and captureZoomedScreenshot; falls back to full screenshot when unset.
  • Handler provides a zoomed screenshot via CDP clip and treats zoom as a no‑op action.
  • Adds unit tests for tool definition, action conversion, crop capture, and handler wiring.

^{Written for commit fac3ca1. Summary will update on new commits. Review in cubic}

[github-actions]

This mirrored PR tracks external contributor PR #1913 at source SHA fac3ca1f7204243475a47fce084d15e2b7083241, approved by @miguelg719.
Original PR: #1913

When the external PR gets new commits, this same internal PR will be refreshed in place after the latest external commit is approved.

[changeset-bot]

⚠️ No Changeset found

Latest commit: fac3ca1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

1 issue found across 4 files

Confidence score: 4/5

• This PR is likely safe to merge: the only reported issue is a test coverage gap (severity 5/10) rather than a confirmed runtime defect.
• In packages/core/tests/unit/cua-handler-zoom.test.ts, the test currently verifies method existence but skips the Anthropic-specific setZoomedScreenshotProvider path, so zoom wiring regressions could slip through CI.
• Pay close attention to packages/core/tests/unit/cua-handler-zoom.test.ts - strengthen assertions around the Anthropic zoom-provider path to catch real zoom behavior regressions.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/tests/unit/cua-handler-zoom.test.ts">

<violation number="1" location="packages/core/tests/unit/cua-handler-zoom.test.ts:169">
P2: This test is not validating the new zoom behavior: it only checks method existence and bypasses the handler’s Anthropic-specific `setZoomedScreenshotProvider` path, so regressions in zoom wiring can pass undetected.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED]</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Claude as Claude (Anthropic API)
    participant Client as AnthropicCUAClient
    participant Handler as V3CuaAgentHandler
    participant Browser as Browser (Playwright)

    Note over Client,Handler: Initialization
    Handler->>Client: NEW: setZoomedScreenshotProvider(callback)
    
    Note over Claude,Browser: Request Cycle
    Client->>Claude: getAction() with NEW: enable_zoom: true
    Claude-->>Client: tool_use: computer (action: "zoom", region: [x1, y1, x2, y2])
    
    Client->>Client: NEW: convertToolUseToAction()
    
    Client->>Handler: actionHandler({ type: "zoom" })
    Note right of Handler: CHANGED: Handled as no-op success
    Handler-->>Client: { success: true }

    Client->>Client: NEW: captureZoomedScreenshot(region)
    
    alt NEW: zoomedScreenshotProvider is set
        Client->>Handler: Invoke provider callback(region)
        Handler->>Browser: NEW: page.screenshot({ clip: { x, y, width, height } })
        Browser-->>Handler: Cropped Screenshot Buffer
        Handler-->>Client: base64 string
    else Fallback
        Client->>Handler: screenshotProvider()
        Handler->>Browser: page.screenshot({ fullPage: false })
        Browser-->>Handler: Full Screenshot Buffer
        Handler-->>Client: base64 string
    end

    Client->>Claude: tool_result (contains zoomed image content)

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.}

[cubic-dev-ai]

P2: This test is not validating the new zoom behavior: it only checks method existence and bypasses the handler’s Anthropic-specific setZoomedScreenshotProvider path, so regressions in zoom wiring can pass undetected.

(Based on your team's feedback about adding unit tests for new behavior.)

View Feedback

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/tests/unit/cua-handler-zoom.test.ts, line 169:

<comment>This test is not validating the new zoom behavior: it only checks method existence and bypasses the handler’s Anthropic-specific `setZoomedScreenshotProvider` path, so regressions in zoom wiring can pass undetected.

(Based on your team's feedback about adding unit tests for new behavior.) </comment>

<file context>
@@ -0,0 +1,257 @@
+      createHandler();
+
+      // Since our mock won't match instanceof, let's verify the method exists
+      expect(typeof fakeCuaClient.setZoomedScreenshotProvider).toBe("function");
+    });
+  });
</file context>