feat: support cross-origin iframe snapshots and interactions via Target.setAutoAttach#949
Merged
ctate merged 4 commits intovercel-labs:mainfrom Mar 20, 2026
Conversation
Contributor
|
@hyunjinee is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
jin-2-kakaoent
force-pushed
the
feat/cross-origin-iframe-snapshot
branch
3 times, most recently
from
9a74ccb to
8a6d04c
Compare
jin-2-kakaoent
changed the title
…et.setAutoAttach (vercel-labs#925) Enable Target.setAutoAttach with flatten: true on page sessions so Chrome auto-creates dedicated CDP sessions for cross-origin iframe targets. - Add DrainedEvents struct, iframe_sessions map, attach/detach event handling - Extract resolve_ax_session (shared) and resolve_frame_session helpers - resolve_element_object_id returns (object_id, effective_session) tuple - resolve_element_center returns (x, y, effective_session) tuple - Input dispatch (click/hover/tap) uses effective session for correct coordinates - Thread iframe_sessions through element.rs, interaction.rs, screenshot.rs - Clear iframe sessions on navigate, tab switch, tab new, tab close - Ignore Target.setAutoAttach failure for non-Chrome backends (Lightpanda) - Unit tests for session resolution logic
jin-2-kakaoent
force-pushed
the
feat/cross-origin-iframe-snapshot
branch
from
8da4487 to
0d91ef7
Compare
jin-2-kakaoent
force-pushed
the
feat/cross-origin-iframe-snapshot
branch
from
bd65a1c to
cca30a3
Compare
Collaborator
|
So good. Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #925.
agent-browsercannot interact with cross-origin iframes —snapshotfails withCDP error (Accessibility.getFullAXTree): Frame with the given frameId is not found. This is because cross-origin iframes exist behind a security boundary that prevents the parent page's CDP session from accessing them viaframeId.This PR enables
Target.setAutoAttachwithflatten: trueon page sessions so Chrome auto-creates dedicated CDP sessions for cross-origin iframe targets. These sessions are tracked and used for accessibility tree queries, element resolution, and interactions, bypassing the security boundary.Reproduction
Snapshot
Before (main):
After (this PR):
Interaction (click, hover)
Before (main): Cannot access elements inside cross-origin iframes — only the iframe element itself is visible as
[ref=e2].After (this PR):
$ agent-browser frame @e2 ✓ Done $ agent-browser click @e2 # clicks "Learn more" link inside iframe ✓ Done $ agent-browser hover @e2 ✓ DoneTest plan
snapshot -i -coutputframe @ref+snapshotworks for cross-origin iframes (previously:Frame with the given frameId is not found)frame main+snapshotreturns to parent pageclickon cross-origin iframe element workshoveron cross-origin iframe element worksresolve_ax_sessionandresolve_frame_sessioncargo test,cargo fmt,cargo clippyall pass