Skip to content

Commit a0fbd82

Browse files
pavelfeldmanpavelfeldman
authored
chore: roll playwright to latest (#1869)
1 parent ace82db commit a0fbd82

File tree

7 files changed

+327
-7
lines changed

7 files changed

+327
-7
lines changed

‎nodejs/docs/api/class-testconfig.mdx‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -405,7 +405,7 @@ export default defineConfig({
405405

406406
<font size="2" style={{position: "relative", top: "-20px"}}>Added in: v1.10</font><x-search>testConfig.metadata</x-search>
407407

408-
Metadata contains key-value pairs to be included in the report. For example, HTML report will display it as key-value pairs, and JSON report will include metadata serialized as json.
408+
Metadata contains key-value pairs to be included in the report. For example, the JSON report will include metadata serialized as JSON.
409409

410410
**Usage**
411411

@@ -923,7 +923,7 @@ export default defineConfig({
923923

924924
Whether to update expected snapshots with the actual results produced by the test run. Defaults to `'missing'`.
925925
* `'all'` - All tests that are executed will update snapshots.
926-
* `'changed'` - All tests that are executed will update snapshots that did not match. Matching snapshots will not be updated.
926+
* `'changed'` - All tests that are executed will update snapshots that did not match. Matching snapshots will not be updated. Also creates missing snapshots.
927927
* `'missing'` - Missing snapshots are created, for example when authoring a new test and running it for the first time. This is the default.
928928
* `'none'` - No snapshots are updated.
929929

‎nodejs/docs/test-agents.mdx‎

Lines changed: 318 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,318 @@
1+
---
2+
id: test-agents
3+
title: "Agents"
4+
---
5+
import Tabs from '@theme/Tabs';
6+
import TabItem from '@theme/TabItem';
7+
import HTMLCard from '@site/src/components/HTMLCard';
8+
import ProgressiveImage from '@theme/ProgressiveImage';
9+
10+
# Playwright Agents
11+
12+
## Test Coverage in 1-2-3
13+
14+
Playwright’s agentic workflow makes it possible to generate test coverage in three straightforward steps. These steps can be performed independently, manually, or as chained calls in an agentic loop.
15+
1. **Plan**: A planning agent explores the app and produces a test plan in `specs/*.md`.
16+
17+
2. **Generate**: A generating agent transforms the plan into `tests/*.spec.ts` files. It executes actions live to verify selectors and flows, then emits testing code and assertions.
18+
19+
3. **Heal**: A healing agent executes the test suite and automatically repairs failing tests by applying diffs in place.
20+
21+
## 1. Plan
22+
23+
**Input**
24+
* A clear request (e.g., “Generate a plan for guest checkout.”)
25+
* A live entry point (URL) or a seed Playwright test that sets up the environment
26+
* PRD (optional)
27+
28+
**Prompt**
29+
30+
```markdown
31+
Ask `playwright-test-planner` agent to generate a test plan for "Guest Checkout" scenario.
32+
Use `seed.spec.ts` as a seed test for the plan.
33+
```
34+
35+
**Output**
36+
* A Markdown test plan saved to `specs/guest-checkout.md`. The plan is human-readable but precise enough for test generation.
37+
38+
<details>
39+
40+
<summary>Example: specs/guest-checkout.md</summary>
41+
42+
```markdown
43+
# Feature: Guest Checkout
44+
45+
## Purpose
46+
Allow a user to purchase without creating an account.
47+
48+
## Preconditions
49+
- Test seed `tests/seed.spec.ts`.
50+
- Payment sandbox credentials available via env vars.
51+
52+
## Scenarios
53+
54+
### SC-1: Add single item to cart and purchase
55+
**Steps**
56+
1. Open home page.
57+
2. Search for "Wireless Mouse".
58+
3. Open product page and add to cart.
59+
4. Proceed to checkout as guest.
60+
5. Fill shipping and payment details.
61+
6. Confirm order.
62+
63+
**Expected**
64+
- Cart count increments after item is added.
65+
- Checkout page shows item, price, tax, and total.
66+
- Order confirmation number appears; status is "Processing".
67+
68+
### SC-2: Tax and shipping recalculation on address change
69+
**Steps**
70+
1. Start checkout with a CA address.
71+
2. Change state to NY.
72+
73+
**Expected**
74+
- Tax and shipping values recalculate.
75+
76+
## Data
77+
- Product SKU: `WM-123`
78+
- Payment: sandbox card `4111 1111 1111 1111`, valid expiry, CVV `123`.
79+
80+
## Methodology
81+
*Optional notes about testing methodology*
82+
```
83+
84+
</details>
85+
86+
## 2. Generate
87+
88+
The generating agent uses the Markdown plan to produce executable Playwright tests. It verifies selectors and assertions live against the application. Playwright supports generation hints and provides a catalog of assertions for efficient structural and behavioral validation.
89+
90+
**Input**
91+
* Markdown plan from `specs/`
92+
93+
**Prompt**
94+
95+
```markdown
96+
Ask `playwright-test-generator` to generate tests for the guest checkout plan under `specs/`.
97+
```
98+
99+
**Output**
100+
* A test suite under `tests/`
101+
* Tests may include initial errors that can be healed automatically
102+
103+
<details>
104+
105+
<summary>Example: tests/guest-checkout.spec.ts</summary>
106+
107+
```ts
108+
import { test, expect } from '@playwright/test';
109+
110+
test.describe('Guest Checkout', () => {
111+
test('SC-1: add item and purchase', async ({ page }) => {
112+
await page.goto('/');
113+
await page.getByRole('searchbox', { name: /search/i }).fill('Wireless Mouse');
114+
await page.getByRole('button', { name: /search/i }).click();
115+
116+
await page.getByRole('link', { name: /wireless mouse/i }).click();
117+
await page.getByRole('button', { name: /add to cart/i }).click();
118+
119+
// Assertion: cart badge increments
120+
await expect(page.getByTestId('cart-badge')).toHaveText('1');
121+
122+
await page.getByRole('link', { name: /checkout/i }).click();
123+
await page.getByRole('button', { name: /continue as guest/i }).click();
124+
125+
// Fill checkout form
126+
await page.getByLabel('Email').fill(process.env.CHECKOUT_EMAIL!);
127+
await page.getByLabel('Full name').fill('Alex Guest');
128+
await page.getByLabel('Address').fill('1 Market St');
129+
await page.getByLabel('City').fill('San Francisco');
130+
await page.getByLabel('State').selectOption('CA');
131+
await page.getByLabel('ZIP').fill('94105');
132+
133+
// Payment (sandbox)
134+
const frame = page.frameLocator('[data-testid="card-iframe"]');
135+
await frame.getByLabel('Card number').fill('4111111111111111');
136+
await frame.getByLabel('MM / YY').fill('12/30');
137+
await frame.getByLabel('CVC').fill('123');
138+
139+
await page.getByRole('button', { name: /pay/i }).click();
140+
141+
// Assertions: confirmation invariants
142+
await expect(page).toHaveURL(/\/orders\/\w+\/confirmation/);
143+
await expect(page.getByRole('heading', { name: /thank you/i })).toBeVisible();
144+
await expect(page.getByTestId('order-status')).toHaveText(/processing/i);
145+
146+
// Optional visual check
147+
await expect(page.locator('[data-testid="order-summary"]')).toHaveScreenshot();
148+
});
149+
});
150+
```
151+
152+
</details>
153+
154+
## 3. Heal
155+
156+
When a test fails, the healing agent:
157+
* Replays the failing steps
158+
* Inspects the current UI to locate equivalent elements or flows
159+
* Suggests a patch (e.g., locator update, wait adjustment, data fix)
160+
* Re-runs the test until it passes or until guardrails stop the loop
161+
162+
**Input**
163+
* Failing test name
164+
165+
**Prompt**
166+
167+
```markdown
168+
Ask `playwright-test-healer` to fix all failing tests for the guest checkout scenario.
169+
```
170+
171+
**Output**
172+
* A passing test, or a skipped test if the functionality is broken
173+
174+
## Artifacts and Conventions
175+
176+
Follow a simple, auditable structure:
177+
178+
```bash
179+
repo/
180+
.{claude|copilot|vscode|...}/ # agent definitions, tools, guardrails
181+
specs/ # human-readable test plans
182+
checkout-guest.md
183+
account-settings.md
184+
tests/ # generated Playwright tests
185+
seed.spec.ts
186+
checkout-guest.spec.ts
187+
account-settings.spec.ts
188+
playwright.config.ts
189+
```
190+
191+
### Agents definitions
192+
193+
Agent definitions are collections of instructions and MCP tools. They are provided by Playwright and should be regenerated whenever Playwright is updated.
194+
195+
Example for Claude Code subagents:
196+
197+
```bash
198+
npx playwright init-agents --claude
199+
```
200+
201+
### Specs in `specs/`
202+
203+
Specs are structured plans describing scenarios in human-readable terms. They include steps, expected outcomes, and data. Specs can start from scratch or extend a seed test.
204+
205+
### Tests in `tests/`
206+
207+
Generated Playwright tests, aligned one-to-one with specs wherever feasible.
208+
209+
### Seed tests `seed.spec.ts`
210+
211+
Seed tests provide a ready-to-use `page` context to bootstrap execution.
212+
213+
214+
[Accessibility]: /api/class-accessibility.mdx "Accessibility"
215+
[Android]: /api/class-android.mdx "Android"
216+
[AndroidDevice]: /api/class-androiddevice.mdx "AndroidDevice"
217+
[AndroidInput]: /api/class-androidinput.mdx "AndroidInput"
218+
[AndroidSocket]: /api/class-androidsocket.mdx "AndroidSocket"
219+
[AndroidWebView]: /api/class-androidwebview.mdx "AndroidWebView"
220+
[APIRequest]: /api/class-apirequest.mdx "APIRequest"
221+
[APIRequestContext]: /api/class-apirequestcontext.mdx "APIRequestContext"
222+
[APIResponse]: /api/class-apiresponse.mdx "APIResponse"
223+
[APIResponseAssertions]: /api/class-apiresponseassertions.mdx "APIResponseAssertions"
224+
[Browser]: /api/class-browser.mdx "Browser"
225+
[BrowserContext]: /api/class-browsercontext.mdx "BrowserContext"
226+
[BrowserServer]: /api/class-browserserver.mdx "BrowserServer"
227+
[BrowserType]: /api/class-browsertype.mdx "BrowserType"
228+
[CDPSession]: /api/class-cdpsession.mdx "CDPSession"
229+
[Clock]: /api/class-clock.mdx "Clock"
230+
[ConsoleMessage]: /api/class-consolemessage.mdx "ConsoleMessage"
231+
[Coverage]: /api/class-coverage.mdx "Coverage"
232+
[Dialog]: /api/class-dialog.mdx "Dialog"
233+
[Download]: /api/class-download.mdx "Download"
234+
[Electron]: /api/class-electron.mdx "Electron"
235+
[ElectronApplication]: /api/class-electronapplication.mdx "ElectronApplication"
236+
[ElementHandle]: /api/class-elementhandle.mdx "ElementHandle"
237+
[FileChooser]: /api/class-filechooser.mdx "FileChooser"
238+
[Frame]: /api/class-frame.mdx "Frame"
239+
[FrameLocator]: /api/class-framelocator.mdx "FrameLocator"
240+
[GenericAssertions]: /api/class-genericassertions.mdx "GenericAssertions"
241+
[JSHandle]: /api/class-jshandle.mdx "JSHandle"
242+
[Keyboard]: /api/class-keyboard.mdx "Keyboard"
243+
[Locator]: /api/class-locator.mdx "Locator"
244+
[LocatorAssertions]: /api/class-locatorassertions.mdx "LocatorAssertions"
245+
[Logger]: /api/class-logger.mdx "Logger"
246+
[Mouse]: /api/class-mouse.mdx "Mouse"
247+
[Page]: /api/class-page.mdx "Page"
248+
[PageAssertions]: /api/class-pageassertions.mdx "PageAssertions"
249+
[Playwright]: /api/class-playwright.mdx "Playwright"
250+
[PlaywrightAssertions]: /api/class-playwrightassertions.mdx "PlaywrightAssertions"
251+
[Request]: /api/class-request.mdx "Request"
252+
[Response]: /api/class-response.mdx "Response"
253+
[Route]: /api/class-route.mdx "Route"
254+
[Selectors]: /api/class-selectors.mdx "Selectors"
255+
[SnapshotAssertions]: /api/class-snapshotassertions.mdx "SnapshotAssertions"
256+
[TimeoutError]: /api/class-timeouterror.mdx "TimeoutError"
257+
[Touchscreen]: /api/class-touchscreen.mdx "Touchscreen"
258+
[Tracing]: /api/class-tracing.mdx "Tracing"
259+
[Video]: /api/class-video.mdx "Video"
260+
[WebError]: /api/class-weberror.mdx "WebError"
261+
[WebSocket]: /api/class-websocket.mdx "WebSocket"
262+
[WebSocketRoute]: /api/class-websocketroute.mdx "WebSocketRoute"
263+
[Worker]: /api/class-worker.mdx "Worker"
264+
[Fixtures]: /api/class-fixtures.mdx "Fixtures"
265+
[FullConfig]: /api/class-fullconfig.mdx "FullConfig"
266+
[FullProject]: /api/class-fullproject.mdx "FullProject"
267+
[Location]: /api/class-location.mdx "Location"
268+
[Test]: /api/class-test.mdx "Test"
269+
[TestConfig]: /api/class-testconfig.mdx "TestConfig"
270+
[TestInfo]: /api/class-testinfo.mdx "TestInfo"
271+
[TestInfoError]: /api/class-testinfoerror.mdx "TestInfoError"
272+
[TestOptions]: /api/class-testoptions.mdx "TestOptions"
273+
[TestProject]: /api/class-testproject.mdx "TestProject"
274+
[TestStepInfo]: /api/class-teststepinfo.mdx "TestStepInfo"
275+
[WorkerInfo]: /api/class-workerinfo.mdx "WorkerInfo"
276+
[Reporter]: /api/class-reporter.mdx "Reporter"
277+
[Suite]: /api/class-suite.mdx "Suite"
278+
[TestCase]: /api/class-testcase.mdx "TestCase"
279+
[TestError]: /api/class-testerror.mdx "TestError"
280+
[TestResult]: /api/class-testresult.mdx "TestResult"
281+
[TestStep]: /api/class-teststep.mdx "TestStep"
282+
[Element]: https://developer.mozilla.org/en-US/docs/Web/API/element "Element"
283+
[EvaluationArgument]: /evaluating.mdx#evaluation-argument "EvaluationArgument"
284+
[Promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise "Promise"
285+
[iterator]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols "Iterator"
286+
[origin]: https://developer.mozilla.org/en-US/docs/Glossary/Origin "Origin"
287+
[selector]: https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors "selector"
288+
[Serializable]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#Description "Serializable"
289+
[UIEvent.detail]: https://developer.mozilla.org/en-US/docs/Web/API/UIEvent/detail "UIEvent.detail"
290+
[UnixTime]: https://en.wikipedia.org/wiki/Unix_time "Unix Time"
291+
[xpath]: https://developer.mozilla.org/en-US/docs/Web/XPath "xpath"
292+
293+
[Array]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array "Array"
294+
[boolean]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Boolean_type "Boolean"
295+
[Buffer]: https://nodejs.org/api/buffer.html#buffer_class_buffer "Buffer"
296+
[ChildProcess]: https://nodejs.org/api/child_process.html "ChildProcess"
297+
[Date]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date "Date"
298+
[Error]: https://nodejs.org/api/errors.html#errors_class_error "Error"
299+
[EventEmitter]: https://nodejs.org/api/events.html#events_class_eventemitter "EventEmitter"
300+
[function]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function "Function"
301+
[FormData]: https://developer.mozilla.org/en-US/docs/Web/API/FormData "FormData"
302+
[Map]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map "Map"
303+
[Metadata]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object "Object&lt;string, any&gt;"
304+
[null]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/null "null"
305+
[number]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#Number_type "Number"
306+
[Object]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object "Object"
307+
[Promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise "Promise"
308+
[Readable]: https://nodejs.org/api/stream.html#stream_class_stream_readable "Readable"
309+
[ReadStream]: https://nodejs.org/api/fs.html#class-fsreadstream "ReadStream"
310+
[RegExp]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp "RegExp"
311+
[string]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#String_type "string"
312+
[void]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/undefined "void"
313+
[URL]: https://nodejs.org/api/url.html "URL"
314+
[URLSearchParams]: https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams "URLSearchParams"
315+
316+
[all available image tags]: https://mcr.microsoft.com/en-us/product/playwright/about "all available image tags"
317+
[Microsoft Artifact Registry]: https://mcr.microsoft.com/en-us/product/playwright/about "Microsoft Artifact Registry"
318+
[Dockerfile.noble]: https://github.com/microsoft/playwright/blob/main/utils/docker/Dockerfile.noble "Dockerfile.noble"

‎nodejs/docs/test-configuration.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
id: test-configuration
3-
title: "Test configuration"
3+
title: "Configuration"
44
---
55
import Tabs from '@theme/Tabs';
66
import TabItem from '@theme/TabItem';

‎nodejs/docs/test-reporters.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,7 @@ HTML report supports the following configuration options and environment variabl
253253
| `PLAYWRIGHT_HTML_HOST` | `host` | When report opens in the browser, it will be served bound to this hostname. | `localhost`
254254
| `PLAYWRIGHT_HTML_PORT` | `port` | When report opens in the browser, it will be served on this port. | `9323` or any available port when `9323` is not available.
255255
| `PLAYWRIGHT_HTML_ATTACHMENTS_BASE_URL` | `attachmentsBaseURL` | A separate location where attachments from the `data` subdirectory are uploaded. Only needed when you upload report and `data` separately to different locations. | `data/`
256+
| `PLAYWRIGHT_HTML_NO_COPY_PROMPT` | `noCopyPrompt` | If true, disable rendering of the Copy prompt for errors. Supports `true`, `1`, `false`, and `0`. | `false`
256257
| `PLAYWRIGHT_HTML_NO_SNIPPETS` | `noSnippets` | If true, disable rendering code snippets in the action log. If there is a top level error, that report section with code snippet will still render. Supports `true`, `1`, `false`, and `0`. | `false`
257258

258259
### Blob reporter

‎nodejs/docs/test-use-options.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
id: test-use-options
3-
title: "Test use options"
3+
title: "Configuration (use)"
44
---
55
import Tabs from '@theme/Tabs';
66
import TabItem from '@theme/TabItem';

‎nodejs/sidebars.js‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,11 @@ module.exports = {
2020
type: 'category',
2121
label: 'Playwright Test',
2222
items: [
23-
{ type: 'doc', id: 'test-configuration' },
24-
{ type: 'doc', id: 'test-use-options' },
23+
{ type: 'doc', id: 'test-agents' },
2524
{ type: 'doc', id: 'test-annotations' },
2625
{ type: 'doc', id: 'test-cli' },
26+
{ type: 'doc', id: 'test-configuration' },
27+
{ type: 'doc', id: 'test-use-options' },
2728
{ type: 'doc', id: 'emulation' },
2829
{ type: 'doc', id: 'test-fixtures' },
2930
{ type: 'doc', id: 'test-global-setup-teardown' },

‎src/generator.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ const { generateTabGroups, renderHTMLCard } = require('./format_utils');
2929
/** @typedef {import('./markdown').MarkdownNode} MarkdownNode */
3030
/** @export @typedef {{ name: string, link: string, usages: string[], args: docs.Member[], signatures?: string[] }} FormattedMember */
3131

32-
const commonSnippets = new Set(['txt', 'html', 'xml', 'yml', 'yaml', 'json', 'groovy', 'html', 'bash', 'sh', 'ini', 'Dockerfile', 'css']);
32+
const commonSnippets = new Set(['txt', 'html', 'xml', 'yml', 'yaml', 'json', 'groovy', 'html', 'bash', 'sh', 'ini', 'Dockerfile', 'css', 'markdown']);
3333

3434
/** @typedef {"header"|"link"|"usage"} FormatMode */
3535

0 commit comments

Comments
 (0)