From 070b128e7ee7f236a1796ba402f1b7fb0c477d61 Mon Sep 17 00:00:00 2001 From: "microsoft-playwright-automation[bot]" <203992400+microsoft-playwright-automation[bot]@users.noreply.github.com> Date: Tue, 21 Oct 2025 10:10:31 +0000 Subject: [PATCH] feat(roll): roll to ToT Playwright (21-10-25) --- dotnet/docs/trace-viewer.mdx | 2 +- java/docs/trace-viewer.mdx | 2 +- nodejs/docs/api/class-fullconfig.mdx | 17 +++++++++++++++ nodejs/docs/api/class-reporter.mdx | 4 ++++ nodejs/docs/api/class-test.mdx | 2 +- nodejs/docs/api/class-testconfig.mdx | 23 +++++++++++++++++++++ nodejs/docs/codegen.mdx | 2 +- nodejs/docs/test-reporters.mdx | 31 ++++++++++++++++++++++++++-- nodejs/docs/test-sharding.mdx | 17 ++++++++++++++- nodejs/docs/trace-viewer.mdx | 2 +- python/docs/trace-viewer.mdx | 2 +- 11 files changed, 95 insertions(+), 9 deletions(-) diff --git a/dotnet/docs/trace-viewer.mdx b/dotnet/docs/trace-viewer.mdx index 3aa5c5dff36..7d09a7fa95a 100644 --- a/dotnet/docs/trace-viewer.mdx +++ b/dotnet/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ pwsh bin/Debug/netX/playwright.ps1 show-trace trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/java/docs/trace-viewer.mdx b/java/docs/trace-viewer.mdx index 54fd37baa51..bcb0b0fb7eb 100644 --- a/java/docs/trace-viewer.mdx +++ b/java/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="sh ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/nodejs/docs/api/class-fullconfig.mdx b/nodejs/docs/api/class-fullconfig.mdx index 39899fdd35f..8dca0328d75 100644 --- a/nodejs/docs/api/class-fullconfig.mdx +++ b/nodejs/docs/api/class-fullconfig.mdx @@ -321,6 +321,23 @@ fullConfig.shard --- +### tags {#full-config-tags} + +Added in: v1.57fullConfig.tags + +Resolved global tags. See [testConfig.tag](/api/class-testconfig.mdx#test-config-tag). + +**Usage** + +```js +fullConfig.tags +``` + +**Type** +- [Array]<[string]> + +--- + ### updateSnapshots {#full-config-update-snapshots} Added in: v1.10fullConfig.updateSnapshots diff --git a/nodejs/docs/api/class-reporter.mdx b/nodejs/docs/api/class-reporter.mdx index 20dea760129..f160789aac9 100644 --- a/nodejs/docs/api/class-reporter.mdx +++ b/nodejs/docs/api/class-reporter.mdx @@ -107,6 +107,10 @@ Additionally, [reporter.onStdOut()](/api/class-reporter.mdx#reporter-on-std-out) If your custom reporter does not print anything to the terminal, implement [reporter.printsToStdio()](/api/class-reporter.mdx#reporter-prints-to-stdio) and return `false`. This way, Playwright will use one of the standard terminal reporters in addition to your custom reporter to enhance user experience. +**Reporter errors** + +Playwright will swallow any errors thrown in your custom reporter methods. If you need to detect or fail on reporter errors, you must wrap and handle them yourself. + **Merged report API notes** When merging multiple [`blob`](../test-reporters#blob-reporter) reports via [`merge-reports`](../test-sharding#merge-reports-cli) CLI command, the same [Reporter] API is called to produce final reports and all existing reporters should work without any changes. There some subtle differences though which might affect some custom reporters. diff --git a/nodejs/docs/api/class-test.mdx b/nodejs/docs/api/class-test.mdx index 9f46d902823..9cd6820f69f 100644 --- a/nodejs/docs/api/class-test.mdx +++ b/nodejs/docs/api/class-test.mdx @@ -1419,7 +1419,7 @@ function step(target: Function, context: ClassMethodDecoratorContext) { const name = this.constructor.name + '.' + (context.name as string); return test.step(name, async () => { return await target.call(this, ...args); - }); + }, { box: true }); }; } diff --git a/nodejs/docs/api/class-testconfig.mdx b/nodejs/docs/api/class-testconfig.mdx index 4ef4d7d3711..d32a8b7bdef 100644 --- a/nodejs/docs/api/class-testconfig.mdx +++ b/nodejs/docs/api/class-testconfig.mdx @@ -805,6 +805,29 @@ In this config: --- +### tag {#test-config-tag} + +Added in: v1.57testConfig.tag + +Tag or tags prepended to each test in the report. Useful for tagging your test run to differentiate between [CI environments](../test-sharding.mdx#merging-reports-from-multiple-environments). + +Note that each tag must start with `@` symbol. Learn more about [tagging](../test-annotations.mdx#tag-tests). + +**Usage** + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" +}); +``` + +**Type** +- [string] | [Array]<[string]> + +--- + ### testDir {#test-config-test-dir} Added in: v1.10testConfig.testDir diff --git a/nodejs/docs/codegen.mdx b/nodejs/docs/codegen.mdx index f38a1d9f376..7f2543e3866 100644 --- a/nodejs/docs/codegen.mdx +++ b/nodejs/docs/codegen.mdx @@ -56,7 +56,7 @@ In the test file in VS Code you will see your new generated actions added to you ### Generating locators You can generate locators with the test generator. -- Click on the **Pick locator** button form the testing sidebar and then hover over elements in the browser window to see the [locator](./locators.mdx) highlighted underneath each element. +- Click on the **Pick locator** button from the testing sidebar and then hover over elements in the browser window to see the [locator](./locators.mdx) highlighted underneath each element. - Click the element you require and it will now show up in the **Pick locator** box in VS Code. - Press Enter on your keyboard to copy the locator into the clipboard and then paste anywhere in your code. Or press 'escape' if you want to cancel. diff --git a/nodejs/docs/test-reporters.mdx b/nodejs/docs/test-reporters.mdx index 52a074f90b8..1863b2704f4 100644 --- a/nodejs/docs/test-reporters.mdx +++ b/nodejs/docs/test-reporters.mdx @@ -263,16 +263,43 @@ Blob reports contain all the details about the test run and can be used later to npx playwright test --reporter=blob ``` -By default, the report is written into the `blob-report` directory in the package.json directory or current working directory (if no package.json is found). The report file name looks like `report-.zip` or `report--.zip` when [sharding](./test-sharding.mdx) is used. The hash is an optional value computed from `--grep`, `--grepInverted`, `--project` and file filters passed as command line arguments. The hash guarantees that running Playwright with different command line options will produce different but stable between runs report names. The output file name can be overridden in the configuration file or pass as `'PLAYWRIGHT_BLOB_OUTPUT_FILE'` environment variable. +By default, the report is written into the `blob-report` directory in the package.json directory or current working directory (if no package.json is found). + +The report file name looks like `report-.zip` or `report--.zip` when [sharding](./test-sharding.mdx) is used. The hash is an optional value computed from `--grep`, `--grepInverted`, `--project`, [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) and file filters passed as command line arguments. The hash guarantees that running Playwright with different command line options will produce different but stable between runs report names. The output file name can be overridden in the configuration file or passed as `'PLAYWRIGHT_BLOB_OUTPUT_FILE'` environment variable. + + + + + +When using blob report to merge multiple shards, you don't have to pass any options. ```js title="playwright.config.ts" import { defineConfig } from '@playwright/test'; export default defineConfig({ - reporter: [['blob', { outputFile: `./blob-report/report-${os.platform()}.zip` }]], + reporter: 'blob', }); ``` + + + + +When running tests in different environments, you might want to use [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) to add a global tag corresponding to the environment. This tag will bring clarity to the merged report, and it will be used to produce a unique blob report name. + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + reporter: 'blob', + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" or "@linux" +}); +``` + + + + + Blob report supports following configuration options and environment variables: | Environment Variable Name | Reporter Config Option| Description | Default diff --git a/nodejs/docs/test-sharding.mdx b/nodejs/docs/test-sharding.mdx index 259822b1507..2525eb6a081 100644 --- a/nodejs/docs/test-sharding.mdx +++ b/nodejs/docs/test-sharding.mdx @@ -59,7 +59,7 @@ export default defineConfig({ }); ``` -Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into `blob-report` directory. +Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into `blob-report` directory. You can learn about [blob report options here](./test-reporters.mdx#blob-reporter). To merge reports from multiple shards, put the blob report files into a single directory, for example `all-blob-reports`. Blob report names contain shard number, so they will not clash. @@ -159,6 +159,21 @@ You can now see the reports have been merged and a combined HTML report is avail image +## Merging reports from multiple environments + +If you want to run the same tests in multiple environments, as opposed to shard your tests onto multiple machines, you need to differentiate these enviroments. + +In this case, it is useful to specify the [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) property, to tag all tests with the environment name. This tag will be automatically picked up by the blob report and later on by the merge tool. + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + reporter: process.env.CI ? 'blob' : 'html', + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" +}); +``` + ## Merge-reports CLI `npx playwright merge-reports path/to/blob-reports-dir` reads all blob reports from the passed directory and merges them into a single report. diff --git a/nodejs/docs/trace-viewer.mdx b/nodejs/docs/trace-viewer.mdx index 1ae622a96b6..93bab98eac6 100644 --- a/nodejs/docs/trace-viewer.mdx +++ b/nodejs/docs/trace-viewer.mdx @@ -25,7 +25,7 @@ npx playwright show-trace path/to/trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/python/docs/trace-viewer.mdx b/python/docs/trace-viewer.mdx index da4e340f287..96cee446994 100644 --- a/python/docs/trace-viewer.mdx +++ b/python/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ playwright show-trace trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally.