diff --git a/dotnet/docs/trace-viewer.mdx b/dotnet/docs/trace-viewer.mdx index 3aa5c5dff3..7d09a7fa95 100644 --- a/dotnet/docs/trace-viewer.mdx +++ b/dotnet/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ pwsh bin/Debug/netX/playwright.ps1 show-trace trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/java/docs/trace-viewer.mdx b/java/docs/trace-viewer.mdx index 54fd37baa5..bcb0b0fb7e 100644 --- a/java/docs/trace-viewer.mdx +++ b/java/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="sh ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/nodejs/docs/api/class-fullconfig.mdx b/nodejs/docs/api/class-fullconfig.mdx index 39899fdd35..8dca0328d7 100644 --- a/nodejs/docs/api/class-fullconfig.mdx +++ b/nodejs/docs/api/class-fullconfig.mdx @@ -321,6 +321,23 @@ fullConfig.shard --- +### tags {#full-config-tags} + +Added in: v1.57fullConfig.tags + +Resolved global tags. See [testConfig.tag](/api/class-testconfig.mdx#test-config-tag). + +**Usage** + +```js +fullConfig.tags +``` + +**Type** +- [Array]<[string]> + +--- + ### updateSnapshots {#full-config-update-snapshots} Added in: v1.10fullConfig.updateSnapshots diff --git a/nodejs/docs/api/class-reporter.mdx b/nodejs/docs/api/class-reporter.mdx index 20dea76012..f160789aac 100644 --- a/nodejs/docs/api/class-reporter.mdx +++ b/nodejs/docs/api/class-reporter.mdx @@ -107,6 +107,10 @@ Additionally, [reporter.onStdOut()](/api/class-reporter.mdx#reporter-on-std-out) If your custom reporter does not print anything to the terminal, implement [reporter.printsToStdio()](/api/class-reporter.mdx#reporter-prints-to-stdio) and return `false`. This way, Playwright will use one of the standard terminal reporters in addition to your custom reporter to enhance user experience. +**Reporter errors** + +Playwright will swallow any errors thrown in your custom reporter methods. If you need to detect or fail on reporter errors, you must wrap and handle them yourself. + **Merged report API notes** When merging multiple [`blob`](../test-reporters#blob-reporter) reports via [`merge-reports`](../test-sharding#merge-reports-cli) CLI command, the same [Reporter] API is called to produce final reports and all existing reporters should work without any changes. There some subtle differences though which might affect some custom reporters. diff --git a/nodejs/docs/api/class-test.mdx b/nodejs/docs/api/class-test.mdx index 9f46d90282..9cd6820f69 100644 --- a/nodejs/docs/api/class-test.mdx +++ b/nodejs/docs/api/class-test.mdx @@ -1419,7 +1419,7 @@ function step(target: Function, context: ClassMethodDecoratorContext) { const name = this.constructor.name + '.' + (context.name as string); return test.step(name, async () => { return await target.call(this, ...args); - }); + }, { box: true }); }; } diff --git a/nodejs/docs/api/class-testconfig.mdx b/nodejs/docs/api/class-testconfig.mdx index 4ef4d7d371..d32a8b7bde 100644 --- a/nodejs/docs/api/class-testconfig.mdx +++ b/nodejs/docs/api/class-testconfig.mdx @@ -805,6 +805,29 @@ In this config: --- +### tag {#test-config-tag} + +Added in: v1.57testConfig.tag + +Tag or tags prepended to each test in the report. Useful for tagging your test run to differentiate between [CI environments](../test-sharding.mdx#merging-reports-from-multiple-environments). + +Note that each tag must start with `@` symbol. Learn more about [tagging](../test-annotations.mdx#tag-tests). + +**Usage** + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" +}); +``` + +**Type** +- [string] | [Array]<[string]> + +--- + ### testDir {#test-config-test-dir} Added in: v1.10testConfig.testDir diff --git a/nodejs/docs/codegen.mdx b/nodejs/docs/codegen.mdx index f38a1d9f37..7f2543e386 100644 --- a/nodejs/docs/codegen.mdx +++ b/nodejs/docs/codegen.mdx @@ -56,7 +56,7 @@ In the test file in VS Code you will see your new generated actions added to you ### Generating locators You can generate locators with the test generator. -- Click on the **Pick locator** button form the testing sidebar and then hover over elements in the browser window to see the [locator](./locators.mdx) highlighted underneath each element. +- Click on the **Pick locator** button from the testing sidebar and then hover over elements in the browser window to see the [locator](./locators.mdx) highlighted underneath each element. - Click the element you require and it will now show up in the **Pick locator** box in VS Code. - Press Enter on your keyboard to copy the locator into the clipboard and then paste anywhere in your code. Or press 'escape' if you want to cancel. diff --git a/nodejs/docs/test-reporters.mdx b/nodejs/docs/test-reporters.mdx index 52a074f90b..1863b2704f 100644 --- a/nodejs/docs/test-reporters.mdx +++ b/nodejs/docs/test-reporters.mdx @@ -263,16 +263,43 @@ Blob reports contain all the details about the test run and can be used later to npx playwright test --reporter=blob ``` -By default, the report is written into the `blob-report` directory in the package.json directory or current working directory (if no package.json is found). The report file name looks like `report-.zip` or `report--.zip` when [sharding](./test-sharding.mdx) is used. The hash is an optional value computed from `--grep`, `--grepInverted`, `--project` and file filters passed as command line arguments. The hash guarantees that running Playwright with different command line options will produce different but stable between runs report names. The output file name can be overridden in the configuration file or pass as `'PLAYWRIGHT_BLOB_OUTPUT_FILE'` environment variable. +By default, the report is written into the `blob-report` directory in the package.json directory or current working directory (if no package.json is found). + +The report file name looks like `report-.zip` or `report--.zip` when [sharding](./test-sharding.mdx) is used. The hash is an optional value computed from `--grep`, `--grepInverted`, `--project`, [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) and file filters passed as command line arguments. The hash guarantees that running Playwright with different command line options will produce different but stable between runs report names. The output file name can be overridden in the configuration file or passed as `'PLAYWRIGHT_BLOB_OUTPUT_FILE'` environment variable. + + + + + +When using blob report to merge multiple shards, you don't have to pass any options. ```js title="playwright.config.ts" import { defineConfig } from '@playwright/test'; export default defineConfig({ - reporter: [['blob', { outputFile: `./blob-report/report-${os.platform()}.zip` }]], + reporter: 'blob', }); ``` + + + + +When running tests in different environments, you might want to use [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) to add a global tag corresponding to the environment. This tag will bring clarity to the merged report, and it will be used to produce a unique blob report name. + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + reporter: 'blob', + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" or "@linux" +}); +``` + + + + + Blob report supports following configuration options and environment variables: | Environment Variable Name | Reporter Config Option| Description | Default diff --git a/nodejs/docs/test-sharding.mdx b/nodejs/docs/test-sharding.mdx index 259822b150..2525eb6a08 100644 --- a/nodejs/docs/test-sharding.mdx +++ b/nodejs/docs/test-sharding.mdx @@ -59,7 +59,7 @@ export default defineConfig({ }); ``` -Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into `blob-report` directory. +Blob report contains information about all the tests that were run and their results as well as all test attachments such as traces and screenshot diffs. Blob reports can be merged and converted to any other Playwright report. By default, blob report will be generated into `blob-report` directory. You can learn about [blob report options here](./test-reporters.mdx#blob-reporter). To merge reports from multiple shards, put the blob report files into a single directory, for example `all-blob-reports`. Blob report names contain shard number, so they will not clash. @@ -159,6 +159,21 @@ You can now see the reports have been merged and a combined HTML report is avail image +## Merging reports from multiple environments + +If you want to run the same tests in multiple environments, as opposed to shard your tests onto multiple machines, you need to differentiate these enviroments. + +In this case, it is useful to specify the [testConfig.tag](/api/class-testconfig.mdx#test-config-tag) property, to tag all tests with the environment name. This tag will be automatically picked up by the blob report and later on by the merge tool. + +```js title="playwright.config.ts" +import { defineConfig } from '@playwright/test'; + +export default defineConfig({ + reporter: process.env.CI ? 'blob' : 'html', + tag: process.env.CI_ENVIRONMENT_NAME, // for example "@APIv2" +}); +``` + ## Merge-reports CLI `npx playwright merge-reports path/to/blob-reports-dir` reads all blob reports from the passed directory and merges them into a single report. diff --git a/nodejs/docs/trace-viewer.mdx b/nodejs/docs/trace-viewer.mdx index 1ae622a96b..93bab98eac 100644 --- a/nodejs/docs/trace-viewer.mdx +++ b/nodejs/docs/trace-viewer.mdx @@ -25,7 +25,7 @@ npx playwright show-trace path/to/trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally. diff --git a/python/docs/trace-viewer.mdx b/python/docs/trace-viewer.mdx index da4e340f28..96cee44699 100644 --- a/python/docs/trace-viewer.mdx +++ b/python/docs/trace-viewer.mdx @@ -22,7 +22,7 @@ playwright show-trace trace.zip ### Using [trace.playwright.dev](https://trace.playwright.dev) -[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload trace files using drag and drop or via the `Select file(s)` button. +[trace.playwright.dev](https://trace.playwright.dev) is a statically hosted variant of the Trace Viewer. You can upload a trace file using drag and drop or via the `Select file` button. Trace Viewer loads the trace entirely in your browser and does not transmit any data externally.