pytest-playwright-json

Generate Playwright-compatible JSON reports from your Python tests

Quick Start

1. Install

pip install pytest-playwright-json

2. Run Tests

pytest --playwright-json=test-results/report.json

That's it! Your test results are now in Playwright's JSON format.

Why Use This?

• TestDino Integration: Upload Python test results to TestDino dashboard
• Playwright Format: Use tools designed for Playwright's native JSON reporter
• Automatic Attachments: Screenshots, videos, and traces are included automatically
• Zero Config: Works out of the box with sensible defaults

Installation

pip install pytest-playwright-json

Requirements:

• Python 3.9 or higher
• pytest 7.0 or higher
• pytest-playwright (optional, for browser tests)

Usage

Basic Usage

# Generate JSON report
pytest --playwright-json=test-results/report.json

# With Playwright browser tests
pytest --playwright-json=test-results/report.json --browser=chromium

With pytest-html (Recommended)

pip install pytest-html

pytest \
  --playwright-json=test-results/report.json \
  --html=test-results/index.html \
  --self-contained-html

Configuration File

Add to your pyproject.toml:

[tool.pytest.ini_options]
addopts = [
    "--playwright-json=test-results/report.json",
    "--playwright-json-test-results-dir=test-results",
]

Or pytest.ini:

[pytest]
addopts = --playwright-json=test-results/report.json

Example config files: See examples/pyproject.toml.example and examples/pytest.ini.example for complete configuration examples including sharding support.

Options

Option	Description
--playwright-json=PATH	Output path for JSON report
--playwright-json-test-results-dir=PATH	Directory with test artifacts (auto-detected, rarely needed)
--playwright-json-include-attachments	Include attachments in report (default: enabled)

Smart Defaults: The plugin automatically finds test artifacts by:

1. Using pytest-playwright's --output directory if set
2. Using the parent directory of your JSON report path
3. Falling back to test-results

Attachments

The plugin automatically includes these attachments in your report:

Type	Extensions
Screenshots	.png, .jpg, .jpeg, .gif, .webp
Videos	.webm, .mp4
Traces	.zip

Running Tests with Sharding

When running tests in parallel across multiple shards, each shard should generate its own report with a unique name to prevent overwriting.

Configuration for Sharded Tests

Add to your pyproject.toml:

[tool.pytest.ini_options]
addopts = [
    "--playwright-json=test-results/report.json",
    "--html=test-results/index.html",
    "--self-contained-html",
    "-p no:selenium",
]

Or pytest.ini:

[pytest]
addopts = 
    --playwright-json=test-results/report.json
    --html=test-results/index.html
    --self-contained-html
    -p no:selenium

Using pytest-shard

Install pytest-shard:

pip install pytest-shard

Run tests with sharding:

# Shard 1 of 5 (0-indexed)
pytest --shard-id=0 --num-shards=5 --playwright-json=test-results/report-1.json

# Shard 2 of 5
pytest --shard-id=1 --num-shards=5 --playwright-json=test-results/report-2.json

Merging Reports from Multiple Shards

After all shards complete, merge the individual reports into a single report:

# Merge specific report files
pytest-playwright-json-merge report-1.json report-2.json report-3.json -o merged.json

# Find and merge all reports in a directory (recursive)
# Automatically finds report.json, report-1.json, report-2.json, etc.
pytest-playwright-json-merge -d test-results -o test-results/report.json

# With verbose output (shows stats from each file)
pytest-playwright-json-merge -d test-results -o test-results/report.json -v

Options:

• -o, --output PATH: Output path for merged report (required)
• -d, --directory PATH: Directory to recursively search for report*.json files
• -v, --verbose: Print detailed information including stats from each report file
• --version: Show version number

Complete GitHub Actions Example

name: Playwright Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4, 5]
        shardTotal: [5]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          pip install pytest pytest-playwright pytest-playwright-json pytest-html pytest-shard
          playwright install --with-deps chromium
      
      - name: Run Playwright Tests
        env:
          SHARD_INDEX: ${{ matrix.shardIndex }}
          SHARD_TOTAL: ${{ matrix.shardTotal }}
        run: |
          mkdir -p test-results
          SHARD_ID=$(( $SHARD_INDEX - 1 ))
          pytest \
            --shard-id=$SHARD_ID \
            --num-shards=$SHARD_TOTAL \
            --playwright-json=test-results/report-${{ matrix.shardIndex }}.json \
            --html=test-results/index.html \
            --self-contained-html \
            -p no:selenium \
            -v
      
      - name: Upload test results
        uses: actions/upload-artifact@v4
        with:
          name: test-results-${{ matrix.shardIndex }}
          path: test-results/
      
      - name: Cache test metadata
        if: always()
        run: |
          testdino cache --working-dir test-results --token="${{ secrets.TESTDINO_TOKEN }}"
  
  merge-and-upload:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          pip install pytest-playwright-json testdino
      
      - name: Download all test results
        uses: actions/download-artifact@v4
        with:
          pattern: test-results-*
          merge-multiple: true
          path: combined-test-results
      
      - name: Merge JSON reports from all shards
        run: |
          python3 -m pytest_playwright_json.merge -d combined-test-results -o combined-test-results/report.json -v
      
      - name: Upload to TestDino
        run: |
          testdino upload ./combined-test-results --token="${{ secrets.TESTDINO_TOKEN }}"

Upload to TestDino

Use with testdino CLI to upload results:

# Install testdino
pip install testdino

# Run tests
pytest --playwright-json=test-results/report.json

# Upload to TestDino
testdino upload test-results --token="your-token"

Example Test

from playwright.sync_api import Page, expect

def test_homepage(page: Page):
    page.goto("https://example.com")
    expect(page).to_have_title("Example Domain")

def test_navigation(page: Page):
    page.goto("https://example.com")
    page.click("a")
    expect(page).to_have_url("https://www.iana.org/domains/example")

Run with:

pytest tests/ --playwright-json=test-results/report.json --browser=chromium

Test Results

After running, you'll find:

test-results/
  report.json          # JSON report (Playwright format)
  index.html           # HTML report (if using pytest-html)
  test-name-chromium/  # Attachments for failed tests
    screenshot.png
    video.webm
    trace.zip

Troubleshooting

No attachments in report?

Make sure your JSON report is in the same directory as test artifacts:

# Recommended: Put report.json inside test-results/
pytest --playwright-json=test-results/report.json --output=test-results

pytest-playwright not found?

Install it:

pip install pytest-playwright
playwright install chromium

Support

• Documentation: testdino.com/docs
• Issues: GitHub Issues

---

Made with love by the TestDino team