Pytest Coverage Comment

A GitHub Action that adds pytest coverage reports as comments to your pull requests, helping you track and improve test coverage with visual feedback.

🎯 Features

📊 Visual Coverage Reports - Automatically comments on PRs with detailed coverage tables
🏷️ Coverage Badges - Dynamic badges showing coverage percentage with color coding
📈 Test Statistics - Shows passed, failed, skipped tests with execution time
🔗 Direct File Links - Click to view uncovered lines directly in your repository
📁 Multiple Reports - Support for monorepo with multiple coverage reports
🎨 Customizable - Flexible titles, badges, and display options
📝 XML Support - Works with both text and XML coverage formats
🚀 Smart Updates - Updates existing comments instead of creating duplicates

📋 Table of Contents

Click to expand

Pytest Coverage Comment

📦 Prerequisites

Before using this action, ensure you have the following installed in your Python environment:

Python - Version 3.6+ (Python 3.9+ recommended for latest pytest/pytest-cov versions)
pytest - Python testing framework
pytest-cov - Coverage plugin for pytest (provides --cov and --cov-report flags)

pip install pytest pytest-cov

Note: The --cov and --cov-report flags used in the examples below are provided by pytest-cov, not pytest itself. If you see an error like pytest: error: unrecognized arguments: --cov, you need to install pytest-cov.

Python version compatibility

Python 3.9+: Supported by latest pytest (8.4+) and pytest-cov (6.0+) versions
Python 3.8: Use pytest-cov < 6.0.0 (e.g., pytest-cov 5.x)
Python 3.7: Use pytest-cov < 5.0.0 (e.g., pytest-cov 4.x)
Python 3.6 and older: Use older versions of pytest and pytest-cov

For most users, we recommend using Python 3.9+ with the latest versions of pytest and pytest-cov to get the latest features and security updates.

🚀 Quick Start

Add this action to your workflow:

- name: Pytest coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

📖 Complete workflow example

name: pytest-coverage-comment
on:
  pull_request:
    branches:
      - '*'

permissions:
  contents: read
  pull-requests: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: 3.11

      - name: Install dependencies
        run: |
          pip install pytest pytest-cov

      - name: Run tests with coverage
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=src tests/ | tee pytest-coverage.txt

      - name: Pytest coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml

⚙️ Configuration

Inputs

📝 Core Inputs

Name	Required	Default	Description
`github-token`	✓	`${{github.token}}`	GitHub token for API access to create/update comments
`pytest-coverage-path`		`./pytest-coverage.txt`	Path to pytest text coverage output (from `--cov-report=term-missing`)
`pytest-xml-coverage-path`			Path to XML coverage report (from `--cov-report=xml:coverage.xml`)
`junitxml-path`			Path to JUnit XML file for test statistics (passed/failed/skipped)
`issue-number`			Pull request number to comment on (required for workflow_dispatch/workflow_run events)

🎨 Display Options

Name	Default	Description
`title`	`Coverage Report`	Main title for the coverage comment (useful for monorepo projects)
`badge-title`	`Coverage`	Text shown on the coverage percentage badge
`junitxml-title`		Title for the test summary section from JUnit XML
`hide-badge`	`false`	Hide the coverage percentage badge from the comment
`hide-report`	`false`	Hide the detailed coverage table (show only summary and badge)
`hide-comment`	`false`	Skip creating PR comment entirely (useful for using outputs only)
`report-only-changed-files`	`false`	Show only files changed in the current pull request
`xml-skip-covered`	`false`	Hide files with 100% coverage from XML coverage reports
`remove-link-from-badge`	`false`	Remove hyperlink from coverage badge (badge becomes plain image)
`remove-links-to-files`	`false`	Remove file links from coverage table to reduce comment size
`remove-links-to-lines`	`false`	Remove line number links from coverage table to reduce comment size
`text-instead-badge`	`false`	Use simple text instead of badge images for coverage display

🔧 Advanced Options

Name	Default	Description
`create-new-comment`	`false`	Create new comment on each run instead of updating existing comment
`unique-id-for-comment`		Unique identifier for matrix builds to update separate comments (e.g., `${{ matrix.python-version }}`)
`default-branch`	`main`	Base branch name for file links in coverage report (e.g., main, master)
`coverage-path-prefix`		Prefix to add to file paths in coverage report links
`multiple-files`		Generate single comment with multiple coverage reports (useful for monorepos)

Outputs

📤 Available Outputs

Name	Example	Description
`coverage`	`85%`	Coverage percentage from pytest report
`color`	`green`	Badge color based on coverage percentage (red/orange/yellow/green/brightgreen)
`coverageHtml`	HTML string	Full HTML coverage report with clickable links to uncovered lines
`summaryReport`	Markdown string	Test summary in markdown format with statistics (tests/skipped/failures/errors/time)
`warnings`	`42`	Number of coverage warnings from pytest-cov
`tests`	`109`	Total number of tests run (from JUnit XML)
`skipped`	`2`	Number of skipped tests (from JUnit XML)
`failures`	`0`	Number of failed tests (from JUnit XML)
`errors`	`0`	Number of test errors (from JUnit XML)
`time`	`12.5`	Test execution time in seconds (from JUnit XML)
`notSuccessTestInfo`	JSON string	JSON details of failed, errored, and skipped tests (from JUnit XML)

📚 Usage Examples

Basic Usage

Standard PR Comment

- name: Run tests
  run: |
    pytest --junitxml=pytest.xml --cov-report=term-missing:skip-covered --cov=src tests/ | tee pytest-coverage.txt

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

Coverage from XML

Using coverage.xml instead of text output

- name: Generate XML coverage
  run: |
    pytest --cov-report=xml:coverage.xml --cov=src tests/

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-xml-coverage-path: ./coverage.xml
    junitxml-path: ./pytest.xml

Monorepo Support

Multiple coverage reports in a single comment

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    multiple-files: |
      Backend API, ./backend/pytest-coverage.txt, ./backend/pytest.xml
      Frontend SDK, ./frontend/pytest-coverage.txt, ./frontend/pytest.xml
      Data Pipeline, ./pipeline/pytest-coverage.txt, ./pipeline/pytest.xml

This creates a consolidated table showing all coverage reports:

Title	Coverage	Tests	Time
Backend API	85%	156	23.4s
Frontend SDK	92%	89	12.1s
Data Pipeline	78%	234	45.6s

Output: Combined table showing coverage and test results for all packages.

Docker Workflows

Running tests inside Docker containers

- name: Run tests in Docker
  run: |
    docker run -v /tmp:/tmp $IMAGE_TAG \
      python -m pytest \
        --cov-report=term-missing:skip-covered \
        --junitxml=/tmp/pytest.xml \
        --cov=src tests/ | tee /tmp/pytest-coverage.txt

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: /tmp/pytest-coverage.txt
    junitxml-path: /tmp/pytest.xml

Matrix Builds

Separate comments for each matrix combination

strategy:
  matrix:
    python-version: ['3.9', '3.10', '3.11']
    os: [ubuntu-latest, windows-latest]

steps:
  - name: Coverage comment
    uses: MishaKav/pytest-coverage-comment@main
    with:
      pytest-coverage-path: ./pytest-coverage.txt
      junitxml-path: ./pytest.xml
      unique-id-for-comment: ${{ matrix.python-version }}-${{ matrix.os }}
      title: Coverage for Python ${{ matrix.python-version }} on ${{ matrix.os }}

Auto-update README Badge

Keep coverage badge in README always up-to-date

First, add placeholders to your README.md:

<!-- Pytest Coverage Comment:Begin -->
<!-- Pytest Coverage Comment:End -->

Then use this workflow:

name: Update Coverage Badge
on:
  push:
    branches: [main]

permissions:
  contents: write

jobs:
  update-badge:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          persist-credentials: false
          fetch-depth: 0

      - name: Run tests
        run: |
          pytest --junitxml=pytest.xml --cov-report=term-missing --cov=src tests/ | tee pytest-coverage.txt

      - name: Coverage comment
        id: coverage
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml
          hide-comment: true

      - name: Update README
        run: |
          sed -i '/<!-- Pytest Coverage Comment:Begin -->/,/<!-- Pytest Coverage Comment:End -->/c\<!-- Pytest Coverage Comment:Begin -->\n${{ steps.coverage.outputs.coverageHtml }}\n<!-- Pytest Coverage Comment:End -->' ./README.md

      - name: Commit changes
        uses: stefanzweifel/git-auto-commit-action@v5
        with:
          commit_message: 'docs: update coverage badge'
          file_pattern: README.md

📋 Output Example

Here's what the generated coverage comment looks like:

Coverage Report

File	Stmts	Miss	Cover	Missing
functions/example_completed
example_completed.py	64	19	70%	33, 39–45, 48–51, 55–58, 65–70, 91–92
functions/example_manager
example_manager.py	44	11	75%	31–33, 49–55, 67–69
example_static.py	40	2	95%	60–61
functions/my_exampels
example.py	20	20	0%	1–31
functions/resources
resources.py	26	26	0%	1–37
TOTAL	1055	739	30%

Tests	Skipped	Failures	Errors	Time
109	2 💤	1 ❌	0 🔥	0.583s ⏱️

🔬 Advanced Features

📝 Text-Based Coverage Display

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
    text-instead-badge: true

Displays coverage as 85% (42/50) instead of a badge image.

📊 Using Output Variables

- name: Coverage comment
  id: coverage
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml

- name: Dynamic Badges
  uses: schneegans/dynamic-badges-action@v1.7.0
  with:
    auth: ${{ secrets.GIST_SECRET }}
    gistID: your-gist-id
    filename: coverage.json
    label: Coverage
    message: ${{ steps.coverage.outputs.coverage }}
    color: ${{ steps.coverage.outputs.color }}

- name: Fail if coverage too low
  if: ${{ steps.coverage.outputs.coverage < 80 }}
  run: |
    echo "Coverage is below 80%!"
    exit 1

🎯 Show Only Changed Files

- name: Coverage comment (changed files only)
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
    report-only-changed-files: true

This is particularly useful for large codebases where you want to focus on coverage for files modified in the PR.

🔀 Workflow Dispatch Support

name: Manual Coverage Report
on:
  workflow_dispatch:
    inputs:
      pr_number:
        description: 'Pull Request number'
        required: true

jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - name: Coverage comment
        uses: MishaKav/pytest-coverage-comment@main
        with:
          pytest-coverage-path: ./pytest-coverage.txt
          junitxml-path: ./pytest.xml
          issue-number: ${{ github.event.inputs.pr_number }}

⚡ Performance Optimization

For large coverage reports that might exceed GitHub's comment size limits:

- name: Coverage comment
  uses: MishaKav/pytest-coverage-comment@main
  with:
    pytest-coverage-path: ./pytest-coverage.txt
    junitxml-path: ./pytest.xml
    hide-report: true # Show only summary and badge
    xml-skip-covered: true # Skip files with 100% coverage
    report-only-changed-files: true # Only show changed files
    remove-links-to-files: true # Remove clickable file links
    remove-links-to-lines: true # Remove clickable line number links

Link Removal Options:

remove-links-to-files: true - Removes clickable links to files. Instead of [example.py](link), shows plain example.py
remove-links-to-lines: true - Removes clickable links to line numbers. Instead of [14-18](link), shows plain 14-18

These options significantly reduce comment size while preserving all coverage information.

🎨 Badge Colors

Coverage badges automatically change color based on the percentage:

Coverage	Badge	Color
0-40%		Red
40-60%		Orange
60-80%		Yellow
80-90%		Green
90-100%		Bright Green

🔄 Auto-Updating Badge on README

If you want auto-update the coverage badge on your README, you can see the workflow example above.

📸 Result Examples

View example outputs

Standard Comment (Collapsed)

Expanded Coverage Report

Multiple Files (Monorepo)

Text Mode (text-instead-badge: true)

With text-instead-badge: true, coverage displays as simple text:

85% (42/50)

Instead of a badge image:

🔧 Troubleshooting

Common Issues and Solutions

Comment Not Appearing

Issue: The action runs successfully but no comment appears on the PR.

Root Cause: This is usually caused by insufficient GitHub token permissions. The GITHUB_TOKEN needs write access to create/update PR comments.

Common Error Messages:

Error: Resource not accessible by integration
HttpError: Resource not accessible by integration
403 Forbidden errors in the action logs

Solutions:

Add permissions block to your workflow (Recommended):

permissions:
  contents: read        # Required for checkout and comparing commits
  pull-requests: write  # Required for creating/updating PR comments

For push events with commit comments, use:

permissions:
  contents: write       # Required for creating commit comments
  pull-requests: write  # If you also want PR comments

Repository/Organization Settings (Admin access required):
- Go to Settings > Actions > General
- Under "Workflow permissions", select "Read and write permissions"
- Note: This affects all workflows, so adding permissions to individual workflows is more secure
Other checks:
- For workflow_dispatch events, provide the issue-number input
- Verify hide-comment is not set to true
- Check branch protection rules aren't blocking automated comments

Why it works on forks but not main repos: Forks often have different default permission settings than the main repository. Organizations frequently set restrictive defaults for security.

Unrecognized Arguments Error

Issue: pytest: error: unrecognized arguments: --cov --cov-report

Root Cause: The pytest-cov plugin is not installed. The --cov and --cov-report flags are provided by pytest-cov, not pytest itself.

Solution:

Install the pytest-cov package in your Python environment:

pip install pytest-cov

Or add it to your requirements.txt or pyproject.toml:

# requirements.txt
pytest>=8.0.0
pytest-cov>=5.0.0

# pyproject.toml
[project]
dependencies = [
    "pytest>=8.0.0",
    "pytest-cov>=5.0.0",
]

Make sure the installation step runs before executing pytest commands in your workflow:

- name: Install dependencies
  run: |
    pip install pytest pytest-cov

- name: Run tests with coverage
  run: |
    pytest --cov=src --cov-report=term-missing tests/

Coverage Report Too Large

Issue: "Comment is too long (maximum is 65536 characters)"

Solutions:

Use xml-skip-covered: true to hide fully covered files
Enable report-only-changed-files: true
Set hide-report: true to show only summary
Use remove-links-to-files: true to remove clickable file links
Use remove-links-to-lines: true to remove clickable line number links
Use --cov-report=term-missing:skip-covered in pytest

GitHub Step Summary Too Large

Issue: "GitHub Action Summary too big" (exceeds 1MB limit)

Solution: As of v1.1.55, the action automatically truncates summaries that exceed GitHub's 1MB limit.

Files Not Found

Issue: "No such file or directory" errors

Solutions:

Use absolute paths or paths relative to $GITHUB_WORKSPACE
For Docker workflows, ensure volumes are mounted correctly
Check that coverage files are generated before the action runs

Wrong File Links

Issue: Links in the coverage report point to wrong files or 404

Solutions:

Set default-branch to your repository's main branch
Use coverage-path-prefix if your test paths differ from repository structure
Ensure the action runs on the correct commit SHA

🤝 Contributing

We welcome all contributions! Please feel free to submit pull requests or open issues for bugs, feature requests, or improvements.

Development Setup

# Clone the repository
git clone https://github.com/MishaKav/pytest-coverage-comment.git
cd pytest-coverage-comment

# Install dependencies
npm install

# Run tests (if available)
npm test

# Build the action
npm run build

👥 Contributors

📄 License

🔗 Similar Actions

For JavaScript/TypeScript projects using Jest: Check out jest-coverage-comment - a similar action with even more features for Jest test coverage.

If you find this action helpful, please consider giving it a ⭐ on GitHub!

Name		Name	Last commit message	Last commit date
Latest commit History 141 Commits
.github		.github
.vscode		.vscode
data		data
dist		dist
src		src
.gitignore		.gitignore
.nvmrc		.nvmrc
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
SUPPORT.md		SUPPORT.md
action.yml		action.yml
eslint.config.js		eslint.config.js
package-lock.json		package-lock.json
package.json		package.json

Uh oh!

License

MishaKav/pytest-coverage-comment

Folders and files

Latest commit

History

Repository files navigation

Pytest Coverage Comment

🎯 Features

📋 Table of Contents

📦 Prerequisites

🚀 Quick Start

⚙️ Configuration

Inputs

Outputs

📚 Usage Examples

Basic Usage

Coverage from XML

Monorepo Support

Docker Workflows

Matrix Builds

Auto-update README Badge

📋 Output Example

🔬 Advanced Features

🎨 Badge Colors

🔄 Auto-Updating Badge on README

📸 Result Examples

Standard Comment (Collapsed)

Expanded Coverage Report

Multiple Files (Monorepo)

Text Mode (text-instead-badge: true)

🔧 Troubleshooting

Comment Not Appearing

Unrecognized Arguments Error

Coverage Report Too Large

GitHub Step Summary Too Large

Files Not Found

Wrong File Links

🤝 Contributing

Development Setup

👥 Contributors

📄 License

🔗 Similar Actions

About

Topics

Resources

License

Code of conduct

Contributing

Security policy

Uh oh!

Stars

Watchers

Forks

Releases 62

Sponsor this project

Uh oh!

Packages 0

Used by 1.9k

Contributors 23

Languages

Packages