Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 57 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,28 @@ Default client fixture names the plugin will look for (in order):

If you use a different fixture name, you can provide one or more names via the CLI flag `--api-cov-client-fixture-names` (repeatable) or in `pyproject.toml` under `[tool.pytest_api_cov]` as `client_fixture_names` (a list).

#### Option 1: Helper Function

#### Option 1: Configuration-Based (recommended for most users)

Configure one or more existing fixture names to be discovered and wrapped automatically by the plugin.

Example `pyproject.toml`:

```toml
[tool.pytest_api_cov]
# Provide a list of candidate fixture names the plugin should try (order matters)
client_fixture_names = ["my_custom_client"]
```

Or use the CLI flag multiple times:

```bash
pytest --api-cov-report --api-cov-client-fixture-names=my_custom_client --api-cov-client-fixture-names=another_fixture
```

If the configured fixture(s) are not found, the plugin will try to use an `app` fixture (if present) to create a tracked client. If neither is available or the plugin cannot extract the app from a discovered client fixture, the tests will still run — coverage will simply be unavailable and a warning will be logged.

#### Option 2: Helper Function

Use the `create_coverage_fixture` helper to create a custom fixture name:

Expand Down Expand Up @@ -221,25 +242,6 @@ def test_with_flask_client(flask_client):

The helper returns a pytest fixture you can assign to a name in `conftest.py`.

#### Option 2: Configuration-Based (recommended for most users)

Configure one or more existing fixture names to be discovered and wrapped automatically by the plugin.

Example `pyproject.toml`:

```toml
[tool.pytest_api_cov]
# Provide a list of candidate fixture names the plugin should try (order matters)
client_fixture_names = ["my_custom_client"]
```

Or use the CLI flag multiple times:

```bash
pytest --api-cov-report --api-cov-client-fixture-names=my_custom_client --api-cov-client-fixture-names=another_fixture
```

If the configured fixture(s) are not found, the plugin will try to use an `app` fixture (if present) to create a tracked client. If neither is available or the plugin cannot extract the app from a discovered client fixture, the tests will still run — coverage will simply be unavailable and a warning will be logged.

### Configuration Options

Expand All @@ -258,12 +260,14 @@ show_excluded_endpoints = false
# Exclude endpoints from coverage using wildcard patterns with negation support
# Use * for wildcard matching, all other characters are matched literally
# Use ! at the start to negate a pattern (include what would otherwise be excluded)
# Optionally prefix a pattern with one or more HTTP methods to target only those methods,
exclusion_patterns = [
"/health",
"/metrics",
"/docs/*",
"/admin/*",
"!/admin/public",
"GET,POST /users/*"
]

# Save detailed JSON report
Expand All @@ -281,6 +285,39 @@ client_fixture_names = ["my_custom_client"]
# Group HTTP methods by endpoint for legacy behavior (default: false)
group_methods_by_endpoint = false

```
Notes on exclusion patterns

- Method prefixes (optional): If a pattern starts with one or more HTTP method names followed by whitespace, the pattern applies only to those methods. Methods may be comma-separated and are matched case-insensitively. Example: `GET,POST /users/*`.
- Path-only patterns (default): If no method is specified the pattern applies to all methods for the matching path (existing behaviour).
- Wildcards: Use `*` to match any characters in the path portion (not a regex; dots and other characters are treated literally unless `*` is used).
- Negation: Prefix a pattern with `!` to override earlier exclusions and re-include a path (or method-specific path). Negations can also include method prefixes (e.g. `!GET /admin/health`).
- Matching: Patterns are tested against both the full `METHOD /path` string and the `/path` portion to remain compatible with existing configurations.

Examples (pyproject or CLI):

- Exclude the `/health` path for all methods:

```toml
exclusion_patterns = ["/health"]
```

- Exclude only GET requests to `/health`:

```toml
exclusion_patterns = ["GET /health"]
```

- Exclude GET and POST for `/users/*` but re-include GET /users/42:

```toml
exclusion_patterns = ["GET,POST /users/*", "!GET /users/42"]
```

Or using the CLI flags (repeatable):

```bash
pytest --api-cov-report --api-cov-exclusion-patterns="GET,POST /users/*" --api-cov-exclusion-patterns="!GET /users/42"
```

### Command Line Options
Expand Down
16 changes: 0 additions & 16 deletions example/conftest.py

This file was deleted.

24 changes: 24 additions & 0 deletions example/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""example/conftest.py"""

import pytest
from fastapi.testclient import TestClient

from example.src.main import app as fastapi_app
from pytest_api_cov.plugin import create_coverage_fixture


@pytest.fixture
def original_client():
"""Original FastAPI test client fixture.

This fixture demonstrates an existing user-provided client that we can wrap
with `create_coverage_fixture` so tests can continue to use the familiar
`client` fixture name while gaining API coverage tracking.
"""
return TestClient(fastapi_app)


# Create a wrapped fixture named 'client' that wraps the existing 'original_client'.
# Tests can continue to request the `client` fixture as before and coverage will be
# collected when pytest is run with --api-cov-report.
client = create_coverage_fixture("client", "original_client")
20 changes: 10 additions & 10 deletions example/tests/test_endpoints.py
Original file line number Diff line number Diff line change
@@ -1,33 +1,33 @@
"""example/tests/test_endpoints.py"""


def test_root_path(coverage_client):
def test_root_path(client):
"""Test the root endpoint."""
response = coverage_client.get("/")
response = client.get("/")
assert response.status_code == 200
assert response.json() == {"message": "Hello World"}


def test_items_path(coverage_client):
def test_items_path(client):
"""Test the items endpoint."""
response = coverage_client.get("/items/42")
response = client.get("/items/42")
assert response.status_code == 200
assert response.json() == {"item_id": 42}


def test_create_item(coverage_client):
def test_create_item(client):
"""Test creating an item."""
response = coverage_client.post("/items", json={"name": "test item"})
response = client.post("/items", json={"name": "test item"})
assert response.status_code == 200
assert response.json()["message"] == "Item created"


def test_xyz_and_root_path(coverage_client):
def test_xyz_and_root_path(client):
"""Test the xyz endpoint."""
response = coverage_client.get("/xyz/123")
response = client.get("/xyz/123")
assert response.status_code == 404
response = coverage_client.get("/xyzzyx")
response = client.get("/xyzzyx")
assert response.status_code == 200
response = coverage_client.get("/")
response = client.get("/")
assert response.status_code == 200
assert response.json() == {"message": "Hello World"}
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "pytest-api-cov"
version = "1.2.0"
version = "1.2.1"
description = "Pytest Plugin to provide API Coverage statistics for Python Web Frameworks"
readme = "README.md"
authors = [{ name = "Barnaby Gill", email = "barnabasgill@gmail.com" }]
Expand Down
87 changes: 57 additions & 30 deletions src/pytest_api_cov/report.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import re
from pathlib import Path
from re import Pattern
from typing import Any, Dict, List, Set, Tuple
from typing import Any, Dict, List, Optional, Set, Tuple

from rich.console import Console

Expand All @@ -30,11 +30,14 @@ def categorise_endpoints(
) -> Tuple[List[str], List[str], List[str]]:
"""Categorise endpoints into covered, uncovered, and excluded.

Exclusion patterns support simple wildcard matching with negation:
Exclusion patterns support simple wildcard matching with negation and optional
HTTP method prefixes:
- Use * for wildcard (matches any characters)
- Use ! at the start to negate a pattern (include what would otherwise be excluded)
- Optionally prefix a pattern with one or more HTTP methods to target only those methods,
e.g. "GET /health" or "GET,POST /users/*" (methods are case-insensitive)
- All other characters are matched literally
- Examples: "/admin/*", "/health", "!users/bob" (negates exclusion)
- Examples: "/admin/*", "/health", "!users/bob", "GET /health", "GET,POST /users/*"
- Pattern order matters: exclusions are applied first, then negations override them
"""
covered, uncovered, excluded = [], [], []
Expand All @@ -47,39 +50,63 @@ def categorise_endpoints(
exclusion_only = [p for p in exclusion_patterns if not p.startswith("!")]
negation_only = [p[1:] for p in exclusion_patterns if p.startswith("!")] # Remove the '!' prefix

compiled_exclusions = (
[re.compile("^" + re.escape(pattern).replace(r"\*", ".*") + "$") for pattern in exclusion_only]
if exclusion_only
else None
)
compiled_negations = (
[re.compile("^" + re.escape(pattern).replace(r"\*", ".*") + "$") for pattern in negation_only]
if negation_only
else None
)
def compile_patterns(patterns: List[str]) -> List[Tuple[Optional[Set[str]], Pattern[str]]]:
compiled: List[Tuple[Optional[Set[str]], Pattern[str]]] = []
for pat in patterns:
path_pattern = pat.strip()
methods: Optional[Set[str]] = None
# Detect method prefix
m = re.match(r"^([A-Za-z,]+)\s+(.+)$", pat)
if m:
methods = {mname.strip().upper() for mname in m.group(1).split(",") if mname.strip()}
path_pattern = m.group(2)
# Build regex from the path part
regex = re.compile("^" + re.escape(path_pattern).replace(r"\*", ".*") + "$")
compiled.append((methods, regex))
return compiled

compiled_exclusions = compile_patterns(exclusion_only) if exclusion_only else None
compiled_negations = compile_patterns(negation_only) if negation_only else None

for endpoint in endpoints:
# Check exclusion patterns against both full "METHOD /path" and just "/path"
is_excluded = False
if compiled_exclusions:
# Extract path from "METHOD /path" format for pattern matching
if " " in endpoint:
_, path_only = endpoint.split(" ", 1)
is_excluded = any(p.match(endpoint) for p in compiled_exclusions) or any(
p.match(path_only) for p in compiled_exclusions
)
else:
is_excluded = any(p.match(endpoint) for p in compiled_exclusions)
endpoint_method = None
path_only = endpoint
if " " in endpoint:
endpoint_method, path_only = endpoint.split(" ", 1)
endpoint_method = endpoint_method.upper()

# Check negation patterns - these override exclusions
if compiled_exclusions:
for methods_set, regex in compiled_exclusions:
if methods_set:
if not endpoint_method:
continue
if endpoint_method not in methods_set:
continue
if regex.match(path_only) or regex.match(endpoint):
is_excluded = True
break
# No methods specified
elif regex.match(path_only) or regex.match(endpoint):
is_excluded = True
break

# Negation patterns
if is_excluded and compiled_negations:
if " " in endpoint:
_, path_only = endpoint.split(" ", 1)
is_negated = any(p.match(endpoint) for p in compiled_negations) or any(
p.match(path_only) for p in compiled_negations
)
else:
is_negated = any(p.match(endpoint) for p in compiled_negations)
is_negated = False
for methods_set, regex in compiled_negations:
if methods_set:
if not endpoint_method:
continue
if endpoint_method not in methods_set:
continue
if regex.match(path_only) or regex.match(endpoint):
is_negated = True
break
elif regex.match(path_only) or regex.match(endpoint):
is_negated = True
break

if is_negated:
is_excluded = False # Negation overrides exclusion
Expand Down
33 changes: 33 additions & 0 deletions tests/unit/test_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,39 @@ def test_categorise_complex_exclusion_negation_scenario(self):
assert set(uncovered) == {"/api/v2/users", "/api/v2/admin", "/docs"}
assert set(excluded_out) == {"/api/v1/admin", "/metrics"}

def test_categorise_with_method_specific_exclusion(self):
"""Exclude a specific HTTP method for an endpoint using 'METHOD /path' patterns."""
discovered = ["GET /items", "POST /items", "GET /health"]
called = {"POST /items"}
patterns = ["GET /items"] # Exclude only GET /items

covered, uncovered, excluded_out = categorise_endpoints(discovered, called, patterns)
assert set(covered) == {"POST /items"}
assert set(uncovered) == {"GET /health"}
assert set(excluded_out) == {"GET /items"}

def test_categorise_with_multiple_method_prefixes(self):
"""Support comma-separated method prefixes to exclude multiple methods for a path."""
discovered = ["GET /users/1", "POST /users/1", "PUT /users/1"]
called = {"PUT /users/1"}
patterns = ["GET,POST /users/*"] # Exclude GET and POST for users

covered, uncovered, excluded_out = categorise_endpoints(discovered, called, patterns)
assert set(covered) == {"PUT /users/1"}
assert set(uncovered) == set()
assert set(excluded_out) == {"GET /users/1", "POST /users/1"}

def test_categorise_method_prefixed_negation(self):
"""Negation with a method prefix should re-include only that method."""
discovered = ["GET /users/alice", "POST /users/alice", "GET /users/bob"]
called = {"GET /users/bob"}
patterns = ["/users/*", "!GET /users/bob"] # Exclude all users but re-include GET /users/bob

covered, uncovered, excluded_out = categorise_endpoints(discovered, called, patterns)
assert set(covered) == {"GET /users/bob"}
assert set(uncovered) == set()
assert set(excluded_out) == {"GET /users/alice", "POST /users/alice"}


class TestCoverageCalculationAndReporting:
"""Tests for coverage computation and report generation."""
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading