Skip to content

Commit 937c7c3

Browse files
authored
Merge pull request #11 from BarnabasG/initial
1.2.0 migrating from app fixture to test client extraction
2 parents 8eb8193 + 43a8243 commit 937c7c3

File tree

12 files changed

+519
-777
lines changed

12 files changed

+519
-777
lines changed

README.md

Lines changed: 65 additions & 124 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,9 @@ A **pytest plugin** that measures **API endpoint coverage** for FastAPI and Flas
55
## Features
66

77
- **Zero Configuration**: Plug-and-play with Flask/FastAPI apps - just install and run
8+
- **Client-Based Discovery**: Automatically extracts app from your existing test client fixtures
89
- **Terminal Reports**: Rich terminal output with detailed coverage information
910
- **JSON Reports**: Export coverage data for CI/CD integration
10-
- **Setup Wizard**: Interactive setup wizard for complex projects
1111

1212
## Quick Start
1313

@@ -28,25 +28,13 @@ pytest --api-cov-report
2828

2929
### App Location Flexibility
3030

31-
**Zero Config**: Works automatically if your app is in `app.py`, `main.py`, or `server.py`
31+
Discovery in this plugin is client-based: the plugin extracts the application instance from your test client fixtures, or from an `app` fixture when present. This means the plugin integrates with the test clients or fixtures you already use in your tests rather than relying on background file scanning.
3232

33-
**Any Location**: Place your app anywhere in your project - just create a `conftest.py`:
33+
How discovery works (in order):
3434

35-
```python
36-
import pytest
37-
from my_project.backend.api import my_app # Any import path!
38-
39-
@pytest.fixture
40-
def app():
41-
return my_app
42-
```
43-
44-
The plugin will automatically discover your Flask/FastAPI app if it's in common locations:
45-
- `app.py` (with variable `app`, `application`, or `main`)
46-
- `main.py` (with variable `app`, `application`, or `main`)
47-
- `server.py` (with variable `app`, `application`, or `server`)
48-
49-
**Your app can be located anywhere!** If it's not in a standard location, just create a `conftest.py` file to tell the plugin where to find it.
35+
1. If you configure one or more candidate client fixture names (see configuration below), the plugin will try each in order and wrap the first matching fixture it finds.
36+
2. If no configured client fixture is found, the plugin will look for a standard `app` fixture and use that to create a tracked client.
37+
3. If neither a client fixture nor an `app` fixture is available (or the plugin cannot extract an app from the client), coverage tracking will be skipped and a helpful message is shown.
5038

5139
### Example
5240

@@ -97,7 +85,7 @@ API Coverage Report
9785
Uncovered Endpoints:
9886
❌ GET /health
9987
100-
Total API Coverage: 66.67%
88+
Total API Coverage: 75.0%
10189
```
10290

10391
Or running with advanced options:
@@ -120,6 +108,16 @@ Total API Coverage: 50.0%
120108
JSON report saved to api_coverage.json
121109
```
122110

111+
### See examples
112+
113+
```bash
114+
# Print an example pyproject.toml configuration snippet
115+
pytest-api-cov show-pyproject
116+
117+
# Print an example conftest.py for a known app module
118+
pytest-api-cov show-conftest FastAPI src.main app
119+
```
120+
123121
## HTTP Method-Aware Coverage
124122

125123
By default, pytest-api-cov tracks coverage for **each HTTP method separately**. This means `GET /users` and `POST /users` are treated as different endpoints for coverage purposes.
@@ -160,19 +158,6 @@ Total API Coverage: 100.0% # All endpoints have at least one method tested
160158

161159
## Advanced Configuration
162160

163-
### Setup Wizard
164-
165-
If auto-discovery doesn't work for your project, use the interactive setup wizard:
166-
167-
```bash
168-
pytest-api-cov init
169-
```
170-
171-
This will:
172-
- Detect your framework and app location
173-
- Create a `conftest.py` fixture if needed
174-
- Generate suggested `pyproject.toml` configuration
175-
176161
### Manual Configuration
177162

178163
Create a `conftest.py` file to specify your app location (works with **any** file path or structure):
@@ -194,7 +179,15 @@ This approach works with any project structure - the plugin doesn't care where y
194179

195180
### Custom Test Client Fixtures
196181

197-
You have several options for using custom client fixtures:
182+
The plugin can wrap existing test client fixtures automatically. Recent changes allow you to specify one or more candidate fixture names (the plugin will try them in order) instead of a single configured name.
183+
184+
Default client fixture names the plugin will look for (in order):
185+
- `client`
186+
- `test_client`
187+
- `api_client`
188+
- `app_client`
189+
190+
If you use a different fixture name, you can provide one or more names via the CLI flag `--api-cov-client-fixture-names` (repeatable) or in `pyproject.toml` under `[tool.pytest_api_cov]` as `client_fixture_names` (a list).
198191

199192
#### Option 1: Helper Function
200193

@@ -226,34 +219,28 @@ def test_with_flask_client(flask_client):
226219
assert response.status_code == 200
227220
```
228221

229-
#### Option 2: Configuration-Based
222+
The helper returns a pytest fixture you can assign to a name in `conftest.py`.
230223

231-
Configure an existing fixture to be wrapped automatically:
224+
#### Option 2: Configuration-Based (recommended for most users)
232225

233-
```python
234-
import pytest
235-
from fastapi.testclient import TestClient
236-
from your_app import app
226+
Configure one or more existing fixture names to be discovered and wrapped automatically by the plugin.
237227

238-
@pytest.fixture
239-
def my_custom_client():
240-
"""Custom test client with authentication."""
241-
client = TestClient(app)
242-
client.headers.update({"Authorization": "Bearer test-token"})
243-
return client
244-
245-
def test_endpoint(coverage_client):
246-
response = coverage_client.get("/protected-endpoint")
247-
assert response.status_code == 200
248-
```
249-
250-
Configure it in `pyproject.toml`:
228+
Example `pyproject.toml`:
251229

252230
```toml
253231
[tool.pytest_api_cov]
254-
client_fixture_name = "my_custom_client"
232+
# Provide a list of candidate fixture names the plugin should try (order matters)
233+
client_fixture_names = ["my_custom_client"]
255234
```
256235

236+
Or use the CLI flag multiple times:
237+
238+
```bash
239+
pytest --api-cov-report --api-cov-client-fixture-names=my_custom_client --api-cov-client-fixture-names=another_fixture
240+
```
241+
242+
If the configured fixture(s) are not found, the plugin will try to use an `app` fixture (if present) to create a tracked client. If neither is available or the plugin cannot extract the app from a discovered client fixture, the tests will still run — coverage will simply be unavailable and a warning will be logged.
243+
257244
### Configuration Options
258245

259246
Add configuration to your `pyproject.toml`:
@@ -272,13 +259,11 @@ show_excluded_endpoints = false
272259
# Use * for wildcard matching, all other characters are matched literally
273260
# Use ! at the start to negate a pattern (include what would otherwise be excluded)
274261
exclusion_patterns = [
275-
"/health", # Exact match
276-
"/metrics", # Exact match
277-
"/docs/*", # Wildcard: matches /docs/swagger, /docs/openapi, etc.
278-
"/admin/*", # Wildcard: matches all admin endpoints
279-
"!/admin/public", # Negation: include /admin/public even though /admin/* excludes it
280-
"/api/v1.0/*", # Exact version match (won't match /api/v1x0/*)
281-
"!/api/v1.0/health" # Negation: include /api/v1.0/health even though /api/v1.0/* excludes it
262+
"/health",
263+
"/metrics",
264+
"/docs/*",
265+
"/admin/*",
266+
"!/admin/public",
282267
]
283268

284269
# Save detailed JSON report
@@ -290,13 +275,12 @@ force_sugar = true
290275
# Force no Unicode symbols in output
291276
force_sugar_disabled = true
292277

293-
# Wrap an existing custom test client fixture with coverage tracking
294-
client_fixture_name = "my_custom_client"
278+
# Provide candidate fixture names (in priority order).
279+
client_fixture_names = ["my_custom_client"]
295280

296281
# Group HTTP methods by endpoint for legacy behavior (default: false)
297-
# When true: treats GET /users and POST /users as one "/users" endpoint
298-
# When false: treats them as separate "GET /users" and "POST /users" endpoints (recommended)
299282
group_methods_by_endpoint = false
283+
300284
```
301285

302286
### Command Line Options
@@ -323,8 +307,8 @@ pytest --api-cov-report --api-cov-report-path=api_coverage.json
323307
# Exclude specific endpoints (supports wildcards and negation)
324308
pytest --api-cov-report --api-cov-exclusion-patterns="/health" --api-cov-exclusion-patterns="/docs/*"
325309

326-
# Exclude with negation (exclude all admin except admin/public)
327-
pytest --api-cov-report --api-cov-exclusion-patterns="/admin/*" --api-cov-exclusion-patterns="!/admin/public"
310+
# Specify one or more existing client fixture names (repeatable)
311+
pytest --api-cov-report --api-cov-client-fixture-names=my_custom_client --api-cov-client-fixture-names=another_fixture
328312

329313
# Verbose logging (shows discovery process)
330314
pytest --api-cov-report -v
@@ -450,74 +434,31 @@ jobs:
450434
451435
### No App Found
452436
453-
If you see "No API app found", you have several options:
454-
455-
**Option 1 - Auto-discovery (Zero Config)**
456-
Place your app in a standard location with a standard name:
457-
- Files: `app.py`, `main.py`, `server.py`, `wsgi.py`, `asgi.py`
458-
- Variable names: `app`, `application`, `main`, `server`
459-
460-
**Option 2 - Custom Location (Any File/Path)**
461-
Create a `conftest.py` file to specify your app location:
462-
463-
```python
464-
import pytest
465-
from my_project.api.server import my_flask_app # Any import path
466-
# or from src.backend.main import fastapi_instance
467-
# or from anywhere import your_app
468-
469-
@pytest.fixture
470-
def app():
471-
return my_flask_app # Return your app instance
472-
```
473-
474-
**Option 3 - Override Auto-discovery**
475-
If you have multiple auto-discoverable files or want to use a different app:
476-
477-
```python
478-
# Even if you have app.py, you can override it
479-
import pytest
480-
from main import my_real_app # Use this instead of app.py
481-
482-
@pytest.fixture
483-
def app():
484-
return my_real_app
485-
```
486-
487-
**Option 4 - Setup Wizard**
488-
Run the interactive setup: `pytest-api-cov init`
489-
490-
The plugin will automatically find your app using the `app` fixture first, then fall back to auto-discovery in common locations. This means you can place your app **anywhere** as long as you create the fixture.
491-
492-
### Multiple App Files
493-
494-
If you have multiple files that could be auto-discovered (e.g., both `app.py` and `main.py`), the plugin will use the **first valid app it finds** in this priority order:
495-
496-
1. `app.py`
497-
2. `main.py`
498-
3. `server.py`
499-
4. `wsgi.py`
500-
5. `asgi.py`
437+
If coverage is not running because the plugin could not locate an app, check the following:
501438
502-
To use a specific app when multiple exist, create a `conftest.py` with an `app` fixture pointing to your preferred app.
439+
- Ensure you are running pytest with `--api-cov-report` enabled.
440+
- Confirm you have a test client fixture (e.g. `client`, `test_client`, `api_client`) or an `app` fixture in your test suite.
441+
- If you use a custom client fixture, add its name to `client_fixture_names` in `pyproject.toml` or pass it via the CLI using `--api-cov-client-fixture-names` (repeatable) so the plugin can find and wrap it.
442+
- If the plugin finds the client fixture but cannot extract the underlying app (for example the client type is not supported or wrapped in an unexpected way), you will see a message like "Could not extract app from client" — in that case either provide an `app` fixture directly or wrap your existing client using `create_coverage_fixture`.
503443

504-
### No Endpoints Discovered
444+
### No endpoints Discovered
505445

506-
If you see "No endpoints discovered":
446+
If you still see no endpoints discovered:
507447

508-
1. Check that your app is properly instantiated
509-
2. Verify your routes/endpoints are defined
510-
3. Ensure the `coverage_client` fixture is working in your tests
511-
4. Use `-v` or `-vv` for debug information
448+
1. Check that your app is properly instantiated inside the fixture or client.
449+
2. Verify your routes/endpoints are defined and reachable by the test client.
450+
3. Ensure the `coverage_client` fixture is being used in your tests (or that your configured client fixture is listed and discovered).
451+
4. Use `-v` or `-vv` for debug logging to see why the plugin skipped discovery or wrapping.
512452

513453
### Framework Not Detected
514454

515455
The plugin supports:
516-
- **FastAPI**: Detected by `from fastapi import` or `import fastapi`
517-
- **Flask**: Detected by `from flask import` or `import flask`
456+
- **FastAPI**: Detected by `FastAPI` class
457+
- **Flask**: Detected by `Flask` class
458+
- **FlaskOpenAPI3**: Detected by `FlaskOpenAPI3` class
518459

519460
Other frameworks are not currently supported.
520461

521462
## License
522463

523-
This project is licensed under the Apache License 2.0.
464+
This project is licensed under the Apache License 2.0.

example/conftest.py

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,16 @@
11
"""example/conftest.py"""
22

33
import pytest
4+
from fastapi.testclient import TestClient
5+
46
from example.src.main import app as fastapi_app
57

68

79
@pytest.fixture
8-
def app():
9-
"""FastAPI app fixture for testing."""
10-
return fastapi_app
10+
def client():
11+
"""Standard FastAPI test client fixture.
12+
13+
The pytest-api-cov plugin will automatically discover this fixture,
14+
extract the app from it, and wrap it with coverage tracking.
15+
"""
16+
return TestClient(fastapi_app)

pyproject.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[project]
22
name = "pytest-api-cov"
3-
version = "1.1.5"
3+
version = "1.2.0"
44
description = "Pytest Plugin to provide API Coverage statistics for Python Web Frameworks"
55
readme = "README.md"
66
authors = [{ name = "Barnaby Gill", email = "barnabasgill@gmail.com" }]
@@ -24,7 +24,7 @@ Source = "https://github.com/BarnabasG/api-coverage"
2424
[dependency-groups]
2525
dev = [
2626
"mypy>=1.17.0",
27-
"path>=17.1.1",
27+
"path>=16.0.0",
2828
"pytest-cov>=6.2.1",
2929
"pytest-sugar>=1.0.0",
3030
"pytest-xdist>=3.8.0",

0 commit comments

Comments
 (0)