feat: Coverage

[regularkevvv]

Summary

Adds automatic branch coverage tracking for Jinja templates, enabling developers to identify untested conditional paths in their templates.

Features

Core Coverage Tracking

• Branch discovery via AST walking: {% if %}, {% elif %}, {% else %}, {% for %}, {% macro %}, {% block %}, {% include %}
• Ternary expression coverage: {{ x if cond else y }} with lazy evaluation preserved
• Implicit false branch tracking: Automatically injects {% else %} for bare {% if %} and {% for %} blocks

Pytest Integration

pytest --jt-cov --jt-cov-fail-under=80

Report Formats

• Terminal: --jt-cov-report=term / term-missing / term-verbose
• HTML: --jt-cov-report=html --jt-cov-html=htmlcov/
• JSON: --jt-cov-report=json --jt-cov-json=coverage.json
• JUnit XML: --jt-cov-report=xml --jt-cov-xml=coverage.xml

Configuration (pyproject.toml)

[tool.jinjatest.coverage]
enabled = true
fail_under = 80
report = ["term", "html"]
exclude_patterns = ["**/vendor/**"]

Technical Details

• CoverageEnvironment subclass overrides _generate() for CondExpr AST transformation
• CondExprTransformer wraps ternary branches with _trace_branch() calls
• AutoInstrumenter uses regex + stack-based tracking for implicit else injection
• Thread-safe CoverageCollector singleton aggregates coverage across test session

Files Added

• jinjatest/coverage/ module (discovery, instrumenter, transformer, environment, tracker, collector, reporter, pytest_cov)

Files Modified

• jinjatest/spec.py - Integration with coverage collector
• jinjatest/instrumentation.py - Added trace_branch() method
• jinjatest/markers.py - Type signature fix
• pyproject.toml - Added ty type checker, coverage plugin entry point
• README.md - Coverage documentation section

---

Summary by cubic

Adds branch coverage for Jinja templates with a pytest plugin, auto-instrumentation, and terminal/HTML/JSON/XML reports to help spot untested paths. Includes lazy-safe ternary tracking and an optional fail-under threshold.

• New Features

  • Discovers branches via AST (if/elif/else, for, block, include, macro, ternary) and tracks implicit false branches.
  • Instruments templates automatically and records hits with a thread-safe collector aggregated across the test session.
  • Tracks ternary branches lazily via a custom Environment transformer and trace_branch passthrough.
  • Pytest plugin: enable with --jt-cov, set thresholds with --jt-cov-fail-under, and output reports: term, term-missing/verbose, html, json, xml.
  • Supports pyproject.toml config (enable, fail_under, reports, exclude_patterns).
  • HTML report assets are embedded via an auto-generated _templates.py (built by scripts/hatch_build.py).

• Dependencies

  • Adds pytest plugin entry point (jinjatest_cov) and integrates coverage hooks in TemplateSpec and instrumentation.
  • Adds "ty" type checker (Makefile and pyproject settings) and ignores generated _templates.py.

^{Written for commit fd242f8. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 33 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="jinjatest/spec.py">

<violation number="1" location="jinjatest/spec.py:444">
P2: Coverage is recorded even when test_mode=False because template_path is set whenever coverage is enabled. This can mark renders with empty trace events and reduce coverage despite instrumentation being disabled. Gate template_path on test_mode so coverage only records for test-mode renders.</violation>
</file>

<file name="README.md">

<violation number="1" location="README.md:422">
P3: The CLI options table omits the supported `term-verbose` report type. Update the documentation to include it so it matches the actual CLI behavior.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[Copilot]

Pull request overview

This PR adds a full Jinja template branch coverage subsystem (discovery, instrumentation, aggregation, and reporting), integrates it with TemplateSpec and pytest, and documents how to use it. It also tightens typing and tests around template variable discovery and anchor handling.

Changes:

• Introduces jinjatest.coverage (discovery, instrumenter, tracker, collector, reporters, pytest plugin) to support branch/ternary coverage and multiple report formats (terminal, JSON, HTML, JUnit XML).
• Integrates coverage into TemplateSpec lifecycle (from string/file and render) and instrumentation, so branch hits are recorded via the coverage collector during tests.
• Adds extensive test suites for coverage discovery/instrumentation/collection/reporting/pytest integration and updates existing tests/docs, tooling (ty), and Makefile to support and validate the new functionality.

Reviewed changes

Copilot reviewed 10 out of 33 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/test_coverage_boost.py	Adjusts get_undeclared_variables expectations to reflect that string templates now keep their source and can report undeclared variables like file-based templates.
tests/test_basic.py	Adds explicit None-checks for anchor sections to satisfy stricter typing and avoid Optional[str] misuse in tests.
tests/coverage/init.py	Marks the new tests.coverage package so coverage-related tests are discoverable by pytest.
tests/coverage/test_tracker.py	Tests TemplateCoverage and TemplateCoverageStats behavior (branch hit tracking, coverage percentages, covered/uncovered branch lists).
tests/coverage/test_reporter.py	Validates terminal, JSON, and HTML reporters (including summary stats, uncovered branch details, filenames, and helper methods) and the unified CoverageReporter wrapper.
tests/coverage/test_pytest_integration.py	Exercises end‑to‑end integration between TemplateSpec, manual coverage collector enablement, and the pytest plugin options at a behavioral level.
tests/coverage/test_instrumenter.py	Covers AutoInstrumenter and InstrumentationResult, including instrumentation of if/elif/else, for, macro, block, whitespace control, and implicit false/else injection behavior.
tests/coverage/test_discovery.py	Tests BranchDiscovery, BranchInfo, and DiscoveryResult for multiple construct types (if/elif/for/macro/include/blocks) and has_else semantics.
tests/coverage/test_condexpr.py	Verifies branch discovery, AST transformation, and runtime semantics (including lazy evaluation and side‑effects) for ternary (CondExpr) expressions via CoverageEnvironment and instrumentation.
tests/coverage/test_collector.py	Tests CoverageCollector, CoverageSummary, and singleton helpers (get_coverage_collector, reset_coverage_collector, set_coverage_collector) for enable/disable, registration, recording, and summary aggregation.
scripts/build_templates.py	Adds a build script to embed HTML/CSS templates from jinjatest/coverage/templates/ into a generated _templates.py module used by HTML reporting.
pyproject.toml	Registers the new pytest plugin entry point (jinjatest_cov), configures the ty type checker (including test overrides), and keeps pytest default settings.
jinjatest/spec.py	Integrates coverage with TemplateSpec: chooses CoverageEnvironment when coverage is enabled, wires _trace_branch, passes original source and a coverage path to the collector, and records coverage hits after each render.
jinjatest/markers.py	Expands type hints so marker loading can accept both TestInstrumentation and ProductionInstrumentation contexts.
jinjatest/instrumentation.py	Adds trace_branch to both test and production instrumentation, enabling ternary branch tracking while preserving values and lazy evaluation semantics.
jinjatest/coverage/init.py	Exposes the coverage API (collector, discovery, instrumenter, reporter, tracker types) as a cohesive public module.
jinjatest/coverage/collector.py	Implements a thread‑safe global coverage collector and CoverageSummary to manage and aggregate TemplateCoverage across all templates, with exclude pattern support.
jinjatest/coverage/discovery.py	Implements AST‑based branch discovery (BranchDiscovery, BranchInfo, DiscoveryResult) for if/elif/for/macro/block/include/CondExpr nodes.
jinjatest/coverage/environment.py	Adds CoverageEnvironment, an Environment subclass that instruments ternary expressions by running CondExprTransformer during code generation.
jinjatest/coverage/instrumenter.py	Provides AutoInstrumenter and InstrumentationResult to inject jt.trace calls for branches and to synthesize implicit else branches for bare if and for blocks.
jinjatest/coverage/pytest_cov.py	Adds a pytest plugin to enable coverage from CLI/pyproject configuration, generate all report formats, and integrate coverage into pytest’s session lifecycle.
jinjatest/coverage/reporter.py	Implements TerminalReporter, JSONReporter, HTMLReporter, JUnitReporter, and CoverageReporter to render collected coverage data into multiple output formats.
jinjatest/coverage/templates/index.html	Template for the main HTML coverage index page summarizing per‑template and overall coverage.
jinjatest/coverage/templates/row.html	Template for a single template row in the HTML index table.
jinjatest/coverage/templates/source_line.html	Template for individual source lines in per‑template HTML views with coverage classes.
jinjatest/coverage/templates/style.css	Stylesheet for HTML coverage reports, including classes for coverage levels and line highlighting.
jinjatest/coverage/templates/template_page.html	Template for per‑template HTML coverage pages showing summary, uncovered branches, and annotated source.
jinjatest/coverage/tracker.py	Implements TemplateCoverage and BranchCoverage/TemplateCoverageStats to tie together discovery, instrumentation, and hit counting for a single template.
jinjatest/coverage/transformer.py	Adds CondExprTransformer, a Jinja2 AST transformer that wraps ternary branches with _trace_branch calls while preserving lazy evaluation.
jinjatest/init.py	Documents that coverage functionality is available via jinjatest.coverage and keeps the public __all__ list consistent.
README.md	Updates installation instructions to use uv and adds a “Template Coverage” section documenting CLI usage, configuration, and what is tracked.
Makefile	Integrates ty type checking into the lint target alongside ruff checks/formatting.
.gitignore	Ignores the generated jinjatest/coverage/_templates.py module and keeps generic coverage outputs ignored.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.