Skip to content

Tests: cover issue snapshots and add 8.5/next set coverage#23

Merged
f3l1x merged 7 commits intocontributte:masterfrom
ohmyfelix:chore/ruleset-analysis
Feb 23, 2026
Merged

Tests: cover issue snapshots and add 8.5/next set coverage#23
f3l1x merged 7 commits intocontributte:masterfrom
ohmyfelix:chore/ruleset-analysis

Conversation

@ohmyfelix
Copy link
Contributor

@ohmyfelix ohmyfelix commented Feb 23, 2026

Summary

  • add a new IssueTest data-provider suite so snapshot fixtures in tests/Issue/* are executed by PHPUnit
  • add tests/Sets/8.5 and tests/Sets/next with ruleset wrappers, fixtures, and generated snapshots to keep versioned set coverage aligned with available rulesets
  • harden SniffTest and RulesetTest process handling by asserting successful phpcs execution and strict JSON decoding while keeping non-zero lint findings from failing process exit
  • add docs/CI follow-ups: generate ruleset-8.5 docs, update ruleset examples, guard snapshot drift in workflow, and prevent filename-key collisions in Codesniffer::normalize()
  • add P2 automation: ruleset/set parity validation and generated ruleset diff report artifact in CI
  • add P3 regression tests for Codesniffer::normalize() duplicate-basename behavior

TODO

  • Fork and clone repository, create chore/ruleset-analysis branch
  • Run baseline test suite (phpunit, paratest) and verify green state
  • Add PHPUnit coverage for issue snapshots via tests/Cases/IssueTest.php
  • Add missing ruleset set coverage for 8.5 and next under tests/Sets
  • Harden PHPCS process handling and JSON decoding in snapshot tests
  • Update docs generation to include ruleset-8.5 in Makefile and .docs/rulesets
  • Remove basename collision risk in tests/Toolkit/Codesniffer.php by using deterministic unique keys
  • Add CI guard for snapshot drift (php bin/snapshots must be clean)
  • Add parity check between ruleset-*.xml and tests/Sets/*
  • Add ruleset diff report generation and upload it as CI artifact
  • Add direct unit tests for Codesniffer::normalize() key behavior

Testing

  • vendor/bin/phpunit
  • vendor/bin/paratest
  • php bin/snapshots
  • bash .dev/check-ruleset-sets-parity.sh
  • bash .dev/ruleset-diff-report.sh

@ohmyfelix
Copy link
Contributor Author

Added a machine-readable sniff coverage matrix and wired it to CI artifacts.

What is now generated in CI (Snapshots job):

  • .reports/sniff-coverage-matrix.csv
  • .reports/sniff-coverage-summary.md

How it works:

  • reads enabled sniffs from vendor/bin/phpcs --standard=ruleset.xml -e
  • maps sniff refs from tests/Sniffs/**/*.ruleset.xml
  • outputs per-enabled-sniff fields:
    • enabled_sniff
    • has_tests
    • fixture_count
    • has_good
    • has_bad
    • custom_count
    • directories

Latest local run summary:

  • Total enabled sniffs: 185
  • With dedicated tests: 185
  • Missing dedicated tests: 0
  • Weak examples: 2
    • Generic.Files.ByteOrderMark
    • Generic.Files.LineEndings

Once checks finish, download artifact: sniff-coverage-matrix.

@f3l1x f3l1x merged commit d98a852 into contributte:master Feb 23, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants