test(qa): add contention and secret-detection coverage with CI gates#35
Merged
test(qa): add contention and secret-detection coverage with CI gates#35
Conversation
7 tasks
There was a problem hiding this comment.
Pull request overview
Adds QA coverage and CI hooks to validate AgentMesh’s coordination/provenance behavior under contention and to regression-test content-based private/secret detection (including a policy mechanism to exempt known synthetic fixtures from content scanning).
Changes:
- Added new pytest suites for contention scenarios and secret/sensitive-content detection using fixtures and inline content.
- Extended
classify_path()policy to supportcontent_scan_exempt_globsand added a repo policy to exempt synthetic secret fixtures/tests. - Added/updated GitHub Actions workflows for PR QA slices, nightly simulations, and release checks.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_secret_detection.py | New regression tests for content-based private pattern detection using fixtures + inline cases. |
| tests/test_public_private.py | Adds regression test ensuring content-scan exemptions bypass secret detection. |
| tests/test_contention.py | New concurrency/contention simulations for claims, stealing, and heavy weave appends. |
| tests/fixtures/secrets/private_key.pem | Secret-like fixture to exercise private-key detection. |
| tests/fixtures/secrets/pricing_doc.md | Business-sensitive fixture to exercise pricing detection. |
| tests/fixtures/secrets/ghp_token.py | Secret-like fixture to exercise GitHub token detection. |
| tests/fixtures/secrets/edge_ghp_in_comment.py | Secret-like fixture to ensure comment content is scanned. |
| tests/fixtures/secrets/clean_public.py | Clean control fixture. |
| tests/fixtures/secrets/aws_key.py | Secret-like fixture to exercise AWS key detection. |
| src/agentmesh/public_private.py | Adds content_scan_exempt_globs to skip content scanning for matching paths. |
| .github/workflows/release-agentmesh-check.yml | New release-time build/install/CLI verification + full test run. |
| .github/workflows/pr-agentmesh-qa.yml | New PR workflow running contention + secret-detection slices. |
| .github/workflows/nightly-agentmesh-simulations.yml | New nightly workflow running full suite (matrix) + contention slice. |
| .agentmesh/policy.json | Adds repo policy exemptions so synthetic fixtures/tests don’t trip the public-private guard. |
Comments suppressed due to low confidence (1)
src/agentmesh/public_private.py:136
- When
content_scan_exempt_globsmatches, content scanning is silently skipped, so callers can't tell whether a file was deemed PUBLIC/REVIEW because it was clean vs because scanning was bypassed. Consider appending an explicit reason (e.g., that content scan was skipped due to an exemption) to improve auditability/debugging of classification results.
content_scan_exempt = _policy_list(cfg.get("content_scan_exempt_globs"))
rel = _rel_path(path, repo_root)
reasons: list[str] = []
if _has_match(rel, private_globs):
reasons.append("path matches private pattern")
content_marker = None
if path.exists() and path.is_file() and not _has_match(rel, content_scan_exempt):
try:
text = path.read_text(errors="ignore")
except OSError:
text = ""
content_marker = _content_has_private_marker(text, private_patterns)
if content_marker:
reasons.append(f"content matches private pattern: {content_marker}")
You can also share your feedback on Copilot code review. Take the survey.
| @@ -0,0 +1,2 @@ | |||
| # This file contains a leaked GitHub PAT for testing secret detection. | |||
| API_TOKEN = "ghp_R8x2mN4vL6pQ9wK1jT3yF5bA7cE0hU2sG4nM" | |||
| @@ -0,0 +1,2 @@ | |||
| # This file contains a leaked AWS access key for testing secret detection. | |||
| AWS_ACCESS_KEY_ID = "AKIAIOSFODNN7EXAMPLE" | |||
Comment on lines
+1
to
+4
| -----BEGIN RSA PRIVATE KEY----- | ||
| MIIBogIBAAJBALRiMLAHudeSA/x3hB2f+2NRkJLA/FAKEFAKEFAKEFAKEFAKE | ||
| FAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKE1234 | ||
| -----END RSA PRIVATE KEY----- |
|
|
||
| on: | ||
| release: | ||
| types: [created] |
|
|
||
|
|
||
| def do_work() -> None: | ||
| # TODO: remove this token ghp_A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6 |
Comment on lines
+122
to
+124
| (repo / "tests" / "fixtures" / "secrets" / "token.py").write_text( | ||
| 'API_TOKEN = "ghp_R8x2mN4vL6pQ9wK1jT3yF5bA7cE0hU2sG4nM"\n' | ||
| ) |
Comment on lines
+4
to
+6
| "tests/fixtures/secrets/**", | ||
| "tests/test_secret_detection.py", | ||
| "tests/test_public_private.py" |
Add multi-agent contention simulation tests, secret-detection fixture corpus, classifier content_scan_exempt_globs policy, exemption regression test, and PR/nightly/release CI workflows. Signed-off-by: Timothy Haserjian <timmymacbookpro@users.noreply.github.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> AgentMesh-Episode: ep_019cf5ac9f767b127bf41c8c AgentMesh-KeyID: mesh_a08cfb329abb0105 AgentMesh-Witness: sha256:5d30fa11c27ca2ae719768120593615498f009d0890217bd10cf2cfbe05c7a2e AgentMesh-Sig: UsLr28bZoUijQiIicOS50REK7FaY9FeUrwG4SbJhTbOcTYQ1fUsRsTfYJ-7Ed9shCDPpz6YcWPBNwMzCwR4aCw== AgentMesh-Witness-Encoding: gzip+base64url AgentMesh-Witness-Chunk-Count: 4 AgentMesh-Witness-Chunk: H4sIAIu5t2kC_0WST4_TMBDFv4uvbKux47HjSFw4gBBCC2KREJdoPB43YdOkatJ2YdXvjtPVwnX-_N6bZz8r2sm4tH1SjeKBTklayBBzQFZ3Sg79PJXSrS2HFnTgjMQhe-ejNj5mq7leR3M_yNzydBoX1Wj7Wuho7sru3JFB1zAKa5eM8UQx AgentMesh-Witness-Chunk: UMrJhsBcY6icqRxFWxVpXTEGMeKwLnz0tsaY0Xtjis6BFu5u2PYsx0hLv__Pz5TqGouK1eSyRkcFgylmnaniyD4k7Tix1SaRN8EmFk8OWYM4i-4fv0_tvFAcpLBRB6QALkWIwmIJEkUo3iVjqDVrDbXLt7xm7mRPq7G5n8Y10ou0Z712-t0o AgentMesh-Witness-Chunk: R9U8Kxp207FfutW2JIMFX_qP8vsl5b2UywhqzrEygWIEDbj6OsWh57bMlaHTt8dlnNpfPz7h-PlheTpfLvdff8of9-XD00cNm-H-4N6lMLz_Dm_V9U6VkKTcsz-UXQPGbaDaaPcAdQO2MX4L5QW8fQPQABStZZqG1yPmm-f1k6zOyj5swxbU AgentMesh-Witness-Chunk: 9foXEOkULzsCAAA=
6f1deb0 to
b41e790
Compare
AgentMesh Lineage CheckLineage coverage: 1/1 commits (100%)
Commit Details
Witness Verification Details
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
content_scan_exempt_globsto classifier policy so synthetic test fixtures and inline-pattern test files don't self-block the PR public-private guardSupersedes #34 (missing
Signed-off-by+ incomplete classifier exemption). Supersedes #33 (mixed QA + bridge scope).Commits
7e6b027test_contention.py,test_secret_detection.py, 6 fixture files, 3 workflow YAMLsb29332dcontent_scan_exempt_globsinpublic_private.py+policy.jsonfor fixturesd07cda26f1deb0test_secret_detection.py,test_public_private.py)Test plan
Signed-off-bytrailer🤖 Generated with Claude Code