Skip to content

test(qa): add contention and secret-detection coverage with CI gates#35

Merged
Haserjian merged 1 commit intomainfrom
test/qa-contention-secret-ci-signed
Mar 16, 2026
Merged

test(qa): add contention and secret-detection coverage with CI gates#35
Haserjian merged 1 commit intomainfrom
test/qa-contention-secret-ci-signed

Conversation

@Haserjian
Copy link
Owner

Summary

  • Add multi-agent contention simulation tests: same-file claim race, port double-claim, steal-against-fresh-holder, heavy concurrent weave append (50 writers / 16 threads)
  • Add secret-detection fixture corpus (6 specimens covering all 7 default private patterns) with parameterized regression tests
  • Add content_scan_exempt_globs to classifier policy so synthetic test fixtures and inline-pattern test files don't self-block the PR public-private guard
  • Wire new QA slices into three CI workflows: PR gates, nightly simulations, release checks

Supersedes #34 (missing Signed-off-by + incomplete classifier exemption). Supersedes #33 (mixed QA + bridge scope).

Commits

Commit What
7e6b027 QA slice: test_contention.py, test_secret_detection.py, 6 fixture files, 3 workflow YAMLs
b29332d Classifier exemption: content_scan_exempt_globs in public_private.py + policy.json for fixtures
d07cda2 Regression test locking the exemption behavior
6f1deb0 Widen exemption to cover inline-pattern test files (test_secret_detection.py, test_public_private.py)

Test plan

  • 320 tests passing locally, 1 pre-existing skip
  • Contention tests: 4/4 passing (S1, S2, S4, S11)
  • Secret detection tests: 11/11 passing
  • Exemption regression test: 1/1 passing
  • Existing public-private tests: 8/8 passing
  • All commits include Signed-off-by trailer
  • Diff against main is QA-only (14 files, +517/-1, no bridge code)
  • CI: public-private-guard passes with widened exemption
  • CI: DCO check passes with sign-off trailers

🤖 Generated with Claude Code

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds QA coverage and CI hooks to validate AgentMesh’s coordination/provenance behavior under contention and to regression-test content-based private/secret detection (including a policy mechanism to exempt known synthetic fixtures from content scanning).

Changes:

  • Added new pytest suites for contention scenarios and secret/sensitive-content detection using fixtures and inline content.
  • Extended classify_path() policy to support content_scan_exempt_globs and added a repo policy to exempt synthetic secret fixtures/tests.
  • Added/updated GitHub Actions workflows for PR QA slices, nightly simulations, and release checks.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/test_secret_detection.py New regression tests for content-based private pattern detection using fixtures + inline cases.
tests/test_public_private.py Adds regression test ensuring content-scan exemptions bypass secret detection.
tests/test_contention.py New concurrency/contention simulations for claims, stealing, and heavy weave appends.
tests/fixtures/secrets/private_key.pem Secret-like fixture to exercise private-key detection.
tests/fixtures/secrets/pricing_doc.md Business-sensitive fixture to exercise pricing detection.
tests/fixtures/secrets/ghp_token.py Secret-like fixture to exercise GitHub token detection.
tests/fixtures/secrets/edge_ghp_in_comment.py Secret-like fixture to ensure comment content is scanned.
tests/fixtures/secrets/clean_public.py Clean control fixture.
tests/fixtures/secrets/aws_key.py Secret-like fixture to exercise AWS key detection.
src/agentmesh/public_private.py Adds content_scan_exempt_globs to skip content scanning for matching paths.
.github/workflows/release-agentmesh-check.yml New release-time build/install/CLI verification + full test run.
.github/workflows/pr-agentmesh-qa.yml New PR workflow running contention + secret-detection slices.
.github/workflows/nightly-agentmesh-simulations.yml New nightly workflow running full suite (matrix) + contention slice.
.agentmesh/policy.json Adds repo policy exemptions so synthetic fixtures/tests don’t trip the public-private guard.
Comments suppressed due to low confidence (1)

src/agentmesh/public_private.py:136

  • When content_scan_exempt_globs matches, content scanning is silently skipped, so callers can't tell whether a file was deemed PUBLIC/REVIEW because it was clean vs because scanning was bypassed. Consider appending an explicit reason (e.g., that content scan was skipped due to an exemption) to improve auditability/debugging of classification results.
    content_scan_exempt = _policy_list(cfg.get("content_scan_exempt_globs"))

    rel = _rel_path(path, repo_root)
    reasons: list[str] = []

    if _has_match(rel, private_globs):
        reasons.append("path matches private pattern")

    content_marker = None
    if path.exists() and path.is_file() and not _has_match(rel, content_scan_exempt):
        try:
            text = path.read_text(errors="ignore")
        except OSError:
            text = ""
        content_marker = _content_has_private_marker(text, private_patterns)
        if content_marker:
            reasons.append(f"content matches private pattern: {content_marker}")

You can also share your feedback on Copilot code review. Take the survey.

@@ -0,0 +1,2 @@
# This file contains a leaked GitHub PAT for testing secret detection.
API_TOKEN = "ghp_R8x2mN4vL6pQ9wK1jT3yF5bA7cE0hU2sG4nM"
@@ -0,0 +1,2 @@
# This file contains a leaked AWS access key for testing secret detection.
AWS_ACCESS_KEY_ID = "AKIAIOSFODNN7EXAMPLE"
Comment on lines +1 to +4
-----BEGIN RSA PRIVATE KEY-----
MIIBogIBAAJBALRiMLAHudeSA/x3hB2f+2NRkJLA/FAKEFAKEFAKEFAKEFAKE
FAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKEFAKE1234
-----END RSA PRIVATE KEY-----

on:
release:
types: [created]


def do_work() -> None:
# TODO: remove this token ghp_A1B2C3D4E5F6G7H8I9J0K1L2M3N4O5P6
Comment on lines +122 to +124
(repo / "tests" / "fixtures" / "secrets" / "token.py").write_text(
'API_TOKEN = "ghp_R8x2mN4vL6pQ9wK1jT3yF5bA7cE0hU2sG4nM"\n'
)
Comment on lines +4 to +6
"tests/fixtures/secrets/**",
"tests/test_secret_detection.py",
"tests/test_public_private.py"
Add multi-agent contention simulation tests, secret-detection fixture
corpus, classifier content_scan_exempt_globs policy, exemption regression
test, and PR/nightly/release CI workflows.

Signed-off-by: Timothy Haserjian <timmymacbookpro@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

AgentMesh-Episode: ep_019cf5ac9f767b127bf41c8c
AgentMesh-KeyID: mesh_a08cfb329abb0105
AgentMesh-Witness: sha256:5d30fa11c27ca2ae719768120593615498f009d0890217bd10cf2cfbe05c7a2e
AgentMesh-Sig: UsLr28bZoUijQiIicOS50REK7FaY9FeUrwG4SbJhTbOcTYQ1fUsRsTfYJ-7Ed9shCDPpz6YcWPBNwMzCwR4aCw==
AgentMesh-Witness-Encoding: gzip+base64url
AgentMesh-Witness-Chunk-Count: 4
AgentMesh-Witness-Chunk: H4sIAIu5t2kC_0WST4_TMBDFv4uvbKux47HjSFw4gBBCC2KREJdoPB43YdOkatJ2YdXvjtPVwnX-_N6bZz8r2sm4tH1SjeKBTklayBBzQFZ3Sg79PJXSrS2HFnTgjMQhe-ejNj5mq7leR3M_yNzydBoX1Wj7Wuho7sru3JFB1zAKa5eM8UQx
AgentMesh-Witness-Chunk: UMrJhsBcY6icqRxFWxVpXTEGMeKwLnz0tsaY0Xtjis6BFu5u2PYsx0hLv__Pz5TqGouK1eSyRkcFgylmnaniyD4k7Tix1SaRN8EmFk8OWYM4i-4fv0_tvFAcpLBRB6QALkWIwmIJEkUo3iVjqDVrDbXLt7xm7mRPq7G5n8Y10ou0Z712-t0o
AgentMesh-Witness-Chunk: R9U8Kxp207FfutW2JIMFX_qP8vsl5b2UywhqzrEygWIEDbj6OsWh57bMlaHTt8dlnNpfPz7h-PlheTpfLvdff8of9-XD00cNm-H-4N6lMLz_Dm_V9U6VkKTcsz-UXQPGbaDaaPcAdQO2MX4L5QW8fQPQABStZZqG1yPmm-f1k6zOyj5swxbU
AgentMesh-Witness-Chunk: 9foXEOkULzsCAAA=
@Haserjian Haserjian force-pushed the test/qa-contention-secret-ci-signed branch from 6f1deb0 to b41e790 Compare March 16, 2026 08:04
@github-actions
Copy link

AgentMesh Lineage Check

Lineage coverage: 1/1 commits (100%)
Witness coverage: 1/1 commits (100%)

Metric Value
Commits in PR 1
With episode trailer 1
Lineage coverage 100%
Unique episodes 1
Files changed 14
Witness trailers present 1
Witness signatures verified 1
Witness coverage 100%
Commit Details
SHA Message Episode
b41e790 test(qa): add contention and secret-detection coverage wi... ep_019cf5ac9f767b127bf41c8c
Witness Verification Details
SHA Witness Status
b41e790 VERIFIED

Checked by agentmesh-action | What is lineage coverage?

@Haserjian Haserjian merged commit 5da793a into main Mar 16, 2026
13 checks passed
@Haserjian Haserjian deleted the test/qa-contention-secret-ci-signed branch March 16, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants