Skip to content

Add DIFC_FILTERED event support to gateway log parsing#21681

Merged
pelikhan merged 5 commits intomainfrom
copilot/update-gateway-log-javascript
Mar 19, 2026
Merged

Add DIFC_FILTERED event support to gateway log parsing#21681
pelikhan merged 5 commits intomainfrom
copilot/update-gateway-log-javascript

Conversation

Copy link
Contributor

Copilot AI commented Mar 18, 2026

Summary

Adds support for DIFC_FILTERED events (DIFC integrity/secrecy check failures) across the gateway log parsing stack — both in the JavaScript step-summary renderer and the Go audit/logs reporters.

Changes

JavaScript (actions/setup/js/parse_mcp_gateway_log.cjs)

  • Added parseGatewayJsonlForDifcFiltered(jsonlContent) — parses gateway.jsonl line by line and extracts all DIFC_FILTERED entries
  • Added generateDifcFilteredSummary(filteredEvents) — renders a collapsible <details> Markdown table showing Server, Tool, Reason, User (login + association), and Resource (linked issue or description)
  • Updated main() to always read gateway.jsonl for DIFC events and append the section to the step summary, regardless of whether gateway.md or legacy log files are used
  • Updated exports

JavaScript tests (actions/setup/js/parse_mcp_gateway_log.test.cjs)

  • Tests for parseGatewayJsonlForDifcFiltered: extraction, no-match, empty input, malformed lines, blank lines
  • Tests for generateDifcFilteredSummary: empty/null inputs, table structure, field rendering (tool code format, server, reason, user, linked resource, pipe escaping), multi-event count

Go (pkg/cli/gateway_logs.go)

  • Extended GatewayLogEntry with DIFC-specific JSON fields: ServerID, Reason, SecrecyTags, IntegrityTags, AuthorAssociation, AuthorLogin, HTMLURL, Number, Description
  • New DifcFilteredEvent struct for structured filtered event records
  • Added TotalFiltered int and FilteredEvents []DifcFilteredEvent to GatewayMetrics
  • Added FilteredCount int to GatewayServerMetrics
  • processGatewayLogEntry now handles type == "DIFC_FILTERED" — increments counters and appends to FilteredEvents; also improved timestamp parsing to support RFC3339Nano
  • renderGatewayMetricsTable shows Total DIFC Filtered in summary, adds a Filtered column to the server table when events exist, and renders a dedicated DIFC Filtered Events table
  • displayAggregatedGatewayMetrics aggregates TotalFiltered, FilteredCount, and FilteredEvents across runs
  • extractMCPToolUsageData populates FilteredEvents from gateway metrics

Go (pkg/cli/audit_report.go)

  • Added FilteredEvents []DifcFilteredEvent to MCPToolUsageData (surfaced in gh aw audit JSON output)

Go (pkg/cli/logs_models.go)

  • Added FilteredEvents []DifcFilteredEvent to MCPToolUsageSummary

Go (pkg/cli/logs_report.go)

  • buildMCPToolUsageSummary now collects and aggregates FilteredEvents across all processed runs; updated nil-return guard to also check filtered events; updated log message

Security Summary

No new security vulnerabilities introduced. DIFC_FILTERED events are read-only log entries; no user input is executed or used in sensitive operations.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot download logs until you find a log with DIFC_FILTERED event; then try out the filtering. Print out the log run url in the generated comment.

…gateway.jsonl)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 19, 2026

@copilot download logs until you find a log with DIFC_FILTERED event; then try out the filtering. Print out the log run url in the generated comment.

Found run with DIFC_FILTERED events: https://github.com/github/gh-aw/actions/runs/23272536774

That run has 4 DIFC_FILTERED events in rpc-messages.jsonl (not gateway.jsonl — the gateway format uses rpc-messages.jsonl in practice):

github / pull_request_read — Resource 'pr:github/gh-aw#21582' has lower integrity than agent
requires. Agent would need to drop integrity tags [unapproved:all approved:all] to trust this resource.

Filtering worked correctly. The generated step summary section looks like:

🔒 DIFC Filtered Events (4)

The following tool calls were blocked by DIFC integrity or secrecy checks:

| Time | Server | Tool | Reason | User | Resource |
|------|--------|------|--------|------|----------|
| 2026-03-18 23:45:22Z | github | `pull_request_read` | Resource 'pr:github/gh-aw#21582' has lower integrity... | - | pr:github/gh-aw#21582 |
...

Two fixes were needed vs. the initial implementation (commit d5d5a6e):

  1. JavaScript: main() now falls back to rpc-messages.jsonl when gateway.jsonl is absent, since DIFC_FILTERED events are interleaved there
  2. Go (parseRPCMessages): Extended RPCMessageEntry with DIFC-specific fields and added a case entry.Type == "DIFC_FILTERED" branch so filtered events are captured when parsing rpc-messages.jsonl

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh auth status (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/actions/runs?per_page=5 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/23272536774
    • Triggering command: /usr/bin/gh gh api repos/{owner}/{repo}/actions/runs/23272536774 --jq {databaseId: .id, number: .run_number, url: .html_url, status: .status, conclusion: .conclusion, workflowName: .name, workflowPath: .path, createdAt: .created_at, startedAt: .run_started_at, updatedAt: .updated_at, event: .event, headBranch: .head_branch, (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow Smoke Copilot --limit 100 --created &gt;=2026-03-12T00:05:54Z (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow Smoke Copilot --limit 90 (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan pelikhan marked this pull request as ready for review March 19, 2026 00:23
Copilot AI review requested due to automatic review settings March 19, 2026 00:23
const trimmed = line.trim();
if (!trimmed) continue;
try {
const entry = JSON.parse(trimmed);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot do a quick string contains before parsing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 3edcb80. parseGatewayJsonlForDifcFiltered now short-circuits with !trimmed.includes("DIFC_FILTERED") before calling JSON.parse.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end support for DIFC_FILTERED events so integrity/secrecy blocks are surfaced in both GitHub Actions step summaries (JS) and CLI audit/log reporting (Go).

Changes:

  • Parse gateway.jsonl/rpc-messages.jsonl for DIFC_FILTERED events and render a dedicated markdown summary section in the Actions step summary.
  • Extend Go gateway metrics to count and retain structured DIFC filtered event records, including aggregation across runs.
  • Surface filtered events through CLI data models (MCPToolUsageData / MCPToolUsageSummary) so they appear in JSON outputs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
actions/setup/js/parse_mcp_gateway_log.cjs Adds JSONL scanning for DIFC_FILTERED and renders a collapsible markdown table in the step summary.
actions/setup/js/parse_mcp_gateway_log.test.cjs Adds unit tests for DIFC JSONL parsing and summary rendering.
pkg/cli/gateway_logs.go Extends gateway parsing/metrics to count and collect DIFC filtered events and render them in console tables.
pkg/cli/audit_report.go Surfaces filtered events in MCPToolUsageData JSON output for gh aw audit.
pkg/cli/logs_models.go Adds FilteredEvents to MCPToolUsageSummary model for aggregated logs output.
pkg/cli/logs_report.go Aggregates FilteredEvents across runs into the overall MCP tool usage summary.
Comments suppressed due to low confidence (1)

pkg/cli/gateway_logs.go:637

  • renderGatewayMetricsTable now conditionally adds a Filtered column and also renders a dedicated "DIFC Filtered Events" table, but existing tests for renderGatewayMetricsTable don’t exercise the TotalFiltered > 0 path. Consider adding/adjusting tests to assert the extra column/table appears (and is omitted when there are no filtered events).
		if hasFiltered {
			output.WriteString(console.RenderTable(console.TableConfig{
				Title:   "Server Usage",
				Headers: []string{"Server", "Requests", "Tool Calls", "Avg Time", "Errors", "Filtered"},
				Rows:    serverRows,
			}))

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

const tool = event.tool_name ? `\`${event.tool_name}\`` : "-";
const reason = (event.reason || "-").replace(/\n/g, " ").replace(/\|/g, "\\|");
const user = event.author_login ? `${event.author_login} (${event.author_association || "NONE"})` : "-";
const resource = event.html_url ? `[#${event.number}](${event.html_url})` : event.description || "-";
Comment on lines +123 to +127
core.summary.addRaw(gatewayMdContent);

// Append DIFC_FILTERED section if any events found
if (difcFilteredEvents.length > 0) {
const difcSummary = generateDifcFilteredSummary(difcFilteredEvents);
Comment on lines +126 to +130
ToolName string `json:"tool_name,omitempty"`
Description string `json:"description,omitempty"`
Reason string `json:"reason,omitempty"`
SecrecyTags []string `json:"secrecy_tags,omitempty"`
IntegrityTags []string `json:"integrity_tags,omitempty"`
Comment on lines +443 to +447
// Handle DIFC_FILTERED events
if entry.Type == "DIFC_FILTERED" {
metrics.TotalFiltered++
// DIFC_FILTERED events use server_id; fall back to server_name for compatibility
serverKey := entry.ServerID
Comment on lines +735 to +736
// Aggregate DIFC filtered events
allFilteredEvents = append(allFilteredEvents, pr.MCPToolUsage.FilteredEvents...)
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot review all comments

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 19, 2026

@copilot review all comments

Addressed all review comments in 77a8701:

  • JS [#undefined]: Resource link now uses filter(Boolean).pop() to safely extract a URL path segment when number is absent (also handles trailing-slash URLs correctly)
  • JS newline: gatewayMdContent is guaranteed to end with \n before the DIFC <details> block is appended
  • Go RPCMessageEntry: Added author_association, author_login, html_url, number fields and populate them in parseRPCMessages's DIFC_FILTERED branch — full parity with gateway.jsonl handling
  • Tests: Added TestProcessGatewayLogEntryDifcFiltered and TestParseRPCMessagesDifcFiltered in gateway_logs_test.go; added TestBuildMCPToolUsageSummaryFilteredEvents in logs_mcp_tool_usage_test.go

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name &#34;prettier&#34; --wriGOINSECURE node 64/bin/go tierignore scripts/**/*.js (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha prettier --write 64/bin/go --ignore-path .prettierignore --log-level=erronpx prettier --check &#39;**/*.cjs&#39; &#39;**/*.ts&#39; &#39;**/*.json&#39; --ignore-path ../../../.pr**/*.json sh -c &#34;prettier&#34; --wriGOSUMDB pkg/cli/actions_GOWORK (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build4130826668/b405/importcfg -pack /tmp/go-build4130826668/b405/_testmain.go env ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha prettier --write 64/bin/go !../../../pkg/wonpx --ignore-path ../../../.pretti--check go env ath ../../../.pr.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 0/x64/lib/node_modules/npm/node_-nolocalimports GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 0/x64/bin/node GOINSECURE GOMOD igFiles,SwigCXXF/home/REDACTED/work/gh-aw/gh-aw/.github/workflows/archie.md go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE rk/cnjHJRsC75-1TI1_m0Vn/ftDIZiDi2O2pqeyPdHW4 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x86_64/sh GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ub/workflows GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE n-dir/sh GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path &#34;prettier&#34; --wriGOINSECURE /bin/sh 64/bin/go tierignore sh /sh node /hom�� --write ../../../**/*.js**/*.json 64/bin/go --ignore-path ../../../.prettilist 64/bin/go go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 ../../../.pretticonfig modules/@npmcli/extensions.objectformat go /pre�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go /pre�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE bd034f0ad4deeaa2-d GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env re GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x86_64/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go .prettierignore --log-level=erro-c 64/bin/go node /hom�� --write ../../../**/*.jsGOWORK 64/bin/go --ignore-path ../../../.pretticonfig modules/@npmcli/extensions.objectformat go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go .prettierignore --log-level=erro-c 64/bin/go node /hom�� --write ../../../**/*.jsGOWORK 64/bin/go --ignore-path ../../../.pretticat-file 64/bin/go go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build4130826668/b399/cli.test /tmp/go-build4130826668/b399/cli.test -test.testlogfile=/tmp/go-build4130826668/b399/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true **/*.json --ignore-path ../../../.pretticd actions/setup/js &amp;&amp; npm run lint:cjs sh -c &#34;prettier&#34; --wriGOSUMDB go 64/bin/go rror GO111MODULE 64/bin/go go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name &#34;prettier&#34; --wriGOINSECURE prettier 64/bin/go scripts/**/*.js --ignore-path 64/bin/go sh -c &#34;prettier&#34; --wriGOSUMDB go 64/bin/go -json GO111MODULE 64/bin/go go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan pelikhan merged commit 1743e41 into main Mar 19, 2026
53 checks passed
@pelikhan pelikhan deleted the copilot/update-gateway-log-javascript branch March 19, 2026 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants