[smoke-detector] 🔍 Smoke Test Investigation - Recurring Bash Syntax Error in Copilot Workflows

[github-actions]

🔍 Smoke Test Investigation - Run #6

Summary

CRITICAL RECURRING ISSUE: Bash syntax error blocking ALL Copilot smoke tests. This is the 2nd occurrence within 6 minutes, affecting both regular and firewall-enabled Copilot workflows.

Failure Details

• Run: 18810373316
• Workflow: Smoke Copilot Firewall
• Commit: b650b2a
• Trigger: schedule
• Duration: 53 seconds (failed immediately)
• Pattern ID: COPILOT_BASH_SHELL_SYNTAX_PARENTHESES
• First Occurrence: Run remove max-runs #65 (18810304059) at 2025-10-26T00:12:23Z
• This Occurrence: Run add cli flag to guard dropping a agentic workflow instructinos file #6 (18810373316) at 2025-10-26T00:18:44Z

Root Cause Analysis

The workflow compiler generates bash commands with unescaped parentheses in tool permission flags:

--allow-tool 'shell(cat)' --allow-tool 'shell(date)' --allow-tool 'shell(echo)' ...

When GitHub Actions executes this bash script, the parentheses are interpreted as subshell syntax rather than literal characters, causing:

/home/runner/work/_temp/7e4fde52-21b3-44da-8339-da15cdb339c9.sh: line 5: syntax error near unexpected token `('

Failed Jobs and Errors

Agent Job (FAILED)

• Error Location: Step "Execute GitHub Copilot CLI"
• Exit Code: 2 (bash syntax error)
• Time: Failed at 31 seconds

Primary Error:

syntax error near unexpected token `('

Command That Failed:

npx -y `@github/copilot`@0.0.351 \
  --allow-tool github \
  --allow-tool safeoutputs \
  --allow-tool 'shell(cat)' \
  --allow-tool 'shell(date)' \
  --allow-tool 'shell(echo)' \
  --allow-tool 'shell(grep)' \
  --allow-tool 'shell(head)' \
  --allow-tool 'shell(ls)' \
  --allow-tool 'shell(pwd)' \
  --allow-tool 'shell(sort)' \
  --allow-tool 'shell(tail)' \
  --allow-tool 'shell(uniq)' \
  --allow-tool 'shell(wc)' \
  --allow-tool 'shell(yq)' \
  --allow-all-paths \
  --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"

Downstream Jobs

• detection: skipped
• create_issue: skipped
• missing_tool: skipped

Investigation Findings

Key Observations

1. ✅ Activation job succeeded - workflow setup is correct
2. ❌ Agent job fails immediately (31s) - bash script cannot execute
3. ⚠️ No Copilot logs generated - the CLI never started
4. 🔁 Second occurrence - confirms systemic issue
5. 🌐 Affects all variants - both regular and firewall-enabled workflows

Technical Details

• Copilot CLI Version: 0.0.351
• Platform: ubuntu-latest
• Firewall Mode: Enabled (allowed domains configured)
• Shell Tools: 12 tools configured with shell(command) syntax

Error Pattern

The bash parser sees:

--allow-tool 'shell(cat)'
       ^          ^   ^
       |          |   |
   start quote    (   )  <-- Interpreted as subshell!

Instead of treating the entire string 'shell(cat)' as a literal argument, bash tries to execute cat in a subshell.

Recommended Actions

Immediate Fix (Choose One)

• Option 1: Escape parentheses in compiler

  --allow-tool 'shell\(cat\)'

  • Modify workflow compiler to escape special characters
  • Location: pkg/workflow/compiler.go or agent command generation

• Option 2: Change quoting strategy

  --allow-tool "shell(cat)"

  • Use double quotes instead of single quotes
  • Test to ensure proper escaping through YAML → GitHub Actions → Bash

• Option 3: Change tool naming syntax

  --allow-tool shell-cat
  --allow-tool shell_cat

  • Avoid special characters entirely
  • Cons: Breaking change requiring updates across codebase

Investigation Steps

1. Locate command generation code

   grep -r "allow-tool" pkg/workflow/

2. Add proper shell escaping

   • Find where --allow-tool 'shell(cat)' is constructed
   • Apply proper escaping for bash special characters: ( ) [ ] { } $ \ " '`

3. Add integration tests

   • Test workflow compilation with special characters in tool names
   • Validate generated bash syntax before execution
   • Add CI check that runs shellcheck on generated scripts

4. Test with all engines

   • Copilot (affected)
   • Claude (verify not affected)
   • Codex (verify not affected)

Prevention Strategies

1. Add shellcheck validation to workflow compilation pipeline
2. Create integration tests for special characters in tool names
3. Implement pre-execution bash validation in GitHub Actions
4. Document safe tool naming conventions
5. Add linter rules for detecting unescaped special characters
6. Run smoke tests immediately after any compilation changes

Historical Context

Similar Past Failures

This is the first time this specific pattern has been observed in the smoke test investigation history. However:

• First occurrence: 2025-10-26T00:12:23Z (run 18810304059, "Smoke Copilot")
• Second occurrence: 2025-10-26T00:18:44Z (run 18810373316, "Smoke Copilot Firewall")

The rapid recurrence (6 minutes apart) across different workflow variants indicates a recent change likely introduced by:

• Workflow compiler update
• Changes to shell tool permission syntax
• Updated Copilot CLI integration

Related Workflows

All Copilot-based workflows with shell tool permissions are affected:

• Smoke Copilot ❌
• Smoke Copilot Firewall ❌
• Any workflow using tools.bash with shell(command) syntax

Impact Assessment

Severity: 🔴 CRITICAL

• Blocks: All Copilot smoke tests
• Scope: Systemic workflow compilation issue
• Workaround: None (requires compiler fix)
• User Impact: Cannot validate Copilot integration
• CI/CD Impact: Smoke tests failing on every scheduled run

Files to Investigate

pkg/workflow/compiler.go        # Likely location of command generation
pkg/workflow/agent.go           # Agent execution setup
pkg/workflow/copilot.go         # Copilot-specific tool configuration
pkg/workflow/tools.go           # Tool permission handling

Related Issues

• None found (this is a new failure pattern)

---

Investigation completed by: Smoke Detector (automated investigator)
Investigation ID: 2025-10-26-18810373316
Pattern Database: Updated with 2nd occurrence
Investigation Report: Stored in cache-memory

🤖 Generated by Smoke Detector Investigation

AI generated by Smoke Detector - Smoke Test Failure Investigator

[pelikhan]

/q implement option 1

[github-actions]

Agentic Q triggered by this issue comment.

[github-actions]

✅ Option 1 Implemented

I've implemented the fix for the bash syntax error by escaping parentheses in the workflow compiler.

Root Cause Analysis

The issue was in the shellEscapeCommandString function. When it wrapped the entire Copilot command in double quotes, any single quotes inside became literal quote characters rather than quote delimiters. This meant the parentheses in 'shell(cat)' were not protected and bash interpreted them as subshell syntax.

The Fix

Modified pkg/workflow/shell.go to escape parentheses when wrapping commands in double quotes:

// Escape parentheses (to prevent subshell interpretation inside double quotes)
escaped = strings.ReplaceAll(escaped, "(", "\\(")
escaped = strings.ReplaceAll(escaped, ")", "\\)")

Before:

"npx `@github/copilot` --allow-tool 'shell(cat)' --allow-tool 'shell(date)' ..."

After:

"npx `@github/copilot` --allow-tool 'shell\\(cat\\)' --allow-tool 'shell\\(date\\)' ..."

Files Modified

1. pkg/workflow/shell.go - Added parentheses escaping
2. pkg/workflow/shell_test.go - Updated 3 test cases to expect escaped parentheses

Pull Request

Created PR with the fix. The changes are minimal and surgical, only affecting the shell escaping logic used when wrapping commands for AWF (firewall mode).

This should resolve all Copilot smoke test failures related to bash syntax errors with shell tool permissions.

Related Items

• Pull Request: #2493

AI generated by Q for #2485