-
Notifications
You must be signed in to change notification settings - Fork 2
Add CLI workflow E2E test (init → validate → audit → generate) #26
Copy link
Copy link
Open
Labels
P1-credibilityFix before or shortly after launchFix before or shortly after launchenhancementNew feature or requestNew feature or request
Description
Problem
The critical user journey `init → validate → health → audit → generate` has unit tests per command but no integration test that runs the full sequence. A breaking change in the manifest schema could break the entire flow without CI catching it.
Proposed fix
Add an E2E test (can be a shell script or vitest integration test) that:
- Runs `agentspec init --yes` in a temp directory
- Runs `agentspec validate agent.yaml` — asserts exit 0
- Runs `agentspec health agent.yaml --no-model --no-mcp --no-memory` — asserts exit 0
- Runs `agentspec audit agent.yaml` — asserts exit 0 and score > 0
- Runs `agentspec generate agent.yaml --framework langgraph --output ./out/ --dry-run` — asserts exit 0 (dry-run avoids needing ANTHROPIC_API_KEY in CI)
This test should run in CI on every PR.
Acceptance criteria
- E2E test exists and passes locally
- Runs in CI (no external API keys required)
- Covers the happy path a new user follows from the quick-start
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P1-credibilityFix before or shortly after launchFix before or shortly after launchenhancementNew feature or requestNew feature or request