Skip to content

Add CLI workflow E2E test (init → validate → audit → generate) #26

@skokaina

Description

@skokaina

Problem

The critical user journey `init → validate → health → audit → generate` has unit tests per command but no integration test that runs the full sequence. A breaking change in the manifest schema could break the entire flow without CI catching it.

Proposed fix

Add an E2E test (can be a shell script or vitest integration test) that:

  1. Runs `agentspec init --yes` in a temp directory
  2. Runs `agentspec validate agent.yaml` — asserts exit 0
  3. Runs `agentspec health agent.yaml --no-model --no-mcp --no-memory` — asserts exit 0
  4. Runs `agentspec audit agent.yaml` — asserts exit 0 and score > 0
  5. Runs `agentspec generate agent.yaml --framework langgraph --output ./out/ --dry-run` — asserts exit 0 (dry-run avoids needing ANTHROPIC_API_KEY in CI)

This test should run in CI on every PR.

Acceptance criteria

  • E2E test exists and passes locally
  • Runs in CI (no external API keys required)
  • Covers the happy path a new user follows from the quick-start

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1-credibilityFix before or shortly after launchenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions