Skip to content

Agent scenario: inspect/retry with real Hookdeck traffic #7

@leggetter

Description

@leggetter

Summary

Extend agent-scenario testing with a scenario that exercises the iterate workflow using real requests, events, and attempts in Hookdeck. Currently receive-webhooks uses a documentation-only Stage 04: the agent documents how to list request → event → attempt and retry; no live traffic is required.

Goal

A scenario (or variant) where:

  1. Traffic exists: After the agent sets up the app and hookdeck listen, at least one request is sent to the Source URL (e.g. via curl in the prompt or via test harness) so Hookdeck has real requests, events, and attempts.
  2. Agent runs CLI: The agent runs hookdeck gateway request list, event list, attempt list, and event retry (or equivalent) against that live data.
  3. Assessment: The assessor can optionally check that the run log contains actual command output (list/retry) rather than only written documentation.

Considerations

  • Credentials / project: Requires a real Hookdeck project and connection in the test environment (e.g. secrets for CI).
  • Failure for retry: To test retry meaningfully, there may need to be a failed attempt (e.g. temporarily wrong destination URL or down endpoint).
  • Flakiness: Live API and traffic add variability; may be better as a separate "integration" or "live" scenario run on-demand or weekly rather than on every PR.

References

  • receive-webhooks Stage 04: documentation-only iterate stage in scenarios.yaml, tools/agent-scenario-tester/src/assess.ts, TESTING.md.
  • Phase F content: skills/event-gateway/references/04-iterate.md, cli-workflows.md, monitoring-debugging.md.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions