Agent/CLI-first discovery: steer agents toward CLI for querying and analytics

[leggetter]

Summary

When an agent (Claude) was asked "which connections had the most events?", it used the Metrics API directly instead of the CLI, even though the CLI would have solved the problem and is the better option for ad-hoc queries. This issue captures feedback on what would have made the agent look at the CLI first, so we can improve agent-skills and docs to steer agents and users toward the CLI for querying and analytics.

Context

• Query: "Which connections had the most events?" (e.g. in Hookdeck prod org default project)
• Actual behavior: Agent called the Metrics API to query event counts.
• Preferred behavior: Agent uses hookdeck gateway metrics events (and related commands) for this type of ad-hoc analytics query.

Feedback: What would make agents look at the CLI first?

1. Direct CLI documentation reference — The Event Gateway skill (SKILL.md) mentions the CLI for hookdeck listen and development workflows, but there isn’t an obvious reference to CLI commands for querying/analytics. A section like "Query events via CLI" or examples showing hookdeck gateway metrics (and related commands) with filters would be a clear signal.

2. CLI-first workflow guidance — The references read were focused on setup, listening, and API patterns. Guidance like "For querying event data and metrics, use the CLI commands first" or a reference such as references/cli-queries.md or references/analytics.md would point agents and users in the right direction.

3. Use case mapping — For "which connections had the most events" (an analytics/reporting query), the default assumption was "API = programmatic queries, CLI = development workflows". If the docs explicitly said the CLI is good for ad-hoc queries and analytics (not only dev workflows), that would shift behavior toward using the CLI for this kind of question.

4. CLI command reference — A quick reference showing commands like hookdeck gateway metrics events, hookdeck gateway connection list, etc. with filtering options would make it obvious the CLI can handle this type of query.

Scope

• In scope: agent-skills repo (event-gateway SKILL.md and references), and website docs (Metrics page, CLI reference) — any "when to use CLI vs API" or "querying/analytics" guidance.
• Out of scope: CLI binary changes (those stay in hookdeck-cli#223).

Possible improvements

• Add a "Query events and metrics via CLI" (or similar) callout in event-gateway SKILL.md that points to metrics and list commands for analytics-style questions.
• In monitoring-debugging or a dedicated reference, add explicit guidance: "For ad-hoc querying of event data and metrics, use the CLI first; use the API when you need programmatic or automated access."
• In SKILL.md or reference material, add a short use-case mapping: CLI for ad-hoc queries, analytics, and exploration; API for scripts and automation.
• Ensure the CLI reference (or a quick-reference table) in the skill includes hookdeck gateway metrics and filtering so agents discover it when considering "how do I get event counts per connection?"