Skip to content

Implement logging strategy for dev and production #164

@vrknetha

Description

@vrknetha

Context

Full logging strategy doc: docs/logging-strategy.md

Currently both proxy and registry log 100% of everything (head_sampling_rate: 1, invocation_logs: true, all log levels). This works for dev but will burn through Cloudflare's free 10M events/month in production.

Problem

With 1,000 active agents at ~50 messages/agent/day:

  • Current: ~300K events/day → 9M/month (hitting free plan limit)
  • After strategy: ~700 events/day → 21K/month (99.7% reduction)

Changes Required

1. Add min log level to createLogger (@clawdentity/sdk)

Add a minLevel parameter that suppresses lower-priority logs:

  • debug (0) < info (1) < warn (2) < error (3)
  • Dev: debug (everything)
  • Production: warn (only failures)

This kills the highest-volume info logs in production:

  • proxy.auth.verified (every request)
  • proxy.hooks.agent.delivered_to_relay (every message)
  • proxy.relay.connect (every WS connect)
  • request.completed (every request)

2. Conditional request logging middleware

Add onlyErrors option to createRequestLoggingMiddleware:

  • Dev: log every request
  • Production: only log 4xx/5xx responses and slow requests (>3s)

3. Wrangler observability config per environment

Proxy production:

"observability": {
  "enabled": true,
  "logs": {
    "enabled": true,
    "invocation_logs": false,
    "head_sampling_rate": 1
  }
}

Key: invocation_logs: false in proxy production. Keep head_sampling_rate: 1 so errors aren't sampled away.

Registry: Keep full logging (low traffic service).

4. DO logging gap

AgentRelaySession has zero console.log calls. Add warn-level logs for:

  • Queue overflow (message dropped)
  • Retry budget exhausted
  • WebSocket delivery failures after all retries

Implementation Checklist

  • Add minLevel to createLogger in SDK
  • Pass ENVIRONMENT to logger creation in proxy/registry
  • Set proxy + registry production to warn level
  • Add onlyErrors to request logging middleware
  • Add per-env observability config in wrangler.jsonc
  • Disable invocation_logs for proxy production
  • Add warn-level logs inside AgentRelaySession DO

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions