Skip to content

Improve jobs control plane and status API#4

Merged
kraftaa merged 9 commits intomainfrom
streaming
Apr 4, 2026
Merged

Improve jobs control plane and status API#4
kraftaa merged 9 commits intomainfrom
streaming

Conversation

@kraftaa
Copy link
Copy Markdown
Owner

@kraftaa kraftaa commented Mar 24, 2026

Summary

  • Add control-plane schema and CLI for scheduled sync jobs:
    • init-jobs, add-job, worker, force-job, reset-state
  • Add status API and UI:
    • JSON + HTML views, status filter, auto-refresh
    • job actions in UI: force run, retry failed, reset state
    • endpoints: /jobs, /jobs/html, /jobs/summary, /logs, /health, /health/worker
  • Record job runs with extra metadata:
    • severity, metrics_json, duration_ms
    • optional DQ hook via RUSTREAM_DQ_CMD
  • Improve reliability and migration behavior:
    • schema backfill on startup for control DB columns
    • invalid saved watermark/cursor state is detected and reset
    • --reset-state / reset-state --table ... for explicit recovery
  • Production template skeletons:
    • examples/production-template (docker-compose, Helm, Terraform stub)
    • GHCR Docker publish workflow

Testing

  • cargo fmt -- --check
  • cargo test
  • cargo clippy --all-targets -- -D warnings
  • Manual smoke:
    • init-jobs, add-job, worker, status-api
    • verified /jobs, /jobs/html, /logs, /health, /health/worker
    • verified UI actions (force/retry/reset)

@kraftaa kraftaa self-assigned this Mar 24, 2026
@kraftaa kraftaa merged commit 9e12522 into main Apr 4, 2026
1 check passed
@kraftaa kraftaa deleted the streaming branch April 4, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant