feat(fn-42): Quickstart documentation and demo infrastructure refactor #110
Merged
feat(fn-42): Quickstart documentation and demo infrastructure refactor #110
Conversation
- Create infra/Dockerfile.backend with multi-stage build - Use uv for fast dependency installation - Run as non-root user (dataing) for security - Include migrations at /app/migrations - Health check on /health endpoint - Default CMD starts API server, worker via command override Image size ~2GB due to ML dependencies (sentence-transformers, torch) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create infra/Dockerfile.frontend with Vite build + nginx serving - Add infra/nginx.conf with SPA fallback, API proxy, gzip compression - Uses runtime DNS resolution for API upstream (works standalone) - Image size: 93.4MB - Includes security headers and health check endpoint Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create docker-compose.yml with 7 services: postgres, redis, temporal,
temporal-ui, api, worker, frontend
- Add infra/init-temporal-db.sh for Temporal database initialization
- Use pgvector/pgvector:pg16 for shared PostgreSQL
- Configure Temporal auto-setup with postgres12 driver
- Wire environment variables with ${VAR:-default} pattern
- Add health checks for postgres, redis, and temporal
- Include observability profile for optional Jaeger/OTEL
Services verified:
- postgres, redis, temporal, temporal-ui start successfully
- Health checks pass
- Temporal uses shared PostgreSQL with separate databases
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…fn-41.4) - Add health checks to all services: - temporal-ui: wget spider check on port 8080 - frontend: wget check on /nginx-health endpoint - worker: Python Temporal client connection check - Enable Redis AOF persistence with --appendonly yes - Rename volumes to dataing-pgdata and dataing-redis - Create infra/healthcheck-worker.sh for reference All services now have health checks with appropriate intervals, timeouts, and start periods for proper startup sequencing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create .env.example with all documented environment variables - Add infra/check-env.sh for startup validation - Add env_file directive to api and worker services - Update .gitignore to track .env.example Variables organized by category: - Required: ANTHROPIC_API_KEY, DATADR_ENCRYPTION_KEY - Database: POSTGRES_PASSWORD - Temporal: TEMPORAL_NAMESPACE, TEMPORAL_TASK_QUEUE - API: LLM_MODEL - Optional: observability settings Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create infra/smoke-test.sh for automated health verification - Fix health checks to use 127.0.0.1 instead of localhost - Cross-platform compatible (works on macOS without timeout command) Verified on macOS Apple Silicon (arm64): - All 7 services start and become healthy - Total idle memory: ~595MB - api: 170MB, worker: 187MB, postgres: 116MB - temporal: 94MB, frontend: 9MB, temporal-ui: 9MB, redis: 9MB - Data persistence across restarts verified - Clean volume removal with -v verified - All smoke tests pass Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace outdated Arq/Redis worker instructions with Docker Compose - Add dual-path presentation (Docker vs pip) using MkDocs tabs - Include demo datasource connection with pre-seeded anomalies - Add verification commands with expected output - Streamlined to 5-minute flow Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Step 4-6: trigger, watch, and view investigation results - Include curl and Python SDK examples with tabbed content - Show expected JSON responses at each step - Add "What just happened?" explainer for investigation phases - Mention Temporal UI for workflow visualization Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- API key not set, Docker daemon, port conflicts - Temporal startup timing, memory requirements - Database migrations, stuck investigations - Platform-specific notes for macOS/Linux/Windows - Getting Help section with support links Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create package-quickstart-fixtures.sh to build tarball - Include baseline, null_spike, volume_drop scenarios - Exclude events.parquet (18MB each) to keep tarball under 5MB - Create quickstart-load.sql for standalone use - Tarball is build artifact (not committed), generates to ~3.1MB Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create docs/test-quickstart.sh for automated E2E validation - Fix broken link to self-host guide in quickstart.md - Script tests: env setup, stack startup, health check, investigation trigger Note: Full E2E validation requires ANTHROPIC_API_KEY and running Docker stack Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add db-migrate init service to run SQL migrations before API starts - Add DATADR_DEMO_MODE=true to seed demo data (tenant, API key, datasource) - Create infra/init-app-db.sh migration runner script - API and worker now depend on db-migrate completing successfully This fixes the "api_keys table does not exist" error on fresh Docker starts. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The investigation API requires expected_value, actual_value, deviation_pct, and anomaly_date fields. Update: - docs/docs/quickstart.md curl and Python examples - docs/test-quickstart.sh E2E test script Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move observability configs to infra/ (otel-collector, jaeger-ui) - Create root docker-compose.demo.yml override with pg_duckdb service - Add INCLUDE_SEEDS env var to control seed migrations - Add tenant to seed migration for datasource foreign key - Register capture_snapshot activity in temporal worker - Simplify justfile demo recipes to use compose - Update demo README with new workflow instructions Demo workflow: 1. just demo (starts all services including pg_duckdb on port 5433) 2. Login: demo@dataing.io / demo123456 3. Add datasource: Type=PostgreSQL, Host=duckdb, Port=5432, DB=demo Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Keep demo-related files together in the demo/ directory. Update justfile and volume paths accordingly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements fn-42 (Quickstart Documentation) with a complete refactor of the demo infrastructure for a cleaner separation of concerns.
Changes
Demo Infrastructure Refactor
docker-compose.demo.ymlintodemo/directoryotel-collector-config.yaml,jaeger-ui-config.json) toinfra/INCLUDE_SEEDSenv var toinit-app-db.shto control seed migrationsDuckDB Integration
init-pgduckdb.sqlto load demo fixtures on startupBug Fixes
capture_snapshotactivity to temporal worker (was missing)Documentation
demo/README.mdwith clear demo workflow instructionsDemo Workflow