feat: migrate from MySQL to Postgres with pgschema by alecthomas · Pull Request #114 · block/sprout

alecthomas · 2026-03-19T03:34:12Z

Up to you if you want to take this one, but I thought I'd throw it out there.

Replace MySQL 8.0 with Postgres 17 across the entire stack:

Add declarative schema/schema.sql for pgschema (merged from 21 migrations)
Delete migrations/ directory (sqlx migrations no longer used)
Update all sprout-db queries: positional $N params, ON CONFLICT,
string_agg, array_position, encode(), pg_catalog partitions
Update sprout-audit: pg_advisory_lock replaces GET_LOCK
Update docker-compose, .env.example, justfile, CI, and scripts
Cargo.toml: sqlx mysql feature → postgres

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
Ai-assisted: true

Replace MySQL 8.0 with Postgres 17 across the entire stack: - Add declarative schema/schema.sql for pgschema (merged from 21 migrations) - Delete migrations/ directory (sqlx migrations no longer used) - Update all sprout-db queries: positional $N params, ON CONFLICT, string_agg, array_position, encode(), pg_catalog partitions - Update sprout-audit: pg_advisory_lock replaces GET_LOCK - Update docker-compose, .env.example, justfile, CI, and scripts - Cargo.toml: sqlx mysql feature → postgres Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Ai-assisted: true

The relay no longer runs migrations itself — pgschema must be invoked explicitly after Postgres is healthy and before the relay starts. Co-authored-by: Claude Code <noreply@anthropic.com> Ai-assisted: true

…postgres * origin/main: fix(mcp): send threaded replies via WebSocket to prevent self-mention echo (#115)

tlongwell-block · 2026-03-19T14:36:21Z

Fixed the two runtime bugs that were failing CI:

UUID columns: sqlx maps Vec<u8> to BYTEA, not Postgres UUID. Replaced all byte-based UUID handling with native uuid::Uuid across sprout-db.
Enum columns: sqlx String doesn't match custom Postgres enum OIDs. Added ::text casts to all enum columns in SELECT statements.

Also added events_p_past and delivery_log_p_past partitions for historical timestamps, removed the dead uuid_from_bytes helper, and cleaned up remaining MySQL references.

17 files, net -105 lines. All quality gates green.

…s, partitions The MySQL→Postgres migration (688846e) introduced two runtime type-mapping bugs that broke the relay on first DB query, failing CI's Desktop E2E Integration job. Bug 1 — UUID columns decoded as Vec<u8>: sqlx maps Vec<u8> to BYTEA, not Postgres native UUID. Every query touching a UUID column (channels.id, workflows.id, channel_members. channel_id, etc.) crashed with ColumnDecode at runtime. Fixed by reading/writing UUID columns as uuid::Uuid directly and removing the now-unnecessary uuid_from_bytes helper. Bug 2 — Postgres custom enums decoded as String: sqlx's String type is only compatible with TEXT/VARCHAR, not custom enum OIDs (channel_type, channel_visibility, member_role, workflow_ status, run_status, approval_status, channel_add_policy). Fixed by adding ::text casts to all enum columns in SELECT statements. Also: - Add events_p_past and delivery_log_p_past partitions (MINVALUE) so historical timestamps don't fail with 'no partition found' - Clean up all remaining MySQL references in code, comments, and docs - Update TESTING.md for Postgres (pgschema, psql, container names) - Remove dead uuid_from_bytes helper and all orphaned imports 18 files changed. All quality gates green: cargo check (0 warnings), clippy -D warnings, fmt, 142 unit tests pass.

Postgres requires ORDER BY expressions to appear in the SELECT list when using SELECT DISTINCT. The LEFT JOIN on channel_members with a specific pubkey bind can't produce duplicates (PK is channel_id+pubkey), so DISTINCT was unnecessary.

Postgres won't auto-cast text bind parameters to custom enum types (only string literals get auto-cast). Add explicit casts: - $N::channel_type, $N::channel_visibility in channel INSERT - $N::member_role in channel_members INSERT - $N::workflow_status, $N::run_status, $N::approval_status in workflow UPDATEs - $N::channel_add_policy in user UPDATE Found via live relay testing — channel creation was returning 500.

tlongwell-block · 2026-03-19T15:42:03Z

Three additional fixes from live testing:

SELECT DISTINCT + ORDER BY — Postgres requires ORDER BY expressions in the SELECT list when using DISTINCT. The LEFT JOIN on channel_members(channel_id, pubkey) can't produce duplicates, so DISTINCT was unnecessary. Removed it.
Bound enum params — Postgres won't auto-cast text bind parameters to custom enum types (only string literals get auto-cast). Added explicit $N::enum_type casts on all INSERT/UPDATE statements that bind enum values.
Docs — Fixed stale expected output in TESTING.md (channel_add_policy is an enum, not text) and updated a comment that still referenced DISTINCT.

Live-tested with three ACP-harnessed agents (Alice, Bob, Charlie) against a fresh Postgres instance. Channel creation, membership, DMs, mentions, threads, fan-out — all working. 110 events, 6 channels, 4 users in the DB.

MySQL: 'Duplicate entry' / error 1062 Postgres: 'duplicate key value' / error 23505 Found via e2e_rest_api test_nip05_duplicate_handle_conflict.

Postgres won't auto-cast text bind parameters to jsonb (only string literals get auto-cast). The workflow definition is passed as &str and bound to a JSONB column — add ::jsonb casts on INSERT and UPDATE. Found via e2e_workflows integration tests.

tlongwell-block · 2026-03-19T16:08:11Z

Full e2e integration test results against a live Postgres relay:

Suite	Result
REST API (`e2e_rest_api`)	51 passed, 0 failed
WebSocket relay (`e2e_relay`)	27 passed, 0 failed
Workflows (`e2e_workflows`)	6 passed, 1 failed (pre-existing on main)
MCP server (`e2e_mcp`)	7 passed, 7 failed (pre-existing on main)

The 1 workflow failure (test_approval_gate_stub_fails_gracefully) and 7 MCP failures are all pre-existing on origin/main — confirmed by checking out main and running the same tests against the same relay. Not introduced by this PR.

Latest fix: workflow definition column is JSONB but was bound as text — added ::jsonb cast. Same class of bug as the enum casts (Postgres won't auto-cast text bind params to jsonb). Also fixed MySQL-specific unique constraint error strings (Duplicate entry/1062 → duplicate key value/23505).

…el to steps - The API returns 'error_message' not 'error' for workflow run failures - send_message steps need an explicit 'channel' when trigger has no channel context (webhook)

tlongwell-block · 2026-03-19T16:21:17Z

All 99 e2e integration tests passing on Postgres:

Suite	Result
REST API	51/51 ✅
WebSocket relay	27/27 ✅
Workflows	7/7 ✅
MCP server	14/14 ✅

Also fixed the 8 pre-existing test failures (stale tool names from PR #115, wrong error field name in workflow approval test). Those were test bugs, not relay bugs.

alecthomas · 2026-03-19T19:56:47Z

🎉

alecthomas and others added 3 commits March 19, 2026 14:32

fix: add pgschema apply step to CI before relay start

e533dc9

The relay no longer runs migrations itself — pgschema must be invoked explicitly after Postgres is healthy and before the relay starts. Co-authored-by: Claude Code <noreply@anthropic.com> Ai-assisted: true

Merge remote-tracking branch 'origin/main' into aat/migrate-mysql-to-…

12f1ab4

…postgres * origin/main: fix(mcp): send threaded replies via WebSocket to prevent self-mention echo (#115)

tlongwell-block force-pushed the aat/migrate-mysql-to-postgres branch from 9f00160 to ffd2b89 Compare March 19, 2026 14:40

tlongwell-block added 3 commits March 19, 2026 10:43

docs: fix stale DISTINCT comment and TESTING.md enum type

f3910e4

style: rustfmt

69f6a5d

tlongwell-block marked this pull request as ready for review March 19, 2026 15:48

tlongwell-block requested a review from wesbillman as a code owner March 19, 2026 15:48

tlongwell-block added 2 commits March 19, 2026 11:56

fix: use Postgres error strings for unique constraint violations

0073a96

MySQL: 'Duplicate entry' / error 1062 Postgres: 'duplicate key value' / error 23505 Found via e2e_rest_api test_nip05_duplicate_handle_conflict.

tlongwell-block added 2 commits March 19, 2026 12:17

test: update MCP e2e tests for renamed tools (PR #115 compat)

613263a

test: fix workflow approval test — use error_message field, add chann…

cfe9f8d

…el to steps - The API returns 'error_message' not 'error' for workflow run failures - send_message steps need an explicit 'channel' when trigger has no channel context (webhook)

style: rustfmt test files

58fdb0f

tlongwell-block merged commit 6093fc6 into main Mar 19, 2026
8 checks passed

tlongwell-block deleted the aat/migrate-mysql-to-postgres branch March 19, 2026 16:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: migrate from MySQL to Postgres with pgschema#114

feat: migrate from MySQL to Postgres with pgschema#114
tlongwell-block merged 13 commits intomainfrom
aat/migrate-mysql-to-postgres

alecthomas commented Mar 19, 2026 •

edited

Loading

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

Uh oh!

alecthomas commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

alecthomas commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

tlongwell-block commented Mar 19, 2026

Uh oh!

Uh oh!

alecthomas commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alecthomas commented Mar 19, 2026 •

edited

Loading