Store `account_id` fields as BYTEA by aditya1702 · Pull Request #488 · stellar/wallet-backend

aditya1702 · 2026-02-03T22:15:45Z

What

This pull request converts storage of account_id fields from TEXT to BYTEA in the database. The change optimizes storage and query performance for Stellar addresses by storing them in their native 33-byte binary format (1 version byte + 32 raw key bytes) rather than as 56-character StrKey strings.

Changes:

Introduced AddressBytea and NullAddressBytea types with SQL Scanner/Valuer interfaces for encoding/decoding between StrKey strings and BYTEA storage
Updated database migrations to change account_id columns from TEXT to BYTEA in accounts, transactions_accounts, operations_accounts, and state_changes tables
Modified data access layer to convert addresses to/from BYTEA when inserting or querying
Updated tests to use valid Stellar addresses generated from keypairs instead of hardcoded placeholder strings

Why

Optimizes storage while not sacrificing query latency

Known limitations

N/A

Issue that this PR addresses

Closes #490 #491 #493

- Add StellarAddress type with sql.Scanner/driver.Valuer for automatic conversion between string addresses (G.../C...) and 33-byte BYTEA - Update accounts table: stellar_address TEXT -> BYTEA - Update transactions_accounts table: account_id TEXT -> BYTEA

- Use StellarAddress type for Get, Insert, Delete operations - Convert []string to [][]byte for BatchGetByIDs ANY() clause - Handle mixed BYTEA/VARCHAR in IsAccountFeeBumpEligible query

- Convert addresses to []byte for BatchInsert UNNEST($11::bytea[]) - Use raw []byte instead of pgtype.Text for BatchCopy

- Pass StellarAddress type which implements driver.Valuer for auto-conversion

- Test nil, valid, and error cases for both Scan and Value - Test roundtrip conversion preserves address

- Fix IsAccountFeeBumpEligible to use separate EXISTS checks instead of UNION (avoids type mismatch between BYTEA and VARCHAR columns) - Update test fixtures to use types.StellarAddress for BYTEA columns - Use types.StellarAddress for scanning account_id from transactions_accounts

This is the first step in converting the operations_accounts table to use BYTEA storage for account IDs, matching the pattern already used by transactions_accounts.

All account_id columns now use BYTEA storage format, so the string case handling for TEXT columns is no longer needed.

All account_id columns now use BYTEA format, so the conditional conversion logic is no longer needed. Always use AddressBytea.

Both operations_accounts and transactions_accounts now use BYTEA for account_id, so the field is no longer needed in the config.

Convert addresses to BYTEA format using AddressBytea.Value() when copying to operations_accounts, matching the transactions pattern.

Convert accountAddr to BYTEA format in HasOperationForAccount query since operations_accounts.account_id is now BYTEA.

- operations_test.go: Use AddressBytea type for scanning account_id - accounts_test.go: Convert address to BYTEA in INSERT statement - test_utils.go: Use AddressBytea directly (already has Value() method)

- Change SQL query to use UNNEST($9::bytea[]) instead of text - Convert addresses to []byte using AddressBytea.Value()

- Use AddressBytea type for scanning account_id in both BatchInsert and BatchCopy test verification - Use AddressBytea when inserting test accounts

Update migration to use BYTEA type for 7 account_id columns: - account_id - signer_account_id, spender_account_id - sponsored_account_id, sponsor_account_id - deployer_account_id, funder_account_id

Helper function to convert nullable address strings to bytes for BYTEA insert operations. Returns nil for invalid/empty addresses.

Update the StateChange struct to use AddressBytea type for account_id field to match the BYTEA database column type.

Convert 7 account_id columns from text arrays to bytea arrays: - account_id (required) uses AddressBytea.Value() - 6 nullable columns use pgtypeBytesFromNullStringAddress helper

Convert 7 account_id columns from pgtype.Text to raw []byte: - account_id (required) uses AddressBytea.Value() - 6 nullable columns use pgtypeBytesFromNullStringAddress helper

Convert accountAddress parameter to AddressBytea for BYTEA column query.

Copilot

Pull request overview

This pull request converts storage of account_id fields from TEXT to BYTEA in the database. The change optimizes storage and query performance for Stellar addresses by storing them in their native 33-byte binary format (1 version byte + 32 raw key bytes) rather than as 56-character StrKey strings.

Changes:

Introduced AddressBytea and NullAddressBytea types with SQL Scanner/Valuer interfaces for encoding/decoding between StrKey strings and BYTEA storage
Updated database migrations to change account_id columns from TEXT to BYTEA in accounts, transactions_accounts, operations_accounts, and state_changes tables
Modified data access layer to convert addresses to/from BYTEA when inserting or querying
Updated tests to use valid Stellar addresses generated from keypairs instead of hardcoded placeholder strings

Reviewed changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
internal/indexer/types/types.go	Added AddressBytea and NullAddressBytea types with Scan/Value implementations for BYTEA conversion
internal/indexer/types/types_test.go	Added comprehensive tests for AddressBytea Scan/Value/String methods and roundtrip conversion
internal/db/migrations/*.sql	Updated CREATE TABLE statements to use BYTEA for all account_id columns
internal/data/accounts.go	Updated queries to convert string addresses to BYTEA; refactored IsAccountFeeBumpEligible to use OR instead of UNION
internal/data/transactions.go	Added address-to-BYTEA conversion for transactions_accounts link table insertions
internal/data/operations.go	Added address-to-BYTEA conversion for operations_accounts link table insertions
internal/data/statechanges.go	Added conversion for account_id and nullable address fields to BYTEA
internal/data/query_utils.go	Added pgtypeBytesFromNullAddressBytea helper for nullable address conversions
internal/utils/sql.go	Added NullAddressBytea helper function for creating nullable address values
internal/services/ingest_test.go	Replaced hardcoded test addresses with valid addresses from keypairs
internal/serve/graphql/resolvers/*.go	Updated resolvers to convert AddressBytea to string for GraphQL responses
internal/indexer/*.go	Updated to handle AddressBytea type conversions
internal/integrationtests/infrastructure/backfill_helpers.go	Updated queries to use AddressBytea for account address parameters

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/services/ingest_test.go

internal/serve/graphql/resolvers/test_utils.go

internal/data/statechanges.go

internal/data/transactions.go

internal/data/operations.go

internal/data/transactions.go

internal/db/migrations/2024-04-29.1-accounts.sql

internal/data/accounts.go

internal/data/operations.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…let-backend into accountid-bytea-txns

aditya1702 added 28 commits February 3, 2026 17:02

Update accounts.go for BYTEA stellar_address column

fe40b54

- Use StellarAddress type for Get, Insert, Delete operations - Convert []string to [][]byte for BatchGetByIDs ANY() clause - Handle mixed BYTEA/VARCHAR in IsAccountFeeBumpEligible query

Update transactions.go for BYTEA account_id column

dee8de5

- Convert addresses to []byte for BatchInsert UNNEST($11::bytea[]) - Use raw []byte instead of pgtype.Text for BatchCopy

Update query_utils.go for BYTEA account_id conversion

12b192b

- Pass StellarAddress type which implements driver.Valuer for auto-conversion

Add tests for StellarAddress Scan/Value methods

a5df424

- Test nil, valid, and error cases for both Scan and Value - Test roundtrip conversion preserves address

Fix tests

da4c44e

StellarAddress -> AddressBytea

f23fd75

Change operations_accounts.account_id from TEXT to BYTEA

096a08e

This is the first step in converting the operations_accounts table to use BYTEA storage for account IDs, matching the pattern already used by transactions_accounts.

Simplify AddressBytea.Scan() to only handle BYTEA

705d2a4

All account_id columns now use BYTEA storage format, so the string case handling for TEXT columns is no longer needed.

Remove AccountIDBytea from query_utils.go

dacec16

All account_id columns now use BYTEA format, so the conditional conversion logic is no longer needed. Always use AddressBytea.

Remove AccountIDBytea field from operations.go and transactions.go

8487254

Both operations_accounts and transactions_accounts now use BYTEA for account_id, so the field is no longer needed in the config.

Update BatchCopy to write account_id as BYTEA

fa60b23

Convert addresses to BYTEA format using AddressBytea.Value() when copying to operations_accounts, matching the transactions pattern.

Update backfill_helpers.go to use BYTEA for account_id

ac3b07e

Convert accountAddr to BYTEA format in HasOperationForAccount query since operations_accounts.account_id is now BYTEA.

Update test files for BYTEA operations_accounts.account_id

331461b

- operations_test.go: Use AddressBytea type for scanning account_id - accounts_test.go: Convert address to BYTEA in INSERT statement - test_utils.go: Use AddressBytea directly (already has Value() method)

Update BatchInsert to write account_id as BYTEA

01b06d3

- Change SQL query to use UNNEST($9::bytea[]) instead of text - Convert addresses to []byte using AddressBytea.Value()

Fix operations tests for BYTEA account_id

9021603

- Use AddressBytea type for scanning account_id in both BatchInsert and BatchCopy test verification - Use AddressBytea when inserting test accounts

Fix make check issues

db17bf3

Change state_changes account_id columns from TEXT to BYTEA

ba42d49

Update migration to use BYTEA type for 7 account_id columns: - account_id - signer_account_id, spender_account_id - sponsored_account_id, sponsor_account_id - deployer_account_id, funder_account_id

Add pgtypeBytesFromNullStringAddress helper for BYTEA address conversion

9c1e726

Helper function to convert nullable address strings to bytes for BYTEA insert operations. Returns nil for invalid/empty addresses.

Change StateChange.AccountID type from string to AddressBytea

5a10c3b

Update the StateChange struct to use AddressBytea type for account_id field to match the BYTEA database column type.

Update BatchInsert to write account_id columns as BYTEA

e209dc5

Convert 7 account_id columns from text arrays to bytea arrays: - account_id (required) uses AddressBytea.Value() - 6 nullable columns use pgtypeBytesFromNullStringAddress helper

Update BatchCopy to write account_id columns as BYTEA

c6c22e9

Convert 7 account_id columns from pgtype.Text to raw []byte: - account_id (required) uses AddressBytea.Value() - 6 nullable columns use pgtypeBytesFromNullStringAddress helper

Update BatchGetByAccountAddress to query BYTEA account_id

3bbc0a7

Convert accountAddress parameter to AddressBytea for BYTEA column query.

Fix all tests

e4199c3

Update backfill_helpers.go

4677df9

Update backfill_helpers.go

af12032

Use NullAddressBytea method for nullable fields of state changes

39e24ec

aditya1702 changed the title ~~accounts, transactions_accounts: Store account_id as BYTEA~~ Store account_id fields as BYTEA Feb 5, 2026

fix tests

6fb0b76

aditya1702 added 3 commits February 5, 2026 10:29

fix more tests

7c5089e

fix more tests again

4ad1ede

Update account_service_test.go

4d825e1

aditya1702 requested a review from Copilot February 5, 2026 15:50

Copilot started reviewing on behalf of aditya1702 February 5, 2026 15:51 View session

Copilot AI reviewed Feb 5, 2026

View reviewed changes

aditya1702 marked this pull request as ready for review February 11, 2026 17:56

aditya1702 and others added 6 commits February 20, 2026 12:09

Update internal/data/accounts.go

60f8f90

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fix comments

d6e4da0

Update internal/data/statechanges.go

bc7fa4f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/data/transactions.go

b5aa6df

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/data/operations.go

ac5376e

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'accountid-bytea-txns' of https://github.com/stellar/wal…

9b72b5b

…let-backend into accountid-bytea-txns

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Store `account_id` fields as BYTEA#488

Store `account_id` fields as BYTEA#488
aditya1702 wants to merge 38 commits intodb-indexesfrom
accountid-bytea-txns

aditya1702 commented Feb 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

Conversation

aditya1702 commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Known limitations

Issue that this PR addresses

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aditya1702 commented Feb 3, 2026 •

edited

Loading