`state_changes` table: Replace JSONB columns with typed columns by aditya1702 · Pull Request #476 · stellar/wallet-backend

aditya1702 · 2026-01-30T20:12:15Z

What

Refactors the state_changes table schema to replace JSONB columns with explicit typed columns for better query performance, type safety and data integrity.

Note that the API still returns a response as key-value JSON with old and new keys. Only the storage design has changed

Why

The previous schema used JSONB columns (flags, key_value, signer_weights, thresholds, trustline_limit, offer_id) for storing
variable structured data. This approach had several drawbacks:

Query Performance: JSONB fields require parsing and cannot leverage efficient PostgreSQL indexes for common query patterns
Type Safety: JSONB allows arbitrary data structures, making schema validation at the application layer necessary
Storage Efficiency: JSONB has higher storage overhead compared to native column types

This PR introduces explicit typed columns:

New Column(s)	Replaces	Type
`signer_weight_old`, `signer_weight_new`	`signer_weights` JSONB	SMALLINT
`threshold_old`, `threshold_new`	`thresholds` JSONB	SMALLINT
`trustline_limit_old`, `trustline_limit_new`	`trustline_limit` JSONB	TEXT
`flags`	`flags` JSON array	SMALLINT (bitmask)
`claimable_balance_id`, `liquidity_pool_id`, `sponsored_data`	Extracted from `key_value` JSONB	TEXT

Additionally:

Removes unused offer_id column
Retains key_value JSONB only for truly variable data (data entries, home domain) that cannot be flattened into a fixed schema

Known limitations

N/A

Issue that this PR addresses

Closes #477

…hanges

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ance

Split trustline operations from AccountTokensModel into dedicated model. - TrustlineBalance struct (renamed from Trustline) - TrustlineBalanceModelInterface with GetByAccount, BatchUpsert, BatchInsert - Updated table name to account_trustline_balances

Split contract operations from AccountTokensModel into dedicated model. - AccountContractTokensModelInterface with GetByAccount, BatchInsert - Manages account_contracts junction table

- Renamed account_trustlines to account_trustline_balances - Merged balance columns from separate migration into main table creation - Removed redundant migration file (2026-01-13.0-add-trustline-balance-fields.sql)

Replace AccountTokens with: - TrustlineBalance (TrustlineBalanceModelInterface) - AccountContractTokens (AccountContractTokensModelInterface)

- account_trustline_balances.sql for TrustlineBalanceModel - account_contract_tokens.sql for AccountContractTokensModel - Removed combined migration file

- Replace accountTokensModel with trustlineBalanceModel and accountContractTokensModel - Update trustlineBatch to use TrustlineBalance struct - Update all model method calls to use new interfaces

- Replace AccountTokensReader with BalanceReader + AccountContractTokensModelInterface - Create balanceReaderAdapter to wrap TrustlineBalanceModelInterface - Update queries.resolvers.go to use new interface methods - Update accountKeyInfo to use TrustlineBalance type

- Replace AccountTokensModel with TrustlineBalanceModel and AccountContractTokensModel - Use NewBalanceReader adapter for GraphQL resolver injection

Pass TrustlineBalance and AccountContractTokens models to NewTokenIngestionService

Copilot

Pull request overview

This PR refactors the state_changes table schema by replacing JSONB columns with explicit typed columns to improve query performance, type safety, and storage efficiency. The changes maintain backward compatibility at the API layer by formatting the typed columns back to JSON in GraphQL resolvers.

Changes:

Replaces JSONB/JSON columns (flags, signer_weights, thresholds, trustline_limit) with typed columns (SMALLINT for numeric values, TEXT for limits)
Extracts entity identifiers (claimable_balance_id, liquidity_pool_id, sponsored_data) from the key_value JSONB into dedicated columns
Implements bitmask encoding/decoding for flags instead of JSON arrays
Updates GraphQL resolvers to format typed values as JSON for backward compatibility
Removes the unused offer_id column
Fixes spelling of "authorized_to_maintain_liabilities" (was "authorized_to_maintain_liabilites" in several places)

Reviewed changes

Copilot reviewed 21 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
internal/db/migrations/2025-06-10.4-create_indexer_table_state_changes.sql	Defines new table schema with typed columns instead of JSONB
internal/indexer/types/types.go	Adds bitmask constants/helpers, removes NullableJSON, updates StateChange struct with typed fields
internal/indexer/processors/state_change_builder.go	Updates builder methods to accept typed values instead of JSONB maps
internal/indexer/processors/effects.go	Updates effects processor to use typed columns and new sponsorship field structure
internal/indexer/processors/effects_horizon.go	Fixes flag name spelling to "authorized_to_maintain_liabilities"
internal/indexer/processors/utils.go	Adds getContractIDFromAssetString helper for sponsorship asset conversion
internal/indexer/processors/contracts/sac.go	Fixes flag name spelling constant
internal/data/statechanges.go	Updates batch insert/copy logic for new column structure
internal/data/query_utils.go	Adds pgtypeInt2FromNullInt16 helper, removes jsonbFromSlice
internal/serve/graphql/schema/statechange.graphqls	Updates GraphQL schema to use new field names (removes keyValue, adds specific fields)
internal/serve/graphql/resolvers/statechange.resolvers.go	Implements resolvers that format typed values back to JSON for compatibility
internal/serve/graphql/resolvers/utils.go	Updates column mapping for old/new paired fields
internal/serve/graphql/generated/generated.go	Auto-generated code updates for schema changes
pkg/wbclient/types/types.go	Removes unused StateChange type
pkg/wbclient/types/statechange.go	Updates client types to match new GraphQL schema
pkg/wbclient/queries.go	Updates GraphQL queries to use new field names
internal/integrationtests/data_validation_test.go	Updates validation functions to use new typed fields
internal/serve/graphql/resolvers/statechange_resolvers_test.go	Updates unit tests for typed field resolvers
internal/indexer/processors/effects_test.go	Updates tests to use typed fields and bitmasks
internal/indexer/processors/contracts/sac_test.go	Updates tests to use bitmask values for flags
internal/data/statechanges_test.go	Updates test data generation with typed fields

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/indexer/processors/effects.go

internal/serve/graphql/resolvers/statechange.resolvers.go

internal/indexer/types/types.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

aditya1702 and others added 30 commits January 14, 2026 17:35

Use ContractBalanceFromContractData function to parse balanc eentry c…

360d38e

…hanges

Update token_transfer.go

57d0bf5

change TrustLines -> Trustlines

6c742eb

Create trustlines_test.go

9c84ccf

Update internal/data/account_tokens.go

ed819f2

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/data/account_tokens.go

93f7116

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/data/trustline_assets.go

afac0ed

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/db/migrations/2025-12-14.0-trustline_assets.sql

26e92fa

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'redis-to-postgres' into calculate-balances-using-changes

40ea015

Merge branch 'calculate-balances-using-changes' into store-native-bal…

a06b341

…ance

Update mocks.go

6e856dc

Remove tokenCacheReader interface

c8e54e9

rename files and remove unused method

f7389d1

Merge branch 'calculate-balances-using-changes' into store-native-bal…

9c5e47a

…ance

Add GetNativeBalance method

8201c58

insert native balances within same batch during checkpoint population

55394f8

Add AccountContractTokensModel for focused contract mapping operations

d7fa1a7

Split contract operations from AccountTokensModel into dedicated model. - AccountContractTokensModelInterface with GetByAccount, BatchInsert - Manages account_contracts junction table

Consolidate migrations: rename table to account_trustline_balances

e40679e

- Renamed account_trustlines to account_trustline_balances - Merged balance columns from separate migration into main table creation - Removed redundant migration file (2026-01-13.0-add-trustline-balance-fields.sql)

Update Models struct to use new focused model interfaces

35673ae

Replace AccountTokens with: - TrustlineBalance (TrustlineBalanceModelInterface) - AccountContractTokens (AccountContractTokensModelInterface)

Split migrations into separate files per model

a0eea75

- account_trustline_balances.sql for TrustlineBalanceModel - account_contract_tokens.sql for AccountContractTokensModel - Removed combined migration file

Update TokenIngestionService to use new focused models

0cba06b

- Replace accountTokensModel with trustlineBalanceModel and accountContractTokensModel - Update trustlineBatch to use TrustlineBalance struct - Update all model method calls to use new interfaces

Update serve.go dependency injection for new model interfaces

872b72d

- Replace AccountTokensModel with TrustlineBalanceModel and AccountContractTokensModel - Use NewBalanceReader adapter for GraphQL resolver injection

Update ingest.go to use new model interfaces

c928381

Pass TrustlineBalance and AccountContractTokens models to NewTokenIngestionService

Update first two test cases to use new model interfaces

02cd9c6

Update success test cases (SAC, SEP-41, mixed, contract, trustline V0)

2fae304

Update error and edge case tests to use new model interfaces

e83f4dd

Update BalancesByAccountAddresses tests to use new model interfaces

74953e2

Fix error message assertion in partial failure test

9e40e7d

aditya1702 added 19 commits January 30, 2026 15:39

Add resolvers for cb id and lp id

b8cdb5d

Update queries.go

3027c5a

Update statechange.go

625b900

Update types.go

e48c293

remove offer ID

409835c

Change ID -> Id

13f2368

Update data_validation_test.go

f712167

Add sponsored data field for reserves change

1ab6a1e

Fix batch copy

5ba65c2

Add LP ID for balance auth change

83e05e5

remove keyValue from balance auth change

72ce607

Update statechange.graphqls

7e3cd12

Add liquidity pool ID field to trustline change

28e8ed1

Update effects_horizon.go

a5c7b65

fix failing test

b834e24

fix test

5a953de

remove keyvalue for reserves change

b07c821

fix failing test - 2

72ac8b0

Update effects.go

31d9c57

aditya1702 marked this pull request as ready for review February 1, 2026 17:13

aditya1702 requested a review from Copilot February 1, 2026 17:17

Copilot started reviewing on behalf of aditya1702 February 1, 2026 17:17 View session

Copilot AI reviewed Feb 1, 2026

View reviewed changes

Base automatically changed from store-failed-ops to main February 4, 2026 14:42

aditya1702 changed the title ~~Replace JSONB columns from state changes table with typed columns~~ state_changes table: Replace JSONB columns with typed columns Feb 5, 2026

aditya1702 and others added 5 commits February 20, 2026 11:52

Update internal/indexer/types/types.go

b5981ba

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/serve/graphql/resolvers/statechange.resolvers.go

d733215

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/serve/graphql/resolvers/statechange.resolvers.go

ec629f6

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update internal/indexer/types/types.go

a19e646

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update effects.go

6e9cd44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

`state_changes` table: Replace JSONB columns with typed columns#476

`state_changes` table: Replace JSONB columns with typed columns#476
aditya1702 wants to merge 447 commits intomainfrom
statechanges-remove-jsonb

aditya1702 commented Jan 30, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Comments

Conversation

aditya1702 commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Known limitations

Issue that this PR addresses

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aditya1702 commented Jan 30, 2026 •

edited

Loading