Skip to content

Conversation

@KyleAMathews
Copy link
Collaborator

This fix allows auto-indexes to be created for nested field paths (e.g., profile.score, metadata.stats.views), not just top-level fields. This resolves performance issues where queries with eq(), gt(), etc. on nested fields were forced to do full table scans instead of using indexes.

Changes:

  • Remove the fieldPath.length !== 1 restriction in extractIndexableExpressions()
  • Update ensureIndexForField() to properly traverse nested paths when creating index accessors
  • Add comprehensive tests for nested path auto-indexing with 1, 2, and 3-level nesting
  • Verify that nested path indexes are properly used by the query optimizer

Fixes #727

🤖 Generated with Claude Code

🎯 Changes

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested this code locally with pnpm test:pr.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

This fix allows auto-indexes to be created for nested field paths
(e.g., `profile.score`, `metadata.stats.views`), not just top-level
fields. This resolves performance issues where queries with `eq()`,
`gt()`, etc. on nested fields were forced to do full table scans
instead of using indexes.

Changes:
- Remove the `fieldPath.length !== 1` restriction in `extractIndexableExpressions()`
- Update `ensureIndexForField()` to properly traverse nested paths when creating index accessors
- Add comprehensive tests for nested path auto-indexing with 1, 2, and 3-level nesting
- Verify that nested path indexes are properly used by the query optimizer

Fixes #727

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@changeset-bot
Copy link

changeset-bot bot commented Oct 29, 2025

🦋 Changeset detected

Latest commit: f44b33a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 12 packages
Name Type
@tanstack/db Patch
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/react-db Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch
todos Patch
@tanstack/db-example-react-todo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Oct 29, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@728

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@728

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@728

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@728

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@728

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@728

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@728

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@728

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@728

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@728

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@728

commit: f44b33a

@github-actions
Copy link
Contributor

github-actions bot commented Oct 29, 2025

Size Change: +63 B (+0.07%)

Total Size: 84.5 kB

Filename Size Change
./packages/db/dist/esm/indexes/auto-index.js 857 B +63 B (+7.93%) 🔍
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.63 kB
./packages/db/dist/esm/collection/changes.js 1.01 kB
./packages/db/dist/esm/collection/events.js 413 B
./packages/db/dist/esm/collection/index.js 3.23 kB
./packages/db/dist/esm/collection/indexes.js 1.16 kB
./packages/db/dist/esm/collection/lifecycle.js 1.8 kB
./packages/db/dist/esm/collection/mutations.js 2.52 kB
./packages/db/dist/esm/collection/state.js 3.8 kB
./packages/db/dist/esm/collection/subscription.js 2.2 kB
./packages/db/dist/esm/collection/sync.js 2.2 kB
./packages/db/dist/esm/deferred.js 230 B
./packages/db/dist/esm/errors.js 3.48 kB
./packages/db/dist/esm/event-emitter.js 798 B
./packages/db/dist/esm/index.js 1.62 kB
./packages/db/dist/esm/indexes/base-index.js 835 B
./packages/db/dist/esm/indexes/btree-index.js 2 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.21 kB
./packages/db/dist/esm/indexes/reverse-index.js 577 B
./packages/db/dist/esm/local-only.js 967 B
./packages/db/dist/esm/local-storage.js 2.42 kB
./packages/db/dist/esm/optimistic-action.js 294 B
./packages/db/dist/esm/proxy.js 3.86 kB
./packages/db/dist/esm/query/builder/functions.js 615 B
./packages/db/dist/esm/query/builder/index.js 4.04 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 938 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.55 kB
./packages/db/dist/esm/query/compiler/expressions.js 760 B
./packages/db/dist/esm/query/compiler/group-by.js 2.04 kB
./packages/db/dist/esm/query/compiler/index.js 2.21 kB
./packages/db/dist/esm/query/compiler/joins.js 2.65 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.43 kB
./packages/db/dist/esm/query/compiler/select.js 1.28 kB
./packages/db/dist/esm/query/ir.js 785 B
./packages/db/dist/esm/query/live-query-collection.js 404 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.54 kB
./packages/db/dist/esm/query/live/collection-registry.js 233 B
./packages/db/dist/esm/query/live/collection-subscriber.js 2.11 kB
./packages/db/dist/esm/query/optimizer.js 3.33 kB
./packages/db/dist/esm/scheduler.js 1.29 kB
./packages/db/dist/esm/SortedMap.js 1.24 kB
./packages/db/dist/esm/transactions.js 3.05 kB
./packages/db/dist/esm/utils.js 1.01 kB
./packages/db/dist/esm/utils/browser-polyfills.js 365 B
./packages/db/dist/esm/utils/btree.js 6.01 kB
./packages/db/dist/esm/utils/comparison.js 754 B
./packages/db/dist/esm/utils/index-optimization.js 1.73 kB

compressed-size-action::db-package-size

@github-actions
Copy link
Contributor

github-actions bot commented Oct 29, 2025

Size Change: 0 B

Total Size: 2.89 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 168 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.41 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.31 kB

compressed-size-action::react-db-package-size

Copy link
Collaborator

@samwillis samwillis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for the naming of the auto indexes - that plus a changeset and it will be good to merge.

return current
},
{
name: `auto_${fieldPath.join(`_`)}`,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I like the naming of the index by replacing of the periods with underscores. It could product a conflict.

Maybe these automatic indexes should be named "auto:path.to.field" - no chance of a conflict.

claude and others added 2 commits October 31, 2025 12:46
Change auto-index naming from 'auto_field_path' to 'auto:field.path'
to prevent ambiguity between nested paths and fields with underscores.

Examples:
- user.profile -> auto:user.profile
- user_profile -> auto:user_profile
(no conflict!)

Co-authored-by: Sam Willis <sam.willis@gmail.com>
@KyleAMathews KyleAMathews changed the title Fix GitHub issue #727 in TanStack DB Enable auto-indexing for nested field paths Oct 31, 2025
@KyleAMathews KyleAMathews merged commit 979a66f into main Oct 31, 2025
6 checks passed
@KyleAMathews KyleAMathews deleted the claude/fix-issue-727-011CUc2REsYmG5e6RfmPDCdY branch October 31, 2025 12:59
@github-actions github-actions bot mentioned this pull request Oct 31, 2025
@github-actions
Copy link
Contributor

🎉 This PR has been released!

Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-index not used for simple eq(date) / eq(dispatchId) filters → heavy “graph run” time on param change

4 participants