From 890a7529df357d7533bc2e7f01625e6537bd4835 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 27 Oct 2025 12:34:24 +0000 Subject: [PATCH 1/7] docs: refactor collection documentation structure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Created dedicated pages for each collection type: - trailbase-collection.md - local-storage-collection.md - local-only-collection.md - derived-collections.md - Simplified overview.md Collections section: - Reduced from ~743 to 408 lines (45% reduction) - Replaced detailed content with brief one-sentence descriptions - Added links to individual collection type pages - Each collection page includes: - Overview and use cases - Installation instructions - Configuration options - Code examples - Complete working examples - Cross-references to related documentation This makes the documentation more navigable and easier to maintain. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/collections/derived-collections.md | 383 +++++++++++++++++++ docs/collections/local-only-collection.md | 313 +++++++++++++++ docs/collections/local-storage-collection.md | 267 +++++++++++++ docs/collections/trailbase-collection.md | 226 +++++++++++ docs/overview.md | 374 +----------------- 5 files changed, 1209 insertions(+), 354 deletions(-) create mode 100644 docs/collections/derived-collections.md create mode 100644 docs/collections/local-only-collection.md create mode 100644 docs/collections/local-storage-collection.md create mode 100644 docs/collections/trailbase-collection.md diff --git a/docs/collections/derived-collections.md b/docs/collections/derived-collections.md new file mode 100644 index 000000000..1c041c00b --- /dev/null +++ b/docs/collections/derived-collections.md @@ -0,0 +1,383 @@ +--- +title: Derived Collections +--- + +# Derived Collections + +Derived collections are collections created from live queries over other collections, acting as materialized views that automatically update when source data changes. + +## Overview + +Live queries in TanStack DB return collections. This powerful feature allows you to: +- Create filtered subsets of existing collections +- Join multiple collections into a unified view +- Aggregate and transform data from source collections +- Chain derivations recursively (derive from other derived collections) +- Maintain materialized views that update incrementally using differential dataflow + +## Installation + +Derived collections use the core TanStack DB package: + +```bash +npm install @tanstack/db +``` + +## Basic Usage + +Create a derived collection using `createLiveQueryCollection`: + +```typescript +import { createLiveQueryCollection } from '@tanstack/db' + +// Source collection +const todoCollection = createCollection({ + // ... config +}) + +// Derived collection: only completed todos +const completedTodoCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => todo.completed), +}) +``` + +## Configuration Options + +The `createLiveQueryCollection` function accepts the following options: + +### Required Options + +- `query`: A function that receives a query builder and returns a query. The query result becomes the collection's data. + +### Optional Options + +- `startSync`: Boolean indicating whether to immediately start the query subscription (default: `false`) +- `id`: Unique identifier for the collection (auto-generated if not provided) + +## Simple Filtering + +Create derived collections that filter source collections: + +```typescript +import { createLiveQueryCollection, eq } from '@tanstack/db' + +const todoCollection = createCollection({ + // ... config +}) + +// Only pending todos +const pendingTodoCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => eq(todo.status, 'pending')), +}) + +// Only high-priority todos +const highPriorityTodoCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => todo.priority > 7), +}) +``` + +## Joining Multiple Collections + +Derive collections from multiple source collections using joins: + +```typescript +import { createLiveQueryCollection, eq } from '@tanstack/db' + +const todoCollection = createCollection({ + // ... config +}) + +const listCollection = createCollection({ + // ... config +}) + +// Todos with their list information +const todosWithListsCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .join( + { list: listCollection }, + ({ todo, list }) => eq(list.id, todo.listId), + 'inner' + ) + .select(({ todo, list }) => ({ + id: todo.id, + text: todo.text, + listName: list.name, + listColor: list.color, + })), +}) +``` + +## Transforming Data + +Use `select` to transform and shape the derived collection data: + +```typescript +import { createLiveQueryCollection } from '@tanstack/db' + +const userCollection = createCollection({ + // ... config +}) + +// Derived collection with computed fields +const userSummaryCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ user: userCollection }) + .select(({ user }) => ({ + id: user.id, + fullName: `${user.firstName} ${user.lastName}`, + initials: `${user.firstName[0]}${user.lastName[0]}`, + isActive: user.lastSeen > Date.now() - 300000, // Active in last 5 min + })), +}) +``` + +## Aggregations + +Create derived collections with aggregated data: + +```typescript +import { createLiveQueryCollection } from '@tanstack/db' + +const orderCollection = createCollection({ + // ... config +}) + +// Orders grouped by status with counts +const orderStatsByStatusCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ order: orderCollection }) + .groupBy(({ order }) => order.status) + .select(({ order }) => ({ + status: order.status, + count: count(order.id), + totalAmount: sum(order.amount), + })), +}) +``` + +## Recursive Derivations + +Derive collections from other derived collections: + +```typescript +import { createLiveQueryCollection, eq } from '@tanstack/db' + +// Base collection +const todoCollection = createCollection({ + // ... config +}) + +// First level: completed todos +const completedTodoCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => todo.completed), +}) + +// Second level: completed high-priority todos +const completedHighPriorityCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: completedTodoCollection }) + .where(({ todo }) => todo.priority > 7), +}) + +// Changes propagate efficiently through the chain +``` + +## Managing Subscriptions + +Control when derived collections start and stop syncing: + +```typescript +const derivedCollection = createLiveQueryCollection({ + startSync: false, // Don't start automatically + query: (q) => q.from({ todo: todoCollection }), +}) + +// Manually start the subscription +derivedCollection.startSync() + +// Later, stop the subscription +derivedCollection.stopSync() + +// Check sync status +if (derivedCollection.status === 'synced') { + console.log('Derived collection is synced') +} +``` + +## Complete Example: Dashboard View + +```typescript +import { createCollection } from '@tanstack/react-db' +import { createLiveQueryCollection, eq } from '@tanstack/db' +import { queryCollectionOptions } from '@tanstack/query-db-collection' + +// Source collections +const tasksCollection = createCollection( + queryCollectionOptions({ + queryKey: ['tasks'], + queryFn: async () => api.tasks.getAll(), + getKey: (item) => item.id, + }) +) + +const usersCollection = createCollection( + queryCollectionOptions({ + queryKey: ['users'], + queryFn: async () => api.users.getAll(), + getKey: (item) => item.id, + }) +) + +const projectsCollection = createCollection( + queryCollectionOptions({ + queryKey: ['projects'], + queryFn: async () => api.projects.getAll(), + getKey: (item) => item.id, + }) +) + +// Derived collection: Active tasks with user and project info +export const dashboardTasksCollection = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ task: tasksCollection }) + .join( + { user: usersCollection }, + ({ task, user }) => eq(user.id, task.assignedTo), + 'left' + ) + .join( + { project: projectsCollection }, + ({ task, project }) => eq(project.id, task.projectId), + 'left' + ) + .where(({ task }) => !task.completed) + .select(({ task, user, project }) => ({ + id: task.id, + title: task.title, + dueDate: task.dueDate, + priority: task.priority, + assigneeName: user?.name || 'Unassigned', + projectName: project?.name || 'No Project', + projectColor: project?.color || '#gray', + })) + .orderBy(({ task }) => task.dueDate, 'asc'), +}) + +// Use in component +function DashboardTasks() { + const { data: tasks } = useLiveQuery((q) => + q.from({ task: dashboardTasksCollection }) + ) + + return ( +
+

Active Tasks

+ {tasks.map((task) => ( +
+

{task.title}

+

Assigned to: {task.assigneeName}

+

Project: {task.projectName}

+

Due: {task.dueDate}

+
+ ))} +
+ ) +} +``` + +## Performance Benefits + +Derived collections use [differential dataflow](https://github.com/electric-sql/d2ts) for incremental updates: + +- **Fast updates**: When source data changes, only the affected parts of the derived collection update +- **Efficient joins**: Join operations update incrementally rather than re-computing the entire join +- **Low latency**: Updates typically complete in sub-millisecond time +- **Memory efficient**: Only materialized results are stored, not intermediate states + +```typescript +// When a single todo is updated in the source collection, +// only that todo's entry in the derived collection updates +const activeTodos = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => !todo.completed) + .orderBy(({ todo }) => todo.createdAt, 'desc'), +}) + +// This is blazing fast even with thousands of todos +todoCollection.update('todo-123', (draft) => { + draft.title = 'Updated title' +}) +``` + +## Querying Derived Collections + +Derived collections are regular collections, so you can query them like any other collection: + +```typescript +const completedTodos = createLiveQueryCollection({ + startSync: true, + query: (q) => + q.from({ todo: todoCollection }) + .where(({ todo }) => todo.completed), +}) + +// Query the derived collection in a component +function CompletedTodosList() { + const { data: todos } = useLiveQuery((q) => + q.from({ todo: completedTodos }) + .where(({ todo }) => todo.priority > 5) + .orderBy(({ todo }) => todo.completedAt, 'desc') + ) + + return +} + +// Or access data directly +const allCompleted = completedTodos.toArray() +``` + +## Use Cases + +Derived collections are perfect for: +- Creating filtered views (active items, favorites, etc.) +- Denormalizing data (joining related collections) +- Computing aggregations (counts, sums, averages) +- Creating dashboard views +- Maintaining sorted/ordered views +- Implementing search results +- Building materialized views for complex queries + +## When Not to Use + +For these scenarios, consider alternatives: +- **One-time transformations**: Use regular `useLiveQuery` directly in components +- **Simple filtering in components**: Use `useLiveQuery` with where clauses +- **Server-side filtering**: Use collection where clauses in `ElectricCollection` or `QueryCollection` + +## Learn More + +- [Live Queries Guide](../guides/live-queries.md) +- [Query Collection](./query-collection.md) +- [Electric Collection](./electric-collection.md) +- [d2ts: Differential Dataflow](https://github.com/electric-sql/d2ts) diff --git a/docs/collections/local-only-collection.md b/docs/collections/local-only-collection.md new file mode 100644 index 000000000..5518e3468 --- /dev/null +++ b/docs/collections/local-only-collection.md @@ -0,0 +1,313 @@ +--- +title: LocalOnly Collection +--- + +# LocalOnly Collection + +LocalOnly collections are designed for in-memory client data or UI state that doesn't need to persist across browser sessions or sync across tabs. + +## Overview + +The `localOnlyCollectionOptions` allows you to create collections that: +- Store data only in memory (no persistence) +- Support optimistic updates with automatic rollback on errors +- Provide optional initial data +- Work perfectly for temporary UI state and session-only data +- Automatically manage the transition from optimistic to confirmed state + +## Installation + +LocalOnly collections are included in the core TanStack DB package: + +```bash +npm install @tanstack/react-db +``` + +## Basic Usage + +```typescript +import { createCollection } from '@tanstack/react-db' +import { localOnlyCollectionOptions } from '@tanstack/react-db' + +const uiStateCollection = createCollection( + localOnlyCollectionOptions({ + id: 'ui-state', + getKey: (item) => item.id, + }) +) +``` + +## Configuration Options + +The `localOnlyCollectionOptions` function accepts the following options: + +### Required Options + +- `id`: Unique identifier for the collection +- `getKey`: Function to extract the unique key from an item + +### Optional Options + +- `schema`: [Standard Schema](https://standardschema.dev) compatible schema (e.g., Zod, Effect) for client-side validation +- `initialData`: Array of items to populate the collection with on creation +- `onInsert`: Optional handler function called before confirming inserts +- `onUpdate`: Optional handler function called before confirming updates +- `onDelete`: Optional handler function called before confirming deletes + +## Initial Data + +Populate the collection with initial data on creation: + +```typescript +const uiStateCollection = createCollection( + localOnlyCollectionOptions({ + id: 'ui-state', + getKey: (item) => item.id, + initialData: [ + { id: 'sidebar', isOpen: false }, + { id: 'theme', mode: 'light' }, + { id: 'modal', visible: false }, + ], + }) +) +``` + +## Mutation Handlers + +Mutation handlers are **completely optional**. When provided, they are called before the optimistic state is confirmed: + +```typescript +const tempDataCollection = createCollection( + localOnlyCollectionOptions({ + id: 'temp-data', + getKey: (item) => item.id, + onInsert: async ({ transaction }) => { + // Custom logic before confirming the insert + console.log('Inserting:', transaction.mutations[0].modified) + }, + onUpdate: async ({ transaction }) => { + // Custom logic before confirming the update + const { original, modified } = transaction.mutations[0] + console.log('Updating from', original, 'to', modified) + }, + onDelete: async ({ transaction }) => { + // Custom logic before confirming the delete + console.log('Deleting:', transaction.mutations[0].original) + }, + }) +) +``` + +## Manual Transactions + +When using LocalOnly collections with manual transactions (created via `createTransaction`), you must call `utils.acceptMutations()` to persist the changes: + +```typescript +import { createTransaction } from '@tanstack/react-db' + +const localData = createCollection( + localOnlyCollectionOptions({ + id: 'form-draft', + getKey: (item) => item.id, + }) +) + +const serverCollection = createCollection( + queryCollectionOptions({ + queryKey: ['items'], + queryFn: async () => api.items.getAll(), + getKey: (item) => item.id, + onInsert: async ({ transaction }) => { + await api.items.create(transaction.mutations[0].modified) + }, + }) +) + +const tx = createTransaction({ + mutationFn: async ({ transaction }) => { + // Server collection mutations are handled automatically + // After server mutations succeed, accept local collection mutations + localData.utils.acceptMutations(transaction) + }, +}) + +// Apply mutations to both collections in one transaction +tx.mutate(() => { + localData.insert({ id: 'draft-1', data: '...' }) + serverCollection.insert({ id: '1', name: 'Item' }) +}) + +await tx.commit() +``` + +## Complete Example: Modal State Management + +```typescript +import { createCollection } from '@tanstack/react-db' +import { localOnlyCollectionOptions } from '@tanstack/react-db' +import { useLiveQuery } from '@tanstack/react-db' +import { z } from 'zod' + +// Define schema +const modalStateSchema = z.object({ + id: z.string(), + isOpen: z.boolean(), + data: z.any().optional(), +}) + +type ModalState = z.infer + +// Create collection +export const modalStateCollection = createCollection( + localOnlyCollectionOptions({ + id: 'modal-state', + getKey: (item) => item.id, + schema: modalStateSchema, + initialData: [ + { id: 'user-profile', isOpen: false }, + { id: 'settings', isOpen: false }, + { id: 'confirm-delete', isOpen: false }, + ], + }) +) + +// Use in component +function UserProfileModal() { + const { data: modals } = useLiveQuery((q) => + q.from({ modal: modalStateCollection }) + .where(({ modal }) => modal.id === 'user-profile') + ) + + const modalState = modals[0] + + const openModal = (data?: any) => { + modalStateCollection.update('user-profile', (draft) => { + draft.isOpen = true + draft.data = data + }) + } + + const closeModal = () => { + modalStateCollection.update('user-profile', (draft) => { + draft.isOpen = false + draft.data = undefined + }) + } + + if (!modalState?.isOpen) return null + + return ( +
+

User Profile

+
{JSON.stringify(modalState.data, null, 2)}
+ +
+ ) +} +``` + +## Complete Example: Form Draft State + +```typescript +import { createCollection } from '@tanstack/react-db' +import { localOnlyCollectionOptions } from '@tanstack/react-db' +import { useLiveQuery } from '@tanstack/react-db' + +type FormDraft = { + id: string + formData: Record + lastModified: Date +} + +// Create collection for form drafts +export const formDraftsCollection = createCollection( + localOnlyCollectionOptions({ + id: 'form-drafts', + getKey: (item) => item.id, + }) +) + +// Use in component +function CreatePostForm() { + const { data: drafts } = useLiveQuery((q) => + q.from({ draft: formDraftsCollection }) + .where(({ draft }) => draft.id === 'new-post') + ) + + const currentDraft = drafts[0] + + const updateDraft = (field: string, value: any) => { + if (currentDraft) { + formDraftsCollection.update('new-post', (draft) => { + draft.formData[field] = value + draft.lastModified = new Date() + }) + } else { + formDraftsCollection.insert({ + id: 'new-post', + formData: { [field]: value }, + lastModified: new Date(), + }) + } + } + + const clearDraft = () => { + if (currentDraft) { + formDraftsCollection.delete('new-post') + } + } + + const submitForm = async () => { + if (!currentDraft) return + + await api.posts.create(currentDraft.formData) + clearDraft() + } + + return ( +
{ e.preventDefault(); submitForm() }}> + updateDraft('title', e.target.value)} + /> + + +
+ ) +} +``` + +## Use Cases + +LocalOnly collections are perfect for: +- Temporary UI state (modals, sidebars, tooltips) +- Form draft data during the current session +- Client-side computed or derived data +- Wizard/multi-step form state +- Temporary filters or search state +- In-memory caches + +## When Not to Use + +For these scenarios, consider other collection types: +- **Data that should persist across sessions**: Use [`LocalStorageCollection`](./local-storage-collection.md) +- **Data that syncs across tabs**: Use [`LocalStorageCollection`](./local-storage-collection.md) +- **Server-synchronized data**: Use [`QueryCollection`](./query-collection.md) or [`ElectricCollection`](./electric-collection.md) +- **Offline-first with sync**: Use [`RxDBCollection`](./rxdb-collection.md) + +## Comparison with LocalStorageCollection + +| Feature | LocalOnly | LocalStorage | +|---------|-----------|--------------| +| Persistence | None (in-memory only) | localStorage | +| Cross-tab sync | No | Yes | +| Survives page reload | No | Yes | +| Performance | Fastest | Fast | +| Size limits | Memory limits | ~5-10MB | +| Best for | Temporary UI state | User preferences | + +## Learn More + +- [Optimistic Mutations](../guides/mutations.md) +- [Live Queries](../guides/live-queries.md) +- [LocalStorage Collection](./local-storage-collection.md) diff --git a/docs/collections/local-storage-collection.md b/docs/collections/local-storage-collection.md new file mode 100644 index 000000000..c4b2f4061 --- /dev/null +++ b/docs/collections/local-storage-collection.md @@ -0,0 +1,267 @@ +--- +title: LocalStorage Collection +--- + +# LocalStorage Collection + +LocalStorage collections store small amounts of local-only state that persists across browser sessions and syncs across browser tabs in real-time. + +## Overview + +The `localStorageCollectionOptions` allows you to create collections that: +- Persist data to localStorage (or sessionStorage) +- Automatically sync across browser tabs using storage events +- Support optimistic updates with automatic rollback on errors +- Store all data under a single localStorage key +- Work with any storage API that matches the localStorage interface + +## Installation + +LocalStorage collections are included in the core TanStack DB package: + +```bash +npm install @tanstack/react-db +``` + +## Basic Usage + +```typescript +import { createCollection } from '@tanstack/react-db' +import { localStorageCollectionOptions } from '@tanstack/react-db' + +const userPreferencesCollection = createCollection( + localStorageCollectionOptions({ + id: 'user-preferences', + storageKey: 'app-user-prefs', + getKey: (item) => item.id, + }) +) +``` + +## Configuration Options + +The `localStorageCollectionOptions` function accepts the following options: + +### Required Options + +- `id`: Unique identifier for the collection +- `storageKey`: The localStorage key where all collection data is stored +- `getKey`: Function to extract the unique key from an item + +### Optional Options + +- `schema`: [Standard Schema](https://standardschema.dev) compatible schema (e.g., Zod, Effect) for client-side validation +- `storage`: Custom storage implementation (defaults to `localStorage`). Can be `sessionStorage` or any object with the localStorage API +- `onInsert`: Optional handler function called when items are inserted +- `onUpdate`: Optional handler function called when items are updated +- `onDelete`: Optional handler function called when items are deleted + +## Cross-Tab Synchronization + +LocalStorage collections automatically sync across browser tabs in real-time: + +```typescript +const settingsCollection = createCollection( + localStorageCollectionOptions({ + id: 'settings', + storageKey: 'app-settings', + getKey: (item) => item.id, + }) +) + +// Changes in one tab are automatically reflected in all other tabs +// This works automatically via storage events +``` + +## Using SessionStorage + +You can use `sessionStorage` instead of `localStorage` for session-only persistence: + +```typescript +const sessionCollection = createCollection( + localStorageCollectionOptions({ + id: 'session-data', + storageKey: 'session-key', + storage: sessionStorage, // Use sessionStorage instead + getKey: (item) => item.id, + }) +) +``` + +## Custom Storage Backend + +Provide any storage implementation that matches the localStorage API: + +```typescript +// Example: Custom storage wrapper with encryption +const encryptedStorage = { + getItem(key: string) { + const encrypted = localStorage.getItem(key) + return encrypted ? decrypt(encrypted) : null + }, + setItem(key: string, value: string) { + localStorage.setItem(key, encrypt(value)) + }, + removeItem(key: string) { + localStorage.removeItem(key) + }, +} + +const secureCollection = createCollection( + localStorageCollectionOptions({ + id: 'secure-data', + storageKey: 'encrypted-key', + storage: encryptedStorage, + getKey: (item) => item.id, + }) +) +``` + +## Mutation Handlers + +Mutation handlers are **completely optional**. Data will persist to localStorage whether or not you provide handlers: + +```typescript +const preferencesCollection = createCollection( + localStorageCollectionOptions({ + id: 'preferences', + storageKey: 'user-prefs', + getKey: (item) => item.id, + // Optional: Add custom logic when preferences are updated + onUpdate: async ({ transaction }) => { + const { modified } = transaction.mutations[0] + console.log('Preference updated:', modified) + // Maybe send analytics or trigger other side effects + }, + }) +) +``` + +## Manual Transactions + +When using LocalStorage collections with manual transactions (created via `createTransaction`), you must call `utils.acceptMutations()` to persist the changes: + +```typescript +import { createTransaction } from '@tanstack/react-db' + +const localData = createCollection( + localStorageCollectionOptions({ + id: 'form-draft', + storageKey: 'draft-data', + getKey: (item) => item.id, + }) +) + +const serverCollection = createCollection( + queryCollectionOptions({ + queryKey: ['items'], + queryFn: async () => api.items.getAll(), + getKey: (item) => item.id, + onInsert: async ({ transaction }) => { + await api.items.create(transaction.mutations[0].modified) + }, + }) +) + +const tx = createTransaction({ + mutationFn: async ({ transaction }) => { + // Server collection mutations are handled automatically + // After server mutations succeed, persist local collection mutations + localData.utils.acceptMutations(transaction) + }, +}) + +// Apply mutations to both collections in one transaction +tx.mutate(() => { + localData.insert({ id: 'draft-1', data: '...' }) + serverCollection.insert({ id: '1', name: 'Item' }) +}) + +await tx.commit() +``` + +## Complete Example + +```typescript +import { createCollection } from '@tanstack/react-db' +import { localStorageCollectionOptions } from '@tanstack/react-db' +import { useLiveQuery } from '@tanstack/react-db' +import { z } from 'zod' + +// Define schema +const userPrefsSchema = z.object({ + id: z.string(), + theme: z.enum(['light', 'dark', 'auto']), + language: z.string(), + notifications: z.boolean(), +}) + +type UserPrefs = z.infer + +// Create collection +export const userPreferencesCollection = createCollection( + localStorageCollectionOptions({ + id: 'user-preferences', + storageKey: 'app-user-prefs', + getKey: (item) => item.id, + schema: userPrefsSchema, + }) +) + +// Use in component +function SettingsPanel() { + const { data: prefs } = useLiveQuery((q) => + q.from({ pref: userPreferencesCollection }) + .where(({ pref }) => pref.id === 'current-user') + ) + + const currentPrefs = prefs[0] + + const updateTheme = (theme: 'light' | 'dark' | 'auto') => { + if (currentPrefs) { + userPreferencesCollection.update(currentPrefs.id, (draft) => { + draft.theme = theme + }) + } else { + userPreferencesCollection.insert({ + id: 'current-user', + theme, + language: 'en', + notifications: true, + }) + } + } + + return ( +
+

Theme: {currentPrefs?.theme}

+ + +
+ ) +} +``` + +## Use Cases + +LocalStorage collections are perfect for: +- User preferences and settings +- UI state that should persist across sessions +- Form drafts +- Recently viewed items +- User-specific configurations +- Small amounts of cached data + +## When Not to Use + +For these scenarios, consider other collection types: +- **Large datasets**: localStorage has size limits (typically 5-10MB) +- **Server-synchronized data**: Use [`QueryCollection`](./query-collection.md) or [`ElectricCollection`](./electric-collection.md) +- **Session-only data without persistence**: Use [`LocalOnlyCollection`](./local-only-collection.md) +- **Offline-first with sync**: Use [`RxDBCollection`](./rxdb-collection.md) + +## Learn More + +- [Optimistic Mutations](../guides/mutations.md) +- [Live Queries](../guides/live-queries.md) +- [LocalOnly Collection](./local-only-collection.md) diff --git a/docs/collections/trailbase-collection.md b/docs/collections/trailbase-collection.md new file mode 100644 index 000000000..1b1d60d42 --- /dev/null +++ b/docs/collections/trailbase-collection.md @@ -0,0 +1,226 @@ +--- +title: TrailBase Collection +--- + +# TrailBase Collection + +TrailBase collections provide seamless integration between TanStack DB and [TrailBase](https://trailbase.io), enabling real-time data synchronization with TrailBase's self-hosted application backend. + +## Overview + +[TrailBase](https://trailbase.io) is an easy-to-self-host, single-executable application backend with built-in SQLite, a V8 JS runtime, auth, admin UIs and sync functionality. + +The `@tanstack/trailbase-db-collection` package allows you to create collections that: +- Automatically sync data from TrailBase Record APIs +- Support real-time subscriptions when `enable_subscriptions` is enabled +- Handle optimistic updates with automatic rollback on errors +- Provide parse/serialize functions for data transformation + +## Installation + +```bash +npm install @tanstack/trailbase-db-collection @tanstack/react-db trailbase +``` + +## Basic Usage + +```typescript +import { createCollection } from '@tanstack/react-db' +import { trailBaseCollectionOptions } from '@tanstack/trailbase-db-collection' +import { initClient } from 'trailbase' + +const trailBaseClient = initClient(`https://your-trailbase-instance.com`) + +const todosCollection = createCollection( + trailBaseCollectionOptions({ + id: 'todos', + recordApi: trailBaseClient.records('todos'), + getKey: (item) => item.id, + }) +) +``` + +## Configuration Options + +The `trailBaseCollectionOptions` function accepts the following options: + +### Required Options + +- `id`: Unique identifier for the collection +- `recordApi`: TrailBase Record API instance created via `trailBaseClient.records()` +- `getKey`: Function to extract the unique key from an item + +### Optional Options + +- `schema`: [Standard Schema](https://standardschema.dev) compatible schema (e.g., Zod, Effect) for client-side validation +- `parse`: Object mapping field names to parsing functions that transform data coming from TrailBase +- `serialize`: Object mapping field names to serialization functions that transform data going to TrailBase +- `onInsert`: Handler function called when items are inserted +- `onUpdate`: Handler function called when items are updated +- `onDelete`: Handler function called when items are deleted + +## Data Transformation + +TrailBase uses different data formats for storage (e.g., Unix timestamps). Use `parse` and `serialize` to handle these transformations: + +```typescript +type SelectTodo = { + id: string + text: string + created_at: number // Unix timestamp from TrailBase + completed: boolean +} + +type Todo = { + id: string + text: string + created_at: Date // JavaScript Date for app usage + completed: boolean +} + +const todosCollection = createCollection( + trailBaseCollectionOptions({ + id: 'todos', + recordApi: trailBaseClient.records('todos'), + getKey: (item) => item.id, + schema: todoSchema, + // Transform TrailBase data to application format + parse: { + created_at: (ts) => new Date(ts * 1000), + }, + // Transform application data to TrailBase format + serialize: { + created_at: (date) => Math.floor(date.valueOf() / 1000), + }, + }) +) +``` + +## Real-time Subscriptions + +TrailBase supports real-time subscriptions when enabled on the server. The collection automatically subscribes to changes and updates in real-time: + +```typescript +const todosCollection = createCollection( + trailBaseCollectionOptions({ + id: 'todos', + recordApi: trailBaseClient.records('todos'), + getKey: (item) => item.id, + // Real-time updates work automatically when + // enable_subscriptions is set in TrailBase config + }) +) + +// Changes from other clients will automatically update +// the collection in real-time +``` + +## Mutation Handlers + +Handle inserts, updates, and deletes by providing mutation handlers: + +```typescript +const todosCollection = createCollection( + trailBaseCollectionOptions({ + id: 'todos', + recordApi: trailBaseClient.records('todos'), + getKey: (item) => item.id, + onInsert: async ({ transaction }) => { + const newTodo = transaction.mutations[0].modified + // TrailBase handles the persistence automatically + // Add custom logic here if needed + }, + onUpdate: async ({ transaction }) => { + const { original, modified } = transaction.mutations[0] + // TrailBase handles the persistence automatically + // Add custom logic here if needed + }, + onDelete: async ({ transaction }) => { + const deletedTodo = transaction.mutations[0].original + // TrailBase handles the persistence automatically + // Add custom logic here if needed + }, + }) +) +``` + +## Complete Example + +```typescript +import { createCollection } from '@tanstack/react-db' +import { trailBaseCollectionOptions } from '@tanstack/trailbase-db-collection' +import { initClient } from 'trailbase' +import { z } from 'zod' + +const trailBaseClient = initClient(`https://your-trailbase-instance.com`) + +// Define schema +const todoSchema = z.object({ + id: z.string(), + text: z.string(), + completed: z.boolean(), + created_at: z.date(), +}) + +type SelectTodo = { + id: string + text: string + completed: boolean + created_at: number +} + +type Todo = z.infer + +// Create collection +export const todosCollection = createCollection( + trailBaseCollectionOptions({ + id: 'todos', + recordApi: trailBaseClient.records('todos'), + getKey: (item) => item.id, + schema: todoSchema, + parse: { + created_at: (ts) => new Date(ts * 1000), + }, + serialize: { + created_at: (date) => Math.floor(date.valueOf() / 1000), + }, + onInsert: async ({ transaction }) => { + const newTodo = transaction.mutations[0].modified + console.log('Todo created:', newTodo) + }, + }) +) + +// Use in component +function TodoList() { + const { data: todos } = useLiveQuery((q) => + q.from({ todo: todosCollection }) + .where(({ todo }) => !todo.completed) + .orderBy(({ todo }) => todo.created_at, 'desc') + ) + + const addTodo = (text: string) => { + todosCollection.insert({ + id: crypto.randomUUID(), + text, + completed: false, + created_at: new Date(), + }) + } + + return ( +
+ {todos.map((todo) => ( +
{todo.text}
+ ))} +
+ ) +} +``` + +## Learn More + +- [TrailBase Documentation](https://trailbase.io/documentation/) +- [TrailBase Record APIs](https://trailbase.io/documentation/apis_record/) +- [Optimistic Mutations](../guides/mutations.md) +- [Live Queries](../guides/live-queries.md) diff --git a/docs/overview.md b/docs/overview.md index b5597a335..ba9d0a51e 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -127,374 +127,40 @@ With an instant inner loop of optimistic state, superseded in time by the slower ### Collections -There are a number of built-in collection types: +TanStack DB provides several built-in collection types for different data sources and use cases. Each collection type has its own detailed documentation page: -1. [`QueryCollection`](#querycollection) to load data into collections using [TanStack Query](https://tanstack.com/query) -2. [`ElectricCollection`](#electriccollection) to sync data into collections using [ElectricSQL](https://electric-sql.com) -3. [`TrailBaseCollection`](#trailbasecollection) to sync data into collections using [TrailBase](https://trailbase.io) -4. [`RxDBCollection`](#rxdbcollection) to integrate with [RxDB](https://rxdb.info) for local persistence and sync -5. [`LocalStorageCollection`](#localstoragecollection) for small amounts of local-only state that syncs across browser tabs -6. [`LocalOnlyCollection`](#localonlycollection) for in-memory client data or UI state +#### Built-in Collection Types -You can also use: +1. **[QueryCollection](./collections/query-collection.md)** — Load data into collections using TanStack Query for REST APIs and data fetching. -- use live collection queries to [derive collections from other collections](#derived-collections) -- the [base Collection](#base-collection) to define your own collection types +2. **[ElectricCollection](./collections/electric-collection.md)** — Sync data into collections from Postgres using ElectricSQL's real-time sync engine. -#### Collection schemas +3. **[TrailBaseCollection](./collections/trailbase-collection.md)** — Sync data into collections using TrailBase's self-hosted backend with real-time subscriptions. -All collections optionally (though strongly recommended) support adding a `schema`. - -If provided, this must be a [Standard Schema](https://standardschema.dev) compatible schema instance, such as a [Zod](https://zod.dev) or [Effect](https://effect.website/docs/schema/introduction/) schema. - -The collection will use the schema to do client-side validation of optimistic mutations. - -The collection will use the schema for its type so if you provide a schema, you can't also pass in an explicit -type (e.g. `createCollection()`). - -#### `QueryCollection` - -[TanStack Query](https://tanstack.com/query) fetches data using managed queries. Use `queryCollectionOptions` to fetch data into a collection using TanStack Query: - -```ts -import { createCollection } from "@tanstack/react-db" -import { queryCollectionOptions } from "@tanstack/query-db-collection" - -const todoCollection = createCollection( - queryCollectionOptions({ - queryKey: ["todoItems"], - queryFn: async () => { - const response = await fetch("/api/todos") - return response.json() - }, - getKey: (item) => item.id, - schema: todoSchema, // any standard schema - }) -) -``` - -The collection will be populated with the query results. - -#### `ElectricCollection` - -[Electric](https://electric-sql.com) is a read-path sync engine for Postgres. It allows you to sync subsets of data out of a Postgres database, [through your API](https://electric-sql.com/blog/2024/11/21/local-first-with-your-existing-api), into a TanStack DB collection. - -Electric's main primitive for sync is a [Shape](https://electric-sql.com/docs/guides/shapes). Use `electricCollectionOptions` to sync a shape into a collection: - -```ts -import { createCollection } from "@tanstack/react-db" -import { electricCollectionOptions } from "@tanstack/electric-db-collection" - -export const todoCollection = createCollection( - electricCollectionOptions({ - id: "todos", - shapeOptions: { - url: "https://example.com/v1/shape", - params: { - table: "todos", - }, - }, - getKey: (item) => item.id, - schema: todoSchema, - }) -) -``` - -The Electric collection requires two Electric-specific options: - -- `shapeOptions` — the Electric [ShapeStreamOptions](https://electric-sql.com/docs/api/clients/typescript#options) that define the [Shape](https://electric-sql.com/docs/guides/shapes) to sync into the collection; this includes the - - `url` to your sync engine; and - - `params` to specify the `table` to sync and any optional `where` clauses, etc. -- `getKey` — identifies the id for the rows being synced into the collection - -A new collections doesn't start syncing until you call `collection.preload()` or you query it. +4. **[RxDBCollection](./collections/rxdb-collection.md)** — Integrate with RxDB for offline-first local persistence with powerful replication and sync capabilities. -Electric shapes allow you to filter data using where clauses: +5. **[LocalStorageCollection](./collections/local-storage-collection.md)** — Store small amounts of local-only state that persists across sessions and syncs across browser tabs. -```ts -export const myPendingTodos = createCollection( - electricCollectionOptions({ - id: "todos", - shapeOptions: { - url: "https://example.com/v1/shape", - params: { - table: "todos", - where: ` - status = 'pending' - AND - user_id = '${user.id}' - `, - }, - }, - getKey: (item) => item.id, - schema: todoSchema, - }) -) -``` - -> [!TIP] -> Shape where clauses, used to filter the data you sync into `ElectricCollection`s, are different from the [live queries](#live-queries) you use to query data in components. -> -> Live queries are much more expressive than shapes, allowing you to query across collections, join, aggregate, etc. Shapes just contain filtered database tables and are used to populate the data in a collection. - -If you need more control over what data syncs into the collection, Electric allows you to [use your API](https://electric-sql.com/blog/2024/11/21/local-first-with-your-existing-api#filtering) as a proxy to both authorize and filter data. - -See the [Electric docs](https://electric-sql.com/docs/intro) for more information. - -#### `TrailBaseCollection` - -[TrailBase](https://trailbase.io) is an easy-to-self-host, single-executable application backend with built-in SQLite, a V8 JS runtime, auth, admin UIs and sync functionality. - -TrailBase lets you expose tables via [Record APIs](https://trailbase.io/documentation/apis_record/) and subscribe to changes when `enable_subscriptions` is set. Use `trailBaseCollectionOptions` to sync records into a collection: - -```ts -import { createCollection } from "@tanstack/react-db" -import { trailBaseCollectionOptions } from "@tanstack/trailbase-db-collection" -import { initClient } from "trailbase" - -const trailBaseClient = initClient(`https://trailbase.io`) +6. **[LocalOnlyCollection](./collections/local-only-collection.md)** — Manage in-memory client data or UI state that doesn't need persistence or cross-tab sync. -export const todoCollection = createCollection( - trailBaseCollectionOptions({ - id: "todos", - recordApi: trailBaseClient.records(`todos`), - getKey: (item) => item.id, - schema: todoSchema, - parse: { - created_at: (ts) => new Date(ts * 1000), - }, - serialize: { - created_at: (date) => Math.floor(date.valueOf() / 1000), - }, - }) -) -``` - -This collection requires the following TrailBase-specific options: - -- `recordApi` — identifies the API to sync. -- `getKey` — identifies the id for the records being synced into the collection. -- `parse` — maps `(v: Todo[k]) => SelectTodo[k]`. -- `serialize` — maps `(v: SelectTodo[k]) => Todo[k]`. - -A new collections doesn't start syncing until you call `collection.preload()` or you query it. - -#### `RxDBCollection` +7. **[Derived Collections](./collections/derived-collections.md)** — Create collections from live queries over other collections, acting as materialized views that update incrementally. -[RxDB](https://rxdb.info) is a client-side database for JavaScript apps with replication, conflict resolution, and offline-first features. -Use `rxdbCollectionOptions` from `@tanstack/rxdb-db-collection` to integrate an RxDB collection with TanStack DB: +#### Collection Schemas -```ts -import { createCollection } from "@tanstack/react-db" -import { rxdbCollectionOptions } from "@tanstack/rxdb-db-collection" -import { createRxDatabase } from "rxdb" - -const db = await createRxDatabase({ - name: "mydb", - storage: getRxStorageMemory(), -}) -await db.addCollections({ - todos: { - schema: { - version: 0, - primaryKey: "id", - type: "object", - properties: { - id: { type: "string", maxLength: 100 }, - text: { type: "string" }, - completed: { type: "boolean" }, - }, - }, - }, -}) - -// Wrap the RxDB collection with TanStack DB -export const todoCollection = createCollection( - rxdbCollectionOptions({ - rxCollection: db.todos, - startSync: true, - }) -) -``` - -With this integration: - -- TanStack DB subscribes to RxDB's change streams and reflects updates, deletes, and inserts in real-time. -- You get local-first sync when RxDB replication is configured. -- Mutation handlers (onInsert, onUpdate, onDelete) are implemented using RxDB's APIs (bulkUpsert, incrementalPatch, bulkRemove). - -This makes RxDB a great choice for apps that need local-first storage, replication, or peer-to-peer sync combined with TanStack DB's live queries and transaction lifecycle. - -#### `LocalStorageCollection` - -localStorage collections store small amounts of local-only state that persists across browser sessions and syncs across browser tabs in real-time. All data is stored under a single localStorage key and automatically synchronized using storage events. - -Use `localStorageCollectionOptions` to create a collection that stores data in localStorage: - -```ts -import { createCollection } from "@tanstack/react-db" -import { localStorageCollectionOptions } from "@tanstack/react-db" - -export const userPreferencesCollection = createCollection( - localStorageCollectionOptions({ - id: "user-preferences", - storageKey: "app-user-prefs", // localStorage key - getKey: (item) => item.id, - schema: userPrefsSchema, - }) -) -``` - -The localStorage collection requires: - -- `storageKey` — the localStorage key where all collection data is stored -- `getKey` — identifies the id for items in the collection - -Mutation handlers (`onInsert`, `onUpdate`, `onDelete`) are completely optional. Data will persist to localStorage whether or not you provide handlers. You can provide alternative storage backends like `sessionStorage` or custom implementations that match the localStorage API. - -```ts -export const sessionCollection = createCollection( - localStorageCollectionOptions({ - id: "session-data", - storageKey: "session-key", - storage: sessionStorage, // Use sessionStorage instead - getKey: (item) => item.id, - }) -) -``` - -> [!TIP] -> localStorage collections are perfect for user preferences, UI state, and other data that should persist locally but doesn't need server synchronization. For server-synchronized data, use [`QueryCollection`](#querycollection) or [`ElectricCollection`](#electriccollection) instead. - -#### `LocalOnlyCollection` - -LocalOnly collections are designed for in-memory client data or UI state that doesn't need to persist across browser sessions or sync across tabs. They provide a simple way to manage temporary, session-only data with full optimistic mutation support. - -Use `localOnlyCollectionOptions` to create a collection that stores data only in memory: - -```ts -import { createCollection } from "@tanstack/react-db" -import { localOnlyCollectionOptions } from "@tanstack/react-db" - -export const uiStateCollection = createCollection( - localOnlyCollectionOptions({ - id: "ui-state", - getKey: (item) => item.id, - schema: uiStateSchema, - // Optional initial data to populate the collection - initialData: [ - { id: "sidebar", isOpen: false }, - { id: "theme", mode: "light" }, - ], - }) -) -``` - -The LocalOnly collection requires: - -- `getKey` — identifies the id for items in the collection - -Optional configuration: - -- `initialData` — array of items to populate the collection with on creation -- `onInsert`, `onUpdate`, `onDelete` — optional mutation handlers for custom logic - -Mutation handlers are completely optional. When provided, they are called before the optimistic state is confirmed. The collection automatically manages the transition from optimistic to confirmed state internally. - -```ts -export const tempDataCollection = createCollection( - localOnlyCollectionOptions({ - id: "temp-data", - getKey: (item) => item.id, - onInsert: async ({ transaction }) => { - // Custom logic before confirming the insert - console.log("Inserting:", transaction.mutations[0].modified) - }, - onUpdate: async ({ transaction }) => { - // Custom logic before confirming the update - const { original, modified } = transaction.mutations[0] - console.log("Updating from", original, "to", modified) - }, - }) -) -``` - -> [!TIP] -> LocalOnly collections are perfect for temporary UI state, form data, or any client-side data that doesn't need persistence. For data that should persist across sessions, use [`LocalStorageCollection`](#localstoragecollection) instead. - -**Using LocalStorage and LocalOnly Collections with Manual Transactions:** - -When using either LocalStorage or LocalOnly collections with manual transactions (created via `createTransaction`), you must call `utils.acceptMutations()` in your transaction's `mutationFn` to persist the changes. This is necessary because these collections don't participate in the standard mutation handler flow for manual transactions. - -```ts -import { createTransaction } from "@tanstack/react-db" - -const localData = createCollection( - localOnlyCollectionOptions({ - id: "form-draft", - getKey: (item) => item.id, - }) -) - -const serverCollection = createCollection( - queryCollectionOptions({ - queryKey: ["items"], - queryFn: async () => api.items.getAll(), - getKey: (item) => item.id, - onInsert: async ({ transaction }) => { - await api.items.create(transaction.mutations[0].modified) - }, - }) -) - -const tx = createTransaction({ - mutationFn: async ({ transaction }) => { - // Server collection mutations are handled by their onInsert handler automatically - // (onInsert will be called and awaited) - - // After server mutations succeed, persist local collection mutations - localData.utils.acceptMutations(transaction) - }, -}) - -// Apply mutations to both collections in one transaction -tx.mutate(() => { - localData.insert({ id: "draft-1", data: "..." }) - serverCollection.insert({ id: "1", name: "Item" }) -}) - -await tx.commit() -``` - -#### Derived collections - -Live queries return collections. This allows you to derive collections from other collections. - -For example: - -```ts -import { createLiveQueryCollection, eq } from "@tanstack/db" +All collections optionally (though strongly recommended) support adding a `schema`. -// Imagine you have a collection of todos. -const todoCollection = createCollection({ - // config -}) +If provided, this must be a [Standard Schema](https://standardschema.dev) compatible schema instance, such as a [Zod](https://zod.dev) or [Effect](https://effect.website/docs/schema/introduction/) schema. -// You can derive a new collection that's a subset of it. -const completedTodoCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }).where(({ todo }) => todo.completed), -}) -``` +The collection will use the schema to do client-side validation of optimistic mutations. -This also works with joins to derive collections from multiple source collections. And it works recursively -- you can derive collections from other derived collections. Changes propagate efficiently using differential dataflow and it's collections all the way down. +The collection will use the schema for its type so if you provide a schema, you can't also pass in an explicit +type (e.g. `createCollection()`). -#### Collection +#### Creating Custom Collection Types -There is a `Collection` interface in [`../packages/db/src/collection.ts`](https://github.com/TanStack/db/blob/main/packages/db/src/collection.ts). You can use this to implement your own collection types. +You can create your own collection types by implementing the `Collection` interface found in [`../packages/db/src/collection.ts`](https://github.com/TanStack/db/blob/main/packages/db/src/collection.ts). -See the existing implementations in [`../packages/db`](https://github.com/TanStack/db/tree/main/packages/db), [`../packages/query-db-collection`](https://github.com/TanStack/db/tree/main/packages/query-db-collection), [`../packages/electric-db-collection`](https://github.com/TanStack/db/tree/main/packages/electric-db-collection) and [`../packages/trailbase-db-collection`](https://github.com/TanStack/db/tree/main/packages/trailbase-db-collection) for reference. +See the existing implementations in [`../packages/db`](https://github.com/TanStack/db/tree/main/packages/db), [`../packages/query-db-collection`](https://github.com/TanStack/db/tree/main/packages/query-db-collection), [`../packages/electric-db-collection`](https://github.com/TanStack/db/tree/main/packages/electric-db-collection) and [`../packages/trailbase-db-collection`](https://github.com/TanStack/db/tree/main/packages/trailbase-db-collection) for reference. Also see the [Collection Options Creator guide](./guides/collection-options-creator.md) for a pattern to create reusable collection configuration factories. ### Live queries @@ -603,8 +269,8 @@ You can use TanStack DB with your existing REST API via TanStack Query. The steps are to: -1. create [`QueryCollection`](#querycollection)s that load data using TanStack Query -2. implement [`mutationFn`](#mutationfn)s that handle mutations by posting them to your API endpoints +1. create [QueryCollection](./collections/query-collection.md)s that load data using TanStack Query +2. implement mutation handlers that handle mutations by posting them to your API endpoints ```tsx import { useLiveQuery, createCollection } from "@tanstack/react-db" From 861f4dc1b2948999c0d1fa2b18051362574c4a2b Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 27 Oct 2025 13:10:04 +0000 Subject: [PATCH 2/7] docs: organize collections by category (fetch, sync, local, derived) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reorganized the collection types in overview.md into logical categories: - Fetch Collections: QueryCollection - Sync Collections: ElectricCollection, TrailBaseCollection, RxDBCollection - Local Collections: LocalStorageCollection, LocalOnlyCollection - Derived Collections: Derived Collections This makes it easier to understand the different collection types at a glance. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/overview.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/docs/overview.md b/docs/overview.md index ba9d0a51e..e6721ca18 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -131,19 +131,27 @@ TanStack DB provides several built-in collection types for different data source #### Built-in Collection Types -1. **[QueryCollection](./collections/query-collection.md)** — Load data into collections using TanStack Query for REST APIs and data fetching. +**Fetch Collections** -2. **[ElectricCollection](./collections/electric-collection.md)** — Sync data into collections from Postgres using ElectricSQL's real-time sync engine. +- **[QueryCollection](./collections/query-collection.md)** — Load data into collections using TanStack Query for REST APIs and data fetching. -3. **[TrailBaseCollection](./collections/trailbase-collection.md)** — Sync data into collections using TrailBase's self-hosted backend with real-time subscriptions. +**Sync Collections** -4. **[RxDBCollection](./collections/rxdb-collection.md)** — Integrate with RxDB for offline-first local persistence with powerful replication and sync capabilities. +- **[ElectricCollection](./collections/electric-collection.md)** — Sync data into collections from Postgres using ElectricSQL's real-time sync engine. -5. **[LocalStorageCollection](./collections/local-storage-collection.md)** — Store small amounts of local-only state that persists across sessions and syncs across browser tabs. +- **[TrailBaseCollection](./collections/trailbase-collection.md)** — Sync data into collections using TrailBase's self-hosted backend with real-time subscriptions. -6. **[LocalOnlyCollection](./collections/local-only-collection.md)** — Manage in-memory client data or UI state that doesn't need persistence or cross-tab sync. +- **[RxDBCollection](./collections/rxdb-collection.md)** — Integrate with RxDB for offline-first local persistence with powerful replication and sync capabilities. -7. **[Derived Collections](./collections/derived-collections.md)** — Create collections from live queries over other collections, acting as materialized views that update incrementally. +**Local Collections** + +- **[LocalStorageCollection](./collections/local-storage-collection.md)** — Store small amounts of local-only state that persists across sessions and syncs across browser tabs. + +- **[LocalOnlyCollection](./collections/local-only-collection.md)** — Manage in-memory client data or UI state that doesn't need persistence or cross-tab sync. + +**Derived Collections** + +- **[Derived Collections](./collections/derived-collections.md)** — Create collections from live queries over other collections, acting as materialized views that update incrementally. #### Collection Schemas From f39d39284cb55b0c9bbf2dbfa5d0fb2355278220 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Oct 2025 12:56:22 +0000 Subject: [PATCH 3/7] docs: address review feedback on collection documentation - Removed derived-collections.md page (covered in live queries docs) - Removed RxDBCollection from overview (being removed from library) - Removed arbitrary "When Not to Use" sections from local collection docs - Added storageEventApi documentation to LocalStorageCollection: - Documents custom event API for cross-tab/process sync - Added to optional options list - Included examples for Electron, WebSocket, and IPC use cases Changes based on feedback from @samwillis --- docs/collections/derived-collections.md | 383 ------------------- docs/collections/local-only-collection.md | 8 - docs/collections/local-storage-collection.md | 42 +- docs/overview.md | 6 - 4 files changed, 34 insertions(+), 405 deletions(-) delete mode 100644 docs/collections/derived-collections.md diff --git a/docs/collections/derived-collections.md b/docs/collections/derived-collections.md deleted file mode 100644 index 1c041c00b..000000000 --- a/docs/collections/derived-collections.md +++ /dev/null @@ -1,383 +0,0 @@ ---- -title: Derived Collections ---- - -# Derived Collections - -Derived collections are collections created from live queries over other collections, acting as materialized views that automatically update when source data changes. - -## Overview - -Live queries in TanStack DB return collections. This powerful feature allows you to: -- Create filtered subsets of existing collections -- Join multiple collections into a unified view -- Aggregate and transform data from source collections -- Chain derivations recursively (derive from other derived collections) -- Maintain materialized views that update incrementally using differential dataflow - -## Installation - -Derived collections use the core TanStack DB package: - -```bash -npm install @tanstack/db -``` - -## Basic Usage - -Create a derived collection using `createLiveQueryCollection`: - -```typescript -import { createLiveQueryCollection } from '@tanstack/db' - -// Source collection -const todoCollection = createCollection({ - // ... config -}) - -// Derived collection: only completed todos -const completedTodoCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => todo.completed), -}) -``` - -## Configuration Options - -The `createLiveQueryCollection` function accepts the following options: - -### Required Options - -- `query`: A function that receives a query builder and returns a query. The query result becomes the collection's data. - -### Optional Options - -- `startSync`: Boolean indicating whether to immediately start the query subscription (default: `false`) -- `id`: Unique identifier for the collection (auto-generated if not provided) - -## Simple Filtering - -Create derived collections that filter source collections: - -```typescript -import { createLiveQueryCollection, eq } from '@tanstack/db' - -const todoCollection = createCollection({ - // ... config -}) - -// Only pending todos -const pendingTodoCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => eq(todo.status, 'pending')), -}) - -// Only high-priority todos -const highPriorityTodoCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => todo.priority > 7), -}) -``` - -## Joining Multiple Collections - -Derive collections from multiple source collections using joins: - -```typescript -import { createLiveQueryCollection, eq } from '@tanstack/db' - -const todoCollection = createCollection({ - // ... config -}) - -const listCollection = createCollection({ - // ... config -}) - -// Todos with their list information -const todosWithListsCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .join( - { list: listCollection }, - ({ todo, list }) => eq(list.id, todo.listId), - 'inner' - ) - .select(({ todo, list }) => ({ - id: todo.id, - text: todo.text, - listName: list.name, - listColor: list.color, - })), -}) -``` - -## Transforming Data - -Use `select` to transform and shape the derived collection data: - -```typescript -import { createLiveQueryCollection } from '@tanstack/db' - -const userCollection = createCollection({ - // ... config -}) - -// Derived collection with computed fields -const userSummaryCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ user: userCollection }) - .select(({ user }) => ({ - id: user.id, - fullName: `${user.firstName} ${user.lastName}`, - initials: `${user.firstName[0]}${user.lastName[0]}`, - isActive: user.lastSeen > Date.now() - 300000, // Active in last 5 min - })), -}) -``` - -## Aggregations - -Create derived collections with aggregated data: - -```typescript -import { createLiveQueryCollection } from '@tanstack/db' - -const orderCollection = createCollection({ - // ... config -}) - -// Orders grouped by status with counts -const orderStatsByStatusCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ order: orderCollection }) - .groupBy(({ order }) => order.status) - .select(({ order }) => ({ - status: order.status, - count: count(order.id), - totalAmount: sum(order.amount), - })), -}) -``` - -## Recursive Derivations - -Derive collections from other derived collections: - -```typescript -import { createLiveQueryCollection, eq } from '@tanstack/db' - -// Base collection -const todoCollection = createCollection({ - // ... config -}) - -// First level: completed todos -const completedTodoCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => todo.completed), -}) - -// Second level: completed high-priority todos -const completedHighPriorityCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: completedTodoCollection }) - .where(({ todo }) => todo.priority > 7), -}) - -// Changes propagate efficiently through the chain -``` - -## Managing Subscriptions - -Control when derived collections start and stop syncing: - -```typescript -const derivedCollection = createLiveQueryCollection({ - startSync: false, // Don't start automatically - query: (q) => q.from({ todo: todoCollection }), -}) - -// Manually start the subscription -derivedCollection.startSync() - -// Later, stop the subscription -derivedCollection.stopSync() - -// Check sync status -if (derivedCollection.status === 'synced') { - console.log('Derived collection is synced') -} -``` - -## Complete Example: Dashboard View - -```typescript -import { createCollection } from '@tanstack/react-db' -import { createLiveQueryCollection, eq } from '@tanstack/db' -import { queryCollectionOptions } from '@tanstack/query-db-collection' - -// Source collections -const tasksCollection = createCollection( - queryCollectionOptions({ - queryKey: ['tasks'], - queryFn: async () => api.tasks.getAll(), - getKey: (item) => item.id, - }) -) - -const usersCollection = createCollection( - queryCollectionOptions({ - queryKey: ['users'], - queryFn: async () => api.users.getAll(), - getKey: (item) => item.id, - }) -) - -const projectsCollection = createCollection( - queryCollectionOptions({ - queryKey: ['projects'], - queryFn: async () => api.projects.getAll(), - getKey: (item) => item.id, - }) -) - -// Derived collection: Active tasks with user and project info -export const dashboardTasksCollection = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ task: tasksCollection }) - .join( - { user: usersCollection }, - ({ task, user }) => eq(user.id, task.assignedTo), - 'left' - ) - .join( - { project: projectsCollection }, - ({ task, project }) => eq(project.id, task.projectId), - 'left' - ) - .where(({ task }) => !task.completed) - .select(({ task, user, project }) => ({ - id: task.id, - title: task.title, - dueDate: task.dueDate, - priority: task.priority, - assigneeName: user?.name || 'Unassigned', - projectName: project?.name || 'No Project', - projectColor: project?.color || '#gray', - })) - .orderBy(({ task }) => task.dueDate, 'asc'), -}) - -// Use in component -function DashboardTasks() { - const { data: tasks } = useLiveQuery((q) => - q.from({ task: dashboardTasksCollection }) - ) - - return ( -
-

Active Tasks

- {tasks.map((task) => ( -
-

{task.title}

-

Assigned to: {task.assigneeName}

-

Project: {task.projectName}

-

Due: {task.dueDate}

-
- ))} -
- ) -} -``` - -## Performance Benefits - -Derived collections use [differential dataflow](https://github.com/electric-sql/d2ts) for incremental updates: - -- **Fast updates**: When source data changes, only the affected parts of the derived collection update -- **Efficient joins**: Join operations update incrementally rather than re-computing the entire join -- **Low latency**: Updates typically complete in sub-millisecond time -- **Memory efficient**: Only materialized results are stored, not intermediate states - -```typescript -// When a single todo is updated in the source collection, -// only that todo's entry in the derived collection updates -const activeTodos = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => !todo.completed) - .orderBy(({ todo }) => todo.createdAt, 'desc'), -}) - -// This is blazing fast even with thousands of todos -todoCollection.update('todo-123', (draft) => { - draft.title = 'Updated title' -}) -``` - -## Querying Derived Collections - -Derived collections are regular collections, so you can query them like any other collection: - -```typescript -const completedTodos = createLiveQueryCollection({ - startSync: true, - query: (q) => - q.from({ todo: todoCollection }) - .where(({ todo }) => todo.completed), -}) - -// Query the derived collection in a component -function CompletedTodosList() { - const { data: todos } = useLiveQuery((q) => - q.from({ todo: completedTodos }) - .where(({ todo }) => todo.priority > 5) - .orderBy(({ todo }) => todo.completedAt, 'desc') - ) - - return -} - -// Or access data directly -const allCompleted = completedTodos.toArray() -``` - -## Use Cases - -Derived collections are perfect for: -- Creating filtered views (active items, favorites, etc.) -- Denormalizing data (joining related collections) -- Computing aggregations (counts, sums, averages) -- Creating dashboard views -- Maintaining sorted/ordered views -- Implementing search results -- Building materialized views for complex queries - -## When Not to Use - -For these scenarios, consider alternatives: -- **One-time transformations**: Use regular `useLiveQuery` directly in components -- **Simple filtering in components**: Use `useLiveQuery` with where clauses -- **Server-side filtering**: Use collection where clauses in `ElectricCollection` or `QueryCollection` - -## Learn More - -- [Live Queries Guide](../guides/live-queries.md) -- [Query Collection](./query-collection.md) -- [Electric Collection](./electric-collection.md) -- [d2ts: Differential Dataflow](https://github.com/electric-sql/d2ts) diff --git a/docs/collections/local-only-collection.md b/docs/collections/local-only-collection.md index 5518e3468..7f58a2b3c 100644 --- a/docs/collections/local-only-collection.md +++ b/docs/collections/local-only-collection.md @@ -287,14 +287,6 @@ LocalOnly collections are perfect for: - Temporary filters or search state - In-memory caches -## When Not to Use - -For these scenarios, consider other collection types: -- **Data that should persist across sessions**: Use [`LocalStorageCollection`](./local-storage-collection.md) -- **Data that syncs across tabs**: Use [`LocalStorageCollection`](./local-storage-collection.md) -- **Server-synchronized data**: Use [`QueryCollection`](./query-collection.md) or [`ElectricCollection`](./electric-collection.md) -- **Offline-first with sync**: Use [`RxDBCollection`](./rxdb-collection.md) - ## Comparison with LocalStorageCollection | Feature | LocalOnly | LocalStorage | diff --git a/docs/collections/local-storage-collection.md b/docs/collections/local-storage-collection.md index c4b2f4061..cac9004df 100644 --- a/docs/collections/local-storage-collection.md +++ b/docs/collections/local-storage-collection.md @@ -52,6 +52,7 @@ The `localStorageCollectionOptions` function accepts the following options: - `schema`: [Standard Schema](https://standardschema.dev) compatible schema (e.g., Zod, Effect) for client-side validation - `storage`: Custom storage implementation (defaults to `localStorage`). Can be `sessionStorage` or any object with the localStorage API +- `storageEventApi`: Event API for subscribing to storage events (defaults to `window`). Enables custom cross-tab, cross-window, or cross-process synchronization - `onInsert`: Optional handler function called when items are inserted - `onUpdate`: Optional handler function called when items are updated - `onDelete`: Optional handler function called when items are deleted @@ -117,6 +118,39 @@ const secureCollection = createCollection( ) ``` +### Cross-Tab Sync with Custom Storage + +The `storageEventApi` option (defaults to `window`) allows the collection to subscribe to storage events for cross-tab synchronization. A custom storage implementation can provide this API to enable custom cross-tab, cross-window, or cross-process sync: + +```typescript +// Example: Custom storage event API for cross-process sync +const customStorageEventApi = { + addEventListener(event: string, handler: (e: StorageEvent) => void) { + // Custom event subscription logic + // Could be IPC, WebSocket, or any other mechanism + myCustomEventBus.on('storage-change', handler) + }, + removeEventListener(event: string, handler: (e: StorageEvent) => void) { + myCustomEventBus.off('storage-change', handler) + }, +} + +const syncedCollection = createCollection( + localStorageCollectionOptions({ + id: 'synced-data', + storageKey: 'data-key', + storage: customStorage, + storageEventApi: customStorageEventApi, // Custom event API + getKey: (item) => item.id, + }) +) +``` + +This enables synchronization across different contexts beyond just browser tabs, such as: +- Cross-process communication in Electron apps +- WebSocket-based sync across multiple browser windows +- Custom IPC mechanisms in desktop applications + ## Mutation Handlers Mutation handlers are **completely optional**. Data will persist to localStorage whether or not you provide handlers: @@ -252,14 +286,6 @@ LocalStorage collections are perfect for: - User-specific configurations - Small amounts of cached data -## When Not to Use - -For these scenarios, consider other collection types: -- **Large datasets**: localStorage has size limits (typically 5-10MB) -- **Server-synchronized data**: Use [`QueryCollection`](./query-collection.md) or [`ElectricCollection`](./electric-collection.md) -- **Session-only data without persistence**: Use [`LocalOnlyCollection`](./local-only-collection.md) -- **Offline-first with sync**: Use [`RxDBCollection`](./rxdb-collection.md) - ## Learn More - [Optimistic Mutations](../guides/mutations.md) diff --git a/docs/overview.md b/docs/overview.md index e6721ca18..6ddd13377 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -141,18 +141,12 @@ TanStack DB provides several built-in collection types for different data source - **[TrailBaseCollection](./collections/trailbase-collection.md)** — Sync data into collections using TrailBase's self-hosted backend with real-time subscriptions. -- **[RxDBCollection](./collections/rxdb-collection.md)** — Integrate with RxDB for offline-first local persistence with powerful replication and sync capabilities. - **Local Collections** - **[LocalStorageCollection](./collections/local-storage-collection.md)** — Store small amounts of local-only state that persists across sessions and syncs across browser tabs. - **[LocalOnlyCollection](./collections/local-only-collection.md)** — Manage in-memory client data or UI state that doesn't need persistence or cross-tab sync. -**Derived Collections** - -- **[Derived Collections](./collections/derived-collections.md)** — Create collections from live queries over other collections, acting as materialized views that update incrementally. - #### Collection Schemas All collections optionally (though strongly recommended) support adding a `schema`. From 22378b68d9ef4243f5a8d90c2a7b9ea88a8e7747 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Oct 2025 13:04:29 +0000 Subject: [PATCH 4/7] docs: fix manual transaction example comments Clarified that server collection mutations are handled by their operation handlers (onInsert, onUpdate, onDelete), not automatically without configuration. Updated comments to match the pattern shown in the mutations guide. Before: "Server collection mutations are handled automatically" After: "Server collection mutations are handled by their onInsert handler automatically (onInsert will be called and awaited first)" --- docs/collections/local-only-collection.md | 4 +++- docs/collections/local-storage-collection.md | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/collections/local-only-collection.md b/docs/collections/local-only-collection.md index 7f58a2b3c..82e87a32d 100644 --- a/docs/collections/local-only-collection.md +++ b/docs/collections/local-only-collection.md @@ -125,7 +125,9 @@ const serverCollection = createCollection( const tx = createTransaction({ mutationFn: async ({ transaction }) => { - // Server collection mutations are handled automatically + // Server collection mutations are handled by their onInsert handler automatically + // (onInsert will be called and awaited first) + // After server mutations succeed, accept local collection mutations localData.utils.acceptMutations(transaction) }, diff --git a/docs/collections/local-storage-collection.md b/docs/collections/local-storage-collection.md index cac9004df..f8bbf10fd 100644 --- a/docs/collections/local-storage-collection.md +++ b/docs/collections/local-storage-collection.md @@ -199,7 +199,9 @@ const serverCollection = createCollection( const tx = createTransaction({ mutationFn: async ({ transaction }) => { - // Server collection mutations are handled automatically + // Server collection mutations are handled by their onInsert handler automatically + // (onInsert will be called and awaited first) + // After server mutations succeed, persist local collection mutations localData.utils.acceptMutations(transaction) }, From 19f515a5ccb41c126912dd85a2792bcb123c032c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Oct 2025 13:08:30 +0000 Subject: [PATCH 5/7] docs: fix incorrect manual transaction mutation handling Fixed a critical misunderstanding about how manual transactions work. When using createTransaction, the collection's operation handlers (onInsert, onUpdate, onDelete) are NOT automatically called. The mutationFn is responsible for handling ALL persistence. Changed examples from: // Server collection mutations are handled automatically To the correct pattern: // Handle server collection mutations explicitly in mutationFn await Promise.all( transaction.mutations .filter((m) => m.collection === serverCollection) .map((m) => api.items.create(m.modified)) ) Fixed in: - docs/guides/mutations.md - docs/collections/local-only-collection.md - docs/collections/local-storage-collection.md --- docs/collections/local-only-collection.md | 8 ++++++-- docs/collections/local-storage-collection.md | 8 ++++++-- docs/guides/mutations.md | 8 ++++++-- 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/collections/local-only-collection.md b/docs/collections/local-only-collection.md index 82e87a32d..f7bde5f1f 100644 --- a/docs/collections/local-only-collection.md +++ b/docs/collections/local-only-collection.md @@ -125,8 +125,12 @@ const serverCollection = createCollection( const tx = createTransaction({ mutationFn: async ({ transaction }) => { - // Server collection mutations are handled by their onInsert handler automatically - // (onInsert will be called and awaited first) + // Handle server collection mutations explicitly in mutationFn + await Promise.all( + transaction.mutations + .filter((m) => m.collection === serverCollection) + .map((m) => api.items.create(m.modified)) + ) // After server mutations succeed, accept local collection mutations localData.utils.acceptMutations(transaction) diff --git a/docs/collections/local-storage-collection.md b/docs/collections/local-storage-collection.md index f8bbf10fd..a8dbeb20c 100644 --- a/docs/collections/local-storage-collection.md +++ b/docs/collections/local-storage-collection.md @@ -199,8 +199,12 @@ const serverCollection = createCollection( const tx = createTransaction({ mutationFn: async ({ transaction }) => { - // Server collection mutations are handled by their onInsert handler automatically - // (onInsert will be called and awaited first) + // Handle server collection mutations explicitly in mutationFn + await Promise.all( + transaction.mutations + .filter((m) => m.collection === serverCollection) + .map((m) => api.items.create(m.modified)) + ) // After server mutations succeed, persist local collection mutations localData.utils.acceptMutations(transaction) diff --git a/docs/guides/mutations.md b/docs/guides/mutations.md index 24e8aba92..a07c7c521 100644 --- a/docs/guides/mutations.md +++ b/docs/guides/mutations.md @@ -814,8 +814,12 @@ const userProfile = createCollection( const tx = createTransaction({ mutationFn: async ({ transaction }) => { - // Server collection mutations are handled by their onUpdate handler automatically - // (onUpdate will be called and awaited first) + // Handle server collection mutations explicitly in mutationFn + await Promise.all( + transaction.mutations + .filter((m) => m.collection === userProfile) + .map((m) => api.profile.update(m.modified)) + ) // After server mutations succeed, accept local collection mutations localSettings.utils.acceptMutations(transaction) From b37c3d81babbbb8ff2297b16aff93b1007597679 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Oct 2025 13:15:45 +0000 Subject: [PATCH 6/7] docs: add RxDBCollection back to sync collections in overview --- docs/overview.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/overview.md b/docs/overview.md index 6ddd13377..dba600b33 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -141,6 +141,8 @@ TanStack DB provides several built-in collection types for different data source - **[TrailBaseCollection](./collections/trailbase-collection.md)** — Sync data into collections using TrailBase's self-hosted backend with real-time subscriptions. +- **[RxDBCollection](./collections/rxdb-collection.md)** — Integrate with RxDB for offline-first local persistence with powerful replication and sync capabilities. + **Local Collections** - **[LocalStorageCollection](./collections/local-storage-collection.md)** — Store small amounts of local-only state that persists across sessions and syncs across browser tabs. From ae450b64f084124542dbc949f5cc6ee2f38593f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 31 Oct 2025 13:23:59 +0000 Subject: [PATCH 7/7] docs: fix ElectricSQL example description Changed 'with a generic ingestion endpoint' to 'for real-time sync with your existing API' to accurately match what the example shows (api.todos.create()). --- docs/overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/overview.md b/docs/overview.md index dba600b33..dad45bfe1 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -262,7 +262,7 @@ See the [Mutations guide](../guides/mutations.md) for comprehensive documentatio Here we illustrate two common ways of using TanStack DB: 1. [using TanStack Query](#1-tanstack-query) with an existing REST API -2. [using the ElectricSQL sync engine](#2-electricsql-sync) with a generic ingestion endpoint +2. [using the ElectricSQL sync engine](#2-electricsql-sync) for real-time sync with your existing API > [!TIP] > You can combine these patterns. One of the benefits of TanStack DB is that you can integrate different ways of loading data and handling mutations into the same app. Your components don't need to know where the data came from or goes.