diff --git a/DOCUMENTATION_GAP_ANALYSIS.md b/DOCUMENTATION_GAP_ANALYSIS.md new file mode 100644 index 0000000..10215db --- /dev/null +++ b/DOCUMENTATION_GAP_ANALYSIS.md @@ -0,0 +1,78 @@ +# Documentation Gap Analysis + +## Summary Comparison + +### ✅ Already Well Covered +1. **Four-layer architecture** - Covered in `intro/index.md` and `core-concepts/architecture.md` +2. **Transaction flow** - Detailed sequence diagram in `architecture.md` +3. **Dual sync strategy** - Explained in `architecture.md` and `nodes.md` +4. **DAG causal ordering** - Visualized in `architecture.md` +5. **Node types** - Coordinator/peer explained in `nodes.md` +6. **Performance characteristics** - Listed in `intro/index.md` +7. **Component map** - Table in `architecture.md` +8. **Applications overview** - Comprehensive in `applications.md` +9. **CRDT collections** - Detailed in `applications.md` +10. **Where to start** - Table in `intro/index.md` + +### ❌ Missing or Incomplete + +#### Critical Gaps (Stub Pages) +1. **`core-concepts/contexts.md`** - Currently just a stub with TODOs +2. **`core-concepts/identity.md`** - Stub with TODOs +3. **`core-concepts/index.md`** - Placeholder content +4. **`getting-started/index.md`** - Placeholder content + +#### Missing Information +1. **Use Cases** - No mention of collaborative editing, decentralized social, P2P gaming, IoT, supply chain, healthcare +2. **Detailed Component Breakdown** - Runtime, SDK, storage, DAG explanations could be more detailed with "what it does" and "why it matters" +3. **Project Structure** - The detailed `core/` directory structure with explanations of what each crate does +4. **What Makes It Unique** - The bullet points exist but could be more prominent and detailed +5. **Core Principles** - Mentioned but not emphasized enough +6. **Key Components Deep Dive** - Each major crate needs a "what/why/how" explanation + +#### Structural Issues +1. **Intro page** - Good but could better emphasize unique value props +2. **Architecture page** - Component map is good but could add "responsibility" column +3. **No "Core Repository Overview"** - Missing a dedicated page explaining the core repo structure + +## Proposed Improvements + +### 1. Enhance `intro/index.md` +- Add prominent "Use Cases" section +- Expand "What Makes Calimero Unique" with more detail +- Add "Core Repository Structure" subsection +- Make "Core Principles" more prominent + +### 2. Fill `core-concepts/contexts.md` +- Explain context isolation model +- Describe lifecycle (create, invite, join, revoke) +- Add diagram showing shared state vs private storage +- Link to merobox workflows + +### 3. Fill `core-concepts/identity.md` +- Explain root vs client keys +- Describe wallet adapters +- Cover authentication flows per chain +- Link to contracts repo + +### 4. Enhance `core-concepts/architecture.md` +- Add "Key Components Deep Dive" section with detailed explanations +- Expand component map with "Responsibility" column +- Add "Project Structure" section explaining core/ directory + +### 5. Create `core-concepts/core-repository.md` (NEW) +- Detailed breakdown of core/ directory structure +- Explanation of each major crate (what/why/how) +- Component relationships diagram +- Links to individual crate READMEs + +### 6. Fill `core-concepts/index.md` +- Replace placeholder with actual overview +- Brief summaries of each concept +- Navigation guidance + +### 7. Fill `getting-started/index.md` +- Replace placeholder with actual content +- Clear progression path +- Links to each getting started page + diff --git a/docs/app-directory/index.md b/docs/app-directory/index.md index 969c11b..d4f2864 100644 --- a/docs/app-directory/index.md +++ b/docs/app-directory/index.md @@ -14,3 +14,12 @@ To get started: 3. Run it within your Calimero environment to test and customize. More detailed guides and configuration examples will be added here soon. + +## Featured Projects + +- `Battleships` — Turn-based multiplayer with private boards and Merobox workflows. [Repo README](https://github.com/calimero-network/battleships#readme) +- `Shared Todo Backlog` — Collaborative backlog with CRDT sync patterns. [Repo README](https://github.com/calimero-network/shared-todo-backlog#readme) +- `Split Bills` — Expense sharing with delegation flows and settlement patterns. [Repo README](https://github.com/calimero-network/split-bills#readme) +- `KV Store` — Minimal read/write example ideal for first integrations. [Repo README](https://github.com/calimero-network/kv-store#readme) +- `Demo Blob App` — File-style payload handling plus frontend wiring. [Repo README](https://github.com/calimero-network/demo-blob-app#readme) +- `Calimero Curb Chat` — Rich messaging UI with cross-device workflows. [Repo README](https://github.com/calimero-network/calimero-curb-chat#readme) diff --git a/docs/builder-directory/index.md b/docs/builder-directory/index.md index afb91c7..12445c0 100644 --- a/docs/builder-directory/index.md +++ b/docs/builder-directory/index.md @@ -14,11 +14,11 @@ Before we dive in, here are the tools we'll be using: - **`mero-devtools-js`** - JavaScript wrappers for Calimero tooling to keep your development workflow simple and consistent. [Source code](https://github.com/calimero-network/mero-devtools-js) -- **`calimero-abi-generator `** - Generates TypeScript clients from your Rust backend, keeping frontend/backend in sync. +- **`@calimero-network/abi-codegen`** - Generates TypeScript clients from your Rust backend, keeping frontend/backend in sync. Don't worry about installing these individually - the setup process will handle it. -## Two Paths to Get Started +## Choose Your Starting Point Choose your adventure: @@ -60,6 +60,19 @@ After setup, follow the README instructions to build and run the app. --- +## Minimal Dev Loop + +1. `pnpm install` — fetch dependencies for root and generated subdirectories. +2. `pnpm logic:build` — compile the Rust WASM and regenerate ABI clients. +3. `pnpm network:bootstrap` — start Merobox, deploy the WASM, capture the **Application ID**. +4. Update your frontend config with the Application ID and context ID. +5. `pnpm dev` — run React/Vite with hot reload alongside `logic:watch`. +6. Open two browser tabs, join the context, and iterate on gameplay or flows. + +Keep this loop handy; the walkthrough below explains each step in detail and shows where the commands live. + +--- + ## The AI-Assisted Path (Walkthrough) This walkthrough uses **Cursor** (an AI-powered IDE) to build a complete Calimero application. We'll use the [Battleships game](https://github.com/calimero-network/battleships) as our example to show you exactly what happens at each step. @@ -536,7 +549,7 @@ After the Rust code is written, the AI runs: ```bash cargo build --target wasm32-unknown-unknown --release -calimero-abi-generator --input logic/res/abi.json --output app/src/api/AbiClient.ts +npx calimero-abi-codegen -i logic/res/abi.json -o app/src/api/AbiClient.ts ``` **What this generates:** @@ -831,7 +844,7 @@ This is why the first step after `pnpm network:bootstrap` is always to grab the "logic:build": "cd logic && cargo build --target wasm32-unknown-unknown --release", "logic:watch": "watchexec -w logic/src 'pnpm logic:build && pnpm logic:sync'", "logic:sync": "cp logic/target/wasm32-unknown-unknown/release/*.wasm workflows/", - "app:generate-client": "calimero-abi-generator --input logic/res/abi.json --output app/src/api/", + "app:generate-client": "calimero-abi-codegen -i logic/res/abi.json -o app/src/api/", "app:dev": "cd app && pnpm dev", "network:bootstrap": "merobox --workflow workflows/local-network.yml", "dev": "concurrently 'pnpm logic:watch' 'pnpm app:dev'" @@ -958,6 +971,9 @@ By following this walkthrough (or just reading it), you now understand: ### Want to Go Deeper? +- **[SDK Guide](sdk-guide.md)** - Complete guide to building with the Calimero Rust SDK +- **[JavaScript SDK Guide](js-sdk-guide.md)** - Build Calimero services using TypeScript/JavaScript +- **[Core Apps Examples](../examples/core-apps-examples.md)** - Reference implementations in `core/apps` - **[Introduction](../intro/index.md)** - Understand Calimero's core concepts and architecture - **[Privacy, Verifiability & Security](../privacy-verifiability-security/index.md)** - Learn about private contexts and data flow - **[App Directory](../app-directory/index.md)** - Discover what's already built on Calimero diff --git a/docs/builder-directory/js-sdk-guide.md b/docs/builder-directory/js-sdk-guide.md new file mode 100644 index 0000000..7083647 --- /dev/null +++ b/docs/builder-directory/js-sdk-guide.md @@ -0,0 +1,858 @@ +# Calimero JavaScript SDK Guide + +Build Calimero services using **TypeScript/JavaScript**. The JavaScript SDK compiles your code to WebAssembly using QuickJS, enabling you to build distributed applications without writing Rust. + +> ⚠️ **Experimental:** The JavaScript SDK is still evolving. Expect breaking changes while we stabilize the toolchain. + +## Overview + +The JavaScript SDK (`calimero-sdk-js`) consists of two main packages: + +- **`@calimero/sdk`** - Decorators, CRDT collections, event system, and environment bindings +- **`@calimero/cli`** - Build toolchain (Rollup → QuickJS → WASM) + +**Key features:** +- Write services in TypeScript/JavaScript instead of Rust +- Same CRDT collections as Rust SDK +- Automatic conflict resolution +- Event-driven architecture +- Private storage for node-local data + +## Architecture + +### Build Pipeline + +```mermaid +flowchart LR + TS[TypeScript
Source] --> BUILD[Build
Bundle] + BUILD --> QJS[QuickJS
Compile] + QJS --> WASM[WASI-SDK
to WASM] + WASM --> OPT[Optimize
~500KB] + + style TS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style BUILD fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style QJS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style WASM fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style OPT fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff +``` + +### Runtime Execution + +```mermaid +flowchart LR + JS[JavaScript
Your code] --> SDK[SDK
Decorators] + SDK --> QJS[QuickJS
Runtime] + QJS --> HOST[Host
Functions] + HOST --> RUNTIME[Calimero
Runtime] + + style JS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style SDK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style QJS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style HOST fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style RUNTIME fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff +``` + +**How it works:** +- Your TypeScript code runs inside QuickJS (a lightweight JavaScript engine) +- CRDT operations call host functions that interact with Rust storage +- State is serialized and synchronized across the network +- Events propagate to all peers automatically + +## Getting Started + +### Prerequisites + +- Node.js 18+ with WASI support +- `pnpm` ≥ 8 (or npm/yarn) +- Access to a Calimero node (`merod`) and CLI (`meroctl`) + +### Installation + +```bash +pnpm add @calimero/sdk +pnpm add -D @calimero/cli typescript +``` + +### Minimal Service + +```typescript +import { State, Logic, Init, View } from '@calimero/sdk'; +import { Counter } from '@calimero/sdk/collections'; +import * as env from '@calimero/sdk/env'; + +@State +export class CounterApp { + count: Counter = new Counter(); +} + +@Logic(CounterApp) +export class CounterLogic extends CounterApp { + @Init + static init(): CounterApp { + env.log('Initializing counter'); + return new CounterApp(); + } + + increment(): void { + env.log('Incrementing counter'); + this.count.increment(); + } + + @View() + getCount(): bigint { + return this.count.value(); + } +} +``` + +### Build & Deploy + +```bash +# Build WASM from TypeScript +npx calimero-sdk build src/index.ts -o build/service.wasm + +# Install on node +meroctl --node node1 app install \ + --path build/service.wasm \ + --context-id + +# Call methods +meroctl --node node1 call \ + --context-id \ + --method increment + +meroctl --node node1 call \ + --context-id \ + --method getCount +``` + +## Core Concepts + +### Decorators + +#### @State + +Marks a class as application state: + +```typescript +import { State } from '@calimero/sdk'; +import { UnorderedMap, Counter } from '@calimero/sdk/collections'; + +@State +export class MyApp { + items: UnorderedMap = new UnorderedMap(); + viewCount: Counter = new Counter(); +} +``` + +**Key points:** +- State is persisted and synchronized across nodes +- Initialize CRDT fields inline (runtime reuses persisted IDs) +- Don't use regular JavaScript objects for synchronized state + +#### @Logic + +Marks a class as application logic (methods): + +```typescript +import { Logic, Init } from '@calimero/sdk'; + +@Logic(MyApp) +export class MyAppLogic extends MyApp { + @Init + static init(): MyApp { + return new MyApp(); + } + + // Mutation method (changes state) + addItem(key: string, value: string): void { + this.items.set(key, value); + } + + // View method (read-only) + @View() + getItem(key: string): string | undefined { + return this.items.get(key); + } +} +``` + +**Key points:** +- Logic class extends State class +- `@Init` marks the initialization method (called once on context creation) +- Methods without `@View()` are mutations (generate deltas) +- Methods with `@View()` are read-only (no delta generated) + +#### @View() + +Marks read-only methods: + +```typescript +@Logic(MyApp) +export class MyAppLogic extends MyApp { + // Mutation - generates delta + setValue(value: string): void { + this.register.set(value); + } + + // View - read-only, no delta + @View() + getValue(): string { + return this.register.get(); + } +} +``` + +**Benefits:** +- Faster execution (no persistence overhead) +- No redundant storage deltas +- Clear intent in API + +#### @Event + +Marks event classes: + +```typescript +import { Event } from '@calimero/sdk'; + +@Event +export class ItemAdded { + constructor( + public key: string, + public value: string, + public timestamp: number + ) {} +} +``` + +#### @Init + +Marks the initialization method: + +```typescript +@Logic(MyApp) +export class MyAppLogic extends MyApp { + @Init + static init(): MyApp { + return new MyApp(); + } +} +``` + +**Requirements:** +- Must be static +- Must return an instance of State class +- Called once when context is created + +### CRDT Collections + +The JavaScript SDK provides the same CRDT collections as the Rust SDK: + +#### UnorderedMap + +Key-value storage with Last-Write-Wins conflict resolution: + +```typescript +import { UnorderedMap } from '@calimero/sdk/collections'; + +const map = new UnorderedMap(); + +// Set value +map.set('key', 'value'); + +// Get value +const value = map.get('key'); // 'value' | undefined + +// Check existence +const exists = map.has('key'); // boolean + +// Remove entry +map.remove('key'); + +// Iterate +const entries = map.entries(); // [['key1', 'value1'], ['key2', 'value2']] +const keys = map.keys(); // ['key1', 'key2'] +const values = map.values(); // ['value1', 'value2'] + +// Size +const size = map.entries().length; +``` + +#### Vector + +Ordered list maintaining insertion order: + +```typescript +import { Vector } from '@calimero/sdk/collections'; + +const vec = new Vector(); + +// Add element +vec.push('first'); +vec.push('second'); + +// Get element +const item = vec.get(0); // 'first' + +// Remove element +const last = vec.pop(); // 'second' + +// Length +const len = vec.len(); // number +``` + +#### Counter + +Distributed counter with automatic summation: + +```typescript +import { Counter } from '@calimero/sdk/collections'; + +const counter = new Counter(); + +// Increment +counter.increment(); +counter.incrementBy(5); + +// Get value +const total = counter.value(); // bigint + +// Decrement +counter.decrement(); +counter.decrementBy(2); +``` + +#### LwwRegister + +Last-Write-Wins register for single values: + +```typescript +import { LwwRegister } from '@calimero/sdk/collections'; + +const register = new LwwRegister(); + +// Set value +register.set('hello'); + +// Get value +const value = register.get(); // 'hello' | null + +// Get timestamp +const timestamp = register.timestamp(); // bigint +``` + +#### UnorderedSet + +Set with union-based merging: + +```typescript +import { UnorderedSet } from '@calimero/sdk/collections'; + +const set = new UnorderedSet(); + +// Add element +set.add('item'); // true on first insert + +// Check membership +const has = set.has('item'); // boolean + +// Remove element +set.delete('item'); + +// Get all values +const all = set.toArray(); // string[] + +// Size +const size = set.size(); // number +``` + +### Nested CRDTs + +CRDTs can be nested arbitrarily: + +```typescript +@State +export class TeamMetrics { + // Map of member → Map of metric → Counter + memberMetrics: UnorderedMap>; + + // Map of team → Set of members + teams: UnorderedMap>; + + // Vector of profiles (each with nested data) + profiles: Vector; + + constructor() { + this.memberMetrics = new UnorderedMap(); + this.teams = new UnorderedMap(); + this.profiles = new Vector(); + } +} +``` + +**Important:** When you get a nested CRDT, you receive a **handle** that retains the CRDT ID. Mutating the handle issues host calls without deserializing the entire structure. + +```typescript +// Get nested map handle (lightweight, ID retained) +const metrics = this.memberMetrics.get('alice'); + +if (metrics) { + // Mutate nested CRDT (incremental host call) + const counter = metrics.get('commits') ?? new Counter(); + counter.increment(); + metrics.set('commits', counter); +} +``` + +### Event System + +Emit events for real-time updates: + +```typescript +import { Event, emit, emitWithHandler } from '@calimero/sdk'; + +// Define event +@Event +export class ItemAdded { + constructor(public key: string, public value: string) {} +} + +@Logic(MyApp) +export class MyAppLogic extends MyApp { + addItem(key: string, value: string): void { + this.items.set(key, value); + + // Emit event without handler + emit(new ItemAdded(key, value)); + + // Or emit with handler (handler executes on receiving nodes) + emitWithHandler(new ItemAdded(key, value), 'onItemAdded'); + } + + // Event handler (runs on peer nodes, not author node) + onItemAdded(event: ItemAdded): void { + this.itemCount.increment(); + env.log(`Item added: ${event.key} = ${event.value}`); + } +} +``` + +**Event lifecycle:** +1. Emitted during method execution +2. Included in delta broadcast to all peers +3. Handlers execute on peer nodes (not author node) +4. Handlers can update state or trigger side effects + +**Handler requirements:** +- **Commutative**: Order-independent operations +- **Independent**: No shared mutable state between handlers +- **Idempotent**: Safe to retry +- **Pure**: No external side effects (only state updates) + +### Private Storage + +For node-local data (secrets, caches, per-node counters): + +```typescript +import { createPrivateEntry } from '@calimero/sdk'; + +// Create private entry +const secrets = createPrivateEntry<{ token: string }>('private:secrets'); + +// Get or initialize +const current = secrets.getOrInit(() => ({ token: '' })); + +// Modify (never synced across nodes) +secrets.modify( + (value) => { + value.token = 'rotated-token'; + }, + () => ({ token: '' }) // Initial value if not exists +); +``` + +**Key properties:** +- Never replicated across nodes +- Stored via `storage_read` / `storage_write` directly +- Never included in CRDT deltas +- Only accessible on the executing node + +## Build Pipeline + +### Development Setup + +```bash +# Create project +mkdir my-calimero-service +cd my-calimero-service +pnpm init + +# Install dependencies +pnpm add @calimero/sdk +pnpm add -D @calimero/cli typescript @types/node + +# Create TypeScript config +cat > tsconfig.json << EOF +{ + "compilerOptions": { + "target": "ES2020", + "module": "ESNext", + "lib": ["ES2020"], + "moduleResolution": "node", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "include": ["src/**/*"] +} +EOF + +# Create source directory +mkdir src +``` + +### Build Process + +```bash +# Build WASM from TypeScript +npx calimero-sdk build src/index.ts -o build/service.wasm + +# With verbose output +npx calimero-sdk build src/index.ts -o build/service.wasm --verbose + +# Validate only (no build) +npx calimero-sdk validate src/index.ts +``` + +### Build Configuration + +The CLI automatically handles: +- TypeScript compilation +- Dependency bundling (Rollup) +- QuickJS bytecode generation +- WASM compilation (WASI-SDK) +- Optimization (wasm-opt) + +**Output:** +- `service.wasm` - Deployable WASM binary (~500KB with QuickJS overhead) + +## Common Patterns + +### Pattern 1: Simple Key-Value Store + +```typescript +import { State, Logic, Init, View } from '@calimero/sdk'; +import { UnorderedMap, LwwRegister } from '@calimero/sdk/collections'; + +@State +export class KvStore { + items: UnorderedMap>; + + constructor() { + this.items = new UnorderedMap(); + } +} + +@Logic(KvStore) +export class KvStoreLogic extends KvStore { + @Init + static init(): KvStore { + return new KvStore(); + } + + set(key: string, value: string): void { + const register = this.items.get(key) ?? new LwwRegister(); + register.set(value); + this.items.set(key, register); + } + + @View() + get(key: string): string | null { + const register = this.items.get(key); + return register ? register.get() : null; + } + + remove(key: string): void { + this.items.remove(key); + } +} +``` + +### Pattern 2: Metrics with Counters + +```typescript +import { State, Logic, Init, View } from '@calimero/sdk'; +import { UnorderedMap, Counter } from '@calimero/sdk/collections'; + +@State +export class Metrics { + pageViews: UnorderedMap; + + constructor() { + this.pageViews = new UnorderedMap(); + } +} + +@Logic(Metrics) +export class MetricsLogic extends Metrics { + @Init + static init(): Metrics { + return new Metrics(); + } + + trackPageView(page: string): void { + const counter = this.pageViews.get(page) ?? new Counter(); + counter.increment(); + this.pageViews.set(page, counter); + } + + @View() + getViews(page: string): bigint { + const counter = this.pageViews.get(page); + return counter ? counter.value() : 0n; + } +} +``` + +### Pattern 3: Event-Driven Updates + +```typescript +import { State, Logic, Init, Event, emitWithHandler } from '@calimero/sdk'; +import { UnorderedMap, Counter } from '@calimero/sdk/collections'; +import * as env from '@calimero/sdk/env'; + +@Event +export class ItemAdded { + constructor(public key: string, public value: string) {} +} + +@State +export class StoreWithEvents { + items: UnorderedMap; + eventCount: Counter; + + constructor() { + this.items = new UnorderedMap(); + this.eventCount = new Counter(); + } +} + +@Logic(StoreWithEvents) +export class StoreWithEventsLogic extends StoreWithEvents { + @Init + static init(): StoreWithEvents { + return new StoreWithEvents(); + } + + addItem(key: string, value: string): void { + this.items.set(key, value); + emitWithHandler(new ItemAdded(key, value), 'onItemAdded'); + } + + // Handler executes on peer nodes + onItemAdded(event: ItemAdded): void { + this.eventCount.increment(); + env.log(`Item added: ${event.key} = ${event.value}`); + } + + @View() + getEventCount(): bigint { + return this.eventCount.value(); + } +} +``` + +## Best Practices + +### 1. Initialize CRDTs Inline + +```typescript +// ✅ GOOD - Runtime reuses persisted IDs +@State +export class MyApp { + items: UnorderedMap = new UnorderedMap(); +} + +// ❌ BAD - Constructor runs every time +@State +export class MyApp { + items: UnorderedMap; + + constructor() { + this.items = new UnorderedMap(); // Not reused! + } +} +``` + +### 2. Use @View() for Read-Only Methods + +```typescript +// ✅ GOOD - No persistence overhead +@View() +getItem(key: string): string | undefined { + return this.items.get(key); +} + +// ❌ BAD - Generates unnecessary deltas +getItem(key: string): string | undefined { + return this.items.get(key); +} +``` + +### 3. Handle Nested CRDTs Correctly + +```typescript +// ✅ GOOD - Use handles, mutate incrementally +const metrics = this.memberMetrics.get('alice'); +if (metrics) { + const counter = metrics.get('commits') ?? new Counter(); + counter.increment(); + metrics.set('commits', counter); +} + +// ❌ BAD - Don't try to clone entire structure +const metrics = this.memberMetrics.get('alice'); +if (metrics) { + // Don't do this - it doesn't work + const cloned = { ...metrics }; // Wrong! +} +``` + +### 4. Make Event Handlers Safe + +```typescript +// ✅ GOOD - Commutative, independent, idempotent +onItemAdded(event: ItemAdded): void { + this.itemCount.increment(); // CRDT operations are safe +} + +// ❌ BAD - Not safe for parallel execution +onItemAdded(event: ItemAdded): void { + // Don't make external API calls! + fetch('/api/notify', { ... }); // Wrong! + + // Don't depend on execution order! + if (this.items.has(event.key)) { // Race condition! + // ... + } +} +``` + +### 5. Use Private Storage for Secrets + +```typescript +// ✅ GOOD - Secrets never leave the node +const secrets = createPrivateEntry<{ token: string }>('private:secrets'); + +// ❌ BAD - Don't put secrets in CRDT state +@State +export class MyApp { + apiToken: string = ''; // Never do this! +} +``` + +## Troubleshooting + +### Build Errors + +**Issue**: TypeScript compilation errors +```bash +# Check TypeScript version +pnpm list typescript + +# Use verbose flag for details +npx calimero-sdk build src/index.ts -o build/service.wasm --verbose +``` + +**Issue**: Missing dependencies +```bash +# Ensure all dependencies are installed +pnpm install + +# Check QuickJS and WASI-SDK are downloaded (CLI handles this) +``` + +### Runtime Errors + +**Issue**: Method not found +- Verify method is in `@Logic` class +- Check method name matches call +- Ensure method is public (not private) + +**Issue**: CRDT operations failing +- Verify CRDT is initialized inline +- Check you're using CRDT collections, not plain objects +- Ensure nested CRDTs are handled as handles + +**Issue**: Events not propagating +- Verify `@Event` decorator on event class +- Check event is emitted during method execution +- Ensure handlers are in `@Logic` class + +## Examples + +The `calimero-sdk-js` repository includes comprehensive examples: + +| Example | Demonstrates | Location | +| --- | --- | --- | +| **counter** | Basic Counter CRDT | `examples/counter` | +| **kv-store** | UnorderedMap + LwwRegister, events | `examples/kv-store` | +| **team-metrics** | Nested CRDTs, mergeable structs | `examples/team-metrics` | +| **private-data** | Private storage patterns | `examples/private-data` | +| **blobs** | Blob management | `examples/blobs` | +| **xcall** | Cross-context calls | `examples/xcall` | + +**Run an example:** +```bash +# Clone repository +git clone https://github.com/calimero-network/calimero-sdk-js +cd calimero-sdk-js + +# Run example workflow +cd examples/counter +merobox bootstrap run workflows/counter-js.yml --log-level=trace +``` + +## Comparison: JavaScript SDK vs Rust SDK + +| Feature | JavaScript SDK | Rust SDK | +| --- | --- | --- | +| **Language** | TypeScript/JavaScript | Rust | +| **Runtime** | QuickJS (in WASM) | Native WASM | +| **Build Size** | ~500KB (includes QuickJS) | ~100KB (optimized) | +| **Performance** | Slower (JS interpreter) | Faster (native code) | +| **Development** | Easier (familiar JS syntax) | More learning curve | +| **Type Safety** | TypeScript | Rust | +| **CRDT Collections** | Same API | Same API | + +**When to use JavaScript SDK:** +- Familiar with JavaScript/TypeScript +- Faster prototyping +- Less performance-critical applications +- Want to leverage existing JS libraries (via Rollup) + +**When to use Rust SDK:** +- Need maximum performance +- Already familiar with Rust +- Complex algorithms or computations +- Minimal binary size requirements + +## Deep Dives + +For detailed JavaScript SDK documentation: + +- **Repository**: [`calimero-network/calimero-sdk-js`](https://github.com/calimero-network/calimero-sdk-js) - Full source code +- **Getting Started**: [`docs/getting-started.md`](https://github.com/calimero-network/calimero-sdk-js/blob/master/docs/getting-started.md) - Step-by-step guide +- **Architecture**: [`docs/architecture.md`](https://github.com/calimero-network/calimero-sdk-js/blob/master/docs/architecture.md) - Build pipeline and runtime +- **Collections**: [`docs/collections.md`](https://github.com/calimero-network/calimero-sdk-js/blob/master/docs/collections.md) - CRDT usage guide +- **Events**: [`docs/events.md`](https://github.com/calimero-network/calimero-sdk-js/blob/master/docs/events.md) - Event patterns and handlers +- **Troubleshooting**: [`docs/troubleshooting.md`](https://github.com/calimero-network/calimero-sdk-js/blob/master/docs/troubleshooting.md) - Common issues + +## Related Topics + +- [SDK Guide (Rust)](sdk-guide.md) - Building with Rust SDK +- [Applications](../core-concepts/applications.md) - Application architecture overview +- [Core Apps Examples](../examples/core-apps-examples.md) - Rust SDK examples +- [Getting Started](../getting-started/index.md) - Complete getting started guide + diff --git a/docs/builder-directory/sdk-guide.md b/docs/builder-directory/sdk-guide.md new file mode 100644 index 0000000..5515268 --- /dev/null +++ b/docs/builder-directory/sdk-guide.md @@ -0,0 +1,424 @@ +# Calimero SDK Guide for Builders + +The Calimero SDK (`core/crates/sdk` and `core/crates/storage`) provides everything you need to build distributed, peer-to-peer applications with automatic conflict-free synchronization. + +## Overview + +The SDK consists of two main components: + +- **Application SDK** (`core/crates/sdk`): Macros, event system, private storage, and runtime integration +- **Storage SDK** (`core/crates/storage`): CRDT collections with automatic merge semantics + +Together, they enable you to build applications with: +- Automatic conflict resolution via CRDTs +- Real-time event propagation +- Private node-local storage +- Type-safe state management + +## Core Concepts + +### State Definition + +Applications define **state** using the `#[app::state]` macro: + +```rust +use calimero_sdk::app; +use calimero_storage::collections::UnorderedMap; +use calimero_sdk::borsh::{BorshSerialize, BorshDeserialize}; + +#[app::state(emits = Event)] +#[derive(BorshSerialize, BorshDeserialize)] +#[borsh(crate = "calimero_sdk::borsh")] +pub struct MyApp { + // CRDT-backed state (automatically synchronized) + items: UnorderedMap, + + // Can nest CRDTs arbitrarily + nested: UnorderedMap>, +} +``` + +**Key points:** +- State is persisted and synchronized across nodes +- Must derive `BorshSerialize` and `BorshDeserialize` for persistence +- Use `#[app::state(emits = Event)]` to enable event emission + +### Logic Implementation + +Implement **logic** using the `#[app::logic]` macro: + +```rust +#[app::logic] +impl MyApp { + // Initialize state (called once on context creation) + #[app::init] + pub fn init() -> MyApp { + MyApp { + items: UnorderedMap::new(), + nested: UnorderedMap::new(), + } + } + + // Mutation method (changes state, generates delta) + pub fn add_item(&mut self, key: String, value: String) -> app::Result<()> { + self.items.insert(key, value)?; + Ok(()) + } + + // View method (read-only, no delta generated) + #[app::view] + pub fn get_item(&self, key: &str) -> app::Result> { + self.items.get(key)?.map(|v| v.get().clone()).ok_or_else(|| app::Error::NotFound) + } +} +``` + +**Key points:** +- `#[app::init]` marks the initialization function +- Mutation methods (`&mut self`) generate deltas and sync +- View methods (`#[app::view]`) are read-only and faster +- Use `app::Result` for error handling + +### CRDT Collections + +The SDK provides several CRDT collection types: + +#### UnorderedMap + +Key-value storage with automatic conflict resolution: + +```rust +use calimero_storage::collections::UnorderedMap; + +let mut map: UnorderedMap = UnorderedMap::new(); + +// Insert value (conflict-free) +map.insert("key".to_string(), "value".to_string())?; + +// Get value +let value = map.get("key")?; // Returns Option + +// Remove value +map.remove("key")?; + +// Check existence +if map.contains("key")? { + // ... +} + +// Iterate entries +for (key, value) in map.entries()? { + // ... +} +``` + +#### Vector + +Ordered list with element-wise merging: + +```rust +use calimero_storage::collections::Vector; + +let mut vec: Vector = Vector::new(); + +// Append element +vec.push("item".to_string())?; + +// Get element by index +let item = vec.get(0)?; // Returns Option + +// Insert at position +vec.insert(0, "first".to_string())?; + +// Remove element +vec.remove(0)?; +``` + +#### Counter + +Distributed counter with automatic summation: + +```rust +use calimero_storage::collections::Counter; + +let mut counter = Counter::new(); + +// Increment +counter.increment()?; + +// Decrement +counter.decrement()?; + +// Get value +let value = counter.value(); // Returns i64 +``` + +#### LwwRegister + +Last-Write-Wins register for single values: + +```rust +use calimero_storage::collections::LwwRegister; + +let mut register: LwwRegister = LwwRegister::new("initial".to_string()); + +// Set value (latest timestamp wins) +register.set("updated".to_string())?; + +// Get value +let value = register.get().clone(); +``` + +#### UnorderedSet + +Set with union-based merging: + +```rust +use calimero_storage::collections::UnorderedSet; + +let mut set: UnorderedSet = UnorderedSet::new(); + +// Insert element +set.insert("item".to_string())?; + +// Check membership +if set.contains("item")? { + // ... +} + +// Remove element +set.remove("item")?; +``` + +### Event System + +Applications can emit events for real-time updates: + +```rust +#[app::state(emits = Event)] +pub struct MyApp { + items: UnorderedMap, +} + +// Define event types +#[app::event] +pub enum Event<'a> { + ItemAdded { + key: &'a str, + value: &'a str, + }, + ItemRemoved { + key: &'a str, + }, +} + +#[app::logic] +impl MyApp { + pub fn add_item(&mut self, key: String, value: String) -> app::Result<()> { + self.items.insert(key.clone(), value.clone())?; + + // Emit event (propagated to all peers) + app::emit!(Event::ItemAdded { + key: &key, + value: &value, + }); + + Ok(()) + } +} +``` + +**Event lifecycle:** +1. Emitted during method execution +2. Included in delta broadcast +3. Handlers execute on peer nodes (not author node) +4. Handlers can update UI or trigger side effects + +### Private Storage + +For node-local data (secrets, caches, per-node counters): + +```rust +use calimero_sdk::private_storage; + +pub fn use_private_storage() { + // Create private entry + let secrets = private_storage::entry::("my-secrets"); + + // Read value + let current = secrets.get_or_init(|| Secrets::default()); + + // Modify value (never synced, stays on node) + secrets.write(|s| { + s.token = "rotated-token".to_string(); + }); +} +``` + +**Key properties:** +- Never replicated across nodes +- Stored via `storage_read` / `storage_write` directly +- Never included in CRDT deltas +- Only accessible on the executing node + +## Common Patterns + +### Pattern 1: Simple Key-Value Store + +```rust +#[app::state(emits = Event)] +pub struct KvStore { + items: UnorderedMap>, +} + +#[app::logic] +impl KvStore { + #[app::init] + pub fn init() -> KvStore { + KvStore { + items: UnorderedMap::new(), + } + } + + pub fn set(&mut self, key: String, value: String) -> app::Result<()> { + self.items.insert(key, value.into())?; + Ok(()) + } + + pub fn get(&self, key: &str) -> app::Result> { + Ok(self.items.get(key)?.map(|v| v.get().clone())) + } +} +``` + +### Pattern 2: Counter with Metrics + +```rust +#[app::state] +pub struct Metrics { + page_views: UnorderedMap, +} + +#[app::logic] +impl Metrics { + #[app::init] + pub fn init() -> Metrics { + Metrics { + page_views: UnorderedMap::new(), + } + } + + pub fn track_page_view(&mut self, page: String) -> app::Result<()> { + if let Some(counter) = self.page_views.get(&page)? { + counter.increment()?; + } else { + let mut counter = Counter::new(); + counter.increment()?; + self.page_views.insert(page, counter)?; + } + Ok(()) + } + + pub fn get_views(&self, page: &str) -> app::Result { + Ok(self.page_views.get(page)?.map(|c| c.value()).unwrap_or(0)) + } +} +``` + +### Pattern 3: Nested Structures + +```rust +#[app::state] +pub struct TeamMetrics { + // Map of team → Map of member → Counter + teams: UnorderedMap>, +} + +#[app::logic] +impl TeamMetrics { + pub fn increment_metric( + &mut self, + team: String, + member: String, + ) -> app::Result<()> { + let members = self.teams + .entry(team)? + .or_insert_with(|| UnorderedMap::new()); + + let counter = members + .entry(member)? + .or_insert_with(|| Counter::new()); + + counter.increment()?; + Ok(()) + } +} +``` + +## Building Applications + +### Project Setup + +```bash +# Create new Rust project +cargo new my-calimero-app +cd my-calimero-app + +# Add dependencies to Cargo.toml +[dependencies] +calimero-sdk = { path = "../../core/crates/sdk" } +calimero-storage = { path = "../../core/crates/storage" } +calimero-sdk-macros = { path = "../../core/crates/sdk/macros" } +borsh = { version = "1.0", features = ["derive"] } + +[lib] +crate-type = ["cdylib"] + +[dependencies.calimero-sdk] +features = ["macro"] +``` + +### Build to WASM + +```bash +# Add WASM target +rustup target add wasm32-unknown-unknown + +# Build WASM binary +cargo build --target wasm32-unknown-unknown --release + +# Output: target/wasm32-unknown-unknown/release/my_calimero_app.wasm +``` + +### Extract ABI + +```bash +# Extract ABI from WASM +calimero-abi extract \ + target/wasm32-unknown-unknown/release/my_calimero_app.wasm \ + -o abi.json +``` + +## Best Practices + +1. **Always use CRDTs**: Don't use regular Rust collections for synchronized state +2. **Use `&self` for views**: Methods with `&self` are read-only and faster (no attribute needed) +3. **Handle errors properly**: Use `app::Result` and meaningful error types +4. **Use private storage for secrets**: Never put secrets in CRDT state +5. **Emit events for UI updates**: Enable real-time updates across nodes +6. **Test with multiple nodes**: Verify sync behavior in multi-node scenarios + +## Deep Dives + +For detailed SDK documentation: + +- **Application SDK**: [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) - Complete API reference +- **Storage Collections**: [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) - CRDT types and semantics +- **Examples**: [`core/apps`](https://github.com/calimero-network/core/tree/master/apps) - Working application examples + +## Related Topics + +- [Getting Started](../getting-started/index.md) - Complete getting started guide +- [Applications](../core-concepts/applications.md) - Application architecture overview +- [Core Apps Examples](../examples/core-apps-examples.md) - Reference implementations + diff --git a/docs/core-concepts/applications.md b/docs/core-concepts/applications.md new file mode 100644 index 0000000..c781663 --- /dev/null +++ b/docs/core-concepts/applications.md @@ -0,0 +1,170 @@ +# Applications + +Calimero **applications** are WASM (WebAssembly) modules that run inside the Calimero node runtime. They use the Calimero SDK to manage CRDT-backed state, emit events, and interact with the network. + +## Application Model + +Applications are compiled from Rust (or other supported languages) to WebAssembly and executed in a sandboxed environment. They use the Calimero SDK to: + +- Define **state** using CRDT collections +- Implement **logic** that mutates state +- Emit **events** for real-time updates +- Access **private storage** for node-local data + +## Quick Start + +See [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) for complete SDK documentation. + +**Minimal example:** + +```rust +use calimero_sdk::app; +use calimero_storage::collections::UnorderedMap; + +#[app::state] +pub struct MyApp { + items: UnorderedMap, +} + +#[app::logic] +impl MyApp { + pub fn add_item(&mut self, key: String, value: String) { + self.items.insert(key, value); + } +} +``` + +Build and deploy with [`meroctl`](../tools-apis/meroctl-cli.md). See [SDK Guide](../builder-directory/sdk-guide.md) for details. + +## Architecture + +### WASM Execution Flow + +```mermaid +flowchart LR + CLIENT[Client] --> NODE[Node] + NODE --> WASM[WASM] + WASM --> STORAGE[Storage] + STORAGE --> NODE + NODE --> NETWORK[Network] + NODE --> CLIENT + + style CLIENT fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NODE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style WASM fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STORAGE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style NETWORK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +### Execution Model + +1. **Deterministic execution**: WASM code runs deterministically for consistent results across nodes +2. **Sandboxing**: Isolated execution environment with resource limits +3. **State mutation**: CRDT operations generate actions that are collected in DELTA_CONTEXT +4. **Event emission**: Applications can emit events for real-time updates + +## CRDT State Management + +Applications use CRDT collections for conflict-free state. See [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) for complete CRDT documentation. + +**Available collections:** +- `Counter` - Distributed counters (sum) +- `LwwRegister` - Last-write-wins registers +- `UnorderedMap` - Key-value maps +- `Vector` - Ordered lists +- `UnorderedSet` - Unique sets (union) +- `Option` - Optional CRDTs + +CRDTs can be nested arbitrarily for complex data structures. + +## Event System + +Applications emit events for real-time updates. Events propagate to all peers automatically. + +**Event lifecycle:** +1. Emitted during method execution +2. Broadcast to all peers via delta +3. Handlers execute on peer nodes +4. Handlers can update state or trigger side effects + +See [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) for event examples. + +## Private Storage + +For node-local data (secrets, caches) that never syncs across nodes: + +```rust +use calimero_sdk::private_storage; + +let secrets = private_storage::entry::("my-secrets"); +secrets.write(|s| { s.token = "rotated".to_string(); }); +``` + +See [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) for private storage details. + +## Views vs Mutations + +In Rust, methods using `&self` are read-only views (no deltas generated), while `&mut self` methods are mutations: + +```rust +// View method (read-only, no delta generated) +pub fn get_item(&self, key: &str) -> app::Result> { + self.items.get(key)?.map(|v| v.get().clone()) +} + +// Mutation method (generates delta) +pub fn set_item(&mut self, key: String, value: String) -> app::Result<()> { + self.items.insert(key, value.into())?; + Ok(()) +} +``` + +**Benefits**: +- View methods (`&self`) don't generate storage deltas +- Faster execution (no persistence overhead) +- Clear intent in API (Rust's type system enforces immutability) + +## Resource Limits + +WASM execution is bounded: + +- **Memory**: Configurable limits (default: ~128MB) +- **Stack size**: Bounded to prevent stack overflow +- **Execution time**: Metered with gas-like system +- **Register limits**: Number and size of storage registers + +See [`core/crates/runtime/README.md`](https://github.com/calimero-network/core/blob/master/crates/runtime/README.md) for detailed limits. + +## ABI Generation + +Applications export an ABI (Application Binary Interface) that clients use: + +1. **Build WASM**: Compile Rust code to WASM +2. **Generate ABI**: Extract method signatures, types, events +3. **Client bindings**: Generate TypeScript/Python clients from ABI +4. **Type safety**: Full type information for client calls + +Tools: +- **`calimero-abi`**: Rust tool for ABI generation +- **`@calimero/abi-codegen`**: TypeScript client generator + +## Example Applications + +- **kv-store**: Simple key-value store ([`core/apps/kv-store`](https://github.com/calimero-network/core/tree/master/apps/kv-store)) +- **blobs**: File/blob sharing ([`core/apps/blobs`](https://github.com/calimero-network/core/tree/master/apps/blobs)) +- **battleships**: Multiplayer game ([`battleships`](https://github.com/calimero-network/battleships)) + +## Deep Dives + +For detailed application development: + +- **SDK Documentation**: [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) - API reference and examples +- **Storage Collections**: [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) - CRDT types and merge semantics +- **Runtime**: [`core/crates/runtime/README.md`](https://github.com/calimero-network/core/blob/master/crates/runtime/README.md) - WASM execution engine +- **Integration Guide**: [`core/crates/node/readme/integration-guide.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/integration-guide.md) - Building applications + +## Related Topics + +- [Getting Started](../getting-started/index.md) - Complete getting started guide +- [Contexts](contexts.md) - Application instances +- [Architecture Overview](architecture.md) - How applications fit into the system diff --git a/docs/core-concepts/architecture.md b/docs/core-concepts/architecture.md new file mode 100644 index 0000000..787ef06 --- /dev/null +++ b/docs/core-concepts/architecture.md @@ -0,0 +1,183 @@ +# Architecture Overview + +Calimero's architecture consists of four main layers that work together to enable distributed, peer-to-peer applications with automatic conflict-free synchronization. + +## Four-Layer Architecture + +```mermaid +flowchart LR + APP[Application
WASM apps & SDK] --> NODE[Node
Sync & execution] + NODE --> STORAGE[Storage
CRDT & DAG] + STORAGE --> NETWORK[Network
P2P & APIs] + NETWORK -.-> NODE + + style APP fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NODE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STORAGE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NETWORK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +## Transaction Flow + +**Transaction Flow:** +```mermaid +flowchart LR + CLIENT[Client
RPC call] --> NODE[Node
Execute] + NODE --> WASM[WASM
Process] + WASM --> STORAGE[Storage
CRDT ops] + STORAGE --> NODE + NODE --> NETWORK[Network
Broadcast] + NETWORK --> PEER[Peer
~100ms] + NODE --> CLIENT + + style CLIENT fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NODE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style WASM fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STORAGE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NETWORK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style PEER fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +See [`core/crates/runtime/README.md`](https://github.com/calimero-network/core/blob/master/crates/runtime/README.md) for execution details. + +## Synchronization Flow + +Calimero uses a dual-path synchronization strategy: + +### Path 1: Gossipsub Broadcast (Primary) + +Fast real-time propagation (~100-200ms): + +```mermaid +flowchart LR + TXN[Execute] --> DELTA[Create Delta] + DELTA --> GOSSIP[Gossipsub] + GOSSIP --> RECEIVE[Peers receive] + RECEIVE --> READY{Ready?} + READY -->|Yes| APPLY[Apply] + READY -->|No| BUFFER[Buffer] + APPLY --> EVENTS[Events] + + style TXN fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style DELTA fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style GOSSIP fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style RECEIVE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style READY fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style APPLY fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style EVENTS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style BUFFER fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +### Path 2: Periodic P2P Sync (Fallback) + +Catch-up every 10-30 seconds: + +```mermaid +flowchart LR + TIMER[Timer] --> PEER[Select Peer] + PEER --> STREAM[Open Stream] + STREAM --> HEADS[Exchange Heads] + HEADS --> DIFF{Differ?} + DIFF -->|Yes| REQUEST[Request Deltas] + DIFF -->|No| SYNC[Synced] + REQUEST --> APPLY2[Apply] + APPLY2 --> SYNC + + style TIMER fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style PEER fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STREAM fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style HEADS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style DIFF fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style REQUEST fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style APPLY2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style SYNC fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff +``` + +**Why both paths?** +- **Gossipsub**: Fast (~100-200ms), reliable in good network conditions +- **Periodic sync**: Ensures eventual consistency even with packet loss, partitions, or downtime + +See [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) for sync configuration details. + +## DAG-Based Causal Ordering + +The DAG ensures deltas are applied in correct causal order: + +```mermaid +flowchart LR + ROOT[Root] --> A[Delta A] + ROOT --> B[Delta B] + A --> MERGE[Merge] + B --> MERGE + + style ROOT fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style A fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style B fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style MERGE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff +``` + +**Key properties**: +- Deltas can arrive in any order +- System buffers deltas until their parent dependencies are ready +- Once parents are available, deltas are applied automatically in causal order +- Concurrent updates create forks that merge automatically + +## Key Components + +```mermaid +flowchart LR + SERVER[Server] --> NODE[Node] + NODE --> RUNTIME[Runtime] + NODE --> NETWORK[Network] + SDK[SDK] --> RUNTIME + RUNTIME --> STORAGE[Storage] + STORAGE --> DAG[DAG] + + style SDK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style RUNTIME fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STORAGE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style DAG fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style NODE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style NETWORK fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style SERVER fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +| Component | Purpose | Repository | +| --- | --- | --- | +| **SDK** | `#[app::state]`, `#[app::logic]`, CRDT collections, events | [`core/crates/sdk`](https://github.com/calimero-network/core/blob/master/crates/sdk) | +| **Runtime** | WASM execution (Wasmer), sandboxing | [`core/crates/runtime`](https://github.com/calimero-network/core/blob/master/crates/runtime) | +| **Storage** | CRDT collections, merge semantics | [`core/crates/storage`](https://github.com/calimero-network/core/blob/master/crates/storage) | +| **DAG** | Causal delta tracking, dependency resolution | [`core/crates/dag`](https://github.com/calimero-network/core/blob/master/crates/dag) | +| **Node** | NodeManager orchestrates sync, events, blobs | [`core/crates/node`](https://github.com/calimero-network/core/blob/master/crates/node) | +| **Network** | libp2p P2P (Gossipsub, streams, DHT) | [`core/crates/network`](https://github.com/calimero-network/core/blob/master/crates/network) | +| **Server** | JSON-RPC API, WebSocket/SSE | [`core/crates/server`](https://github.com/calimero-network/core/blob/master/crates/server) | +| **merod** | Node binary (coordinator/peer) | [`core/crates/merod`](https://github.com/calimero-network/core/blob/master/crates/merod) | +| **meroctl** | CLI for node operations | [`core/crates/meroctl`](https://github.com/calimero-network/core/blob/master/crates/meroctl) | + +See [`core/README.md`](https://github.com/calimero-network/core#readme) for complete architecture. + +## Component Details + +For detailed information on each component, see their README files: + +- **SDK**: [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) - Macros, CRDTs, events +- **Runtime**: [`core/crates/runtime/README.md`](https://github.com/calimero-network/core/blob/master/crates/runtime/README.md) - WASM execution, sandboxing +- **Storage**: [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) - CRDT collections, merging +- **DAG**: [`core/crates/dag/README.md`](https://github.com/calimero-network/core/blob/master/crates/dag/README.md) - Causal ordering, dependency resolution +- **Node**: [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) - NodeManager, sync, events +- **Network**: [`core/crates/network/README.md`](https://github.com/calimero-network/core/blob/master/crates/network/README.md) - libp2p, Gossipsub, P2P +- **Server**: [`core/crates/server/README.md`](https://github.com/calimero-network/core/blob/master/crates/server/README.md) - JSON-RPC, WebSocket, SSE + +## Deep Dives + +For detailed architecture information: + +- **DAG Logic**: [`core/crates/dag/README.md`](https://github.com/calimero-network/core/blob/master/crates/dag/README.md) - Causal ordering algorithms +- **Node Architecture**: [`core/crates/node/readme/architecture.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/architecture.md) - Complete system design +- **Sync Protocol**: [`core/crates/node/readme/sync-protocol.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/sync-protocol.md) - Delta propagation details +- **Storage**: [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) - CRDT types and merge semantics +- **Network**: [`core/crates/network/README.md`](https://github.com/calimero-network/core/blob/master/crates/network/README.md) - P2P protocols and configuration + +--- + +**Next**: Learn about specific concepts: [Contexts](contexts.md) | [Identity](identity.md) | [Applications](applications.md) | [Nodes](nodes.md) diff --git a/docs/core-concepts/contexts.md b/docs/core-concepts/contexts.md new file mode 100644 index 0000000..0b879ad --- /dev/null +++ b/docs/core-concepts/contexts.md @@ -0,0 +1,177 @@ +# Contexts + +A **context** is an isolated instance of an application running on the Calimero network. Each context has its own state, members, and access controls. Think of it as a "workspace" or "room" where a specific application instance operates. + +## What Are Contexts? + +Contexts provide: + +- **State Isolation**: Each context maintains its own CRDT-backed state, completely separate from other contexts +- **Member Management**: Invite-only access control with cryptographic identities +- **Multi-Chain Support**: Each context can be associated with a blockchain protocol (NEAR, Ethereum, ICP, etc.) +- **Lifecycle Management**: Create, invite members, join, and manage contexts independently + +## Context Lifecycle + +```mermaid +flowchart LR + START[Start] --> CREATED[Created
WASM installed] + CREATED --> INVITED[Invited
Invitation sent] + INVITED --> JOINED[Joined
Accept invitation] + JOINED --> ACTIVE[Active
Members can call] + ACTIVE --> END[End
Delete context] + + style START fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CREATED fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style INVITED fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style JOINED fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style ACTIVE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style END fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +### 1. Creation + +A context is created when an application is installed. See [`core/crates/meroctl/README.md`](https://github.com/calimero-network/core/blob/master/crates/meroctl/README.md) for CLI details. + +**What happens:** +- Application WASM is installed on the node +- Initial context state is created (via `#[app::init]` method) +- Context creator becomes the first member +- Context ID is generated (unique identifier) + +### 2. Invitation Flow + +Context creators can invite other identities to join: + +```bash +# Generate identity for invitee +meroctl identity create --node node2 + +# Create invitation +meroctl context invite \ + --context-id \ + --grantee-id +``` + +**Invitation contains:** +- Context ID +- Grantee public key +- Permissions (read/write/execute) +- Expiration (optional) + +### 3. Joining a Context + +Invited members can join using the invitation: + +```bash +# Join context using invitation +meroctl context join \ + --context-id \ + --invitation +``` + +**What happens:** +- Invitation is validated +- Member is added to context membership +- Member receives current state snapshot +- Member can now call methods and receive updates + +### 4. Membership & Permissions + +Context members have different permission levels: + +- **Creator**: Full control, can invite/revoke members +- **Member**: Can call methods, read state, receive events +- **Read-only**: Can query state but cannot mutate + +## State Isolation Model + +```mermaid +flowchart LR + CTX1[Context A
Shared State] --> PRIVATE1[Private
Storage] + CTX2[Context B
Shared State] --> PRIVATE2[Private
Storage] + CTX1 -.->|isolated| CTX2 + + style CTX1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CTX2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style PRIVATE1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style PRIVATE2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +**Key points:** +- **Shared CRDT State**: Replicated across all member nodes, automatically synchronized +- **Private Storage**: Node-local data that never leaves the executing node +- **Complete Isolation**: Context A cannot access Context B's state + +See [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) for CRDT implementation details. + +## Multi-Chain Integration + +Contexts can be associated with blockchain protocols: + +| Protocol | Use Case | Identity Source | +| --- | --- | --- | +| **NEAR** | NEAR-based applications | NEAR account IDs | +| **Ethereum** | Ethereum dApps | Ethereum addresses | +| **ICP** | Internet Computer apps | ICP principals | +| **Stellar** | Stellar-based apps | Stellar accounts | + +Each protocol provides: +- **Identity verification**: Cryptographic proof of ownership +- **Cross-chain attestations**: Verifiable claims about on-chain state +- **Relayer integration**: Bridge between Calimero and blockchain + +## Context Operations + +**Query state:** +```bash +meroctl call --context-id --method get_item --args '{"key": "hello"}' +``` + +**Mutate state:** +```bash +meroctl call --context-id --method add_item --args '{"key": "hello", "value": "world"}' --executor-public-key +``` + +**Subscribe to events:** +- WebSocket: Connect to `ws://localhost:2528/ws` +- Use `subscribe` method with `context_id` and `event_type` +- See [API Reference](../reference/index.md) for details + +See [`core/crates/meroctl/README.md`](https://github.com/calimero-network/core/blob/master/crates/meroctl/README.md) for complete CLI documentation. + +## Context Management + +**List contexts:** +```bash +meroctl context list +``` + +**Revoke access:** +```bash +meroctl context revoke --context-id --member-id +``` + +Revoking access removes the member but preserves state history. + +## Best Practices + +1. **One Context Per Use Case**: Create separate contexts for different purposes (e.g., one per team, one per project) +2. **Minimize Members**: Only invite necessary members to reduce sync overhead +3. **Use Private Storage**: Store secrets and node-local data in private storage, not CRDT state +4. **Context Naming**: Use descriptive context IDs or metadata for easier management + +## Deep Dives + +For detailed context documentation: + +- **Context Management**: [`core/crates/context/README.md`](https://github.com/calimero-network/core/blob/master/crates/context/README.md) - Lifecycle and operations +- **Identity & Permissions**: [Identity](identity.md) - Cryptographic identities and access control +- **Merobox Workflows**: [`merobox` README](https://github.com/calimero-network/merobox#readme) - Automated context creation and management + +## Related Topics + +- [Applications](applications.md) - What runs inside contexts +- [Identity](identity.md) - Who can access contexts +- [Nodes](nodes.md) - Where contexts run +- [Architecture Overview](architecture.md) - How contexts fit into the system diff --git a/docs/core-concepts/identity.md b/docs/core-concepts/identity.md new file mode 100644 index 0000000..314812f --- /dev/null +++ b/docs/core-concepts/identity.md @@ -0,0 +1,171 @@ +# Identity + +Calimero uses **cryptographic identities** to manage access control and authentication across the network. Each participant has one or more identities that prove ownership and grant permissions. + +## Identity Model + +Calimero supports a hierarchical identity model: + +```mermaid +flowchart LR + ROOT[Root Key
alice.near] --> CLIENT1[Client Key A] + ROOT --> CLIENT2[Client Key B] + ROOT --> CLIENT3[Client Key C] + + CLIENT1 --> CTX1[Context A] + CLIENT2 --> CTX2[Context B] + CLIENT3 --> CTX3[Context C] + + style ROOT fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style CLIENT1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CLIENT2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CLIENT3 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CTX1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CTX2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CTX3 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +### Root Keys + +A **root key** is the master identity for a user or node. It's typically: + +- Generated from a blockchain wallet (NEAR, Ethereum, etc.) +- Used for high-level operations (creating contexts, managing memberships) +- Stored securely (hardware wallet, keychain, etc.) + +### Client Keys + +**Client keys** are derived from root keys and used for: + +- Executing methods in specific contexts +- Signing transactions and deltas +- Proving membership in contexts + +**Benefits:** +- **Isolation**: Compromise of one client key doesn't affect others +- **Revocation**: Can revoke access per-context without changing root key +- **Privacy**: Different keys for different contexts + +## Identity Generation + +Generate identities with `meroctl`: + +```bash +meroctl identity create --node node1 +``` + +See [`core/crates/meroctl/README.md`](https://github.com/calimero-network/core/blob/master/crates/meroctl/README.md) for CLI details. + +## Blockchain Wallet Integration + +Calimero supports wallet-based authentication: + +| Protocol | Identity Source | +| --- | --- | +| **NEAR** | NEAR account ID + signature | +| **Ethereum** | Ethereum address + signature | +| **ICP** | ICP principal + signature | +| **Stellar** | Stellar account + signature | + +**Flow:** +1. User connects wallet +2. Signs challenge message +3. Calimero verifies signature +4. JWT token issued + +See [`calimero-client-js/README.md`](https://github.com/calimero-network/calimero-client-js#readme) for client authentication examples. + +## Authentication Flows + +For wallet authentication examples, see: +- **JavaScript**: [`calimero-client-js/README.md`](https://github.com/calimero-network/calimero-client-js#readme) - Client-side auth flows +- **Python**: [`calimero-client-py/README.md`](https://github.com/calimero-network/calimero-client-py#readme) - Python client auth + +## JWT Tokens + +After authentication, Calimero issues JWT tokens containing: +- `context_id` - Target context +- `executor_public_key` - Client key for execution +- `permissions` - Access permissions +- `exp` - Expiration timestamp + +**Usage:** +- Include in API requests: `Authorization: Bearer ` +- Tokens expire and can be refreshed +- See [`core/crates/auth/README.md`](https://github.com/calimero-network/core/blob/master/crates/auth/README.md) for details + +## Key Management + +**Hierarchical structure:** +- Root keys delegate to client keys per context +- Each context has separate client keys +- Keys can be revoked independently + +**Revoke access:** +```bash +meroctl context revoke --context-id --member-id +``` + +See [`core/crates/meroctl/README.md`](https://github.com/calimero-network/core/blob/master/crates/meroctl/README.md) for key management commands. + +**What happens:** +- Key is removed from context membership +- Key can no longer sign transactions for that context +- Existing transactions remain valid (immutable history) +- Root key remains unaffected + +## Wallet Adapters + +Calimero provides wallet adapters for easy integration: + +### JavaScript Client + +```typescript +import { ClientLogin } from '@calimero-network/calimero-client'; + +// Automatically handles wallet connection and authentication + navigate('/dashboard')} +/> +``` + +**Supported wallets:** +- NEAR Wallet +- MetaMask (Ethereum) +- WalletConnect +- Internet Identity (ICP) + +### Python Client + +```python +from calimero_client_py import create_connection, AuthMode + +# Connect with wallet authentication +connection = create_connection( + base_url="https://node.calimero.network", + auth_mode=AuthMode.WALLET, + wallet_type="near" # or "ethereum", "icp" +) +``` + +## Best Practices + +1. **Use Client Keys**: Don't use root keys directly for context operations +2. **Rotate Keys**: Periodically rotate client keys for security +3. **Secure Storage**: Store private keys in secure keychains, never in code +4. **Multi-Sig Support**: Use multi-signature wallets for high-value contexts +5. **Key Backup**: Backup root keys securely (hardware wallet, paper backup) + +## Deep Dives + +For detailed identity documentation: + +- **Identity Contracts**: [`contracts` README](https://github.com/calimero-network/contracts#readme) - Smart contract implementations +- **Auth Service**: [`core/crates/auth/README.md`](https://github.com/calimero-network/core/blob/master/crates/auth/README.md) - Authentication service +- **Client SDKs**: [Tools & APIs](../tools-apis/index.md) - Wallet integration guides + +## Related Topics + +- [Contexts](contexts.md) - Where identities are used +- [Applications](applications.md) - What identities can access +- [Architecture Overview](architecture.md) - How identity fits into the system diff --git a/docs/core-concepts/index.md b/docs/core-concepts/index.md new file mode 100644 index 0000000..8dd492a --- /dev/null +++ b/docs/core-concepts/index.md @@ -0,0 +1,133 @@ +# Core Concepts + +Understanding Calimero's core concepts is essential for building and operating applications. This section covers the fundamental building blocks of the platform. + +- **[Contexts](contexts.md)** - Isolated application instances with their own state and members +- **[Identity](identity.md)** - Cryptographic identities for access control and authentication +- **[Applications](applications.md)** - WASM modules that run inside contexts +- **[Nodes](nodes.md)** - Runtime that orchestrates synchronization and execution +- **[Architecture Overview](architecture.md)** - How all components work together + +## Contexts + +**Contexts** are isolated instances of applications running on the Calimero network. Each context has: + +- **Isolated State**: Own CRDT-backed state, completely separate from other contexts +- **Member Management**: Invite-only access control with cryptographic identities +- **Lifecycle**: Create, invite, join, and manage independently + +**Key concepts:** +- State isolation model (shared CRDT state vs private storage) +- Invitation and membership flow +- Multi-chain integration (NEAR, Ethereum, ICP, etc.) + +[Learn more →](contexts.md) + +## Identity + +**Identity** in Calimero uses cryptographic keys for access control: + +- **Root Keys**: Master identities (typically from blockchain wallets) +- **Client Keys**: Derived keys for specific contexts +- **Authentication**: Wallet-based authentication (NEAR, Ethereum, ICP, etc.) + +**Key concepts:** +- Hierarchical key management +- Wallet adapters and integration +- JWT tokens for API authentication + +[Learn more →](identity.md) + +## Applications + +**Applications** are WASM modules that run inside contexts: + +- **WASM Runtime**: Sandboxed execution environment +- **CRDT State**: Conflict-free data types for automatic synchronization +- **Event System**: Real-time event emission and handling +- **Private Storage**: Node-local data that never leaves the node + +**Key concepts:** +- SDK macros (`#[app::state]`, `#[app::logic]`) +- CRDT collections (UnorderedMap, Vector, Counter, etc.) +- Views vs mutations +- ABI generation for client bindings + +[Learn more →](applications.md) + +## Nodes + +**Nodes** (`merod`) orchestrate synchronization and execution: + +- **Node Types**: Coordinator (first node) or Peer (joins network) +- **Synchronization**: Dual-path sync (Gossipsub + periodic P2P) +- **Event Handling**: Automatic event handler execution +- **Blob Distribution**: Content-addressed file sharing + +**Key concepts:** +- NodeManager architecture +- Delta propagation and application +- Admin surfaces (JSON-RPC, WebSocket, SSE) + +[Learn more →](nodes.md) + +## Architecture Overview + +The **Architecture Overview** explains how all components work together: + +- **Four-Layer Architecture**: Application, Node, Storage, Network layers +- **Transaction Flow**: How requests flow through the system +- **Synchronization Flow**: Dual-path sync strategy +- **DAG-Based Ordering**: Causal ordering for out-of-order delivery +- **Component Map**: Detailed breakdown of each crate + +[Learn more →](architecture.md) + +## How Concepts Relate + +```mermaid +flowchart LR + NODE[Node
merod] + CTX1[Context A
App + State] + CTX2[Context B
App + State] + ID1[Identity A
alice.near] + ID2[Identity B
bob.near] + + NODE --> CTX1 + NODE --> CTX2 + ID1 --> CTX1 + ID2 --> CTX2 + CTX1 -.->|isolated| CTX2 + + style NODE fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style CTX1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style CTX2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style ID1 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style ID2 fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +## Learning Path + +**New to Calimero?** Start here: + +1. **[Architecture Overview](architecture.md)** - Understand the big picture +2. **[Applications](applications.md)** - Learn how to build apps +3. **[Contexts](contexts.md)** - Understand application instances +4. **[Identity](identity.md)** - Learn about access control +5. **[Nodes](nodes.md)** - Understand the runtime + +**Ready to build?** Check out: + +- [Getting Started](../getting-started/index.md) - Step-by-step guides +- [Builder Directory](../builder-directory/index.md) - Development resources +- [Examples](../examples/index.md) - Working applications + +**Need to operate?** See: + +- [Running Nodes](../operator-track/run-a-local-network.md) - Setup and deployment +- [Monitoring](../operator-track/monitor-and-debug.md) - Observability +- [API Reference](../reference/index.md) - Complete API documentation + +--- + +**Next**: Start with [Architecture Overview](architecture.md) to understand how everything fits together. diff --git a/docs/core-concepts/nodes.md b/docs/core-concepts/nodes.md new file mode 100644 index 0000000..82507c2 --- /dev/null +++ b/docs/core-concepts/nodes.md @@ -0,0 +1,132 @@ +# Nodes + +A **node** (`merod`) is the core runtime that orchestrates synchronization, event handling, and blob distribution across a distributed network of peers. It wraps the DAG with WASM execution, networking, and lifecycle management. + +## What Nodes Do + +**Core Responsibilities**: +1. **Delta Management**: Apply deltas to WASM state in correct causal order +2. **Network Coordination**: Gossipsub broadcasts + P2P sync streams +3. **Event Execution**: Trigger event handlers on receiving nodes +4. **Blob Distribution**: Content-addressed file sharing +5. **Resource Management**: Memory limits, cleanup, garbage collection + +## Node Types + +### Coordinator Node +- First node in a network +- Handles initial context creation and bootstrap +- Can serve as entry point for new peers + +```bash +merod --node-type coordinator +``` + +### Peer Node +- Joins existing network +- Connects to coordinator or other peers +- Participates in P2P mesh + +```bash +merod --node-type peer --swarm-addrs /ip4/127.0.0.1/tcp/2428 +``` + +## Node Architecture + +```mermaid +flowchart LR + MGR[NodeManager
Orchestrator] --> CLIENTS[Clients
Context & Node] + MGR --> MANAGERS[Managers
Blob & Sync] + MGR --> STATE[State
Cache & Deltas] + + style MGR fill:#000000,stroke:#00ff00,stroke-width:4px,color:#ffffff + style CLIENTS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style MANAGERS fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff + style STATE fill:#1a1a1a,stroke:#00ff00,stroke-width:3px,color:#ffffff +``` + +### Components + +- **NodeManager**: Orchestrates sync, events, lifecycle, periodic timers +- **NodeClients**: ContextClient (WASM execution), NodeClient (P2P operations) +- **NodeManagers**: BlobManager (content storage), SyncManager (periodic sync) +- **NodeState**: Blob cache, delta stores per context + +See [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) for detailed architecture. + +## Synchronization + +Nodes use dual-path sync: +- **Gossipsub** (~100-200ms) - Fast real-time broadcast +- **Periodic P2P** (every 10-30s) - Catch-up and recovery + +See [Architecture Overview](architecture.md#synchronization-flow) for sync flow diagrams. + +## Event Handling + +When deltas are applied: +1. Author node skips handlers (local UI already updated) +2. Peer nodes execute handlers automatically +3. Events propagate via DAG causal ordering + +## Blob Distribution + +Content-addressed blob storage: +- Blobs stored by hash (blob ID) +- LRU cache with eviction (5 min or 100 blobs / 500 MB) +- P2P distribution via streams +- Periodic garbage collection + +See [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) for blob management details. + +## Admin Surfaces + +**JSON-RPC** (port 2528): +- `/jsonrpc` - RPC calls + +**WebSocket** (`/ws`): +- Real-time subscriptions (deltas, events, state changes) +- JSON-RPC over WebSocket + +**SSE** (`/sse`): +- Alternative to WebSocket for event streaming +- Browser-friendly real-time updates + +**Admin API** (`/admin-api/`): +- Node administration, context management, identity operations +- JWT token authentication + +See [`core/crates/server/README.md`](https://github.com/calimero-network/core/blob/master/crates/server/README.md) for API details. + +## Monitoring & Debugging + +Key metrics and logs: + +- **DAG stats**: Pending deltas, applied deltas, DAG heads per context +- **Network stats**: Peer count, Gossipsub mesh health, P2P stream activity +- **Performance**: Delta application latency, sync duration, memory usage +- **Events**: Event handler execution, errors, warnings + +See [Monitor & Debug](../operator-track/monitor-and-debug.md) for detailed guidance. + +## Configuration + +Nodes can be configured via: +- **Command-line flags**: `--node-type`, `--swarm-addrs`, `--config` +- **Configuration file**: TOML file with network, storage, runtime settings +- **Environment variables**: Override specific settings + +## Deep Dives + +For detailed node documentation: + +- **Node Architecture**: [`core/crates/node/readme/architecture.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/architecture.md) - Complete system design +- **Sync Configuration**: [`core/crates/node/readme/sync-configuration.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/sync-configuration.md) - Tuning parameters +- **Event Handling**: [`core/crates/node/readme/event-handling.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/event-handling.md) - Event flow and handlers +- **Troubleshooting**: [`core/crates/node/readme/troubleshooting.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/troubleshooting.md) - Common issues + +## Related Topics + +- [Run a Local Network](../operator-track/run-a-local-network.md) - Getting started with nodes +- [Architecture Overview](architecture.md) - How nodes fit into the system +- [Contexts](contexts.md) - Application instances managed by nodes diff --git a/docs/examples/core-apps-examples.md b/docs/examples/core-apps-examples.md new file mode 100644 index 0000000..69becdb --- /dev/null +++ b/docs/examples/core-apps-examples.md @@ -0,0 +1,346 @@ +# Core Apps Examples + +The `core/apps` directory contains reference implementations demonstrating various SDK patterns and use cases. These examples serve as learning resources and starting points for your own applications. + +## Overview + +| Example | What it demonstrates | Key SDK features | +| --- | --- | --- | +| **kv-store** | Basic CRDT usage | `UnorderedMap`, `LwwRegister`, events | +| **kv-store-init** | Initialization patterns | `#[app::init]`, state setup | +| **kv-store-with-handlers** | Event handling | Event emission and handlers | +| **blobs** | File/blob management | Blob storage and distribution | +| **collaborative-editor** | Text collaboration | `ReplicatedGrowableArray` (RGA) CRDT | +| **private-data** | Private storage | Node-local secrets and data | +| **team-metrics** | Nested CRDTs | Nested maps, counters, metrics | +| **xcall-example** | Cross-context calls | Inter-context communication | + +## kv-store + +**Location**: [`core/apps/kv-store`](https://github.com/calimero-network/core/tree/master/apps/kv-store) + +**What it does:** +Simple key-value store demonstrating basic CRDT operations and event emission. + +**Key features:** +- `UnorderedMap>` for key-value storage +- Event emission for insertions, updates, and removals +- View methods for read-only queries +- Error handling patterns + +**API methods:** +- `set(key: String, value: String)` - Store or update a value +- `get(key: &str) -> Option` - Retrieve a value +- `remove(key: &str) -> Option` - Remove a value +- `clear()` - Remove all entries +- `len() -> usize` - Get number of entries +- `entries() -> BTreeMap` - Get all entries + +**Example usage:** +```bash +# Build +cd core/apps/kv-store +./build.sh + +# Install +meroctl --node node1 app install \ + --path res/kv_store.wasm \ + --application-id kv-store + +# Create context +meroctl --node node1 context create \ + --application-id kv-store \ + --context-id kv-context + +# Set a value +meroctl --node node1 call \ + --context-id kv-context \ + --method set \ + --args '{"key": "hello", "value": "world"}' + +# Get a value +meroctl --node node1 call \ + --context-id kv-context \ + --method get \ + --args '{"key": "hello"}' +``` + +**Learn from this example:** +- Basic CRDT state definition +- Mutations vs views +- Event emission patterns +- Error handling with `app::Result` + +## kv-store-with-handlers + +**Location**: [`core/apps/kv-store-with-handlers`](https://github.com/calimero-network/core/tree/master/apps/kv-store-with-handlers) + +**What it does:** +KV store with event handlers demonstrating real-time event processing. + +**Key features:** +- Same KV operations as basic kv-store +- Event handlers that execute on peer nodes +- Event-driven UI updates +- Handler patterns and best practices + +**Learn from this example:** +- Event handler implementation +- Real-time event propagation +- UI update patterns + +## blobs + +**Location**: [`core/apps/blobs`](https://github.com/calimero-network/core/tree/master/apps/blobs) + +**What it does:** +Demonstrates blob storage and distribution across the network. + +**Key features:** +- Upload files to blob storage +- Content-addressed blob IDs +- Blob listing and retrieval +- Context-aware blob announcements +- Network distribution via P2P + +**API methods:** +- `upload_blob(blob_id: [u8; 32], size: u64)` - Register a blob +- `list_blobs() -> Vec<[u8; 32]>` - List all blobs +- `get_blob_info(blob_id: [u8; 32]) -> Option` - Get blob metadata + +**Example usage:** +```bash +# Upload a blob +meroctl --node node1 blob upload \ + --file ./document.pdf \ + --context-id blob-context + +# Register in application +meroctl --node node1 call \ + --context-id blob-context \ + --method upload_blob \ + --args '{"blob_id": "...", "size": 1024}' + +# List blobs +meroctl --node node1 call \ + --context-id blob-context \ + --method list_blobs +``` + +**Learn from this example:** +- Blob storage patterns +- Content-addressed file sharing +- Large file distribution +- P2P blob discovery + +## collaborative-editor + +**Location**: [`core/apps/collaborative-editor`](https://github.com/calimero-network/core/tree/master/apps/collaborative-editor) + +**What it does:** +Real-time collaborative text editor using RGA (Replicated Growable Array) CRDT. + +**Key features:** +- Character-level collaborative editing +- `ReplicatedGrowableArray` for conflict-free text operations +- Insert and delete operations at any position +- Edit counting with `Counter` CRDT +- Document metadata (title, statistics) + +**API methods:** +- `init() -> EditorState` - Initialize new document +- `set_title(title: String)` - Set document title +- `insert_text(position: usize, text: String)` - Insert text +- `delete_text(start: usize, end: usize)` - Delete text range +- `get_text() -> String` - Get full document text +- `get_title() -> String` - Get document title +- `get_edit_count() -> i64` - Get total edit count + +**Example usage:** +```bash +# Insert text at position 0 +meroctl --node node1 call \ + --context-id editor-context \ + --method insert_text \ + --args '{"position": 0, "text": "Hello"}' + +# Delete text from position 5 to 10 +meroctl --node node1 call \ + --context-id editor-context \ + --method delete_text \ + --args '{"start": 5, "end": 10}' + +# Get full text +meroctl --node node1 call \ + --context-id editor-context \ + --method get_text +``` + +**Learn from this example:** +- RGA CRDT for text editing +- Character-level conflict resolution +- Collaborative editing patterns +- Position-based operations + +## private-data + +**Location**: [`core/apps/private-data`](https://github.com/calimero-network/core/tree/master/apps/private-data) + +**What it does:** +Demonstrates private storage for node-local secrets and data. + +**Key features:** +- Private storage usage patterns +- Node-local data that never syncs +- Secrets management +- Per-node counters and caches + +**API methods:** +- Methods for storing/retrieving private data +- Per-node secret rotation +- Private cache management + +**Example usage:** +See workflow in [`workflows/example.yml`](https://github.com/calimero-network/core/blob/master/apps/private-data/workflows/example.yml) + +**Learn from this example:** +- Private storage API usage +- Secrets management patterns +- When to use private vs shared storage + +## team-metrics + +**Location**: [`core/apps/team-metrics-macro`](https://github.com/calimero-network/core/tree/master/apps/team-metrics-macro) and [`core/apps/team-metrics-custom`](https://github.com/calimero-network/core/tree/master/apps/team-metrics-custom) + +**What it does:** +Demonstrates nested CRDT structures for team metrics tracking. + +**Key features:** +- Nested `UnorderedMap` structures +- Map of team → Map of member → Counter +- Metrics aggregation patterns +- Complex CRDT hierarchies + +**API methods:** +- `increment_metric(team: String, member: String)` - Increment member metric +- `get_team_metrics(team: String) -> HashMap` - Get all team metrics +- `get_member_metric(team: String, member: String) -> i64` - Get specific metric + +**Learn from this example:** +- Nested CRDT patterns +- Multi-level data structures +- Metrics aggregation +- Complex state management + +## xcall-example + +**Location**: [`core/apps/xcall-example`](https://github.com/calimero-network/core/tree/master/apps/xcall-example) + +**What it does:** +Demonstrates cross-context calls (xcall) for inter-context communication. + +**Key features:** +- Cross-context method calls +- Context-to-context communication +- Ping-pong pattern demonstration +- Event emission from xcalls + +**API methods:** +- `ping(target_context: String)` - Send ping to another context +- `pong()` - Handle ping (increments counter) +- `get_counter() -> u64` - Get ping counter +- `reset_counter()` - Reset counter + +**Example usage:** +```bash +# Deploy to Context A and Context B + +# Send ping from Context A to Context B +meroctl --node node1 call \ + --context-id context-a \ + --method ping \ + --args '{"target_context": ""}' + +# Check counter on Context B +meroctl --node node1 call \ + --context-id context-b \ + --method get_counter +``` + +**Learn from this example:** +- Cross-context communication +- xcall patterns +- Inter-context coordination +- Multi-context applications + +## Running Examples + +### Using Merobox Workflows + +Most examples include Merobox workflow files for automated testing: + +```bash +# Run example workflow +cd core/apps/kv-store +merobox bootstrap run workflows/kv-store.yml + +# Or for other examples +cd core/apps/collaborative-editor +merobox bootstrap run workflows/collaborative-editor.yml +``` + +### Manual Testing + +```bash +# 1. Build the example +cd core/apps/kv-store +./build.sh + +# 2. Start nodes (via merobox) +merobox run --count 2 + +# 3. Install application +meroctl --node calimero-node-1 app install \ + --path res/kv_store.wasm \ + --application-id kv-store + +# 4. Create context +meroctl --node calimero-node-1 context create \ + --application-id kv-store \ + --context-id test-context + +# 5. Test methods +meroctl --node calimero-node-1 call \ + --context-id test-context \ + --method set \ + --args '{"key": "hello", "value": "world"}' +``` + +## Learning Path + +**Start here:** +1. **kv-store** - Understand basic CRDT operations +2. **kv-store-with-handlers** - Learn event handling +3. **collaborative-editor** - See advanced CRDT usage (RGA) + +**Then explore:** +- **blobs** - File sharing patterns +- **private-data** - Secrets management +- **team-metrics** - Complex nested structures +- **xcall-example** - Cross-context communication + +## Related Topics + +- [SDK Guide](../builder-directory/sdk-guide.md) - Complete SDK reference +- [Build Your First Application](../getting-started/index.md) - Step-by-step tutorial +- [Applications](../core-concepts/applications.md) - Application architecture + +## Deep Dives + +For detailed example documentation: + +- **kv-store**: [`core/apps/kv-store/README.md`](https://github.com/calimero-network/core/blob/master/apps/kv-store/README.md) +- **collaborative-editor**: [`core/apps/collaborative-editor/README.md`](https://github.com/calimero-network/core/blob/master/apps/collaborative-editor/README.md) +- **blobs**: [`core/apps/blobs/README.md`](https://github.com/calimero-network/core/blob/master/apps/blobs/README.md) +- **xcall-example**: [`core/apps/xcall-example/README.md`](https://github.com/calimero-network/core/blob/master/apps/xcall-example/README.md) + diff --git a/docs/examples/index.md b/docs/examples/index.md new file mode 100644 index 0000000..6ae56ef --- /dev/null +++ b/docs/examples/index.md @@ -0,0 +1,49 @@ +# Examples + +Learn Calimero by exploring working examples. All examples include source code, README files, and Merobox workflow configurations. + +## Core Apps Examples + +**[Core Apps Examples](core-apps-examples.md)** — Reference implementations from `core/apps` demonstrating SDK patterns and CRDT usage. + +These examples live in [`calimero-network/core/apps`](https://github.com/calimero-network/core/tree/master/apps): + +- **kv-store** - Basic CRDT operations, key-value storage +- **blobs** - File/blob management with content addressing +- **collaborative-editor** - Text collaboration with RGA CRDT +- **private-data** - Private storage patterns +- **team-metrics** - Nested CRDT structures +- **xcall-example** - Cross-context communication + +**Quick start:** +```bash +cd core/apps/kv-store +./build.sh +meroctl --node node1 app install --path build/kv_store.wasm +``` + +## Application Examples + +For complete applications, see: + +- **Battleships** - Multiplayer game - [`battleships`](https://github.com/calimero-network/battleships) repository +- **Shared Todo** - Collaborative task list - [`shared-todo-backlog`](https://github.com/calimero-network/shared-todo-backlog) repository +- **KV Store** - Template app - Created via `npx create-mero-app` + +## By Complexity + +Examples organized by difficulty: + +- **Starter**: Basic CRDT usage (kv-store, counters) +- **Intermediate**: Event handling, blob management, nested structures +- **Advanced**: Cross-context calls, complex state machines + +## Getting Started with Examples + +1. Clone or browse the repository +2. Read the README for setup instructions +3. Run `./build.sh` or `pnpm run logic:build` +4. Install on a local node using `meroctl` +5. Explore the code and adapt patterns to your app + +See [Getting Started](../getting-started/index.md) for installation and setup. diff --git a/docs/getting-started/index.md b/docs/getting-started/index.md new file mode 100644 index 0000000..784d606 --- /dev/null +++ b/docs/getting-started/index.md @@ -0,0 +1,282 @@ +# Getting Started + +Welcome to Calimero! This guide will help you get up and running quickly, whether you're building applications, running nodes, or just exploring the platform. + +## Quick Start Paths + +Choose your path based on what you want to do: + +| Goal | Path | Time | +| --- | --- | --- | +| **Run a local network** | Installation → Launch network | 10 min | +| **Build an application** | Installation → Create app → Deploy | 30 min | +| **Explore examples** | Installation → Run example | 15 min | + +## Step 1: Installation + +### Prerequisites + +**Required:** +- **Docker** 20.10+ (for running nodes) +- **Python** 3.8+ (for merobox) +- **Node.js** 18+ (for client SDKs and building JavaScript apps) +- **Rust toolchain** (for building Rust applications) + +**Optional:** +- `merod` and `meroctl` (if building from source) + +### Install Merobox + +Merobox is the easiest way to run local Calimero networks: + +```bash +# Using pipx (recommended) +pipx install merobox + +# Or on macOS with Homebrew +brew install calimero-network/tap/merobox + +# Verify installation +merobox --version +``` + +### Install Rust Toolchain (for building applications) + +```bash +# Install Rust +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh + +# Add WASM target +rustup target add wasm32-unknown-unknown + +# Verify installation +cargo --version +rustc --version +``` + +### Install Node.js SDKs (for client development) + +```bash +# Install Calimero JavaScript SDK +npm install @calimero/sdk @calimero/cli + +# Or with pnpm +pnpm add @calimero/sdk @calimero/cli +``` + +See [`merobox/README.md`](https://github.com/calimero-network/merobox#readme) for complete installation options. + +## Step 2: Run a Local Network + +### Using Merobox (Recommended) + +```bash +# Start a 2-node network +merobox run --count 2 + +# Check status +merobox list + +# View logs +merobox logs node1 --follow +``` + +### Using Docker Directly + +```bash +# Start coordinator +docker run -p 2528:2528 ghcr.io/calimero-network/merod:edge \ + --node-type coordinator + +# Start peer (in another terminal) +docker run -p 2529:2528 ghcr.io/calimero-network/merod:edge \ + --node-type peer \ + --swarm-addrs /ip4/127.0.0.1/tcp/2428 +``` + +### Verify Network + +```bash +# Check node health +meroctl --node node1 health + +# List contexts +meroctl --node node1 context list +``` + +See [Running Nodes](../operator-track/run-a-local-network.md) for detailed node management. + +## Step 3: Build Your First Application + +### Option A: Create from Template + +```bash +# Create a new application +npx create-mero-app my-first-app + +# Navigate to project +cd my-first-app + +# Install dependencies +pnpm install + +# Build Rust backend +pnpm run logic:build + +# This creates: +# - logic/build/my_first_app.wasm (WASM binary) +# - logic/build/my_first_app.abi.json (API definition) +``` + +### Option B: Build from Core Examples + +```bash +# Clone core repository +git clone https://github.com/calimero-network/core +cd core/apps/kv-store + +# Build WASM +./build.sh + +# Install on node +meroctl --node node1 app install \ + --path build/kv_store.wasm +``` + +### Option C: Build from Scratch + +**1. Create Rust project:** +```bash +cargo new --lib my-app +cd my-app +``` + +**2. Add dependencies to `Cargo.toml`:** +```toml +[dependencies] +calimero-sdk = { path = "../../crates/sdk" } +calimero-storage = { path = "../../crates/storage" } +``` + +**3. Write your application in `src/lib.rs`:** +```rust +use calimero_sdk::app; +use calimero_storage::collections::UnorderedMap; + +#[app::state] +pub struct MyApp { + items: UnorderedMap, +} + +#[app::logic] +impl MyApp { + pub fn add_item(&mut self, key: String, value: String) { + self.items.insert(key, value); + } + + pub fn get_item(&self, key: &str) -> app::Result> { + self.items.get(key)?.map(|v| v.get().clone()) + } +} +``` + +**4. Build WASM:** +```bash +cargo build --target wasm32-unknown-unknown --release +``` + +**5. Install on node:** +```bash +meroctl --node-name node1 app install \ + --path target/wasm32-unknown-unknown/release/my_app.wasm +``` + +See [SDK Guide](../builder-directory/sdk-guide.md) or [JavaScript SDK Guide](../builder-directory/js-sdk-guide.md) for detailed development guides. + +## Step 4: Create and Use a Context + +**Create a context:** +```bash +meroctl --node node1 context create \ + --application-id +``` + +**Call methods:** +```bash +# Call a mutation +meroctl --node node1 call \ + --context-id \ + --method add_item \ + --args '{"key": "hello", "value": "world"}' \ + --executor-public-key + +# Call a view +meroctl --node node1 call \ + --context-id \ + --method get_item \ + --args '{"key": "hello"}' +``` + +See [Contexts](../core-concepts/contexts.md) for context management details. + +## Step 5: Explore Examples + +### Core Examples + +Examples in [`core/apps`](https://github.com/calimero-network/core/tree/master/apps): + +- **kv-store** - Simple key-value store (great for learning basics) +- **blobs** - File/blob sharing with content addressing +- **collaborative-editor** - Text collaboration with RGA CRDT +- **private-data** - Private storage patterns +- **team-metrics** - Nested CRDT structures +- **xcall-example** - Cross-context communication + +**Run an example:** +```bash +cd core/apps/kv-store +./build.sh +meroctl --node-name node1 app install --path build/kv_store.wasm +``` + +### Application Examples + +- **Battleships** - Multiplayer game with real-time sync +- **Shared Todo** - Collaborative task list +- **KV Store** - Boilerplate template application + +See [Examples](../examples/index.md) for complete list. + +## What's Next? + +### For Builders + +- **[Core Concepts](../core-concepts/index.md)** - Deep dive into architecture, contexts, identity +- **[Rust SDK Guide](../builder-directory/sdk-guide.md)** - Complete Rust development guide +- **[JavaScript SDK Guide](../builder-directory/js-sdk-guide.md)** - JavaScript/TypeScript development +- **[Examples](../examples/core-apps-examples.md)** - Working example applications + +### For Operators + +- **[Running Nodes](../operator-track/run-a-local-network.md)** - Node setup and management +- **[Monitoring](../operator-track/monitor-and-debug.md)** - Observability and troubleshooting +- **[meroctl CLI](../tools-apis/meroctl-cli.md)** - Command-line reference + +### For Everyone + +- **[Architecture Overview](../core-concepts/architecture.md)** - How Calimero works +- **[Tools & APIs](../tools-apis/index.md)** - SDKs, CLI, developer tools + +## Common Questions + +**Q: Do I need to run my own node?** +A: For local development, yes. Use `merobox` to run nodes locally. For production, you can use hosted nodes or run your own. + +**Q: Which language should I use?** +A: Applications can be written in Rust (compiled to WASM) or JavaScript/TypeScript (using `@calimero/sdk`). Clients can use JavaScript/TypeScript or Python. + +**Q: How do I handle authentication?** +A: Calimero supports wallet-based authentication (NEAR, Ethereum, ICP). See [Identity](../core-concepts/identity.md) and [Client SDKs](../tools-apis/client-sdks.md). + +**Q: Can I use this offline?** +A: Yes! Calimero is offline-first. Apps work offline and sync when online. diff --git a/docs/index.md b/docs/index.md index 85d54bf..dbfe6aa 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,12 +1,52 @@ # Calimero Documentation -Welcome to the Calimero documentation. +## Why Calimero Exists -This site provides an overview of the Calimero network — a decentralized application layer focused on privacy, modularity, and verifiability. +The Internet was designed to be **peer-to-peer**. From its origins in **DARPA’s research on packet switching** — a response to the fragility of circuit-switched networks like telephony — the Internet’s architecture has always favored decentralization. Protocols such as **TCP/IP** and **SMTP** embody this spirit: open, resilient, and without a central authority. Calimero builds upon that same idea. -Use the sections below to explore: +- **Calimero is not a blockchain.** +- **Calimero is an application layer** built on top of the network — a place for collaboration, computation, and coordination between peers. +- Where a blockchain would rely on **consensus**, Calimero uses **CRDTs (Conflict-free Replicated Data Types)** for distributed consistency without global agreement. -- [Introduction](intro/index.md) — the philosophy and core ideas behind Calimero -- [App Directory](app-directory/index.md) — explore and deploy existing applications -- [Builder Directory](builder-directory/index.md) — learn how to build your own Calimero apps -- [Privacy, Verifiability & Security](privacy-verifiability-security/index.md) — understand how Calimero fits into the modular privacy stack +Calimero is the layer you reach for when you *don’t* need the guarantees (or costs) of consensus — when local autonomy and asynchronous coordination are enough. + +## Build Self-Sovereign Applications with CRDT-Powered P2P Sync + +Calimero Network is a framework for distributed, peer-to-peer applications with automatic conflict-free data synchronization, user-owned data, and verifiable off-chain computing. + +| Attribute | What it means | +| --- | --- | +| Local-first by default | Your data stays on your node; you control replication | +| DAG-based CRDT sync | Conflict resolution without coordination, resilient offline | +| Event-driven architecture | Real-time updates emitted across participating nodes | +| Encrypted P2P channels | End-to-end secure sharing between context members | +| WASM runtime | Build applications in Rust, ship deterministic WebAssembly | +| Multi-chain integrations | Connect NEAR, ICP, Ethereum, Stellar, ZKSync for attestations | + +Calimero is a privacy-focused application layer for peer-to-peer collaboration. This site stays concise on purpose: each section orients you in a few minutes, then links directly to the canonical GitHub READMEs for full architecture and workflows. + +## Quick Actions + +| Start here | What you get | +| --- | --- | +| [Launch a local network](builder-directory/index.md#minimal-dev-loop) | Bootstrap `merod` + Merobox and observe a context end-to-end. | +| [Build from a template](builder-directory/index.md#choose-your-starting-point) | Scaffold a Rust + React app with `create-mero-app`. | +| [Explore a reference app](app-directory/index.md#featured-projects) | Learn from maintained examples such as Battleships or Shared Todo. | +| [Understand the architecture](intro/index.md#core-architecture-snapshot) | See how contexts, nodes, state sync, and identity fit together. | + +## Choose Your Path + +| If you are… | Go to… | Why | +| --- | --- | --- | +| New to Calimero | [Introduction](intro/index.md) | Philosophy, architecture snapshot, and repo map. | +| Shipping an application | [Builder Directory](builder-directory/index.md) | Toolchain checklist, dev loop, and SDK links. | +| Evaluating existing apps | [App Directory](app-directory/index.md) | Spot maintained demos with direct README links. | +| Securing deployments | [Privacy · Verifiability · Security](privacy-verifiability-security/index.md) | Isolation model, identity delegation, auditability patterns. | +| Looking for tooling | [Tools & APIs](tools-apis/index.md) | Runtime, admin, SDK, and automation catalog. | + +## Highlights + +- **Contexts as private networks** — CRDT-backed state and scoped storage so teams can collaborate without global consensus. +- **Hierarchical identities** — Root keys delegate client keys per device, integrating with NEAR, Ethereum, Starknet, and ICP wallets. +- **Modular runtime** — `merod` orchestrates networking (libp2p), storage, and WASM apps with JSON-RPC/WebSocket surfaces. +- **Repository-first docs** — Detailed flows live in project READMEs such as [`calimero-network/core`](https://github.com/calimero-network/core#readme) and [`calimero-network/merobox`](https://github.com/calimero-network/merobox#readme). \ No newline at end of file diff --git a/docs/intro/index.md b/docs/intro/index.md index 19411ef..5ae231e 100644 --- a/docs/intro/index.md +++ b/docs/intro/index.md @@ -1,14 +1,160 @@ -# Introduction +# Introduction to Calimero -The Internet was designed to be **peer-to-peer**. From its origins in **DARPA’s research on packet switching** — a response to the fragility of circuit-switched networks like telephony — the Internet’s architecture has always favored decentralization. +## What is Calimero? -Protocols such as **TCP/IP** and **SMTP** embody this spirit: open, resilient, and without a central authority. -Calimero builds upon that same idea. +**Calimero Core** is the runtime for building peer-to-peer applications with automatic conflict-free synchronization. It enables offline-first, distributed apps that sync when online, without central servers. -## What Calimero Is (and Isn’t) +Calimero is an application layer built on top of the network — a place for collaboration, computation, and coordination between peers. Where a blockchain would rely on **consensus**, Calimero uses **CRDTs (Conflict-free Replicated Data Types)** for distributed consistency without global agreement. -- **Calimero is not a blockchain.** -- **Calimero is an application layer** built on top of the network — a place for collaboration, computation, and coordination between peers. -- Where a blockchain would rely on **consensus**, Calimero uses **CRDTs (Conflict-free Replicated Data Types)** for distributed consistency without global agreement. +## Core Value Proposition + +| Feature | What it means | +| --- | --- | +| **CRDT-based Sync** | Automatic conflict resolution without coordination | +| **Offline-first** | Works without connectivity, syncs when online | +| **Event-driven** | Real-time notifications across nodes | +| **WASM Runtime** | Write apps in Rust, run in sandboxed environment | +| **Multi-chain** | Integrates with NEAR, ICP, Ethereum, Stellar, zkSync | + +## Key Design Principles + +1. **Data Ownership** - Your data on your devices, you control access +2. **Privacy by Default** - End-to-end encryption +3. **Distributed by Design** - P2P architecture, no single point of failure +4. **Developer-Friendly** - Simple API with powerful CRDT primitives + +## What Makes Calimero Unique? + +Calimero combines several powerful technologies to deliver a unique distributed computing platform: + +- ✅ **Automatic conflict resolution** via CRDTs — no manual merge code needed. Write natural code with nested data structures; the storage layer handles synchronization automatically. +- ✅ **Causal ordering** via DAG — handles out-of-order network delivery gracefully. Deltas can arrive in any order; the system buffers and applies them in correct causal sequence. +- ✅ **Dual sync strategy** — fast broadcast (~100-200ms) + reliable catch-up. Gossipsub provides real-time propagation, while periodic P2P sync ensures eventual consistency even after network issues. +- ✅ **Event-driven architecture** — real-time updates without polling. Applications emit events that automatically propagate to all peers and trigger handlers. +- ✅ **Local-first** — data owned by users, no central authority required. Your data lives on your devices; you control access and sharing. + +## Use Cases + +Calimero enables a wide range of decentralized applications: + +| Use Case | Why Calimero? | +| --- | --- | +| **Collaborative Editing** | Real-time document collaboration without servers. Multiple users edit simultaneously; conflicts resolve automatically via CRDTs. | +| **Decentralized Social** | User-controlled social networks. Each user runs their own node; data ownership and privacy by default. | +| **P2P Gaming** | Multiplayer games with automatic state sync. Game state synchronizes across players even with network interruptions. | +| **IoT Networks** | Decentralized device coordination. Devices coordinate without central servers; works offline and syncs when online. | +| **Supply Chain** | Transparent, multi-party tracking. All participants maintain their own copy; automatic conflict resolution handles concurrent updates. | +| **Healthcare** | Private, patient-controlled medical records. Patients own their data; selective sharing with healthcare providers via encrypted contexts. | + +See [Example Applications](../examples/index.md) for working implementations. + +## Performance Characteristics + +- **Latency**: 100-200ms delta propagation (Gossipsub) +- **Throughput**: 100-1000 deltas/sec per context +- **Memory**: ~10MB per context (1000 deltas) +- **Scalability**: Horizontal (more contexts = more throughput) + +## Where to Start + +| If you are… | Start here | +| --- | --- | +| **Building apps** | [Getting Started](../getting-started/index.md) → [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) | +| **Running nodes** | [Run a Local Network](../operator-track/run-a-local-network.md) → [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) | +| **Understanding sync** | [Architecture Overview](../core-concepts/architecture.md) → [`core/crates/dag/README.md`](https://github.com/calimero-network/core/blob/master/crates/dag/README.md) | +| **Debugging issues** | [Monitor & Debug](../operator-track/monitor-and-debug.md) → [`core/crates/node/readme/troubleshooting.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/troubleshooting.md) | + +## Core Architecture Layers + +Calimero's architecture consists of four main layers: + +### 1. Application Layer +- WASM apps using the Calimero SDK +- CRDT collections: `UnorderedMap`, `Vector`, `Counter`, `LwwRegister` +- Event emission for real-time updates + +**Documentation**: [Applications](../core-concepts/applications.md) → [`core/crates/sdk/README.md`](https://github.com/calimero-network/core/blob/master/crates/sdk/README.md) + +### 2. Node Layer +- Orchestrates synchronization and execution +- Dual sync paths: Gossipsub (~100-200ms) + Periodic P2P (every 10-30s) +- Event handler execution across the network +- Blob distribution for file sharing + +**Documentation**: [Nodes](../core-concepts/nodes.md) → [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) + +### 3. Storage Layer +- CRDT storage with automatic merging +- DAG (Directed Acyclic Graph) for causal ordering +- Handles out-of-order delivery with dependency resolution +- Merkle trees for efficient state comparison + +**Documentation**: [Architecture Overview](../core-concepts/architecture.md) → [`core/crates/storage/README.md`](https://github.com/calimero-network/core/blob/master/crates/storage/README.md) + +### 4. Network Layer +- libp2p-based P2P (Gossipsub, reliable streams, DHT discovery) +- JSON-RPC server for client interaction +- WebSocket/SSE for real-time subscriptions +- Authentication and authorization + +**Documentation**: [Reference](../reference/index.md) → [`core/crates/network/README.md`](https://github.com/calimero-network/core/blob/master/crates/network/README.md) + +## How It Works (Transaction Flow) + +1. **Client calls WASM method** via JSON-RPC +2. **WASM executes**, generates CRDT actions (inserts, increments, etc.) +3. **Actions committed** as DAG delta with causal parents +4. **Delta broadcast** via Gossipsub to all peers (~100-200ms) +5. **Peers receive delta**, check if parents are ready +6. **If ready**: apply immediately and execute event handlers +7. **If not ready**: buffer until parents arrive (DAG handles this automatically) +8. **Periodic P2P sync** ensures eventual consistency + +For detailed flow diagrams, see [Architecture Overview](../core-concepts/architecture.md). + +## Core Repository Structure {#core-architecture-snapshot} + +The Calimero Core repository (`calimero-network/core`) contains the runtime and all supporting infrastructure: + +``` +core/ +├── crates/ # Core Rust libraries (30+ crates) +│ ├── sdk/ # Application SDK - macros, CRDT collections, event emission +│ ├── runtime/ # WASM execution engine (Wasmer), sandboxing, resource limits +│ ├── storage/ # CRDT storage layer - collections with merge semantics +│ ├── dag/ # DAG logic (pure) - causal delta tracking, dependency resolution +│ ├── node/ # Node runtime - orchestrates sync, events, blob sharing +│ ├── network/ # P2P networking - libp2p integration (Gossipsub, streams, DHT) +│ ├── server/ # JSON-RPC server - API, WebSocket/SSE subscriptions, admin endpoints +│ ├── auth/ # Authentication - identity management, JWT tokens +│ ├── store/ # Persistent storage backend (RocksDB) +│ ├── context/ # Context lifecycle management (application instances) +│ ├── merod/ # Node binary (coordinator or peer) +│ └── meroctl/ # CLI tool - node operations, app installation, method calls +├── apps/ # Example applications +│ ├── kv-store/ # Simple key-value store +│ ├── blobs/ # Blob sharing example +│ └── ... # More examples +├── contracts/ # Cross-chain smart contracts +│ ├── near/ # NEAR Protocol contracts +│ ├── icp/ # Internet Computer Protocol canisters +│ ├── ethereum/ # Solidity contracts +│ └── stellar/ # Stellar contracts +└── e2e-tests/ # End-to-end test suite +``` + +For detailed component explanations, see [Architecture Overview](../core-concepts/architecture.md#key-components). + +## Next Steps + +- 📖 **New to Calimero?** → [Getting Started](../getting-started/index.md) +- 🏗️ **Ready to build?** → [Builder Directory](../builder-directory/index.md) +- 🎓 **Want to learn?** → [Core Concepts](../core-concepts/index.md) +- ⚙️ **Need to operate?** → [Operator Track](../operator-track/index.md) + +--- + +**Built with ❤️ by the Calimero Network team** + +For questions, reach out on [Discord](https://discord.gg/wZRC73DVpU) or [GitHub Issues](https://github.com/calimero-network/core/issues). -Calimero is the layer you reach for when you *don’t* need the guarantees (or costs) of consensus — when local autonomy and asynchronous coordination are enough. diff --git a/docs/javascripts/sidebar.js b/docs/javascripts/sidebar.js new file mode 100644 index 0000000..1552e10 --- /dev/null +++ b/docs/javascripts/sidebar.js @@ -0,0 +1,22 @@ +document$.subscribe(() => { + const hideIfSingle = (selector) => { + const sidebar = document.querySelector(selector); + if (!sidebar) return; + + const list = sidebar.querySelector('nav > ul.md-nav__list'); + if (!list) return; + + const links = Array.from(list.querySelectorAll('li.md-nav__item > a.md-nav__link')).filter( + (link) => link.offsetParent !== null + ); + + if (links.length <= 1) { + sidebar.classList.add('is-hidden-single'); + } else { + sidebar.classList.remove('is-hidden-single'); + } + }; + + hideIfSingle('.md-sidebar--primary'); + hideIfSingle('.md-sidebar--secondary'); +}); diff --git a/docs/operator-track/index.md b/docs/operator-track/index.md new file mode 100644 index 0000000..92d6e88 --- /dev/null +++ b/docs/operator-track/index.md @@ -0,0 +1,35 @@ +# Operating Calimero Nodes + +Guidance for running and managing Calimero nodes in production. + +## Quick Links + +| Task | Guide | Repository Docs | +| --- | --- | --- | +| **Local Development** | [Run a Local Network](run-a-local-network.md) | [`merobox/README.md`](https://github.com/calimero-network/merobox#readme) | +| **Monitoring** | [Monitor & Debug](monitor-and-debug.md) | [`core/crates/node/readme/troubleshooting.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/troubleshooting.md) | + +## Running Nodes + +For complete node documentation, see [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md). + +**Quick start:** +```bash +# Install merod +cargo install --path core/crates/merod + +# Run coordinator +merod --node-type coordinator + +# Run peer +merod --node-type peer --swarm-addrs /ip4/127.0.0.1/tcp/2428 +``` + +**Or use Merobox:** +```bash +merobox run --count 2 +``` + +## Monitoring + +See [Monitor & Debug](monitor-and-debug.md) for observability and troubleshooting. diff --git a/docs/operator-track/monitor-and-debug.md b/docs/operator-track/monitor-and-debug.md new file mode 100644 index 0000000..2c13e22 --- /dev/null +++ b/docs/operator-track/monitor-and-debug.md @@ -0,0 +1,68 @@ +# Monitor & Debug + +Observability and troubleshooting for Calimero nodes. + +## Quick Health Checks + +```bash +# Check node health +meroctl --node node1 health + +# View logs +merobox logs node1 --follow + +# List contexts +meroctl --node node1 context list +``` + +## Admin Dashboard + +Access the web UI at `http://localhost:2528/admin-dashboard` (or your node URL). + +**Features:** +- Context management +- Application installation +- Identity management +- Metrics and stats + +## Monitoring Endpoints + +```bash +# Health check +curl http://localhost:2528/admin-api/health + +# Get peers count +curl http://localhost:2528/admin-api/peers-count + +# List contexts +curl http://localhost:2528/admin-api/contexts +``` + +## Logs + +```bash +# View node logs +merobox logs node1 + +# Follow logs in real-time +merobox logs node1 --follow + +# Or with Docker directly +docker logs calimero-node-1 --follow +``` + +## Troubleshooting + +See [`core/crates/node/readme/troubleshooting.md`](https://github.com/calimero-network/core/blob/master/crates/node/readme/troubleshooting.md) for: +- Common issues and solutions +- Performance tuning +- Network problems +- Storage issues + +## Metrics + +Nodes expose metrics at: +- **Admin API**: `http://localhost:2528/admin-api/metrics` +- **Prometheus**: Configure in node settings + +See [`core/crates/node/README.md`](https://github.com/calimero-network/core/blob/master/crates/node/README.md) for monitoring configuration. diff --git a/docs/operator-track/run-a-local-network.md b/docs/operator-track/run-a-local-network.md new file mode 100644 index 0000000..ed5a8c2 --- /dev/null +++ b/docs/operator-track/run-a-local-network.md @@ -0,0 +1,93 @@ +# Run a Local Network + +Quick guide to running Calimero nodes locally for development and testing. + +## Using Merobox (Recommended) + +Merobox is the easiest way to run local networks. See [`merobox/README.md`](https://github.com/calimero-network/merobox#readme) for complete documentation. + +**Quick start:** +```bash +# Install merobox +pipx install merobox + +# Start 2-node network +merobox run --count 2 + +# Check status +merobox list +merobox health +``` + +**With workflow:** +```bash +merobox bootstrap run workflow.yml +``` + +## Using merod Directly + +For more control, run nodes directly without Docker: + +```bash +# Install merod (from source) +cargo install --path core/crates/merod + +# Or build and use directly +cd core/crates/merod +cargo build --release +``` + +**Initialize and run a single node:** + +```bash +# Initialize a node with default settings +merod --node-name node1 init + +# Or with custom ports +merod --node-name node1 init --server-port 2428 --swarm-port 2528 + +# Run the node +merod --node-name node1 run +``` + +**Run multiple nodes manually:** + +```bash +# Terminal 1: Start first node (coordinator) +merod --node-name node1 init --server-port 2428 --swarm-port 2528 +merod --node-name node1 run + +# Terminal 2: Start second node (peer) +merod --node-name node2 init --server-port 2429 --swarm-port 2529 +merod --node-name node2 config --swarm-addrs /ip4/127.0.0.1/tcp/2528 +merod --node-name node2 run + +# Terminal 3: Start third node (peer) +merod --node-name node3 init --server-port 2430 --swarm-port 2530 +merod --node-name node3 config --swarm-addrs /ip4/127.0.0.1/tcp/2528 +merod --node-name node3 run +``` + +**Configuration:** + +```bash +# Configure node settings +merod --node-name node1 config --server-host 0.0.0.0 --server-port 3000 +merod --node-name node1 config --swarm-host 0.0.0.0 --swarm-port 2428 + +# Use custom home directory +merod --home ~/.calimero-custom --node-name node1 init +``` + +See [`meroctl CLI`](../tools-apis/meroctl-cli.md) for managing contexts and use `merod --help` for all available node options. + +## Requirements + +- Docker 20.10+ (for Merobox) +- Rust toolchain (if building from source) +- Python 3.8+ (for Merobox) + +## Next Steps + +- [Monitor & Debug](monitor-and-debug.md) - Observability and troubleshooting +- [Developer Tools](../tools-apis/developer-tools.md) - Merobox, ABI codegen, scaffolding diff --git a/docs/privacy-verifiability-security/index.md b/docs/privacy-verifiability-security/index.md index 749a690..b29adff 100644 --- a/docs/privacy-verifiability-security/index.md +++ b/docs/privacy-verifiability-security/index.md @@ -13,3 +13,41 @@ In this modular stack: - Other tools like **zero-knowledge proofs (ZK)** can extend privacy guarantees. Calimero is a component of a broader **privacy-oriented architecture** — modular, composable, and adaptable to your needs. + +## Isolation Model at a Glance + +- **Contexts scope visibility** — each context has shared CRDT state plus per-member private storage (`#[app::private]`). +- **Hierarchical identities** — root keys issue client keys per device or integration; revocation cascades from the root. +- **Deterministic runtime** — WASM apps run inside `merod`, so every state transition is deterministic and replayable. +- **Selective disclosure** — events can reveal only hashed or redacted payloads while full data stays on the owner’s node. +- **Audit trail** — every method call is tied to the caller via `executor_id`, enabling tamper-evident logs. + +## Verification Pathways + +| Layer | What is verified | How | +| --- | --- | --- | +| Context membership | Who can read/write state | Anchored invites or role assignments persisted on-chain | +| State synchronization | CRDT merges, Merkle checkpoints | Nodes exchange proofs before accepting remote updates | +| Application integrity | WASM binaries, configuration | Hashes committed to L1, compared during deployment | +| User actions | Caller identity, authorization | Challenge/response over wallet connector + executor audit logs | +| Data access | Private vs shared storage | Storage namespaces tied to caller identity, enforced in runtime | + +## Hardening Checklist + +1. Anchor critical context membership changes to your chosen L1 (NEAR, Ethereum, Starknet, ICP). +2. Enable event payload hashing when emitting sensitive data; share full payloads via authenticated channels only. +3. Rotate client keys on a cadence and revoke stale devices at the root key level. +4. Run periodic Merkle checkpoint comparisons across nodes to detect divergence early. +5. Configure node monitoring (Admin Dashboard, Node Console) to alert on failed syncs or unauthorized method calls. + +## Where to Deep Dive + +| Topic | Reference | Why it matters | +| --- | --- | --- | +| Runtime architecture & security model | [`calimero-network/core` – Architecture](https://github.com/calimero-network/core#architecture) | Details on `merod`, networking layers, and verification primitives | +| Identity delegation & permissions | [`calimero-network/contracts` – Identity module](https://github.com/calimero-network/contracts/blob/main/README.mdx#identity-module) | How root/client keys, invites, and revocations are enforced | +| Context lifecycle & admin API | [`calimero-network/merobox` – Workflows](https://github.com/calimero-network/merobox#workflows) | Managing contexts, capturing Application IDs, production rollouts | +| Authentication adapters & wallet flows | [`calimero-network/core/apps/auth`](https://github.com/calimero-network/core/tree/main/apps/auth#readme) | Challenge/response flows for NEAR, Ethereum, Starknet, ICP | +| Advanced cryptography experiments | [`calimero-network/experiments/mpc-signer`](https://github.com/calimero-network/experiments/tree/main/mpc-signer#readme) | Threshold signing, multi-party custody, and ZK experiments | + +_This page stays high-level. For full setup steps, audit procedures, and API details, follow the linked READMEs._ diff --git a/docs/reference/index.md b/docs/reference/index.md new file mode 100644 index 0000000..b14877a --- /dev/null +++ b/docs/reference/index.md @@ -0,0 +1,53 @@ +# API Reference + +Complete API documentation for Calimero nodes and clients. + +## Server API + +Calimero nodes expose a JSON-RPC API over HTTP with WebSocket support for real-time subscriptions. + +**Base URL**: `http://localhost:2528` (default) + +### Endpoints + +- **JSON-RPC**: `/jsonrpc` - Standard JSON-RPC 2.0 calls +- **WebSocket**: `/ws` - Real-time subscriptions +- **SSE**: `/sse` - Server-Sent Events for subscriptions +- **Admin API**: `/admin-api/` - Node management endpoints + +### API Methods + +Common methods: +- `context.create` - Create a new context +- `context.list` - List all contexts +- `context.call` - Call a method on a context +- `app.install` - Install an application +- `identity.create` - Create an identity + +### Authentication + +Most endpoints require JWT authentication. See [Identity](../core-concepts/identity.md) for authentication flows. + +## Complete Documentation + +For complete API documentation, including all methods, request/response formats, and error codes: + +**→ [`core/crates/server/README.md`](https://github.com/calimero-network/core/blob/master/crates/server/README.md)** + +## Client SDKs + +For easier client development, use the official SDKs: + +- **JavaScript/TypeScript**: [`@calimero/client`](https://github.com/calimero-network/calimero-client-js) - See [Client SDKs](../tools-apis/client-sdks.md) +- **Python**: [`calimero-client-py`](https://github.com/calimero-network/calimero-client-py) - See [Client SDKs](../tools-apis/client-sdks.md) +- **Rust**: [`core/crates/client`](https://github.com/calimero-network/core/blob/master/crates/client/README.md) + +## CLI Reference + +For command-line operations, see [`meroctl` CLI](../tools-apis/meroctl-cli.md). + +## Related Topics + +- [Tools & APIs](../tools-apis/index.md) - SDKs and developer tools +- [Core Concepts](../core-concepts/index.md) - Architecture and concepts +- [Running Nodes](../operator-track/run-a-local-network.md) - Node setup diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index db1cb65..9ed3e04 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -1,20 +1,230 @@ /* Custom Calimero Branding Styles */ /* ============================================ - CUSTOM COLORS + CUSTOM COLORS (Calimero Brand Palette) ============================================ */ :root { - --md-accent-fg-color: #A5FF11; + /* Neutrals */ + --calimero-neutral-900: #131215; + --calimero-neutral-800: #222222; + --calimero-neutral-700: #282828; + --calimero-neutral-600: #404040; + --calimero-neutral-500: #5C5C5C; + --calimero-neutral-400: #8E8E8E; + --calimero-neutral-300: #A0A0A0; + --calimero-neutral-200: #C8C8C8; + --calimero-neutral-100: #F3F3F3; + + /* Brand greens */ + --calimero-green-900: #2D381B; + --calimero-green-800: #8AA200; + --calimero-green-700: #73B30C; + --calimero-green-600: #A5FF11; + --calimero-green-500: #E0F97A; +} + +/* Light theme (default) */ +[data-md-color-scheme="default"] { + --md-default-bg-color: #F5F5F4; /* light neutral backdrop */ + --md-default-fg-color: #1F1F1F; + --md-primary-fg-color: var(--calimero-green-800); + --md-primary-bg-color: #FFFFFF; + --md-accent-fg-color: var(--calimero-green-700); + --md-accent-bg-color: rgba(115, 179, 12, 0.14); + --md-code-bg-color: #EFEFEF; + --md-code-fg-color: #222222; +} + +[data-md-color-scheme="default"] body, +[data-md-color-scheme="default"] .md-main, +[data-md-color-scheme="default"] .md-content, +[data-md-color-scheme="default"] .md-header, +[data-md-color-scheme="default"] .md-sidebar { + background-color: var(--md-default-bg-color); + color: var(--md-default-fg-color); +} + +[data-md-color-scheme="default"] .md-header { + background: linear-gradient(90deg, rgba(255, 255, 255, 0.95), rgba(240, 246, 222, 0.85)); + border-bottom: 1px solid rgba(31, 31, 31, 0.08); + box-shadow: 0 4px 12px rgba(19, 18, 21, 0.05); +} + +[data-md-color-scheme="default"] .md-tabs { + background: linear-gradient(90deg, rgba(240, 246, 222, 0.95), rgba(209, 238, 140, 0.65)); + color: var(--md-default-fg-color); +} + +[data-md-color-scheme="default"] .md-tabs__link { + color: var(--md-default-fg-color); + opacity: 0.8; +} + +[data-md-color-scheme="default"] .md-tabs__item--active .md-tabs__link, +[data-md-color-scheme="default"] .md-tabs__link:hover { + color: var(--calimero-green-800); + opacity: 1; +} + +[data-md-color-scheme="default"] .md-sidebar__scrollwrap { + background-color: #FFFFFF; + border-right: 1px solid rgba(31, 31, 31, 0.06); +} + +[data-md-color-scheme="default"] .md-nav__link--active { + color: #6B8611; +} + +[data-md-color-scheme="default"] .md-nav__title { + color: #4E5F17; +} + +[data-md-color-scheme="default"] .md-typeset a { + color: #58730A; +} + +[data-md-color-scheme="default"] .md-typeset a:hover { + color: #3E5207; +} + +[data-md-color-scheme="default"] .md-typeset h1, +[data-md-color-scheme="default"] .md-typeset h2, +[data-md-color-scheme="default"] .md-typeset h3 { + color: #202320; +} + +[data-md-color-scheme="default"] .md-typeset table:not([class]) th { + background-color: rgba(138, 162, 0, 0.22); + color: #222222; +} + +[data-md-color-scheme="default"] .md-typeset table:not([class]) tbody tr { + background-color: rgba(255, 255, 255, 0.9); +} + +[data-md-color-scheme="default"] .md-typeset table:not([class]) tbody tr:nth-child(2n) { + background-color: rgba(240, 240, 240, 0.9); +} + +[data-md-color-scheme="default"] .md-typeset .admonition, +[data-md-color-scheme="default"] .md-typeset details { + background-color: rgba(209, 238, 140, 0.25); + border-color: rgba(115, 179, 12, 0.6); +} + +[data-md-color-scheme="default"] blockquote { + border-left: 0.25rem solid rgba(115, 179, 12, 0.7); + background-color: rgba(255, 255, 255, 0.85); + color: #262626; +} + +[data-md-color-scheme="default"] .md-button--primary { + background: var(--calimero-green-700); + color: #111; +} + +[data-md-color-scheme="default"] .md-button--primary:hover { + background: var(--calimero-green-600); +} + +/* Dark theme (slate) */ +[data-md-color-scheme="slate"] { + --md-default-bg-color: var(--calimero-neutral-900); + --md-default-fg-color: #E5E5E5; + --md-primary-fg-color: var(--calimero-green-600); + --md-primary-bg-color: var(--calimero-neutral-900); + --md-accent-fg-color: var(--calimero-green-600); + --md-accent-bg-color: rgba(165, 255, 17, 0.1); + --md-code-bg-color: rgba(40, 40, 40, 0.85); + --md-code-fg-color: #E5E5E5; +} + +[data-md-color-scheme="slate"] .md-header { + border-bottom: 1px solid rgba(255, 255, 255, 0.06); +} + +[data-md-color-scheme="slate"] .md-tabs { + background: linear-gradient(90deg, rgba(45, 56, 27, 0.9), rgba(73, 140, 24, 0.6)); + color: var(--md-default-fg-color); +} + +[data-md-color-scheme="slate"] .md-tabs__link { + color: var(--md-default-fg-color); + opacity: 0.75; +} + +[data-md-color-scheme="slate"] .md-tabs__link:hover, +[data-md-color-scheme="slate"] .md-tabs__link--active { + color: var(--calimero-green-600); + opacity: 1; } /* Content body link colors */ .md-typeset a { - color: #8AA200; + color: var(--calimero-green-600); } .md-typeset a:hover { - color: #A5FF11; + color: var(--calimero-green-700); +} + +/* Card and blockquote styling */ +.md-typeset .admonition, +.md-typeset details { + background-color: rgba(45, 56, 27, 0.35); + border-color: var(--calimero-green-800); +} + +blockquote { + border-left: 0.25rem solid var(--calimero-green-700); + background-color: rgba(19, 18, 21, 0.6); +} + +/* Table styling */ +.md-typeset table:not([class]) th { + background-color: rgba(138, 162, 0, 0.15); + color: var(--md-default-fg-color); +} + +.md-typeset table:not([class]) tbody tr { + background-color: rgba(34, 34, 34, 0.5); +} + +.md-typeset table:not([class]) tbody tr:nth-child(2n) { + background-color: rgba(40, 40, 40, 0.6); +} + +/* Tabs navigation */ +.md-tabs { + background: linear-gradient(90deg, rgba(45, 56, 27, 0.9), rgba(73, 140, 24, 0.6)); + color: var(--md-default-fg-color); +} + +.md-tabs__link { + color: var(--md-default-fg-color); + opacity: 0.75; +} + +.md-tabs__link:hover, +.md-tabs__link--active { + color: var(--calimero-green-600); + opacity: 1; +} + +/* Buttons */ +.md-button--primary { + background: var(--calimero-green-700); + color: #0E0F0E; +} + +.md-button--primary:hover { + background: var(--calimero-green-600); +} + +/* Sidebar headings */ +.md-nav__title { + color: var(--calimero-green-500); } /* Light mode: use black logo and proper header colors */ @@ -68,105 +278,40 @@ LAYOUT IMPROVEMENTS ============================================ */ -/* Hide sidebars on desktop only (NOT on mobile - needed for hamburger menu) */ -@media screen and (min-width: 60em) { +/* Swap sidebars so main navigation appears on the left and page table of contents on the right */ +@media screen and (min-width: 76.25rem) { .md-sidebar--primary { - display: none !important; + order: 1; + border-right: 1px solid var(--md-default-fg-color--lightest); } - - .md-sidebar--secondary { - display: none !important; - } -} -/* Wider content area - no sidebars on desktop */ -.md-main__inner { - max-width: none; -} - -.md-content { - max-width: 1200px; - margin: 0 auto; -} - -/* Remove margins for sidebars on desktop */ -@media screen and (min-width: 60em) { - .md-content__inner { - margin-left: 0; - margin-right: 0; - } -} - -/* ============================================ - MOBILE OPTIMIZATIONS - ============================================ */ - -/* Mobile-specific padding to prevent text from touching edges */ -@media screen and (max-width: 59.99em) { - /* Main content padding */ - .md-content__inner { - padding-left: 1.5rem !important; - padding-right: 1.5rem !important; - } - - /* Typography padding */ - .md-typeset { - padding-left: 0 !important; - padding-right: 0 !important; - } - - /* Ensure proper padding for main grid */ - .md-main { - padding-left: 0; - padding-right: 0; - } - - /* Content container padding */ .md-content { - margin-left: 0; - margin-right: 0; - padding-left: 0; - padding-right: 0; - } - - /* Ensure the primary sidebar (navigation drawer) is visible and functional on mobile */ - .md-sidebar--primary { - display: block !important; - } - - /* Style the navigation drawer when it opens */ - .md-sidebar--primary .md-sidebar__scrollwrap { - background-color: var(--md-default-bg-color); + order: 2; } - - /* Ensure hamburger menu button is visible and clickable */ - .md-header__button.md-icon { - display: inline-block; - color: var(--md-default-fg-color); - } -} -/* Tabs navigation - adapt to light/dark mode */ -.md-tabs { - background-color: var(--md-default-bg-color) !important; - color: var(--md-default-fg-color); + .md-sidebar--secondary { + order: 3; + border-left: 1px solid var(--md-default-fg-color--lightest); + } } -.md-tabs__list { - background-color: var(--md-default-bg-color) !important; +/* Hide the secondary sidebar when there is only a single ToC entry */ +@media screen and (min-width: 76.25rem) { + .md-sidebar--secondary ul.md-nav__list > li:first-child:last-child { + display: none; + } } -.md-tabs__link { - color: var(--md-default-fg-color); - opacity: 0.7; +/* Hide the primary sidebar when there is only a single navigation entry */ +@media screen and (min-width: 76.25rem) { + .md-sidebar--primary nav > ul.md-nav__list:not(:has(> li.md-nav__item:nth-child(2))) { + display: none; + } } -.md-tabs__link:hover { - opacity: 1; +/* Utility class used by sidebar.js to hide sidebars with single items */ +.md-sidebar.is-hidden-single { + display: none !important; } -.md-tabs__link--active { - color: var(--md-primary-fg-color); - opacity: 1; -} diff --git a/docs/support/contact-channels.md b/docs/support/contact-channels.md new file mode 100644 index 0000000..b0c723a --- /dev/null +++ b/docs/support/contact-channels.md @@ -0,0 +1,7 @@ +# Contact Channels + +_Stub page — list official support avenues._ + +- TODO: Provide Discord invite, GitHub Issues, Discussions, and email. +- TODO: Set expectations for response times and SLAs. +- TODO: Include guidance on what info to include when requesting help. diff --git a/docs/support/contribution-guide.md b/docs/support/contribution-guide.md new file mode 100644 index 0000000..56e8c2c --- /dev/null +++ b/docs/support/contribution-guide.md @@ -0,0 +1,7 @@ +# Contribution Guide + +_Stub page — explain how to contribute to Calimero docs and repos._ + +- TODO: Outline pull request expectations and review checklist. +- TODO: Reference style guide, linting, and testing requirements. +- TODO: Provide link to contributor license agreement if applicable. diff --git a/docs/support/faq.md b/docs/support/faq.md new file mode 100644 index 0000000..25f6b84 --- /dev/null +++ b/docs/support/faq.md @@ -0,0 +1,7 @@ +# FAQ + +_Stub page — capture top troubleshooting questions._ + +- TODO: Add common setup, build, and deployment issues with quick fixes. +- TODO: Link to detailed README sections for deeper guidance. +- TODO: Reference support channels when escalation is needed. diff --git a/docs/support/index.md b/docs/support/index.md new file mode 100644 index 0000000..f9f4b72 --- /dev/null +++ b/docs/support/index.md @@ -0,0 +1,32 @@ +# Support & Community + +> Placeholder hub — fill in once support pathways are confirmed. + +- [FAQ](faq.md) +- [Contact Channels](contact-channels.md) +- [Contribution Guide](contribution-guide.md) +- [Roadmap & Feedback](roadmap-feedback.md) + +## FAQ + +- Capture top troubleshooting questions. +- Include quick fixes and pointers to READMEs. + +## Contact Channels + +- Discord, GitHub Issues, Discussions, email. +- Note response expectations. + +## Contribution Guide + +- Outline how to propose doc updates and code changes. +- Mention style guide and review checklists. + +## Roadmap & Feedback + +- Link to public roadmap or tracking board. +- Provide feedback form or discussion thread links. + +--- + +_Substitute bullets with finalized copy and links once stakeholders approve messaging._ diff --git a/docs/support/roadmap-feedback.md b/docs/support/roadmap-feedback.md new file mode 100644 index 0000000..3aab83c --- /dev/null +++ b/docs/support/roadmap-feedback.md @@ -0,0 +1,7 @@ +# Roadmap & Feedback + +_Stub page — connect users with planning and feedback loops._ + +- TODO: Link to public roadmap or planning board once defined. +- TODO: Provide form or discussion thread for feedback submissions. +- TODO: Summarize how feedback is triaged and prioritized. diff --git a/docs/tools-apis/client-sdks.md b/docs/tools-apis/client-sdks.md new file mode 100644 index 0000000..2b98a2e --- /dev/null +++ b/docs/tools-apis/client-sdks.md @@ -0,0 +1,768 @@ +# Calimero Client SDKs + +Client SDKs for interacting with Calimero nodes programmatically. These SDKs provide programmatic access to Calimero's admin API, enabling you to build sidecar tools, developer utilities, monitoring scripts, and automation workflows. + +## Overview + +Calimero provides three client SDKs for different language ecosystems: + +| SDK | Language | Repository | Authentication Support | Primary Use Cases | +| --- | --- | --- | --- | --- | +| **Rust Client** | Rust | `core/crates/client` | ⚠️ Not yet supported | Sidecar tools, CLI utilities, developer tools | +| **Python Client** | Python | `calimero-client-py` | ⚠️ Not yet supported | Automation scripts, monitoring tools, developer tools | +| **JavaScript Client** | TypeScript/JavaScript | `calimero-client-js` | ✅ Full support | Web apps, browser extensions, Node.js tools | + +!!! warning "Authentication Status" + **Rust and Python clients do not currently support authentication.** Authentication support is planned for future releases. Current usage is intended for: + - **Sidecar tools** - Local services running alongside Calimero nodes + - **Developer tools** - Scripts and utilities for development/testing + - **Internal automation** - CI/CD pipelines and internal tooling + + **JavaScript client has full authentication support** including JWT token management, wallet-based authentication, and React components for user authentication flows. + +## Use Cases + +### Sidecar Tools + +Tools that run alongside Calimero nodes to provide additional functionality: + +- **Metrics collectors** - Export node metrics to Prometheus, DataDog, etc. +- **Log aggregators** - Process and forward node logs +- **Health checkers** - Monitor node health and alert on issues +- **Backup services** - Periodically backup node state +- **Monitoring dashboards** - Custom dashboards for node status + +### Developer Tools + +Utilities for development and testing: + +- **Test scripts** - Automated testing of Calimero applications +- **Deployment tools** - Scripts for deploying and managing applications +- **Debugging tools** - Utilities for inspecting node state +- **Development helpers** - Scaffolding and code generation tools + +### Automation & CI/CD + +Automated workflows for DevOps: + +- **CI pipelines** - Automated testing and deployment +- **Release automation** - Scripts for packaging and releasing +- **Health monitoring** - Automated health checks and alerts +- **Data migration** - Scripts for migrating data between nodes + +## Rust Client SDK + +The Rust client SDK (`core/crates/client`) provides a trait-based abstraction for interacting with Calimero nodes. It's designed for building command-line tools, sidecar services, and developer utilities. + +### Features + +- **Trait-based design** - Flexible authentication and storage backends +- **Async/await support** - Full async support with `tokio` +- **Comprehensive API** - Access to all Calimero admin endpoints +- **Error handling** - Robust error types and handling +- **Type safety** - Strongly typed with Rust's type system + +### Installation + +Add to your `Cargo.toml`: + +```toml +[dependencies] +calimero-client = { path = "../core/crates/client" } +# Or from crates.io (when published) +# calimero-client = "0.1.0" +``` + +### Quick Start + +```rust +use calimero_client::{create_connection, create_client, AuthMode, ConnectionInfo}; +use calimero_client::traits::{ClientAuthenticator, ClientStorage}; +use url::Url; + +#[tokio::main] +async fn main() -> eyre::Result<()> { + // Create connection + let api_url = Url::parse("http://localhost:2528")?; + let authenticator = CliAuthenticator::new(); + let storage = FileStorage::new(); + + let connection = ConnectionInfo::new( + api_url, + Some("node1".to_string()), + authenticator, + storage, + ); + + // Create client + let client = Client::new(connection)?; + + // List contexts + let contexts = client.list_contexts().await?; + println!("Found {} contexts", contexts.data.len()); + + // List applications + let apps = client.list_applications().await?; + println!("Found {} applications", apps.data.len()); + + Ok(()) +} +``` + +### Authentication + +!!! warning "Authentication Not Yet Supported" + The Rust client currently supports `AuthMode::None` only. Authentication support with JWT tokens is planned for a future release. + +```rust +use calimero_client::{AuthMode, ConnectionInfo}; + +// Currently only AuthMode::None is supported +let connection = ConnectionInfo::new( + api_url, + Some("node1".to_string()), + CliAuthenticator::new(), + FileStorage::new(), +); +``` + +### API Examples + +#### Context Management + +```rust +// List all contexts +let contexts = client.list_contexts().await?; + +// Get specific context +let context = client.get_context(&context_id).await?; + +// Create new context +let create_request = CreateContextRequest { + application_id: app_id, + protocol: "near".to_string(), + params: Some(json!({"network": "testnet"}).to_string()), +}; +let new_context = client.create_context(create_request).await?; + +// Delete context +client.delete_context(&context_id).await?; +``` + +#### Application Management + +```rust +// List applications +let apps = client.list_applications().await?; + +// Get application +let app = client.get_application(&app_id).await?; + +// Install development application +let install_request = InstallDevApplicationRequest { + path: "/path/to/app.wasm".to_string(), + metadata: None, +}; +client.install_dev_application(install_request).await?; + +// Uninstall application +client.uninstall_application(&app_id).await?; +``` + +#### Function Execution + +```rust +use calimero_client::client::Client; + +// Execute function via JSON-RPC +let result = client.execute_function( + &context_id, + "set_value", + r#"{"key": "test", "value": "hello"}"#, + &executor_public_key, +).await?; +``` + +#### Blob Management + +```rust +// Upload blob +let data = b"Hello, Calimero!".to_vec(); +let blob_info = client.upload_blob(data, Some(&context_id)).await?; + +// List blobs +let blobs = client.list_blobs().await?; + +// Get blob info +let info = client.get_blob_info(&blob_id).await?; + +// Delete blob +client.delete_blob(&blob_id).await?; +``` + +### Architecture + +The Rust client uses a trait-based design for flexibility: + +```rust +pub trait ClientAuthenticator { + async fn authenticate(&self, url: &Url) -> Result; + // ... +} + +pub trait ClientStorage { + async fn load_tokens(&self, node_name: &str) -> Result>; + async fn save_tokens(&self, node_name: &str, tokens: &JwtToken) -> Result<()>; + // ... +} + +pub struct Client +where + A: ClientAuthenticator, + S: ClientStorage, +{ + connection: ConnectionInfo, +} +``` + +This allows you to implement custom authenticators and storage backends for your specific use case. + +### Error Handling + +```rust +use calimero_client::errors::ClientError; + +match client.list_contexts().await { + Ok(response) => println!("Success: {:?}", response), + Err(ClientError::Network { message }) => { + eprintln!("Network error: {}", message); + } + Err(ClientError::Authentication { message }) => { + eprintln!("Auth error: {}", message); + } + Err(e) => eprintln!("Error: {:?}", e), +} +``` + +### Related Documentation + +- **Repository**: [`calimero-network/core/crates/client`](https://github.com/calimero-network/core/tree/master/crates/client) +- **Source code**: [`core/crates/client/src`](https://github.com/calimero-network/core/tree/master/crates/client/src) + +## Python Client SDK + +The Python client SDK (`calimero-client-py`) provides Python bindings built with PyO3 for high-performance integration with Calimero nodes. Perfect for automation scripts, monitoring tools, and developer utilities. + +### Features + +- **High performance** - Built with Rust and PyO3 for optimal performance +- **Comprehensive API** - Full access to Calimero Network functionality +- **Type safety** - Strongly typed Python bindings +- **Async support** - Built-in async/await support +- **Easy installation** - Simple `pip install` + +### Installation + +```bash +pip install calimero-client-py +``` + +### Quick Start + +```python +import asyncio +from calimero_client_py import create_connection, create_client, AuthMode + +async def main(): + # Create connection + connection = create_connection( + base_url="http://localhost:2528", + auth_mode=AuthMode.NONE # Authentication not yet supported + ) + + # Create client + client = create_client(connection) + + # List contexts + contexts = await client.list_contexts() + print(f"Found {len(contexts.data)} contexts") + + # List applications + apps = await client.list_applications() + print(f"Found {len(apps.data)} applications") + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### Authentication + +!!! warning "Authentication Not Yet Supported" + The Python client currently supports `AuthMode.NONE` only. Authentication support with JWT tokens is planned for a future release. + +```python +from calimero_client_py import create_connection, AuthMode + +# Currently only AuthMode.NONE is supported +connection = create_connection( + base_url="http://localhost:2528", + auth_mode=AuthMode.NONE +) +``` + +### API Examples + +#### Context Management + +```python +# List all contexts +contexts = await client.list_contexts() + +# Get specific context +context = await client.get_context(context_id) + +# Create new context +context = await client.create_context( + application_id=app_id, + protocol="near", + params='{"network": "testnet"}' +) + +# Delete context +await client.delete_context(context_id) +``` + +#### Application Management + +```python +# List applications +apps = await client.list_applications() + +# Get application +app = await client.get_application(app_id) + +# Install development application +response = await client.install_dev_application( + path="/path/to/app.wasm", + metadata=None +) + +# Uninstall application +await client.uninstall_application(app_id) +``` + +#### Function Execution + +```python +# Execute function via JSON-RPC +result = await client.execute_function( + context_id=context_id, + method="set_value", + args='{"key": "test", "value": "hello"}', + executor_public_key=executor_public_key +) +``` + +#### Blob Management + +```python +# Upload blob +with open("file.dat", "rb") as f: + data = f.read() +blob_info = await client.upload_blob(data, context_id=context_id) + +# List blobs +blobs = await client.list_blobs() + +# Get blob info +info = await client.get_blob_info(blob_id) + +# Delete blob +await client.delete_blob(blob_id) +``` + +### Error Handling + +```python +from calimero_client_py import ClientError + +try: + contexts = await client.list_contexts() +except ClientError as e: + if e.error_type == "Network": + print(f"Network error: {e.message}") + elif e.error_type == "Authentication": + print(f"Auth error: {e.message}") + else: + print(f"Error: {e.message}") +``` + +### Development + +#### Building from Source + +```bash +# Install maturin +pip install maturin + +# Build the package +maturin build --release + +# Install in development mode +maturin develop +``` + +#### Running Tests + +```bash +# Test Python integration +python -m pytest tests/ + +# Test the environment +python example_usage.py +``` + +### Related Documentation + +- **Repository**: [`calimero-network/calimero-client-py`](https://github.com/calimero-network/calimero-client-py) +- **PyPI Package**: [`calimero-client-py`](https://pypi.org/project/calimero-client-py/) +- **README**: [`calimero-client-py/README.md`](https://github.com/calimero-network/calimero-client-py/blob/master/README.md) + +## JavaScript Client SDK + +The JavaScript client SDK (`calimero-client-js`) provides TypeScript/JavaScript bindings with full authentication support. Ideal for web applications, browser extensions, and Node.js tools. + +### Features + +- **Full authentication** - JWT token management, wallet-based auth, React components +- **Real-time updates** - WebSocket and SSE subscriptions +- **TypeScript support** - Full type definitions +- **React components** - Pre-built UI components for authentication +- **Browser & Node.js** - Works in both environments + +### Installation + +```bash +# npm +npm install @calimero-network/calimero-client + +# yarn +yarn add @calimero-network/calimero-client + +# pnpm +pnpm add @calimero-network/calimero-client +``` + +### Quick Start + +#### Basic Setup + +```typescript +import { + setAppEndpointKey, + setApplicationId, + JsonRpcClient, +} from '@calimero-network/calimero-client'; + +// Configure node URL and application ID +setAppEndpointKey('https://your-calimero-node-url.com'); +setApplicationId('your-application-id'); + +// Create RPC client +const rpcClient = new JsonRpcClient( + 'https://your-calimero-node-url.com', + '/jsonrpc' +); + +// Make a query +const response = await rpcClient.query({ + contextId: 'context-id', + method: 'get_value', + argsJson: { key: 'test' }, + executorPublicKey: 'public-key', +}); +``` + +#### Authentication Flow + +```typescript +import { ClientLogin, AccessTokenWrapper } from '@calimero-network/calimero-client'; + +// Use ClientLogin component for authentication +function LoginPage() { + const handleLoginSuccess = () => { + // Navigate to authenticated section + window.location.href = '/dashboard'; + }; + + return ; +} + +// Wrap your app with AccessTokenWrapper for automatic token management +function App() { + return ( + localStorage.getItem('node_url') || ''}> + + + ); +} +``` + +### Authentication + +The JavaScript client has **full authentication support** including: + +- **JWT token management** - Automatic token storage and refresh +- **Wallet-based authentication** - Support for NEAR, Ethereum, and other wallets +- **React components** - Pre-built UI components (`ClientLogin`, `SetupModal`) +- **Manual token handling** - Direct token management APIs + +#### Complete Authentication Flow + +```typescript +import { + SetupModal, + ClientLogin, + AccessTokenWrapper, +} from '@calimero-network/calimero-client'; + +// Step 1: Setup (configure node URL and application ID) +function SetupPage() { + const handleSetupComplete = () => { + navigate('/auth'); + }; + + return ; +} + +// Step 2: Authentication (user login) +function AuthPage() { + const handleLoginSuccess = () => { + navigate('/home'); + }; + + return ; +} + +// Step 3: App with automatic token management +function App() { + return ( + localStorage.getItem('node_url') || ''}> + + } /> + } /> + } /> + + + ); +} +``` + +#### Manual Token Usage + +```typescript +import { + setAccessToken, + getJWTObject, + JsonRpcClient, +} from '@calimero-network/calimero-client'; + +// Set your token +setAccessToken('your-jwt-token-here'); + +// Get contextId and executorPublicKey from the token +const jwt = getJWTObject(); +const contextId = jwt?.context_id; +const executorPublicKey = jwt?.executor_public_key; + +// Use the client +const rpcClient = new JsonRpcClient('your-api-url', '/jsonrpc'); +const response = await rpcClient.query({ + contextId, + method: 'your-method', + argsJson: { /* your args */ }, + executorPublicKey, +}); +``` + +### API Examples + +#### RPC Client + +```typescript +import { JsonRpcClient } from '@calimero-network/calimero-client'; + +const rpcClient = new JsonRpcClient( + process.env.NEXT_PUBLIC_API_URL, + '/jsonrpc' +); + +// Make a query (read-only) +const queryResponse = await rpcClient.query({ + contextId: 'context-id', + method: 'get_posts', + argsJson: { limit: 10 }, + executorPublicKey: 'public-key', +}); + +// Make a mutation (write operation) +const mutateResponse = await rpcClient.mutate({ + contextId: 'context-id', + method: 'create_post', + argsJson: { + title: 'My First Post', + text: 'This is my first post', + }, + executorPublicKey: 'public-key', +}); +``` + +#### WebSocket Subscriptions + +```typescript +import { WsSubscriptionsClient } from '@calimero-network/calimero-client'; + +const subscriptionsClient = new WsSubscriptionsClient( + process.env.NEXT_PUBLIC_API_URL, + '/ws' +); + +// Connect and subscribe +await subscriptionsClient.connect(); +subscriptionsClient.subscribe(['context-id']); + +// Handle incoming events +subscriptionsClient.addCallback((event) => { + console.log('Received event:', event); +}); + +// Clean up +subscriptionsClient.disconnect(); +``` + +#### SSE Subscriptions + +```typescript +import { SseSubscriptionsClient } from '@calimero-network/calimero-client'; + +const sseClient = new SseSubscriptionsClient( + process.env.NEXT_PUBLIC_API_URL, + '/sse' +); + +// Connect to SSE endpoint +await sseClient.connect(); +await sseClient.subscribe(['context-id']); + +// Handle incoming events +sseClient.addCallback((event) => { + console.log('Received SSE event:', event); +}); + +// Clean up +sseClient.disconnect(); +``` + +#### Admin API + +```typescript +import { apiClient } from '@calimero-network/calimero-client'; + +// List contexts +const contexts = await apiClient.node().getContexts(); + +// Create context +const newContext = await apiClient.node().createContext( + applicationId, + 'near' // protocol +); + +// List applications +const apps = await apiClient.node().getApplications(); + +// Get application +const app = await apiClient.node().getApplication(appId); +``` + +### Error Handling + +```typescript +try { + const response = await rpcClient.query(params); + if (response.error) { + // Handle RPC error + console.error('RPC Error:', response.error.message); + } else { + // Process successful response + console.log('Result:', response.result); + } +} catch (error) { + // Handle network or other errors + console.error('Request failed:', error); +} +``` + +### Best Practices + +1. **Token Management** + - Use `AccessTokenWrapper` for automatic token refresh + - Store sensitive information in environment variables + - Never expose tokens in client-side code + +2. **Connection Management** + - Always clean up WebSocket connections when done + - Use unique connection IDs for multiple connections + - Implement reconnection logic for production + +3. **Error Handling** + - Always check for errors in RPC responses + - Implement proper error boundaries in React + - Log errors appropriately for debugging + +### Related Documentation + +- **Repository**: [`calimero-network/calimero-client-js`](https://github.com/calimero-network/calimero-client-js) +- **NPM Package**: [`@calimero-network/calimero-client`](https://www.npmjs.com/package/@calimero-network/calimero-client) +- **README**: [`calimero-client-js/README.md`](https://github.com/calimero-network/calimero-client-js/blob/master/README.md) + +## Comparison + +| Feature | Rust Client | Python Client | JavaScript Client | +| --- | --- | --- | --- | +| **Language** | Rust | Python | TypeScript/JavaScript | +| **Performance** | High (native) | High (Rust bindings) | Good (JavaScript) | +| **Authentication** | ⚠️ Not yet | ⚠️ Not yet | ✅ Full support | +| **Async Support** | ✅ Tokio | ✅ asyncio | ✅ Native | +| **Type Safety** | ✅ Rust types | ✅ Python types | ✅ TypeScript | +| **React Components** | ❌ | ❌ | ✅ | +| **WebSocket** | ✅ | ✅ | ✅ | +| **SSE** | ✅ | ✅ | ✅ | +| **Best For** | CLI tools, sidecars | Scripts, automation | Web apps, browsers | + +## Choosing the Right SDK + +**Choose Rust Client if:** +- Building command-line tools or sidecar services +- Need maximum performance +- Already using Rust in your stack +- Building developer utilities + +**Choose Python Client if:** +- Building automation scripts or monitoring tools +- Working with Python-based tooling +- Need quick prototyping +- Building CI/CD pipelines + +**Choose JavaScript Client if:** +- Building web applications or browser extensions +- Need authentication flows +- Want React components +- Building user-facing applications + +## Related Topics + +- [meroctl CLI](meroctl-cli.md) - Command-line interface for Calimero +- [Introduction](../intro/index.md) - Understanding Calimero's core concepts +- [Contexts](../core-concepts/contexts.md) - Working with contexts +- [Identity](../core-concepts/identity.md) - Authentication and identity management + diff --git a/docs/tools-apis/developer-tools.md b/docs/tools-apis/developer-tools.md new file mode 100644 index 0000000..19911c8 --- /dev/null +++ b/docs/tools-apis/developer-tools.md @@ -0,0 +1,587 @@ +# Calimero Developer Tools + +Developer tools for building, testing, and scaffolding Calimero applications locally. These tools streamline the development workflow from initial project setup to testing and deployment. + +## Overview + +| Tool | Purpose | Language | Installation | +| --- | --- | --- | --- | +| **Merobox** | Local multi-node networks, workflow orchestration, testing | Python | `pipx install merobox` or `brew install merobox` | +| **ABI Codegen** | Generate TypeScript clients from Rust application ABIs | TypeScript/Node.js | `npm install @calimero-network/abi-codegen` | +| **create-mero-app** | Scaffold new Calimero apps from kv-store boilerplate | TypeScript/Node.js | `npx create-mero-app@latest` | + +## Merobox + +Merobox is a Python CLI tool for managing Calimero nodes in Docker containers. It's essential for local development and testing, enabling you to spin up multi-node networks, execute complex workflows, and automate testing scenarios. + +### Features + +- **Node Management** - Start, stop, and monitor Calimero nodes in Docker +- **Multi-Node Networks** - Run multiple nodes locally for testing P2P synchronization +- **Workflow Orchestration** - Execute complex multi-step workflows with YAML files +- **Auth Service Integration** - Traefik proxy and authentication service with nip.io DNS +- **Context Management** - Create and manage blockchain contexts +- **Identity Management** - Generate and manage cryptographic identities +- **Function Calls** - Execute smart contract functions via JSON-RPC +- **Testing Support** - Python testing fixtures for integration tests + +### Installation + +**Option 1: Using pipx (recommended)** + +```bash +pipx install merobox +``` + +**Option 2: Using Homebrew** + +```bash +brew install merobox +``` + +**Option 3: From source** + +```bash +git clone https://github.com/calimero-network/merobox.git +cd merobox +pipx install -e . +``` + +### Quick Start + +#### Basic Node Management + +```bash +# Start a single Calimero node +merobox run --name my-node + +# Start multiple nodes +merobox run --count 2 + +# Start with custom ports +merobox run --name my-node --server-port 2428 --swarm-port 2528 + +# List running nodes +merobox list + +# Check node health +merobox health my-node + +# View node logs +merobox logs my-node --follow + +# Stop a node +merobox stop my-node + +# Remove all node data (destructive!) +merobox nuke my-node +``` + +#### Application Management + +```bash +# Install a WASM application on a node +merobox install my-node /path/to/app.wasm + +# Call a function on an installed application +merobox call my-node '{"arg": "value"}' +``` + +#### Identity & Context Management + +```bash +# Create a new identity +merobox identity create + +# Create a new context +merobox context create my-node + +# Join a node to a context +merobox join my-node +``` + +### Workflow Orchestration + +Merobox supports complex multi-step workflows defined in YAML files: + +```yaml +# workflow.yml +image: "ghcr.io/calimero-network/merod:edge" + +nodes: + count: 2 + prefix: "calimero-node" + +steps: + - name: "Create identities" + type: identity + identity: "alice" + + - name: "Create context" + type: context + application_id: "my-app" + protocol: "near" + params: '{"network": "testnet"}' + + - name: "Install application" + type: install + node: "calimero-node-1" + path: "./app.wasm" + + - name: "Execute function" + type: execute + node: "calimero-node-1" + context_id: "{{ steps.create_context.context_id }}" + method: "set_value" + args: '{"key": "test", "value": "hello"}' + executor_public_key: "{{ steps.create_identities.alice.public_key }}" +``` + +**Execute a workflow:** + +```bash +merobox bootstrap run workflow.yml +``` + +**Workflow steps:** +- `identity` - Create cryptographic identities +- `context` - Create blockchain contexts +- `install` - Install WASM applications +- `execute` - Execute function calls +- `join` - Join contexts +- `invite_open` - Create open invitations +- `join_open` - Join via open invitation +- `wait` - Wait for conditions +- `assert` - Assert expected results +- `repeat` - Repeat steps multiple times +- `script` - Execute shell scripts + +### Auth Service Integration + +Merobox can integrate with authentication services using Traefik proxy: + +```bash +# Start nodes with auth service +merobox run --auth-service --count 2 +``` + +This automatically: +- Starts Traefik proxy (`traefik:v2.10`) +- Starts auth service (`ghcr.io/calimero-network/calimero-auth:latest`) +- Creates Docker networks (`calimero_web`, `calimero_internal`) +- Configures nip.io DNS resolution +- Sets up forward authentication middleware + +**Access patterns:** +- Node 1 API: `http://calimero-node-1.127.0.0.1.nip.io/jsonrpc` (protected) +- Node 1 Dashboard: `http://calimero-node-1.127.0.0.1.nip.io/admin-dashboard` (public) +- Auth Service: `http://localhost/auth/` (authentication endpoints) + +### Testing Support + +Merobox provides Python testing fixtures for integration tests: + +```python +from merobox.testing import cluster + +# Context manager +with cluster(count=2, prefix="ci") as env: + node1 = env.nodes[0] + node2 = env.nodes[1] + + # Use nodes in your tests + response = await client.list_contexts(node1) + assert len(response.data) == 0 + +# Pytest fixture +import pytest +from merobox.testing import cluster_fixture + +@pytest.fixture(scope="session") +def calimero_cluster(cluster_fixture): + yield cluster_fixture(count=3, prefix="test") + +def test_my_app(calimero_cluster): + node = calimero_cluster.nodes[0] + # Run your tests +``` + +### Use Cases + +- **Local Development** - Spin up local nodes for development and testing +- **Integration Testing** - Multi-node test scenarios with automated workflows +- **CI/CD Pipelines** - Automated testing in CI environments +- **Demo Environments** - Quick setup for demonstrations +- **Network Simulation** - Test P2P synchronization with multiple nodes + +### Related Documentation + +- **Repository**: [`calimero-network/merobox`](https://github.com/calimero-network/merobox) +- **PyPI Package**: [`merobox`](https://pypi.org/project/merobox/) +- **README**: [`merobox/README.md`](https://github.com/calimero-network/merobox/blob/master/README.md) + +## ABI Codegen + +ABI Codegen (`@calimero-network/abi-codegen`) generates TypeScript client code and type definitions from Rust application ABI manifests. It parses WASM-ABI v1 manifest files and generates fully-typed TypeScript clients for interacting with your Calimero applications. + +### Features + +- **TypeScript Generation** - Fully-typed client classes and type definitions +- **WASM-ABI v1 Support** - Parses standard ABI manifest format +- **Method Generation** - Generates methods for all Rust functions +- **Event Types** - Generates TypeScript types for events +- **Error Types** - Generates error types for method errors +- **CLI & Programmatic** - Use as CLI tool or import programmatically + +### Installation + +```bash +npm install @calimero-network/abi-codegen +``` + +### Quick Start + +#### CLI Usage + +```bash +# Basic usage +npx calimero-abi-codegen -i abi.json -o src + +# With custom client name +npx calimero-abi-codegen -i abi.json -o src --client-name MyClient + +# Validate ABI manifest only (no code generation) +npx calimero-abi-codegen --validate -i abi.json + +# Derive client name from WASM file +npx calimero-abi-codegen -i abi.json -o src --name-from kv_store.wasm +``` + +#### CLI Options + +- `-i, --input ` - Input ABI JSON file (default: `abi.json`) +- `-o, --outDir ` - Output directory for generated files (default: `src`) +- `--client-name ` - Custom client class name (default: `Client`) +- `--name-from ` - Derive client name from file path (e.g., WASM file) +- `--import-path ` - Custom import path for CalimeroApp and Context (default: `@calimero-network/calimero-client`) +- `--validate` - Validate ABI manifest only (no code generation) +- `-h, --help` - Show help message + +### Programmatic Usage + +```typescript +import { loadAbiManifestFromFile } from '@calimero-network/abi-codegen/parse'; +import { generateTypes } from '@calimero-network/abi-codegen/generate/types'; +import { generateClient } from '@calimero-network/abi-codegen/generate/client'; + +// Load ABI manifest +const manifest = loadAbiManifestFromFile('./abi.json'); + +// Generate TypeScript types +const typesContent = generateTypes(manifest); + +// Generate client class +const clientContent = generateClient(manifest, 'MyClient'); + +// Write to files +await fs.writeFile('src/types.ts', typesContent); +await fs.writeFile('src/MyClient.ts', clientContent); +``` + +### Generated Files + +ABI Codegen generates two files: + +1. **types.ts** - TypeScript type definitions for all types, events, and errors +2. **{ClientName}.ts** - Client class with methods for all Rust functions + +#### Example Generated Client + +```typescript +import { CalimeroApp, Context } from '@calimero-network/calimero-client'; + +// Generated types +export interface SetValueArgs { + key: string; + value: string; +} + +export interface GetValueArgs { + key: string; +} + +export interface ItemAddedEvent { + key: string; + value: string; + timestamp: number; +} + +// Generated client +export class KvStoreClient { + constructor( + private app: CalimeroApp, + private context: Context + ) {} + + async setValue(args: SetValueArgs): Promise { + await this.app.mutate({ + contextId: this.context.id, + method: 'set_value', + argsJson: args, + executorPublicKey: this.context.executorPublicKey, + }); + } + + async getValue(args: GetValueArgs): Promise { + const response = await this.app.query({ + contextId: this.context.id, + method: 'get_value', + argsJson: args, + executorPublicKey: this.context.executorPublicKey, + }); + return response.result as string | null; + } +} +``` + +### ABI Manifest Format + +ABI manifests follow the WASM-ABI v1 specification: + +```json +{ + "version": "1", + "methods": [ + { + "name": "set_value", + "args": { + "key": "string", + "value": "string" + }, + "returns": null, + "errors": [] + }, + { + "name": "get_value", + "args": { + "key": "string" + }, + "returns": "string | null", + "errors": [] + } + ], + "events": [ + { + "name": "ItemAdded", + "payload": { + "key": "string", + "value": "string", + "timestamp": "number" + } + } + ], + "types": { + "SetValueArgs": { + "fields": { + "key": "string", + "value": "string" + } + } + } +} +``` + +### Integration with Build Process + +Add to your `package.json`: + +```json +{ + "scripts": { + "generate:client": "calimero-abi-codegen -i abi.json -o src/generated", + "build": "npm run generate:client && npm run build:app" + } +} +``` + +### Use Cases + +- **Type Safety** - End-to-end type safety from Rust to TypeScript +- **API Generation** - Automatic client generation from Rust applications +- **Developer Experience** - Auto-complete and type checking in IDEs +- **Documentation** - Types serve as documentation for your API + +### Related Documentation + +- **Repository**: [`calimero-network/mero-devtools-js`](https://github.com/calimero-network/mero-devtools-js) +- **NPM Package**: [`@calimero-network/abi-codegen`](https://www.npmjs.com/package/@calimero-network/abi-codegen) +- **README**: [`mero-devtools-js/README.md`](https://github.com/calimero-network/mero-devtools-js/blob/master/README.md) + +## create-mero-app + +`create-mero-app` scaffolds new Calimero applications by cloning the `kv-store` example repository and copying its files (excluding Git artifacts). It provides a ready-to-use boilerplate for building new Calimero applications. + +### Features + +- **Quick Scaffolding** - Generate new apps in seconds +- **kv-store Boilerplate** - Based on the proven kv-store example +- **Clean Output** - Excludes Git artifacts and node_modules +- **Package Name Setup** - Automatically configures package.json name + +### Installation + +No installation required - use via `npx`: + +```bash +npx create-mero-app@latest my-app +``` + +### Quick Start + +```bash +# Create a new app +npx create-mero-app@latest my-kv-store + +# Navigate to the new app +cd my-kv-store + +# Install dependencies +pnpm install + +# Build the WASM application +cd logic +chmod +x ./build.sh +./build.sh + +# Start the frontend +cd ../app +pnpm build +pnpm dev +``` + +### What Gets Generated + +The tool clones the [`calimero-network/kv-store`](https://github.com/calimero-network/kv-store) repository and copies: + +- **Rust Application** (`logic/`) - WASM application with CRDT state +- **React Frontend** (`app/`) - React application with TypeScript +- **Build Scripts** - Build scripts for Rust and TypeScript +- **Configuration** - TypeScript config, package.json, etc. +- **Workflows** (`workflows/`) - Merobox workflows for local testing + +**Excluded:** +- `.git/` and `.github/` - Git artifacts +- `node_modules/` - Dependencies (installed via `pnpm install`) + +### Project Structure + +``` +my-kv-store/ +├── logic/ # Rust WASM application +│ ├── src/ +│ │ └── lib.rs # Main application logic +│ ├── Cargo.toml +│ └── build.sh # Build script +├── app/ # React frontend +│ ├── src/ +│ │ └── App.tsx # Main React component +│ ├── package.json +│ └── vite.config.ts +├── workflows/ # Merobox workflows +│ └── local-network.yml +├── package.json # Root package.json +└── README.md +``` + +### Customization + +After scaffolding, customize your app: + +1. **Update Application Logic** - Modify `logic/src/lib.rs` with your business logic +2. **Update Frontend** - Modify `app/src/App.tsx` with your UI +3. **Update Package Names** - Update `package.json` files with your project name +4. **Add Dependencies** - Install additional npm/Rust packages as needed + +### Next Steps + +1. **Build the WASM** - Compile Rust to WASM: `cd logic && ./build.sh` +2. **Generate TypeScript Client** - Use ABI codegen to generate client types +3. **Start Local Network** - Use Merobox to start local nodes: `merobox bootstrap run workflows/local-network.yml` +4. **Run Frontend** - Start the React app: `cd app && pnpm dev` +5. **Deploy** - Deploy your WASM to Calimero nodes and host your frontend + +### Use Cases + +- **Quick Prototyping** - Rapidly scaffold new application ideas +- **Learning** - Start with a working example to understand Calimero +- **Boilerplate** - Base template for new applications +- **Development** - Ready-to-use development environment + +### Related Documentation + +- **Repository**: [`calimero-network/mero-devtools-js`](https://github.com/calimero-network/mero-devtools-js) +- **NPM Package**: [`create-mero-app`](https://www.npmjs.com/package/create-mero-app) +- **README**: [`mero-devtools-js/create-mero-app/README.md`](https://github.com/calimero-network/mero-devtools-js/blob/master/create-mero-app/README.md) +- **kv-store Example**: [`calimero-network/kv-store`](https://github.com/calimero-network/kv-store) + +## Complete Development Workflow + +A typical development workflow using all three tools: + +### 1. Scaffold New Application + +```bash +npx create-mero-app@latest my-app +cd my-app +pnpm install +``` + +### 2. Build and Test Locally + +```bash +# Build WASM application +cd logic +chmod +x ./build.sh +./build.sh + +# Generate TypeScript client from ABI +cd .. +npx calimero-abi-codegen -i abi.json -o app/src/generated + +# Start local network with Merobox +merobox bootstrap run workflows/local-network.yml +``` + +### 3. Develop and Iterate + +```bash +# Start frontend dev server +cd app +pnpm dev + +# Make changes to Rust logic, rebuild +cd ../logic +./build.sh + +# Regenerate TypeScript client +cd .. +npx calimero-abi-codegen -i abi.json -o app/src/generated +``` + +### 4. Test Workflow + +```bash +# Run Merobox workflow for testing +merobox bootstrap run workflows/test-workflow.yml + +# Or use Merobox in Python tests +python -m pytest tests/ +``` + +## Related Topics + +- [Client SDKs](client-sdks.md) - Client libraries for interacting with nodes +- [meroctl CLI](meroctl-cli.md) - Command-line interface for node management +- [SDK Guide](../builder-directory/sdk-guide.md) - Building Calimero applications +- [Core Apps Examples](../examples/core-apps-examples.md) - Reference implementations + diff --git a/docs/tools-apis/index.md b/docs/tools-apis/index.md index e0c41c4..dc499d7 100644 --- a/docs/tools-apis/index.md +++ b/docs/tools-apis/index.md @@ -1,85 +1,54 @@ -# Tools and APIs +# Tools & APIs -A comprehensive overview of the core tools and APIs in the Calimero ecosystem. Each tool serves a specific purpose in building, deploying, and managing Calimero applications. +!!! info + Every entry below links to its canonical README. Follow those for installation flags, API surfaces, and deeper guides. ---- +Use this directory as a jumping-off point; it shows you **what exists** and **where to learn more** without duplicating repo docs. -## merod +## Runtime & Admin -**Calimero Node Daemon** +| Tool | Reference | Notes | +| --- | --- | --- | +| `merod` | [`calimero-network/core`](https://github.com/calimero-network/core#readme) | Node runtime orchestrating WASM apps, storage, networking, RPC. | +| `meroctl` | [CLI Reference](meroctl-cli.md) | Command-line surface for context lifecycle, deployment, diagnostics. See also [`core/crates/meroctl`](https://github.com/calimero-network/core/tree/master/crates/meroctl) for source code. | +| Admin Dashboard | [`calimero-network/admin-dashboard`](https://github.com/calimero-network/admin-dashboard#readme) | Web UI for member management, metrics, alerts. | -The core node implementation that powers the Calimero network. `merod` is the runtime daemon that manages peer-to-peer connections, application contexts, and CRDT-based state synchronization. +## Developer Tooling -**Key Features:** -- P2P networking and peer discovery -- Context management and isolation -- CRDT-based state replication -- WebAssembly application runtime -- Built-in RPC server +!!! tip "Developer Tools Guide" + For comprehensive documentation on Merobox, ABI Codegen, and create-mero-app, see the [Developer Tools Guide](developer-tools.md). -👉 [View merod on GitHub](https://github.com/calimero-network/core) +| Tool | Reference | Notes | +| --- | --- | --- | +| Developer Tools | [Developer Tools Guide](developer-tools.md) | Comprehensive guide to Merobox (local networks), ABI Codegen (TypeScript generation), and create-mero-app (boilerplate scaffolding). | +| Merobox | [`calimero-network/merobox`](https://github.com/calimero-network/merobox#readme) | Docker workflows for local multi-node networks, testing, and workflow orchestration. | +| ABI Codegen | [`calimero-network/mero-devtools-js`](https://github.com/calimero-network/mero-devtools-js#readme) | Generate TypeScript clients from Rust application ABIs. | +| create-mero-app | [`calimero-network/mero-devtools-js`](https://github.com/calimero-network/mero-devtools-js#readme) | Scaffold new Calimero apps from kv-store boilerplate. | +| Design System | [`calimero-network/design-system`](https://github.com/calimero-network/design-system#readme) | Shared UI components and tokens. | +| Plugins | [`calimero-network/plugins`](https://github.com/calimero-network/plugins#readme) | Automation hooks and extension samples. | ---- +## SDKs & Clients -## meroctl +!!! tip "Client SDKs Guide" + For comprehensive documentation on all three client SDKs (Rust, Python, JavaScript), see the [Client SDKs Guide](client-sdks.md). -**Calimero Command-Line Interface** +| SDK | Reference | Notes | +| --- | --- | --- | +| Client SDKs | [Client SDKs Guide](client-sdks.md) | Comprehensive guide to Rust, Python, and JavaScript client SDKs for interacting with Calimero nodes. | +| JavaScript Client | [`calimero-network/calimero-client-js`](https://github.com/calimero-network/calimero-client-js#readme) | Browser/Node bindings, event streaming, auth helpers. ✅ Full authentication support. | +| Python Client | [`calimero-network/calimero-client-py`](https://github.com/calimero-network/calimero-client-py#readme) | Python bindings, ABI tooling, automation recipes. ⚠️ Authentication support planned. | +| Rust Client | [`calimero-network/core/crates/client`](https://github.com/calimero-network/core/tree/master/crates/client) | Rust client SDK for CLI tools and sidecar services. ⚠️ Authentication support planned. | +| Rust SDK | [`calimero-network/core/crates/sdk`](https://github.com/calimero-network/core/tree/master/crates/sdk) | App macros, storage primitives, state helpers. For building Calimero applications. | -The primary CLI tool for interacting with Calimero nodes. Use `meroctl` to manage contexts, deploy applications, and configure your node. +## Automation & Workflows -**Key Features:** -- Context creation and management -- Application deployment and updates -- Node configuration and monitoring -- Identity and key management -- RPC client for node communication +| Resource | Reference | Notes | +| --- | --- | --- | +| Merobox Workflows | [`workflows/` in example repos](https://github.com/calimero-network/battleships/tree/main/workflows) | Reusable network topologies for local + CI. | +| Docs & CI scripts | [`calimero-network/docs`](https://github.com/calimero-network/docs#readme) | MkDocs site, link policies, CI glue. | -👉 [View meroctl on GitHub](https://github.com/calimero-network/core) - ---- - -## merobox - -**Calimero Development Environment** - -A containerized development environment that bundles everything you need to run a local Calimero network. Perfect for testing and development. - -**Key Features:** -- Pre-configured local node setup -- Hot-reload for rapid development -- Multi-node testing environment -- Isolated development contexts -- Docker-based deployment - -👉 [View merobox on GitHub](https://github.com/calimero-network/merobox) - ---- - -## mero-devtools-js - -**JavaScript/TypeScript Development Kit** - -A comprehensive JavaScript/TypeScript SDK for building Calimero applications. Includes client libraries, type definitions, and development utilities. - -**Key Features:** -- TypeScript-first API client -- CRDT state management utilities -- Context and identity management -- WebSocket and RPC communication -- React hooks and utilities - -👉 [View mero-devtools-js on GitHub](https://github.com/calimero-network/mero-devtools-js) - ---- - -## Getting Started - -1. **Install merod** to run a Calimero node -2. **Use meroctl** to manage your node and deploy applications -3. **Try merobox** for local development and testing -4. **Build with mero-devtools-js** to create JavaScript/TypeScript applications - -For detailed setup instructions, refer to each tool's README in its respective repository. +!!! tip + Pick a tool, follow its README end-to-end, then link back into MKDocs when you need a refresher. These pages stay minimal by design. diff --git a/docs/tools-apis/meroctl-cli.md b/docs/tools-apis/meroctl-cli.md new file mode 100644 index 0000000..c6471aa --- /dev/null +++ b/docs/tools-apis/meroctl-cli.md @@ -0,0 +1,322 @@ +# meroctl CLI Reference + +`meroctl` is the command-line interface for managing Calimero nodes, applications, contexts, and blobs. It provides a complete toolkit for development, deployment, and operations. + +## Installation + +```bash +# From source (requires Rust) +cargo install --path core/crates/meroctl + +# Or via package manager (when available) +# brew install calimero-tap/meroctl +``` + +## Configuration + +### Node Connection + +Connect to a node using one of these methods: + +```bash +# Using node alias (configured in ~/.calimero/config.toml) +meroctl --node node1 + +# Using direct API URL +meroctl --api http://localhost:2528 +``` + +### Environment Variables + +```bash +# Set default config directory +export CALIMERO_HOME=~/.calimero + +# Configure node aliases in ~/.calimero/config.toml +``` + +## Command Categories + +### Applications (`app`) + +Manage WASM applications on nodes: + +```bash +# List all applications +meroctl --node node1 app ls + +# Get application details +meroctl --node node1 app get + +# Install application from WASM file +meroctl --node node1 app install \ + --path ./my-app.wasm \ + --application-id \ + --context-id + +# Watch WASM file and auto-update contexts +meroctl --node node1 app watch --path ./my-app.wasm + +# Uninstall application +meroctl --node node1 app uninstall + +# List packages +meroctl --node node1 app list-packages + +# List versions of a package +meroctl --node node1 app list-versions com.example.myapp + +# Get latest version +meroctl --node node1 app get-latest-version com.example.myapp +``` + +### Contexts (`context`) + +Manage application contexts: + +```bash +# List all contexts +meroctl --node node1 context ls + +# Create new context +meroctl --node node1 context create \ + --application-id + +# Create context in dev mode (with watch) +meroctl --node node1 context create \ + --watch \ + --context-id + +# Get context details +meroctl --node node1 context get + +# Join a context via invitation +meroctl --node node1 context join \ + --context-id \ + --invitation + +# Join via open invitation +meroctl --node node1 context join-open \ + --context-id + +# Invite identity to context +meroctl --node node1 context invite \ + --context-id \ + --grantee-id + +# Create open invitation +meroctl --node node1 context invite-open \ + --context-id + +# Update context +meroctl --node node1 context update \ + --context-id \ + --metadata + +# Delete context +meroctl --node node1 context delete + +# Watch context for changes +meroctl --node node1 context watch + +# Sync context state +meroctl --node node1 context sync + +# Manage context aliases +meroctl --node node1 context alias set +meroctl --node node1 context alias get +meroctl --node node1 context use + +# Manage context identity/permissions +meroctl --node node1 context identity grant \ + --context-id \ + --grantee-id \ + --permission ManageApplication + +meroctl --node node1 context identity revoke \ + --context-id \ + --grantee-id +``` + +### Calling Methods (`call`) + +Execute application methods: + +```bash +# Call a mutation method +meroctl --node node1 call \ + --context-id \ + --method set \ + --args '{"key": "hello", "value": "world"}' \ + --executor-public-key + +# Call a view method (read-only) +meroctl --node node1 call \ + --context-id \ + --method get \ + --args '{"key": "hello"}' +``` + +### Blobs (`blob`) + +Manage content-addressed blobs: + +```bash +# List all blobs +meroctl --node node1 blob ls + +# Upload blob from file +meroctl --node node1 blob upload \ + --file /path/to/file \ + --context-id # Optional: announce to context + +# Download blob to file +meroctl --node node1 blob download \ + --blob-id \ + --output /path/to/output \ + --context-id # Optional: network discovery + +# Get blob information +meroctl --node node1 blob info --blob-id + +# Delete blob +meroctl --node node1 blob delete --blob-id +``` + +### Peers (`peers`) + +Manage peer connections: + +```bash +# List connected peers +meroctl --node node1 peers ls + +# Get peer information +meroctl --node node1 peers get +``` + +### Node (`node`) + +Node management operations: + +```bash +# Get node status +meroctl --node node1 node status + +# Get node information +meroctl --node node1 node info + +# Health check +meroctl --node node1 node health +``` + +## Output Formats + +```bash +# JSON output (default) +meroctl --node node1 context ls --output-format json + +# Table output +meroctl --node node1 context ls --output-format table + +# Plain text +meroctl --node node1 context ls --output-format plain +``` + +## Common Workflows + +### Development Workflow + +```bash +# 1. Start local node (via merobox or Docker) +merobox run --count 1 + +# 2. Install application +meroctl --node calimero-node-1 app install \ + --path ./build/my-app.wasm \ + --application-id my-app + +# 3. Create context +meroctl --node calimero-node-1 context create \ + --application-id my-app \ + --context-id my-context + +# 4. Watch for changes (auto-reload) +meroctl --node calimero-node-1 app watch my-app \ + --path ./build/my-app.wasm + +# 5. Call methods +meroctl --node calimero-node-1 call \ + --context-id my-context \ + --method my_method \ + --args '{"arg": "value"}' +``` + +### Multi-Node Workflow + +```bash +# Node 1: Create context +meroctl --node node1 context create --application-id my-app + +# Node 1: Invite Node 2 +meroctl --node node1 context invite \ + --context-id \ + --grantee-id + +# Node 2: Join context +meroctl --node node2 context join \ + --context-id \ + --invitation + +# Both nodes can now call methods and sync state +``` + +### Blob Sharing Workflow + +```bash +# Node 1: Upload blob and announce to context +meroctl --node node1 blob upload \ + --file ./document.pdf \ + --context-id + +# Output: blob_id: abc123... + +# Node 2: Download blob (discovered via context) +meroctl --node node2 blob download \ + --blob-id abc123... \ + --output ./downloaded.pdf \ + --context-id +``` + +## Troubleshooting + +### Connection Issues + +```bash +# Check node is running +meroctl --api http://localhost:2528 node health + +# Verify authentication +meroctl --node node1 context ls +``` + +### Common Errors + +- **"Node not found"**: Check node alias in `~/.calimero/config.toml` or use `--api` flag +- **"Context not found"**: Verify context ID with `context ls` +- **"Method not found"**: Check ABI with `app get ` +- **"Permission denied"**: Verify executor public key has access to context + +## Deep Dives + +For detailed CLI documentation: + +- **Source Code**: [`core/crates/meroctl`](https://github.com/calimero-network/core/tree/master/crates/meroctl) - Full implementation +- **Examples**: See `EXAMPLES` constants in source files for more usage patterns + +## Related Topics + +- [Applications](../core-concepts/applications.md) - Building applications that work with CLI +- [Contexts](../core-concepts/contexts.md) - Understanding context operations +- [Operator Track](../operator-track/index.md) - Running and managing nodes + diff --git a/mkdocs.yml b/mkdocs.yml index df86cca..99d2f03 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,5 +1,5 @@ # Site Information -site_name: "" +site_name: "Calimero Docs" site_description: Lightweight documentation for the Calimero network and developer stack site_url: https://docs.calimero.network @@ -49,8 +49,10 @@ theme: - navigation.tabs # Top-level sections in tabs - navigation.tabs.sticky # Keep tabs visible when scrolling - navigation.sections # Show sections in sidebar + - navigation.expand # Expand active section in sidebar - navigation.indexes # Allow section index pages - navigation.top # Back to top button + - toc.follow # Keep right-hand TOC in view - search.suggest # Search suggestions - search.highlight # Highlight search terms - content.code.copy # Copy button for code blocks @@ -59,11 +61,52 @@ theme: # Plugins (install with: pip install ) plugins: - search # Built-in search functionality + - mermaid2: # Mermaid diagram support + arguments: + theme: "dark" + themeVariables: + primaryColor: "#1a1a1a" + primaryTextColor: "#ffffff" + primaryBorderColor: "#00ff00" + lineColor: "#00ff00" + secondaryColor: "#00ff00" + tertiaryColor: "#1a1a1a" + background: "#1a1a1a" + mainBkg: "#1a1a1a" + secondBkg: "#2a2a2a" + textColor: "#ffffff" + noteBkgColor: "#2a2a2a" + noteTextColor: "#ffffff" + noteBorderColor: "#00ff00" + actorBorder: "#00ff00" + actorBkg: "#1a1a1a" + actorTextColor: "#ffffff" + actorLineColor: "#00ff00" + signalColor: "#00ff00" + signalTextColor: "#ffffff" + labelBoxBkgColor: "#1a1a1a" + labelBoxBorderColor: "#00ff00" + labelTextColor: "#ffffff" + loopTextColor: "#ffffff" + activationBorderColor: "#00ff00" + activationBkgColor: "#2a2a2a" + sequenceNumberColor: "#ffffff" + sectionBkgColor: "#1a1a1a" + altBkgColor: "#2a2a2a" + clusterBkg: "#1a1a1a" + clusterBorder: "#00ff00" + defaultLinkColor: "#00ff00" + titleColor: "#ffffff" + edgeLabelBackground: "#1a1a1a" + labelBackground: "#1a1a1a" # Custom CSS for branding and logo switching extra_css: - stylesheets/extra.css +extra_javascript: + - javascripts/sidebar.js + # Popular MkDocs Plugins to consider: # - mkdocs-material (already using as theme) # - mkdocs-mermaid2-plugin: For diagrams and flowcharts @@ -78,10 +121,27 @@ extra_css: nav: - Home: index.md - Introduction: intro/index.md - - App Directory: app-directory/index.md - - Builder Directory: builder-directory/index.md - - Tools & APIs: tools-apis/index.md - - Privacy, Verifiability & Security: privacy-verifiability-security/index.md + - Getting Started: getting-started/index.md + - Concepts: + - Overview: core-concepts/index.md + - Architecture: core-concepts/architecture.md + - Applications: core-concepts/applications.md + - Contexts: core-concepts/contexts.md + - Identity: core-concepts/identity.md + - Nodes: core-concepts/nodes.md + - Guides: + - Building Applications: + - Rust SDK: builder-directory/sdk-guide.md + - JavaScript SDK: builder-directory/js-sdk-guide.md + - Examples: examples/index.md + - Running Nodes: + - Run a Local Network: operator-track/run-a-local-network.md + - Monitoring: operator-track/monitor-and-debug.md + - Tools: + - CLI (meroctl): tools-apis/meroctl-cli.md + - Client SDKs: tools-apis/client-sdks.md + - Developer Tools: tools-apis/developer-tools.md + - API Reference: reference/index.md # Extra configuration extra: @@ -99,7 +159,11 @@ markdown_extensions: anchor_linenums: true - pymdownx.inlinehilite - pymdownx.snippets - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tabbed: alternate_style: true - admonition diff --git a/requirements.txt b/requirements.txt index a2eea5e..7e90642 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,3 +4,6 @@ mkdocs-material>=9.5.0 # Markdown extensions pymdown-extensions>=10.0 + +# Mermaid diagrams +mkdocs-mermaid2-plugin>=1.0.0