diff --git a/AGENTS.md b/AGENTS.md index cfcd3a8ff..429c2eab6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,8 +1,6 @@ # Everything Claude Code (ECC) — Agent Instructions -This is a **production-ready AI coding plugin** providing 28 specialized agents, 119 skills, 60 commands, and automated hook workflows for software development. - -**Version:** 1.9.0 +This is a **production-ready AI coding plugin** providing 28 specialized agents, 126 skills, 60 commands, and automated hook workflows for software development. ## Core Principles diff --git a/README.md b/README.md index b1161a5a9..61cdb43c2 100644 --- a/README.md +++ b/README.md @@ -212,7 +212,7 @@ For manual install instructions see the README in the `rules/` folder. /plugin list everything-claude-code@everything-claude-code ``` -✨ **That's it!** You now have access to 28 agents, 119 skills, and 60 commands. +✨ **That's it!** You now have access to 28 agents, 126 skills, and 60 commands. --- @@ -324,6 +324,7 @@ everything-claude-code/ | |-- videodb/ # Video and audio: ingest, search, edit, generate, stream (NEW) | |-- golang-patterns/ # Go idioms and best practices | |-- golang-testing/ # Go testing patterns, TDD, benchmarks +| |-- harper-best-practices/ # Harper schema, APIs, authentication, deployment, and coding best practices (NEW) | |-- cpp-coding-standards/ # C++ coding standards from C++ Core Guidelines (NEW) | |-- cpp-testing/ # C++ testing with GoogleTest, CMake/CTest (NEW) | |-- django-patterns/ # Django patterns, models, views (NEW) @@ -419,6 +420,7 @@ everything-claude-code/ | |-- golang/ # Go specific | |-- swift/ # Swift specific | |-- php/ # PHP specific (NEW) +| |-- harper/ # Harper specific (NEW) | |-- hooks/ # Trigger-based automations | |-- README.md # Hook documentation, recipes, and customization guide @@ -1085,7 +1087,7 @@ The configuration is automatically detected from `.opencode/opencode.json`. |---------|-------------|----------|--------| | Agents | ✅ 28 agents | ✅ 12 agents | **Claude Code leads** | | Commands | ✅ 60 commands | ✅ 31 commands | **Claude Code leads** | -| Skills | ✅ 119 skills | ✅ 37 skills | **Claude Code leads** | +| Skills | ✅ 126 skills | ✅ 37 skills | **Claude Code leads** | | Hooks | ✅ 8 event types | ✅ 11 events | **OpenCode has more!** | | Rules | ✅ 29 rules | ✅ 13 instructions | **Claude Code leads** | | MCP Servers | ✅ 14 servers | ✅ Full | **Full parity** | diff --git a/rules/harper/adding-tables-with-schemas.md b/rules/harper/adding-tables-with-schemas.md new file mode 100644 index 000000000..3849850de --- /dev/null +++ b/rules/harper/adding-tables-with-schemas.md @@ -0,0 +1,40 @@ +--- +name: adding-tables-with-schemas +description: Guidelines for adding tables to a Harper database using GraphQL schemas. +--- + +# Adding Tables with Schemas + +Instructions for the agent to follow when adding tables to a Harper database. + +## When to Use + +Use this skill when you need to define new data structures or modify existing ones in a Harper database. + +## How It Works + +1. **Create Dedicated Schema Files**: Prefer having a dedicated schema `.graphql` file for each table. Check the `config.yaml` file under `graphqlSchema.files` to see how it's configured. It typically accepts wildcards (e.g., `schemas/*.graphql`), but may be configured to point at a single file. +2. **Use Directives**: All available directives for defining your schema are defined in `node_modules/harperdb/schema.graphql`. Common directives include `@table`, `@export`, `@primaryKey`, `@indexed`, and `@relationship`. +3. **Define Relationships**: Link tables together using the `@relationship` directive. For more details, see the [Defining Relationships](defining-relationships.md) skill. +4. **Enable Automatic APIs**: If you add `@table @export` to a schema type, Harper automatically sets up REST and WebSocket APIs for basic CRUD operations against that table. For a detailed list of available endpoints and how to use them, see the [Automatic REST APIs](automatic-apis.md) skill. + - `GET /{TableName}`: Describes the schema itself. + - `GET /{TableName}/`: Lists all records (supports filtering, sorting, and pagination via query parameters). See the [Querying REST APIs](querying-rest-apis.md) skill for details. + - `GET /{TableName}/{id}`: Retrieves a single record by its ID. + - `POST /{TableName}/`: Creates a new record. + - `PUT /{TableName}/{id}`: Updates an existing record. + - `PATCH /{TableName}/{id}`: Performs a partial update on a record. + - `DELETE /{TableName}/`: Deletes all records or filtered records. + - `DELETE /{TableName}/{id}`: Deletes a single record by its ID. +5. **Consider Table Extensions**: If you are going to [extend the table](./extending-tables.md) in your resources, then do not `@export` the table from the schema. + +## Examples + +In a hypothetical `schemas/ExamplePerson.graphql`: + +```graphql +type ExamplePerson @table @export { + id: ID @primaryKey + name: String + tag: String @indexed +} +``` diff --git a/rules/harper/automatic-apis.md b/rules/harper/automatic-apis.md new file mode 100644 index 000000000..a737877d2 --- /dev/null +++ b/rules/harper/automatic-apis.md @@ -0,0 +1,37 @@ +--- +name: automatic-apis +description: How to use Harper's automatically generated REST and WebSocket APIs. +--- + +# Automatic APIs + +Instructions for the agent to follow when utilizing Harper's automatic APIs. + +## When to Use + +Use this skill when you want to interact with Harper tables via REST or WebSockets without writing custom resource logic. This is ideal for basic CRUD operations and real-time updates. + +## How It Works + +1. **Enable Automatic APIs**: Ensure your GraphQL schema includes the `@export` directive for the table. +2. **Access REST Endpoints**: Use the standard endpoints for your table (Note: Paths are case-sensitive). +3. **Use Automatic WebSockets**: Connect to `wss://your-harper-instance/{TableName}` to receive events whenever updates are made to that table. This is the easiest way to add real-time capabilities. (Use `ws://` for local development without SSL). For more complex needs, see [Real-time Apps](real-time-apps.md). +4. **Apply Filtering and Querying**: Use query parameters with `GET /{TableName}/` and `DELETE /{TableName}/`. See the [Querying REST APIs](querying-rest-apis.md) skill for advanced details. +5. **Customize if Needed**: If the automatic APIs don't meet your requirements, [customize the resources](./custom-resources.md). + +## Examples + +### Schema Configuration + +```graphql +type MyTable @table @export { + id: ID @primaryKey + name: String +} +``` + +### Common REST Operations + +- **List Records**: `GET /MyTable/` +- **Create Record**: `POST /MyTable/` +- **Update Record**: `PATCH /MyTable/{id}` diff --git a/rules/harper/caching.md b/rules/harper/caching.md new file mode 100644 index 000000000..88ebe8e7b --- /dev/null +++ b/rules/harper/caching.md @@ -0,0 +1,50 @@ +--- +name: caching +description: How to implement integrated data caching in Harper from external sources. +--- + +# Caching + +Instructions for the agent to follow when implementing caching in Harper. + +## When to Use + +Use this skill when you need high-performance, low-latency storage for data from external sources. It's ideal for reducing API calls to third-party services, preventing cache stampedes, and making external data queryable as if it were native Harper tables. + +## How It Works + +1. **Configure a Cache Table**: Define a table in your `schema.graphql` with an `expiration` (in seconds). +2. **Define an External Source**: Create a Resource class that fetches the data from your source. +3. **Attach Source to Table**: Use `sourcedFrom` to link your resource to the table. +4. **Implement Active Caching (Optional)**: Use `subscribe()` for proactive updates. See [Real-Time Apps](real-time-apps.md). +5. **Implement Write-Through Caching (Optional)**: Define `put` or `post` in your resource to propagate updates upstream. + +## Examples + +### Schema Configuration + +```graphql +type MyCache @table(expiration: 3600) @export { + id: ID @primaryKey +} +``` + +### Resource Implementation + +```js +import { Resource, tables } from 'harperdb'; + +export class ThirdPartyAPI extends Resource { + async get() { + const id = this.getId(); + const response = await fetch(`https://api.example.com/items/${id}`); + if (!response.ok) { + throw new Error('Source fetch failed'); + } + return await response.json(); + } +} + +// Attach source to table +tables.MyCache.sourcedFrom(ThirdPartyAPI); +``` diff --git a/rules/harper/checking-authentication.md b/rules/harper/checking-authentication.md new file mode 100644 index 000000000..189818105 --- /dev/null +++ b/rules/harper/checking-authentication.md @@ -0,0 +1,199 @@ +--- +name: checking-authentication +description: How to handle user authentication and sessions in Harper Resources. +--- + +# Checking Authentication + +Instructions for the agent to follow when handling authentication and sessions. + +## When to Use + +Use this skill when you need to implement sign-in/sign-out functionality, protect specific resource endpoints, or identify the currently logged-in user in a Harper application. + +## How It Works + +1. **Configure Harper for Sessions**: Ensure `harperdb-config.yaml` has sessions enabled and local auto-authorization disabled for testing: + ```yaml + authentication: + authorizeLocal: false + enableSessions: true + ``` +2. **Implement Sign In**: Use `this.getContext().login(username, password)` to create a session: + ```typescript + async post(_target, data) { + const context = this.getContext(); + try { + await context.login(data.username, data.password); + } catch { + return new Response('Invalid credentials', { status: 403 }); + } + return new Response('Logged in', { status: 200 }); + } + ``` +3. **Identify Current User**: Use `this.getCurrentUser()` to access session data: + ```typescript + async get() { + const user = this.getCurrentUser?.(); + if (!user) return new Response(null, { status: 401 }); + return { username: user.username, role: user.role }; + } + ``` +4. **Implement Sign Out**: Use `this.getContext().logout()` or delete the session from context: + ```typescript + async post() { + const context = this.getContext(); + await context.session?.delete?.(context.session.id); + return new Response('Logged out', { status: 200 }); + } + ``` +5. **Protect Routes**: In your Resource, use `allowRead()`, `allowUpdate()`, etc., to enforce authorization logic based on `this.getCurrentUser()`. For privileged actions, verify `user.role.permission.super_user`. + +## Examples + +### Sign In Implementation + +```typescript +async post(_target, data) { + const context = this.getContext(); + try { + await context.login(data.username, data.password); + } catch { + return new Response('Invalid credentials', { status: 403 }); + } + return new Response('Logged in', { status: 200 }); +} +``` + +### Identify Current User + +```typescript +async get() { + const user = this.getCurrentUser?.(); + if (!user) return new Response(null, { status: 401 }); + return { username: user.username, role: user.role }; +} +``` + +### Sign Out Implementation + +```typescript +async post() { + const context = this.getContext(); + await context.session?.delete?.(context.session.id); + return new Response('Logged out', { status: 200 }); +} +``` + +## Status code conventions used here + +- 200: Successful operation. For `GET /me`, a `200` with empty body means “not signed in”. +- 400: Missing required fields (e.g., username/password on sign-in). +- 401: No current session for an action that requires one (e.g., sign out when not signed in). +- 403: Authenticated but not authorized (bad credentials on login attempt, or insufficient privileges). + +## Client considerations + +- Sessions are cookie-based; the server handles setting and reading the cookie via Harper. If you make cross-origin requests, ensure the appropriate `credentials` mode and CORS settings. +- If developing locally, double-check the server config still has `authentication.authorizeLocal: false` to avoid accidental superuser bypass. + +## Token-based auth (JWT + refresh token) for non-browser clients + +Cookie-backed sessions are great for browser flows. For CLI tools, mobile apps, or other non-browser clients, it’s often easier to use **explicit tokens**: + +- **JWT (`operation_token`)**: short-lived bearer token used to authorize API requests. +- **Refresh token (`refresh_token`)**: longer-lived token used to mint a new JWT when it expires. + +This project includes two Resource patterns for that flow: + +### Issuing tokens: `IssueTokens` + +**Description / use case:** Generate `{ refreshToken, jwt }` either: + +- with an existing Authorization token (either Basic Auth or a JWT) and you want to issue new tokens, or +- from an explicit `{ username, password }` payload (useful for direct “login” from a CLI/mobile client). + +```javascript +export class IssueTokens extends Resource { + static loadAsInstance = false; + + async get(target) { + const { refresh_token: refreshToken, operation_token: jwt } = + await databases.system.hdb_user.operation( + { operation: 'create_authentication_tokens' }, + this.getContext(), + ); + return { refreshToken, jwt }; + } + + async post(target, data) { + if (!data.username || !data.password) { + throw new Error('username and password are required'); + } + + const { refresh_token: refreshToken, operation_token: jwt } = + await databases.system.hdb_user.operation({ + operation: 'create_authentication_tokens', + username: data.username, + password: data.password, + }); + return { refreshToken, jwt }; + } +} +``` + +**Recommended documentation notes to include:** + +- `GET` variant: intended for “I already have an Authorization token, give me new tokens”. +- `POST` variant: intended for “I have credentials, give me tokens”. +- Response shape: + - `refreshToken`: store securely (long-lived). + - `jwt`: attach to requests (short-lived). + +### Refreshing a JWT: `RefreshJWT` + +**Description / use case:** When the JWT expires, the client uses the refresh token to get a new JWT without re-supplying username/password. + +```javascript +export class RefreshJWT extends Resource { + static loadAsInstance = false; + + async post(target, data) { + if (!data.refreshToken) { + throw new Error('refreshToken is required'); + } + + const { operation_token: jwt } = await databases.system.hdb_user.operation({ + operation: 'refresh_operation_token', + refresh_token: data.refreshToken, + }); + return { jwt }; + } +} +``` + +**Recommended documentation notes to include:** + +- Requires `refreshToken` in the request body. +- Returns a new `{ jwt }`. +- If refresh fails (expired/revoked), client must re-authenticate (e.g., call `IssueTokens.post` again). + +### Suggested client flow (high-level) + +1. **Sign in (token flow)** + - POST /IssueTokens/ with a body of `{ "username": "your username", "password": "your password" }` or GET /IssueTokens/ with an existing Authorization token. + - Receive `{ jwt, refreshToken }` in the response +2. **Call protected APIs** + - Send the JWT with each request in the Authorization header (as your auth mechanism expects) +3. **JWT expires** + - POST /RefreshJWT/ with a body of `{ "refreshToken": "your refresh token" }`. + - Receive `{ jwt }` in the response and continue + +## Quick checklist + +- [ ] Public endpoints explicitly `allowRead`/`allowCreate` as needed. +- [ ] Sign-in uses `context.login` and handles 400/403 correctly. +- [ ] Protected routes call `ensureSuperUser(this.getCurrentUser())` (or another role check) before doing work. +- [ ] Sign-out verifies a session and deletes it. +- [ ] `authentication.authorizeLocal` is `false` and `enableSessions` is `true` in Harper config. +- [ ] If using tokens: `IssueTokens` issues `{ jwt, refreshToken }`, `RefreshJWT` refreshes `{ jwt }` with a `refreshToken`. diff --git a/rules/harper/creating-a-fabric-account-and-cluster.md b/rules/harper/creating-a-fabric-account-and-cluster.md new file mode 100644 index 000000000..0c5bf616c --- /dev/null +++ b/rules/harper/creating-a-fabric-account-and-cluster.md @@ -0,0 +1,28 @@ +--- +name: creating-a-fabric-account-and-cluster +description: How to create a Harper Fabric account, organization, and cluster. +--- + +# Creating a Harper Fabric Account and Cluster + +Follow these steps to set up your Harper Fabric environment for deployment. + +## How It Works + +1. **Sign Up/In**: Go to [https://fabric.harper.fast/](https://fabric.harper.fast/) and sign up or sign in. +2. **Create an Organization**: Create an organization (org) to manage your projects. +3. **Create a Cluster**: Create a new cluster. This can be on the free tier, no credit card required. +4. **Set Credentials**: During setup, set the cluster username and password to finish configuring it. +5. **Get Application URL**: Navigate to the **Config** tab and copy the **Application URL**. +6. **Configure Environment**: Update your `.env` file or GitHub Actions secrets with cluster-specific credentials. +7. **Next Steps**: See the [deploying-to-harper-fabric](deploying-to-harper-fabric.md) rule for detailed instructions on deploying your application successfully. + +## Examples + +### Environment Configuration + +```bash +CLI_TARGET_USERNAME='YOUR_CLUSTER_USERNAME' +CLI_TARGET_PASSWORD='YOUR_CLUSTER_PASSWORD' +CLI_TARGET='YOUR_CLUSTER_URL' +``` diff --git a/rules/harper/creating-harper-apps.md b/rules/harper/creating-harper-apps.md new file mode 100644 index 000000000..d9cd55f1b --- /dev/null +++ b/rules/harper/creating-harper-apps.md @@ -0,0 +1,49 @@ +--- +name: creating-harper-apps +description: How to initialize a new Harper application using the CLI. +--- + +# Creating Harper Applications + +The fastest way to start a new Harper project is using the `create-harper` CLI tool. This command initializes a project with a standard folder structure, essential configuration files, and basic schema definitions. + +## When to Use + +Use this command when starting a new Harper application or adding a new Harper microservice to an existing architecture. + +## Commands + +Initialize a project using your preferred package manager: + +### NPM + +```bash +npm create harper@latest +``` + +### PNPM + +```bash +pnpm create harper@latest +``` + +### Bun + +```bash +bun create harper@latest +``` + +## Options + +You can specify the project name and template directly: + +```bash +npm create harper@latest my-app --template default +``` + +## Next Steps + +1. **Configure Environment**: Set up your `.env` file with local or cloud credentials. +2. **Define Schema**: Modify `schema.graphql` to fit your application's data model. +3. **Start Development**: Run `npm run dev` to start the local Harper instance. +4. **Deploy**: Use `npm run deploy` to push your application to Harper Fabric. diff --git a/rules/harper/custom-resources.md b/rules/harper/custom-resources.md new file mode 100644 index 000000000..708232107 --- /dev/null +++ b/rules/harper/custom-resources.md @@ -0,0 +1,37 @@ +--- +name: custom-resources +description: How to define custom REST endpoints with JavaScript or TypeScript in Harper. +--- + +# Custom Resources + +Instructions for the agent to follow when creating custom resources in Harper. + +## When to Use + +Use this skill when the automatic CRUD operations provided by `@table @export` are insufficient, and you need custom logic, third-party API integration, or specialized data handling for your REST endpoints. + +## How It Works + +1. **Check if a Custom Resource is Necessary**: Verify if [Automatic APIs](./automatic-apis.md) or [Extending Tables](./extending-tables.md) can satisfy the requirement first. +2. **Create the Resource File**: Create a `.ts` or `.js` file in the directory specified by `jsResource` in `config.yaml` (typically `resources/`). +3. **Define the Resource Class**: Export a class extending `Resource` from `harperdb`: + + ```typescript + import { type RequestTargetOrId, Resource } from 'harperdb'; + + export class MyResource extends Resource { + async get(target?: RequestTargetOrId) { + return { message: 'Hello from custom GET!' }; + } + } + ``` + +4. **Implement HTTP Methods**: Add methods like `get`, `post`, `put`, `patch`, or `delete` to handle corresponding requests. Note that paths are **case-sensitive** and match the class name. +5. **Access Tables (Optional)**: Import and use the `tables` object to interact with your data: + ```typescript + import { tables } from 'harperdb'; + // ... inside a method + const results = await tables.MyTable.list(); + ``` +6. **Configure Loading**: Ensure `config.yaml` points to your resource files (e.g., `jsResource: { files: 'resources/*.ts' }`). diff --git a/rules/harper/defining-relationships.md b/rules/harper/defining-relationships.md new file mode 100644 index 000000000..5471faa9b --- /dev/null +++ b/rules/harper/defining-relationships.md @@ -0,0 +1,33 @@ +--- +name: defining-relationships +description: How to define and use relationships between tables in Harper using GraphQL. +--- + +# Defining Relationships + +Instructions for the agent to follow when defining relationships between Harper tables. + +## When to Use + +Use this skill when you need to link data across different tables, enabling automatic joins and efficient related-data fetching via REST APIs. + +## How It Works + +1. **Identify the Relationship Type**: Determine if it's one-to-one, many-to-one, or one-to-many. +2. **Use the `@relationship` Directive**: Apply it to a field in your GraphQL schema. + - **Many-to-One (Current table holds FK)**: Use `from`. + ```graphql + type Book @table @export { + authorId: ID + author: Author @relationship(from: "authorId") + } + ``` + - **One-to-Many (Related table holds FK)**: Use `to` and an array type. + ```graphql + type Author @table @export { + books: [Book] @relationship(to: "authorId") + } + ``` +3. **Query with Relationships**: Use dot syntax in REST API calls for filtering or the `select()` operator for including related data. + - Example Filter: `GET /Book/?author.name=Harper` + - Example Select: `GET /Author/?select(name,books(title))` diff --git a/rules/harper/deploying-to-harper-fabric.md b/rules/harper/deploying-to-harper-fabric.md new file mode 100644 index 000000000..6a1f2f944 --- /dev/null +++ b/rules/harper/deploying-to-harper-fabric.md @@ -0,0 +1,102 @@ +--- +name: deploying-to-harper-fabric +description: How to deploy a Harper application to the Harper Fabric cloud. +--- + +# Deploying to Harper Fabric + +Instructions for the agent to follow when deploying to Harper Fabric. + +## When to Use + +Use this skill when you are ready to move your Harper application from local development to a cloud-hosted environment. + +## How It Works + +1. **Sign up**: Follow the [creating-a-fabric-account-and-cluster](creating-a-fabric-account-and-cluster.md) rule to create a Harper Fabric account, organization, and cluster. +2. **Configure Environment**: Add your cluster credentials and cluster application URL to `.env`: + ```bash + CLI_TARGET_USERNAME='YOUR_CLUSTER_USERNAME' + CLI_TARGET_PASSWORD='YOUR_CLUSTER_PASSWORD' + CLI_TARGET='YOUR_CLUSTER_URL' + ``` +3. **Deploy From Local Environment**: Run `npm run deploy`. +4. **Set up CI/CD**: Configure `.github/workflows/deploy.yaml` and set repository secrets for automated deployments. + +## Manual Setup for Existing Apps + +If your application was not created with `npm create harper`, you'll need to manually configure the deployment scripts and CI/CD workflow. + +### 1. Update `package.json` + +Add the following scripts and dependencies to your `package.json`: + +```json +{ + "scripts": { + "deploy": "dotenv -- npm run deploy:component", + "deploy:component": "harperdb deploy_component . restart=rolling replicated=true" + }, + "devDependencies": { + "dotenv-cli": "^11.0.0", + "harperdb": "^4.7.20" + } +} +``` + +#### Why split the scripts? + +The `deploy` script is separated from `deploy:component` to ensure environment variables from your `.env` file are properly loaded and passed to the Harper CLI. + +- `deploy`: Uses `dotenv-cli` to load environment variables (like `CLI_TARGET`, `CLI_TARGET_USERNAME`, and `CLI_TARGET_PASSWORD`) before executing the next command. +- `deploy:component`: The actual command that performs the deployment. + +By using `dotenv -- npm run deploy:component`, the environment variables are correctly set in the shell session before `harperdb deploy_component` is called, allowing it to authenticate with your cluster. + +### 2. Configure GitHub Actions + +Create a `.github/workflows/deploy.yaml` file with the following content: + +```yaml +name: Deploy to Harper Fabric +on: + workflow_dispatch: +# push: +# branches: +# - main +concurrency: + group: main + cancel-in-progress: false +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + with: + fetch-depth: 0 + fetch-tags: true + - name: Set up Node.js + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0 + with: + cache: 'npm' + node-version-file: '.nvmrc' + - name: Install dependencies + run: npm ci + - name: Run unit tests + run: npm test + - name: Run lint + run: npm run lint + - name: Deploy + run: npm run deploy + env: + CLI_TARGET: ${{ secrets.CLI_TARGET }} + CLI_TARGET_USERNAME: ${{ secrets.CLI_TARGET_USERNAME }} + CLI_TARGET_PASSWORD: ${{ secrets.CLI_TARGET_PASSWORD }} +``` + +Be sure to set the following repository secrets in your GitHub repository's /settings/secrets/actions: + +- `CLI_TARGET` +- `CLI_TARGET_USERNAME` +- `CLI_TARGET_PASSWORD` diff --git a/rules/harper/extending-tables.md b/rules/harper/extending-tables.md new file mode 100644 index 000000000..a0597c1b6 --- /dev/null +++ b/rules/harper/extending-tables.md @@ -0,0 +1,41 @@ +--- +name: extending-tables +description: How to add custom logic to automatically generated table resources in Harper. +--- + +# Extending Tables + +Instructions for the agent to follow when extending table resources in Harper. + +## When to Use + +Use this skill when you need to add custom validation, side effects (like webhooks), data transformation, or custom access control to the standard CRUD operations of a Harper table. + +## How It Works + +1. **Define the Table in GraphQL**: In your `.graphql` schema, define the table using the `@table` directive. **Do not** use `@export` if you plan to extend it. + ```graphql + type MyTable @table { + id: ID @primaryKey + name: String + } + ``` +2. **Create the Extension File**: Create a `.ts` file in your `resources/` directory. +3. **Extend the Table Resource**: Export a class that extends `tables.YourTableName`: + + ```typescript + import { type RequestTargetOrId, tables } from 'harperdb'; + + export class MyTable extends tables.MyTable { + async post(target: RequestTargetOrId, record: any) { + // Custom logic here + if (!record.name) { + throw new Error('Name required'); + } + return super.post(target, record); + } + } + ``` + +4. **Override Methods**: Override `get`, `post`, `put`, `patch`, or `delete` as needed. Always call `super[method]` to maintain default Harper functionality unless you intend to replace it entirely. +5. **Implement Logic**: Use overrides for validation, side effects, or transforming data before/after database operations. diff --git a/rules/harper/handling-binary-data.md b/rules/harper/handling-binary-data.md new file mode 100644 index 000000000..0b536d1da --- /dev/null +++ b/rules/harper/handling-binary-data.md @@ -0,0 +1,45 @@ +--- +name: handling-binary-data +description: How to store and serve binary data like images or audio in Harper. +--- + +# Handling Binary Data + +Instructions for the agent to follow when handling binary data in Harper. + +## When to Use + +Use this skill when you need to store binary files (images, audio, etc.) in the database or serve them back to clients via REST endpoints. + +## How It Works + +1. **Store Binary Data**: In your resource's `post` or `put` method, convert incoming data to Buffers and then to Blobs using `createBlob`. Include the MIME type if available: + + ```typescript + import { createBlob } from 'harperdb'; + + async post(target, record) { + if (record.data) { + record.data = createBlob(Buffer.from(record.data, record.encoding || 'base64'), { + type: record.contentType || 'application/octet-stream', + }); + } + return super.post(target, record); + } + ``` + +2. **Serve Binary Data**: In your resource's `get` method, return a response object with the appropriate `Content-Type` and the binary data in the `body`: + ```typescript + async get(target) { + const record = await super.get(target); + if (record?.data) { + return { + status: 200, + headers: { 'Content-Type': record.data.type || 'application/octet-stream' }, + body: record.data, + }; + } + return record; + } + ``` +3. **Use the Blob Type**: Ensure your GraphQL schema uses the `Blob` scalar for binary fields. diff --git a/rules/harper/programmatic-table-requests.md b/rules/harper/programmatic-table-requests.md new file mode 100644 index 000000000..70a8a3f39 --- /dev/null +++ b/rules/harper/programmatic-table-requests.md @@ -0,0 +1,39 @@ +--- +name: programmatic-table-requests +description: How to interact with Harper tables programmatically using the `tables` object. +--- + +# Programmatic Table Requests + +Instructions for the agent to follow when interacting with Harper tables via code. + +## When to Use + +Use this skill when you need to perform database operations (CRUD, search, subscribe) from within Harper Resources or scripts. + +## How It Works + +1. **Access the Table**: Use the global `tables` object followed by your table name (e.g., `tables.MyTable`). +2. **Perform CRUD Operations**: + - **Get**: `await tables.MyTable.get(id)` for a single record or `await tables.MyTable.get({ conditions: [...] })` for multiple. + - **Create**: `await tables.MyTable.post(record)` (auto-generates ID) or `await tables.MyTable.put(id, record)`. + - **Update**: `await tables.MyTable.patch(id, partialRecord)` for partial updates. + - **Delete**: `await tables.MyTable.delete(id)`. +3. **Use Updatable Records for Atomic Ops**: Call `update(id)` to get a reference, then use `addTo` or `subtractFrom` for atomic increments/decrements: + ```typescript + const stats = await tables.Stats.update('daily'); + stats.addTo('viewCount', 1); + ``` +4. **Search and Stream**: Use `search(query)` for efficient streaming of large result sets: + ```typescript + for await (const record of tables.MyTable.search({ conditions: [...] })) { + // process record + } + ``` +5. **Real-time Subscriptions**: Use `subscribe(query)` to listen for changes: + ```typescript + for await (const event of tables.MyTable.subscribe(query)) { + // handle event + } + ``` +6. **Publish Events**: Use `publish(id, message)` to trigger subscriptions without necessarily persisting data. diff --git a/rules/harper/querying-rest-apis.md b/rules/harper/querying-rest-apis.md new file mode 100644 index 000000000..70c9e39e8 --- /dev/null +++ b/rules/harper/querying-rest-apis.md @@ -0,0 +1,22 @@ +--- +name: querying-rest-apis +description: How to use query parameters to filter, sort, and paginate Harper REST APIs. +--- + +# Querying REST APIs + +Instructions for the agent to follow when querying Harper's REST APIs. + +## When to Use + +Use this skill when you need to perform advanced data retrieval (filtering, sorting, pagination, joins) using Harper's automatic REST endpoints. + +## How It Works + +1. **Basic Filtering**: Use attribute names as query parameters: `GET /Table/?key=value`. +2. **Use Comparison Operators**: Append operators like `gt`, `ge`, `lt`, `le`, `ne` using FIQL-style syntax: `GET /Table/?price=gt=100`. +3. **Apply Logic and Grouping**: Use `&` for AND, `|` for OR, and `()` for grouping: `GET /Table/?(rating=5|featured=true)&price=lt=50`. +4. **Select Specific Fields**: Use `select()` to limit returned attributes: `GET /Table/?select(name,price)`. +5. **Paginate Results**: Use `limit(count)` or `limit(offset, count)`: `GET /Table/?limit(20, 10)`. +6. **Sort Results**: Use `sort()` with `+` (asc) or `-` (desc): `GET /Table/?sort(-price,+name)`. +7. **Query Relationships**: Use dot syntax for tables linked with `@relationship`: `GET /Book/?author.name=Harper`. diff --git a/rules/harper/real-time-apps.md b/rules/harper/real-time-apps.md new file mode 100644 index 000000000..13d028b10 --- /dev/null +++ b/rules/harper/real-time-apps.md @@ -0,0 +1,43 @@ +--- +name: real-time-apps +description: How to build real-time features in Harper using WebSockets and Pub/Sub. +--- + +# Real-time Applications + +Instructions for the agent to follow when building real-time applications in Harper. + +## When to Use + +Use this skill when you need to stream live updates to clients, implement chat features, or provide real-time data synchronization between the database and a frontend. + +## How It Works + +1. **Check Automatic WebSockets**: If you only need to stream table changes, use [Automatic APIs](automatic-apis.md) which provide a WebSocket endpoint for every `@export`ed table. +2. **Implement `connect` in a Resource**: For custom bi-directional logic, implement the `connect` method. +3. **Use Pub/Sub**: Use `tables.TableName.subscribe(query)` to listen for specific data changes and stream them to the client. +4. **Handle SSE**: Ensure your `connect` method gracefully handles cases where `incomingMessages` is null (Server-Sent Events). +5. **Connect from Client**: Use standard WebSockets (`new WebSocket('wss://...')`) to connect to your resource endpoint. Ensure you use the appropriate scheme (`ws://` for HTTP, `wss://` for HTTPS). + +## Examples + +### Bi-directional WebSocket Resource + +```typescript +import { Resource, tables } from 'harperdb'; + +export class MySocket extends Resource { + async *connect(target, incomingMessages) { + // Subscribe to table changes + const subscription = await tables.MyTable.subscribe(target); + if (!incomingMessages) { + return subscription; // SSE mode + } + + // Handle incoming client messages + for await (let message of incomingMessages) { + yield { received: message }; + } + } +} +``` diff --git a/rules/harper/serving-web-content.md b/rules/harper/serving-web-content.md new file mode 100644 index 000000000..f11d3dd87 --- /dev/null +++ b/rules/harper/serving-web-content.md @@ -0,0 +1,65 @@ +--- +name: serving-web-content +description: How to serve static files and integrated Vite/React applications in Harper. +--- + +# Serving Web Content + +Instructions for the agent to follow when serving web content from Harper. + +## When to Use + +Use this skill when you need to serve a frontend (HTML, CSS, JS, or a React app) directly from your Harper instance. + +## How It Works + +1. **Choose a Method**: Decide between the simple Static Plugin or the integrated Vite Plugin. +2. **Option A: Static Plugin (Simple)**: + - Add to `config.yaml`: + ```yaml + static: + files: 'web/*' + ``` + - Place files in a `web/` folder in the project root. + - Files are served at the root URL (e.g., `http://localhost:9926/index.html`). +3. **Option B: Vite Plugin (Advanced/Development)**: + - Add to `config.yaml`: + ```yaml + '@harperfast/vite-plugin': + package: '@harperfast/vite-plugin' + ``` + - Ensure `vite.config.ts` and `index.html` are in the project root. + + ```javascript + import vue from '@vitejs/plugin-vue'; + import path from 'node:path'; + import { defineConfig } from 'vite'; + + // https://vite.dev/config/ + export default defineConfig({ + plugins: [vue()], + resolve: { + alias: { + '@': path.resolve(import.meta.dirname, './src'), + }, + }, + build: { + outDir: 'web', + emptyOutDir: true, + rolldownOptions: { + external: ['**/*.test.*', '**/*.spec.*'], + }, + }, + }); + ``` + + - Install dependencies: `npm install --save-dev vite @harperfast/vite-plugin`. + - Then `harper run .` will start up Harper and Vite with HMR. Vite does _not_ need to be executed separately. + +4. **Deploy for Production**: For Vite apps, use a build script to generate static files into a `web/` folder and deploy them using the static handler pattern. For example, these scripts in a package.json can perform the necessary steps: + ```json + "build": "vite build", + "deploy": "rm -Rf deploy && npm run build && mkdir deploy && mv web deploy/ && cp -R deploy-template/* deploy/ && cp -R schemas resources deploy/ && dotenv -- npm run deploy:component && rm -Rf deploy", + "deploy:component": "(cd deploy && harper deploy_component . project=web restart=rolling replicated=true)" + ``` + Then in production, the "Static Plugin" option will performantly and securely serve your assets. `npm create harper@latest` scaffolds all of this for you. diff --git a/rules/harper/typescript-type-stripping.md b/rules/harper/typescript-type-stripping.md new file mode 100644 index 000000000..9b09e5044 --- /dev/null +++ b/rules/harper/typescript-type-stripping.md @@ -0,0 +1,32 @@ +--- +name: typescript-type-stripping +description: How to run TypeScript files directly in Harper without a build step. +--- + +# TypeScript Type Stripping + +Instructions for the agent to follow when using TypeScript in Harper. + +## When to Use + +Use this skill when you want to write Harper Resources in TypeScript and have them execute directly in Node.js without an intermediate build or compilation step. + +## How It Works + +1. **Verify Node.js Version**: Ensure you are using Node.js v22.6.0 or higher. +2. **Name Files with `.ts`**: Create your resource files in the `resources/` directory with a `.ts` extension. +3. **Use TypeScript Syntax**: Write your resource classes using standard TypeScript (interfaces, types, etc.). + ```typescript + import { Resource } from 'harperdb'; + export class MyResource extends Resource { + async get(): Promise<{ message: string }> { + return { message: 'Running TS directly!' }; + } + } + ``` +4. **Use Explicit Extensions in Imports**: When importing other local modules, include the `.ts` extension: `import { helper } from './helper.ts'`. +5. **Configure `config.yaml`**: Ensure `jsResource` points to your `.ts` files: + ```yaml + jsResource: + files: 'resources/*.ts' + ``` diff --git a/rules/harper/using-blob-datatype.md b/rules/harper/using-blob-datatype.md new file mode 100644 index 000000000..b980af2b9 --- /dev/null +++ b/rules/harper/using-blob-datatype.md @@ -0,0 +1,36 @@ +--- +name: using-blob-datatype +description: How to use the Blob data type for efficient binary storage in Harper. +--- + +# Using Blob Datatype + +Instructions for the agent to follow when working with the Blob data type in Harper. + +## When to Use + +Use this skill when you need to store unstructured or large binary data (media, documents) that is too large for standard JSON fields. Blobs provide efficient storage and integrated streaming support. + +## How It Works + +1. **Define Blob Fields**: In your GraphQL schema, use the `Blob` type: + ```graphql + type MyTable @table { + id: ID @primaryKey + data: Blob + } + ``` +2. **Create and Store Blobs**: Use `createBlob()` from `harperdb` to wrap Buffers or Streams: + ```javascript + import { createBlob, tables } from 'harperdb'; + const blob = createBlob(largeBuffer); + await tables.MyTable.put('my-id', { data: blob }); + ``` +3. **Use Streaming (Optional)**: For very large files, pass a stream to `createBlob()` to avoid loading the entire file into memory. +4. **Read Blob Data**: Retrieve the record and use `.bytes()` or streaming interfaces on the blob field: + ```javascript + const record = await tables.MyTable.get('my-id'); + const buffer = await record.data.bytes(); + ``` +5. **Ensure Write Completion**: Use `saveBeforeCommit: true` in `createBlob` options if you need the blob fully written before the record is committed. +6. **Handle Errors**: Attach error listeners to the blob object to handle streaming failures. diff --git a/rules/harper/vector-indexing.md b/rules/harper/vector-indexing.md new file mode 100644 index 000000000..b0d16df51 --- /dev/null +++ b/rules/harper/vector-indexing.md @@ -0,0 +1,159 @@ +--- +name: vector-indexing +description: How to enable and query vector indexes for similarity search in Harper. +--- + +# Vector Indexing + +Instructions for the agent to follow when implementing vector search in Harper. + +## When to Use + +Use this skill when you need to perform similarity searches on high-dimensional data, such as AI embeddings for semantic search, recommendations, or image retrieval. + +## How It Works + +1. **Enable Vector Indexing**: In your GraphQL schema, add `@indexed(type: "HNSW")` to a numeric array field: + ```graphql + type Product @table { + id: ID @primaryKey + textEmbeddings: [Float] @indexed(type: "HNSW") + } + ``` +2. **Configure Index Options (Optional)**: Fine-tune the index with parameters like `distance` (`cosine` or `euclidean`), `M`, and `efConstruction`. +3. **Query with Vector Search**: Use `tables.Table.search()` with a `sort` object containing the `target` vector: + ```javascript + const results = await tables.Product.search({ + select: ['name', '$distance'], + sort: { + attribute: 'textEmbeddings', + target: [0.1, 0.2, ...], // query vector + }, + limit: 5, + }); + ``` +4. **Filter by Distance**: Use `conditions` with a `target` vector and a `comparator` (e.g., `lt`) to return results within a similarity threshold: + ```javascript + const results = await tables.Product.search({ + conditions: { + attribute: 'textEmbeddings', + comparator: 'lt', + value: 0.1, + target: searchVector, + }, + }); + ``` +5. **Generate Embeddings**: Use external services (OpenAI, Ollama) to generate the numeric vectors before storing or searching them in Harper. + +```typescript +import OpenAI from 'openai'; +import ollama from 'ollama'; + +const { Product } = tables; +const openai = new OpenAI(); +// the name of the OpenAI embedding model +const OPENAI_EMBEDDING_MODEL = 'text-embedding-3-small'; + +// the name of the Ollama embedding model +const OLLAMA_EMBEDDING_MODEL = 'llama3'; + +const SIMILARITY_THRESHOLD = 0.5; + +export class ProductSearch extends Resource { + // based on env variable we choose the appropriate embedding generator + generateEmbedding = + process.env.EMBEDDING_GENERATOR === 'ollama' + ? this._generateOllamaEmbedding + : this._generateOpenAIEmbedding; + + /** + * Executes a search query using a generated text embedding and returns the matching products. + * + * @param {Object} data - The input data for the request. + * @param {string} data.prompt - The prompt to generate the text embedding from. + * @return {Promise} Returns a promise that resolves to an array of products matching the conditions, + * including fields: name, description, price, and $distance. + */ + async post(data) { + const embedding = await this.generateEmbedding(data.prompt); + + return await Product.search({ + select: ['name', 'description', 'price', '$distance'], + conditions: { + attribute: 'textEmbeddings', + comparator: 'lt', + value: SIMILARITY_THRESHOLD, + target: embedding[0], + }, + limit: 5, + }); + } + + /** + * Generates an embedding using the Ollama API. + * + * @param {string} promptData - The input data for which the embedding is to be generated. + * @return {Promise} A promise that resolves to the generated embedding as an array of numbers. + */ + async _generateOllamaEmbedding(promptData) { + const embedding = await ollama.embed({ + model: OLLAMA_EMBEDDING_MODEL, + input: promptData, + }); + return embedding?.embeddings; + } + + /** + * Generates OpenAI embeddings based on the given prompt data. + * + * @param {string} promptData - The input data used for generating the embedding. + * @return {Promise} A promise that resolves to an array of embeddings, where each embedding is an array of floats. + */ + async _generateOpenAIEmbedding(promptData) { + const embedding = await openai.embeddings.create({ + model: OPENAI_EMBEDDING_MODEL, + input: promptData, + encoding_format: 'float', + }); + + let embeddings = []; + embedding.data.forEach((embeddingData) => { + embeddings.push(embeddingData.embedding); + }); + + return embeddings; + } +} +``` + +## Examples + +Sample request to the `ProductSearch` resource which prompts to find "shorts for the gym": + +```bash +curl -X POST "http://localhost:9926/ProductSearch/" \ +-H "Accept: application/json" \ +-H "Content-Type: application/json" \ +-H "Authorization: Basic " \ +-d '{"prompt": "shorts for the gym"}' +``` + +--- + +## When to Use Vector Indexing + +Vector indexing is ideal when: + +- Storing embedding vectors from ML models +- Performing semantic or similarity-based search +- Working with high-dimensional numeric data +- Exact-match indexes are insufficient + +--- + +## Summary + +- Vector indexing enables fast similarity search on numeric arrays +- Defined using `@indexed(type: "HNSW")` +- Queried using a target vector in search sorting +- Tunable for performance and accuracy diff --git a/skills/harper-best-practices/SKILL.md b/skills/harper-best-practices/SKILL.md new file mode 100644 index 000000000..07ec4124e --- /dev/null +++ b/skills/harper-best-practices/SKILL.md @@ -0,0 +1,412 @@ +--- +name: harper-best-practices +description: > + Best practices for building Harper applications, covering schema definition, + automatic APIs, authentication, custom resources, and data handling. + Triggers on tasks involving Harper database design, API implementation, + and deployment. +origin: ECC +--- + +# Harper Best Practices + +Guidelines for building scalable, secure, and performant applications on Harper. These practices cover everything from initial schema design to advanced deployment strategies. + +## When to Use + +Reference these guidelines when: + +- Defining or modifying database schemas +- Implementing or extending REST/WebSocket APIs +- Handling authentication and session management +- Working with custom resources and extensions +- Optimizing data storage and retrieval (Blobs, Vector Indexing) +- Deploying applications to Harper Fabric + +## Steps + +1. Review the requirements for the task (schema design, API needs, or infrastructure setup). +2. Consult the relevant category under "Rule Categories by Priority" to understand the impact of your decisions. +3. Apply specific rules from the "Rule Details" section below by reading the corresponding rule files in `rules/harper/`. +4. If you're building a new table, prioritize the `schema-` rules. +5. If you're extending functionality, consult the `logic-` and `api-` rules. +6. Validate your implementation against the `ops-` rules before deployment. + +## Rule Categories by Priority + +| Priority | Category | Impact | Prefix | +| -------- | ----------------------- | ------ | --------------- | +| 1 | Schema & Data Design | HIGH | `schema-` | +| 2 | API & Communication | HIGH | `api-` | +| 3 | Logic & Extension | MEDIUM | `logic-` | +| 4 | Infrastructure & Ops | MEDIUM | `ops-` | + +## Rule Details + +> All rule files are located in `rules/harper/`. Each section heading below links directly to its rule file. Read the linked file for full details and code examples before implementing. + +### 1. Schema & Data Design + +**Impact: HIGH** + +#### 1.1 [Adding Tables with Schemas](../../rules/harper/adding-tables-with-schemas.md) + +Instructions for the agent to follow when adding tables to a Harper database. + +##### When to Use +Use this skill when you need to define new data structures or modify existing ones in a Harper database. + +##### How It Works +1. **Create Dedicated Schema Files**: Prefer having a dedicated schema `.graphql` file for each table. Check the `config.yaml` file under `graphqlSchema.files` to see how it's configured. It typically accepts wildcards (e.g., `schemas/*.graphql`), but may be configured to point at a single file. +2. **Use Directives**: All available directives for defining your schema are defined in `node_modules/harperdb/schema.graphql`. Common directives include `@table`, `@export`, `@primaryKey`, `@indexed`, and `@relationship`. +3. **Define Relationships**: Link tables together using the `@relationship` directive. +4. **Enable Automatic APIs**: If you add `@table @export` to a schema type, Harper automatically sets up REST and WebSocket APIs for basic CRUD operations against that table. +5. **Consider Table Extensions**: If you are going to extend the table in your resources, then do not `@export` the table from the schema. + +##### Example +```graphql +type ExamplePerson @table @export { + id: ID @primaryKey + name: String + tag: String @indexed +} +``` + +#### 1.2 [Defining Relationships](../../rules/harper/defining-relationships.md) + +Using the `@relationship` directive to link tables. + +##### When to Use +Use this when you have two or more tables that need to be logically linked (e.g., a "Product" table and a "Category" table). + +##### How It Works +1. **Identify the Relationship Type**: Determine if it's one-to-one, many-to-one, or one-to-many. +2. **Apply the `@relationship` Directive**: In your GraphQL schema, use the `@relationship` directive on the field that links to another table. + - **Many-to-One (Current table holds FK)**: Use `from`. + ```graphql + type Book @table @export { + authorId: ID + author: Author @relationship(from: "authorId") + } + ``` + - **One-to-Many (Related table holds FK)**: Use `to` and an array type. + ```graphql + type Author @table @export { + books: [Book] @relationship(to: "authorId") + } + ``` +3. **Query with Relationships**: Use dot syntax in REST API calls for filtering or the `select()` operator for including related data. + +##### Example +```graphql +type Product @table @export { + id: ID @primaryKey + name: String + categoryId: ID + category: Category @relationship(from: "categoryId") +} + +type Category @table @export { + id: ID @primaryKey + name: String + products: [Product] @relationship(to: "categoryId") +} +``` + +#### 1.3 [Vector Indexing](../../rules/harper/vector-indexing.md) + +How to define and use vector indexes for efficient similarity search. + +##### When to Use +Use this when you need to perform similarity searches on high-dimensional data, such as image embeddings, text embeddings, or any other numeric vectors. + +##### How It Works +1. **Define the Vector Field**: In your GraphQL schema, define a field with a list of floats (e.g., `[Float]`). +2. **Apply the `@indexed` Directive**: Use the `@indexed` directive on the vector field and specify the index type as `vector`. +3. **Configure the Index (Optional)**: You can provide additional configuration for the vector index, such as the distance metric (e.g., `cosine`, `euclidean`). +4. **Querying**: Use the `vector` operator in your REST or programmatic requests to perform similarity searches. + +##### Example +```graphql +type Document @table @export { + id: ID @primaryKey + content: String + embedding: [Float] @indexed(type: "vector", options: { dims: 1536, metric: "cosine" }) +} +``` + +#### 1.4 [Using Blobs](../../rules/harper/using-blob-datatype.md) + +How to store and retrieve large data in Harper. + +##### When to Use +Use this when you need to store large, unstructured data such as files, images, or large text documents that exceed the typical size of a standard database field. + +##### How It Works +1. **Define the Blob Field**: Use the `Blob` scalar type in your GraphQL schema. +2. **Storing Data**: Send the data as a buffer or a stream when creating or updating a record. +3. **Retrieving Data**: Access the blob field, which will return the data as a stream or buffer. + +#### 1.5 [Handling Binary Data](../../rules/harper/handling-binary-data.md) + +How to store and serve binary data like images or MP3s. + +##### When to Use +Use this when your application needs to handle binary files, particularly for storage and retrieval. + +##### How It Works +1. **Use the `Blob` type**: As with general large data, the `Blob` type is best for binary files. Ensure you store and retrieve the appropriate MIME type (e.g., `image/jpeg`, `audio/mpeg`) for the data. +2. **Streaming**: For large files, use streaming to minimize memory usage during upload and download. +3. **MIME Types**: Store the MIME type alongside the binary data to ensure it is served correctly by your application logic. + +--- + +### 2. API & Communication + +**Impact: HIGH** + +#### 2.1 [Automatic REST APIs](../../rules/harper/automatic-apis.md) + +Details on the CRUD endpoints automatically generated for exported tables. + +##### Endpoints +- `GET /{TableName}`: Describes the schema. +- `GET /{TableName}/`: Lists records (supports filtering/sorting). +- `GET /{TableName}/{id}`: Gets a record by ID. +- `POST /{TableName}/`: Creates a record. +- `PUT /{TableName}/{id}`: Updates a record. +- `PATCH /{TableName}/{id}`: Partial update. +- `DELETE /{TableName}/`: Deletes records. +- `DELETE /{TableName}/{id}`: Deletes by ID. + +#### 2.2 [Querying REST APIs](../../rules/harper/querying-rest-apis.md) + +How to use filters, operators, sorting, and pagination in REST requests. + +##### Query Parameters +- `limit`: Number of records to return. +- `offset`: Number of records to skip. +- `sort`: Field to sort by. +- `order`: `asc` or `desc`. +- `filter`: JSON object for filtering. + +#### 2.3 [Real-time Applications](../../rules/harper/real-time-apps.md) + +Implementing WebSockets and Pub/Sub for live data updates. + +##### When to Use +Use this for applications that require live updates, such as chat apps, live dashboards, or collaborative tools. + +##### How It Works +1. **WebSocket Connection**: Connect to the Harper WebSocket endpoint. Use `wss://` for secure connections over HTTPS, or `ws://` for local development. +2. **Subscribing**: Subscribe to table updates or specific records. +3. **Pub/Sub**: Use the internal bus to publish and subscribe to custom events. + +#### 2.4 [Checking Authentication](../../rules/harper/checking-authentication.md) + +How to use sessions to verify user identity and roles. + +##### When to Use +Use this to secure your application by ensuring that only authorized users can access certain resources or perform specific actions. + +##### How It Works +1. **Session Handling**: Access the session object from the request context. +2. **Identity Verification**: Check for the presence of a user ID or token. +3. **Role Checks**: Verify if the user has the required roles for the action. + +--- + +### 3. Logic & Extension + +**Impact: MEDIUM** + +#### 3.1 [Custom Resources](../../rules/harper/custom-resources.md) + +How to define custom REST endpoints using JavaScript or TypeScript. + +##### How It Works +1. **Create Resource File**: Define your logic in a JS or TS file. +2. **Export Handlers**: Export functions like `GET`, `POST`, etc. +3. **Registration**: Ensure the resource is correctly registered in your application configuration. + +#### 3.2 [Extending Table Resources](../../rules/harper/extending-tables.md) + +Adding custom logic to automatically generated table resources. + +##### How It Works +1. **Define Extension**: Create a resource file that targets an existing table. +2. **Intercept Requests**: Use handlers to add custom validation or data transformation. +3. **No `@export`**: If extending, remember not to `@export` the table in the schema. + +#### 3.3 [Programmatic Table Requests](../../rules/harper/programmatic-table-requests.md) + +How to use filters, operators, sorting, and pagination in programmatic table requests. + +##### Usage +When writing custom resources, use the internal API to query tables with full support for advanced filtering and sorting. + +#### 3.4 [TypeScript Type Stripping](../../rules/harper/typescript-type-stripping.md) + +Using TypeScript directly without build tools via Node.js Type Stripping. + +##### Configuration +Harper supports native TypeScript type stripping, allowing you to run `.ts` files directly. Ensure your environment is configured to take advantage of this for faster development cycles. + +#### 3.5 [Caching](../../rules/harper/caching.md) + +How caching is defined and implemented in Harper applications. + +##### Strategies +- **In-memory**: For fast access to frequently used data. +- **Distributed**: For scaling across multiple nodes in Harper Fabric. + +--- + +### 4. Infrastructure & Ops + +**Impact: MEDIUM** + +#### 4.1 [Creating Harper Applications](../../rules/harper/creating-harper-apps.md) + +The fastest way to start a new Harper project is using the `create-harper` CLI tool. This command initializes a project with a standard folder structure, essential configuration files, and basic schema definitions. + +##### When to Use +Use this command when starting a new Harper application or adding a new Harper microservice to an existing architecture. + +##### Commands +Initialize a project using your preferred package manager: + +**NPM** +```bash +npm create harper@latest +``` + +**PNPM** +```bash +pnpm create harper@latest +``` + +**Bun** +```bash +bun create harper@latest +``` + +#### 4.2 [Creating a Fabric Account and Cluster](../../rules/harper/creating-a-fabric-account-and-cluster.md) + +Follow these steps to set up your Harper Fabric environment for deployment. + +##### How It Works + +1. **Sign Up/In**: Go to [https://fabric.harper.fast/](https://fabric.harper.fast/) and sign up or sign in. +2. **Create an Organization**: Create an organization (org) to manage your projects. +3. **Create a Cluster**: Create a new cluster. This can be on the free tier, no credit card required. +4. **Set Credentials**: During setup, set the cluster username and password to finish configuring it. +5. **Get Application URL**: Navigate to the **Config** tab and copy the **Application URL**. +6. **Configure Environment**: Update your `.env` file or GitHub Actions secrets with these cluster-specific credentials: + ```bash + CLI_TARGET_USERNAME='YOUR_CLUSTER_USERNAME' + CLI_TARGET_PASSWORD='YOUR_CLUSTER_PASSWORD' + CLI_TARGET='YOUR_CLUSTER_URL' + ``` + +#### 4.3 [Deploying to Harper Fabric](../../rules/harper/deploying-to-harper-fabric.md) + +Globally scaling your Harper application. + +##### Benefits +- **Global Distribution**: Low latency for users everywhere. +- **Automatic Sync**: Data is synced across the fabric automatically. +- **Free Tier**: Start for free and scale as you grow. + +##### How It Works +1. **Sign up**: Follow the [Creating a Fabric Account and Cluster](#42-creating-a-fabric-account-and-cluster) steps to create a Harper Fabric account, organization, and cluster. +2. **Configure Environment**: Add your cluster credentials and cluster application URL to `.env`: + ```bash + CLI_TARGET_USERNAME='YOUR_CLUSTER_USERNAME' + CLI_TARGET_PASSWORD='YOUR_CLUSTER_PASSWORD' + CLI_TARGET='YOUR_CLUSTER_URL' + ``` +3. **Deploy From Local Environment**: Run `npm run deploy`. +4. **Set up CI/CD**: Configure `.github/workflows/deploy.yaml` and set repository secrets for automated deployments. + +##### Manual Setup for Existing Apps + +If your application was not created with `npm create harper`, you'll need to manually configure the deployment scripts and CI/CD workflow. + +###### 1. Update `package.json` + +Add the following scripts and dependencies to your `package.json`: + +```json +{ + "scripts": { + "deploy": "dotenv -- npm run deploy:component", + "deploy:component": "harperdb deploy_component . restart=rolling replicated=true" + }, + "devDependencies": { + "dotenv-cli": "^11.0.0", + "harperdb": "^4.7.20" + } +} +``` + +####### Why split the scripts? + +The `deploy` script is separated from `deploy:component` to ensure environment variables from your `.env` file are properly loaded and passed to the Harper CLI. + +- `deploy`: Uses `dotenv-cli` to load environment variables (like `CLI_TARGET`, `CLI_TARGET_USERNAME`, and `CLI_TARGET_PASSWORD`) before executing the next command. +- `deploy:component`: The actual command that performs the deployment. + +By using `dotenv -- npm run deploy:component`, the environment variables are correctly set in the shell session before `harperdb deploy_component` is called, allowing it to authenticate with your cluster. + +###### 2. Configure GitHub Actions + +Create a `.github/workflows/deploy.yaml` file with the following content: + +```yaml +name: Deploy to Harper Fabric +on: + workflow_dispatch: +# push: +# branches: +# - main +concurrency: + group: main + cancel-in-progress: false +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + - name: Set up Node.js + uses: actions/setup-node@v4 + with: + cache: 'npm' + node-version: '20' + - name: Install dependencies + run: npm ci + - name: Run unit tests + run: npm test + - name: Run lint + run: npm run lint + - name: Deploy + run: npm run deploy + env: + CLI_TARGET: ${{ secrets.CLI_TARGET }} + CLI_TARGET_USERNAME: ${{ secrets.CLI_TARGET_USERNAME }} + CLI_TARGET_PASSWORD: ${{ secrets.CLI_TARGET_PASSWORD }} +``` + +Be sure to set the following repository secrets in your GitHub repository: +- `CLI_TARGET` +- `CLI_TARGET_USERNAME` +- `CLI_TARGET_PASSWORD` + +#### 4.4 [Serving Web Content](../../rules/harper/serving-web-content.md) + +Two ways to serve web content from a Harper application. + +##### Methods +1. **Static Serving**: Serve HTML, CSS, and JS files directly. If using the Vite plugin for development, ensure Harper is running (e.g., `harperdb run .`) to allow for Hot Module Replacement (HMR). +2. **Dynamic Rendering**: Use custom resources to render content on the fly.