Skip to content

Commit 70e6aca

Browse files
GsCommandGsCommand
committed
feat: harden runtime security controls and add CI smoke coverage
1 parent 98e2960 commit 70e6aca

8 files changed

Lines changed: 685 additions & 18 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
pull_request:
6+
7+
jobs:
8+
checks:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout
12+
uses: actions/checkout@v4
13+
14+
- name: Setup Node
15+
uses: actions/setup-node@v4
16+
with:
17+
node-version: 20
18+
cache: npm
19+
20+
- name: Install dependencies
21+
run: npm ci
22+
23+
- name: Syntax check
24+
run: npm run check
25+
26+
- name: Smoke tests
27+
run: npm test

‎README.md‎

Lines changed: 147 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,152 @@
1-
# CommandLayer Runtime (Commons)
1+
# CommandLayer Runtime
22

3-
Reference runtime for CommandLayer Commons verbs.
3+
Reference Node.js runtime for CommandLayer Commons verbs. This service executes deterministic verb handlers, signs receipts with Ed25519, and verifies receipts using local keys or ENS-discovered public keys.
44

5-
## Endpoints
6-
- GET /health
7-
- GET /debug/env
8-
- POST /fetch/v1.0.0
9-
- POST /verify
5+
## What this service does
106

11-
## Example
12-
RECEIPT=$(curl -s -X POST https://<YOUR_DOMAIN>/fetch/v1.0.0 \
7+
- Exposes `POST /<verb>/v1.0.0` endpoints for Commons verbs (`fetch`, `describe`, `format`, `clean`, `parse`, `summarize`, `convert`, `explain`, `analyze`, `classify`).
8+
- Returns signed receipts containing:
9+
- deterministic result payloads,
10+
- execution trace metadata,
11+
- proof metadata (`alg`, canonical mode, SHA-256 hash, signature).
12+
- Exposes `POST /verify` to verify receipt hash/signature, and optionally validate schema + fetch public key from ENS.
13+
- Includes schema validator caching, warmup queueing, SSRF protections for `fetch`, and runtime safety budgets.
14+
15+
## API overview
16+
17+
### Core routes
18+
19+
- `GET /` — service index with links and enabled verbs.
20+
- `GET /health` — process/service health and signer readiness.
21+
- `POST /<verb>/v1.0.0` — execute a single verb and return a signed receipt.
22+
- `POST /verify` — verify receipt integrity/signature; optional schema and ENS verification.
23+
24+
### Debug routes
25+
26+
> Debug routes are disabled by default. Enable with `DEBUG_ROUTES_ENABLED=1` and optionally protect with `DEBUG_BEARER_TOKEN`.
27+
28+
- `GET /debug/env` — effective runtime configuration.
29+
- `GET /debug/enskey` — ENS TXT key discovery state.
30+
- `GET /debug/schemafetch?verb=<verb>` — computed receipt schema URL.
31+
- `GET /debug/validators` — validator cache and warm-queue state.
32+
- `POST /debug/prewarm` — queue schema validator warmup.
33+
34+
## Quickstart
35+
36+
### 1) Install dependencies
37+
38+
```bash
39+
npm install
40+
```
41+
42+
### 2) Generate an Ed25519 keypair (for local signing)
43+
44+
```bash
45+
openssl genpkey -algorithm Ed25519 -out private.pem
46+
openssl pkey -in private.pem -pubout -out public.pem
47+
48+
export RECEIPT_SIGNING_PRIVATE_KEY_PEM_B64="$(base64 -w0 < private.pem)"
49+
export RECEIPT_SIGNING_PUBLIC_KEY_PEM_B64="$(base64 -w0 < public.pem)"
50+
export RECEIPT_SIGNER_ID="runtime.local"
51+
```
52+
53+
> macOS note: replace `base64 -w0` with `base64 | tr -d '\n'`.
54+
55+
### 3) Start the runtime
56+
57+
```bash
58+
npm start
59+
```
60+
61+
Default port is `8080` (override with `PORT`).
62+
63+
### 4) Verify startup
64+
65+
```bash
66+
curl -s http://localhost:8080/health | jq .
67+
```
68+
69+
You should see `"ok": true` and `"signer_ok": true`.
70+
71+
## Example flow
72+
73+
### Request a fetch receipt
74+
75+
```bash
76+
RECEIPT=$(curl -s -X POST "http://localhost:8080/fetch/v1.0.0" \
77+
-H "Content-Type: application/json" \
78+
-d '{
79+
"x402": {
80+
"entry": "x402://fetchagent.eth/fetch/v1.0.0",
81+
"verb": "fetch",
82+
"version": "1.0.0"
83+
},
84+
"source": "https://example.com"
85+
}')
86+
87+
printf '%s\n' "$RECEIPT" | jq .
88+
```
89+
90+
### Verify the receipt locally
91+
92+
```bash
93+
printf '%s' "$RECEIPT" | curl -s -X POST "http://localhost:8080/verify" \
94+
-H "Content-Type: application/json" \
95+
-d @- | jq .
96+
```
97+
98+
### Verify with ENS public key lookup
99+
100+
```bash
101+
printf '%s' "$RECEIPT" | curl -s -X POST "http://localhost:8080/verify?ens=1" \
13102
-H "Content-Type: application/json" \
14-
-d '{"x402":{"entry":"x402://fetchagent.eth/fetch/v1.0.0","verb":"fetch","version":"1.0.0"},"source":"https://example.com"}')
103+
-d @- | jq .
104+
```
105+
106+
## Verification semantics
107+
108+
`POST /verify` supports query flags:
109+
110+
- `ens=1` — fetch verifier pubkey from ENS TXT record (`VERIFIER_ENS_NAME`, `ENS_PUBKEY_TEXT_KEY`).
111+
- `refresh=1` — bypass ENS cache and refresh lookup.
112+
- `schema=1` — validate receipt against verb schema.
113+
114+
When `VERIFY_SCHEMA_CACHED_ONLY=1` (default), schema validation is edge-safe:
115+
116+
- if validator is warm: request is fully validated,
117+
- if validator is cold: service returns `202` with `validator_not_warmed_yet` and queues async prewarm.
118+
119+
Use `POST /debug/prewarm` and `GET /debug/validators` for schema prewarming workflows.
120+
121+
## Configuration
122+
123+
Detailed environment variable documentation lives in [`docs/CONFIGURATION.md`](docs/CONFIGURATION.md).
124+
125+
## Development checks
126+
127+
```bash
128+
npm run check
129+
npm test
130+
```
131+
132+
`npm test` runs an automated smoke suite for signer readiness, verb execution, verify pass/fail paths, and debug route auth.
133+
134+
## Security notes
135+
136+
- `fetch` only allows `http(s)` URLs.
137+
- SSRF guard blocks localhost/private IP ranges and DNS resolutions to private ranges.
138+
- Optional host allowlist (`ALLOW_FETCH_HOSTS`) can strictly bound outbound `fetch`.
139+
- Request-level limits are capped by server-side `SERVER_MAX_HANDLER_MS`.
140+
- `/verify` execution is bounded by `VERIFY_MAX_MS`.
141+
142+
## Operational notes
143+
144+
- Validator/schema caches are in-memory (per process).
145+
- Prewarm is best-effort and asynchronous.
146+
- In multi-replica deployments, warm each replica independently.
147+
148+
See [`docs/OPERATIONS.md`](docs/OPERATIONS.md) for deployment and runbook guidance.
149+
150+
## Engineering review artifacts
15151

16-
printf '%s' "$RECEIPT" | curl -s -X POST "https://<YOUR_DOMAIN>/verify?ens=1" \
17-
-H "Content-Type: application/json" -d @-
152+
For a focused codebase review (strengths, risks, and prioritized improvements), see [`docs/REVIEW.md`](docs/REVIEW.md).

‎docs/CONFIGURATION.md‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Configuration Reference
2+
3+
This runtime is configured via environment variables.
4+
5+
## Core service identity
6+
7+
| Variable | Default | Purpose |
8+
|---|---|---|
9+
| `PORT` | `8080` | HTTP listen port. |
10+
| `SERVICE_NAME` | `commandlayer-runtime` | Name exposed in index/health metadata. |
11+
| `SERVICE_VERSION` | `1.0.0` | Service version exposed in responses. |
12+
| `API_VERSION` | `1.0.0` | Version segment used in verb route shape. |
13+
| `CANONICAL_BASE_URL` | `https://runtime.commandlayer.org` | Base URL metadata in index/health payloads. |
14+
15+
## CORS controls
16+
17+
| Variable | Default | Purpose |
18+
|---|---|---|
19+
| `CORS_ALLOW_ORIGINS` | empty | CSV allowlist of browser origins. Empty rejects cross-origin browser requests. Use `*` only when intended. |
20+
| `CORS_ALLOW_HEADERS` | `Content-Type, Authorization` | Allowed CORS request headers. |
21+
| `CORS_ALLOW_METHODS` | `GET,POST,OPTIONS` | Allowed CORS methods. |
22+
23+
## Enabled verbs
24+
25+
| Variable | Default |
26+
|---|---|
27+
| `ENABLED_VERBS` | `fetch,describe,format,clean,parse,summarize,convert,explain,analyze,classify` |
28+
29+
Comma-separated list of enabled handlers. Disabled verbs return `404`.
30+
31+
## Signing + verifier identity
32+
33+
| Variable | Default | Purpose |
34+
|---|---|---|
35+
| `RECEIPT_SIGNER_ID` | `runtime` (or `ENS_NAME` when set) | Receipt proof signer identifier. |
36+
| `RECEIPT_SIGNING_PRIVATE_KEY_PEM_B64` | empty | Required for signing receipts. Base64 of PEM private key. |
37+
| `RECEIPT_SIGNING_PUBLIC_KEY_PEM_B64` | empty | Optional local pubkey for `/verify` signature checks. |
38+
| `ENS_NAME` | empty | Optional identity alias fallback. |
39+
40+
## ENS-based verification
41+
42+
| Variable | Default | Purpose |
43+
|---|---|---|
44+
| `ETH_RPC_URL` | empty | Ethereum RPC endpoint for ENS resolver lookups. |
45+
| `VERIFIER_ENS_NAME` | `ENS_NAME` / `RECEIPT_SIGNER_ID` fallback | ENS name queried for TXT pubkey value. |
46+
| `ENS_PUBKEY_TEXT_KEY` | `cl.receipt.pubkey.pem` | ENS TXT key containing PEM-formatted public key. |
47+
48+
## Schema fetching + validation budgets
49+
50+
| Variable | Default | Purpose |
51+
|---|---|---|
52+
| `SCHEMA_HOST` | `https://www.commandlayer.org` | Schema host prefix used to compute receipt schema URLs. |
53+
| `SCHEMA_FETCH_TIMEOUT_MS` | `15000` | Timeout per schema document fetch. |
54+
| `SCHEMA_VALIDATE_BUDGET_MS` | `15000` | Budget for async schema compilation. |
55+
| `VERIFY_SCHEMA_CACHED_ONLY` | `1` | If `1`, `/verify?schema=1` only uses warm validators and returns `202` on cold cache. |
56+
| `REQUEST_SCHEMA_VALIDATION` | `0` | If `1`, verb endpoints validate request payloads against verb request schema before execution. |
57+
58+
## Debug route controls
59+
60+
| Variable | Default | Purpose |
61+
|---|---|---|
62+
| `DEBUG_ROUTES_ENABLED` | `0` | If `1`, enables `/debug/*` endpoints. |
63+
| `DEBUG_BEARER_TOKEN` | empty | Optional bearer token required for `/debug/*` when set. |
64+
65+
## Cache controls
66+
67+
| Variable | Default |
68+
|---|---|
69+
| `MAX_JSON_CACHE_ENTRIES` | `256` |
70+
| `JSON_CACHE_TTL_MS` | `600000` |
71+
| `MAX_VALIDATOR_CACHE_ENTRIES` | `128` |
72+
| `VALIDATOR_CACHE_TTL_MS` | `1800000` |
73+
74+
## Request safety limits
75+
76+
| Variable | Default | Purpose |
77+
|---|---|---|
78+
| `SERVER_MAX_HANDLER_MS` | `12000` | Hard upper bound for verb execution timeout. |
79+
| `VERIFY_MAX_MS` | `30000` | Upper bound for `/verify` request processing. |
80+
81+
## `fetch` hardening
82+
83+
| Variable | Default | Purpose |
84+
|---|---|---|
85+
| `FETCH_TIMEOUT_MS` | `8000` | Timeout for outbound `fetch` HTTP request. |
86+
| `FETCH_MAX_BYTES` | `262144` | Max bytes read from outbound response body. |
87+
| `ENABLE_SSRF_GUARD` | `1` | Enables DNS/IP/local-network SSRF checks. |
88+
| `ALLOW_FETCH_HOSTS` | empty | Optional CSV domain allowlist (`example.com,api.example.com`). |
89+
90+
## Schema prewarm behavior
91+
92+
| Variable | Default | Purpose |
93+
|---|---|---|
94+
| `PREWARM_MAX_VERBS` | `25` | Max verbs accepted in one `/debug/prewarm` call. |
95+
| `PREWARM_TOTAL_BUDGET_MS` | `12000` | Total worker runtime budget. |
96+
| `PREWARM_PER_VERB_BUDGET_MS` | `5000` | Max warm budget per verb. |
97+
98+
## Recommended production baseline
99+
100+
- Set explicit signing keys and verify `signer_ok=true` on `/health`.
101+
- Keep `VERIFY_SCHEMA_CACHED_ONLY=1` for edge stability.
102+
- Restrict egress using both network policy and `ALLOW_FETCH_HOSTS` where possible.
103+
- Tune `FETCH_MAX_BYTES` and timeout budgets based on expected payload sizes.
104+
- Poll `/debug/validators` after deploy and prewarm critical verbs.

‎docs/OPERATIONS.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Operations Runbook
2+
3+
## Deploy checklist
4+
5+
1. Set signing keys:
6+
- `RECEIPT_SIGNING_PRIVATE_KEY_PEM_B64`
7+
- `RECEIPT_SIGNING_PUBLIC_KEY_PEM_B64`
8+
2. Set identity metadata:
9+
- `RECEIPT_SIGNER_ID`
10+
- `SERVICE_NAME`, `SERVICE_VERSION`
11+
3. If using ENS verification:
12+
- `ETH_RPC_URL`
13+
- `VERIFIER_ENS_NAME`
14+
- `ENS_PUBKEY_TEXT_KEY`
15+
4. Set safety limits (`FETCH_TIMEOUT_MS`, `FETCH_MAX_BYTES`, `VERIFY_MAX_MS`).
16+
5. Restrict outbound domains with `ALLOW_FETCH_HOSTS` where possible.
17+
18+
## Post-deploy validation
19+
20+
```bash
21+
curl -s "$BASE_URL/health" | jq .
22+
curl -s "$BASE_URL/debug/env" | jq .
23+
```
24+
25+
Expected:
26+
- `ok=true`
27+
- `signer_ok=true`
28+
- expected `enabled_verbs`
29+
- expected timeouts/cache settings
30+
31+
## Schema prewarm sequence
32+
33+
```bash
34+
curl -s -X POST "$BASE_URL/debug/prewarm" \
35+
-H 'content-type: application/json' \
36+
-d '{"verbs":["fetch","parse","summarize","classify"]}' | jq .
37+
38+
curl -s "$BASE_URL/debug/validators" | jq .
39+
```
40+
41+
Repeat validator polling until required verbs appear under `cached`.
42+
43+
## Verification troubleshooting
44+
45+
### `no public key available`
46+
47+
- Set `RECEIPT_SIGNING_PUBLIC_KEY_PEM_B64` **or** use ENS verification with:
48+
- `ETH_RPC_URL`
49+
- `VERIFIER_ENS_NAME`
50+
- valid PEM at ENS TXT key.
51+
52+
### `validator_not_warmed_yet` with HTTP 202
53+
54+
- Expected when `VERIFY_SCHEMA_CACHED_ONLY=1` and schema validator is cold.
55+
- Trigger `/debug/prewarm` and retry `/verify?schema=1`.
56+
57+
### `schema fetch failed`
58+
59+
- Confirm schema host reachability from runtime environment.
60+
- Check `SCHEMA_HOST`, `SCHEMA_FETCH_TIMEOUT_MS`, outbound egress rules.
61+
62+
## Recommended observability
63+
64+
At minimum, capture and alert on:
65+
- HTTP 5xx rate by endpoint and verb.
66+
- `/verify` latency and timeout count.
67+
- `fetch` timeout/error rates.
68+
- cold-validator 202 rate after deploy.
69+
- cache sizes from `/debug/validators`.
70+
71+
## Hardening notes
72+
73+
- Keep CORS policy constrained if this service is not intended for broad browser access.
74+
- If internet fetch is not required, disable `fetch` verb via `ENABLED_VERBS`.
75+
- Consider process isolation or egress proxy for stricter SSRF containment.

0 commit comments

Comments
 (0)