Skip to content

Commit 3ce77b7

Browse files
authored
docs: update public docs for recent features (#27)
* chore: add .worktrees/ to gitignore * docs: update public docs for maintenance mode, env var secrets, and sensitive fields - fleet.md: add maintenance mode section (toggle, behavior, access control) - security.md: rewrite secret resolution to reflect env var delivery model, add secret name normalization rules, add sensitive fields section - agent.md: update config response docs (checksum includes secrets, maintenance mode returns empty pipelines), clarify env injection - database.md: add maintenanceMode fields to VectorNode, git fields to Environment - pipeline-editor.md: add secret picker note for sensitive fields * docs: fix secret collision tie-break description The code overwrites on collision (last alphabetical match wins), not first match. * docs: clarify that secrets are still transmitted in config response The hint incorrectly implied secrets are not sent over the wire. They are — just separately from the YAML, in the config response secrets dictionary.
1 parent c0b2229 commit 3ce77b7

File tree

6 files changed

+85
-6
lines changed

6 files changed

+85
-6
lines changed

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,4 +56,6 @@ next-env.d.ts
5656
# codeql
5757
.codeql/
5858

59-
/scripts/*
59+
/scripts/*
60+
# worktrees
61+
.worktrees/

docs/public/operations/security.md

Lines changed: 34 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ When you create or update a secret, the value is encrypted with **AES-256-GCM**
1818

1919
### How secrets are resolved
2020

21-
When a pipeline is deployed, VectorFlow generates the Vector configuration file. During generation, it scans the configuration for **secret references** and replaces them with the actual decrypted values.
21+
When a pipeline is deployed, VectorFlow generates the Vector configuration file. During generation, it scans the configuration for **secret references** and converts them into environment variable placeholders. The actual secret values are delivered separately to the agent and injected as environment variables into the Vector process.
2222

2323
Secret references use the syntax:
2424

@@ -34,7 +34,38 @@ sasl:
3434
password: SECRET[KAFKA_PASSWORD]
3535
```
3636
37-
At deploy time, `SECRET[KAFKA_PASSWORD]` is resolved to the decrypted value of the `KAFKA_PASSWORD` secret in the pipeline's environment.
37+
At deploy time, VectorFlow converts this to:
38+
39+
```yaml
40+
sasl:
41+
username: "my-user"
42+
password: "${VF_SECRET_KAFKA_PASSWORD}"
43+
```
44+
45+
The actual decrypted value of `KAFKA_PASSWORD` is delivered to the agent in a separate `secrets` dictionary alongside the YAML. The agent sets `VF_SECRET_KAFKA_PASSWORD` as an environment variable when starting the Vector process, and Vector's built-in environment variable interpolation resolves the placeholder at runtime.
46+
47+
{% hint style="info" %}
48+
Secret values never appear in pipeline YAML -- not in the database, not in config files on disk, and not in the YAML sent to agents. Only environment variable placeholders are embedded in the configuration. The actual values are delivered separately in the config response and injected as process environment variables.
49+
{% endhint %}
50+
51+
#### Secret name normalization
52+
53+
Secret names are converted to valid environment variable names using these rules:
54+
55+
- Prefixed with `VF_SECRET_`
56+
- All non-alphanumeric characters replaced with underscores
57+
- Converted to uppercase
58+
59+
For example, `my-api.key` becomes `VF_SECRET_MY_API_KEY`. If two secrets normalize to the same environment variable name (e.g., `db-password` and `db_password` both become `VF_SECRET_DB_PASSWORD`), VectorFlow logs a warning and the last match in alphabetical order takes precedence.
60+
61+
### Sensitive fields
62+
63+
Pipeline component fields that contain credentials -- passwords, API keys, tokens -- are restricted to **secret references only** in the pipeline editor. You cannot type a plaintext value into a sensitive field. Instead, the editor presents a secret picker that lets you select an existing secret or create a new one inline.
64+
65+
Fields are treated as sensitive when:
66+
67+
- The Vector component schema marks them as `sensitive: true`
68+
- The field name matches a pattern like `password`, `secret`, `token`, or `api_key`
3869

3970
### Certificate references
4071

@@ -59,7 +90,7 @@ VectorFlow encrypts sensitive data before storing it in PostgreSQL:
5990
| Certificates | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
6091
| OIDC client secret | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
6192
| Sensitive node config fields | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
62-
| Git access tokens | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
93+
| Git access tokens (audit trail) | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
6394
| User passwords | bcrypt (cost 12) | Built-in salt |
6495
| TOTP secrets | AES-256-GCM | SHA-256 hash of `NEXTAUTH_SECRET` |
6596
| 2FA backup codes | SHA-256 hash | -- |

docs/public/reference/agent.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -161,11 +161,16 @@ Called on every poll cycle. Returns all deployed pipeline configurations for thi
161161
```
162162

163163
Key fields:
164-
- **`secrets`**: Pre-resolved secret values with `VF_SECRET_` prefix. The agent injects these as environment variables into the Vector process.
164+
- **`configYaml`**: The generated Vector YAML. Secret references are converted to environment variable placeholders (e.g., `${VF_SECRET_AWS_KEY}`) rather than containing decrypted values.
165+
- **`secrets`**: Pre-resolved secret values keyed by their `VF_SECRET_` environment variable name. The agent injects these as environment variables into the Vector process, where Vector's interpolation resolves the placeholders.
166+
- **`checksum`**: Includes both the YAML and the secrets dictionary, so rotating a secret triggers a pipeline restart even if the YAML itself hasn't changed.
165167
- **`certFiles`**: Certificate data written to `<VF_DATA_DIR>/certs/` before starting the pipeline.
166-
- **`checksum`**: Used to detect config changes without re-parsing YAML.
167168
- **`pendingAction`**: Server-initiated action (currently only `self_update`).
168169

170+
{% hint style="info" %}
171+
When a node is in **maintenance mode**, the config response returns an empty `pipelines` array. The agent stops all running pipelines but continues sending heartbeats. See [Fleet Management](../user-guide/fleet.md#maintenance-mode) for details.
172+
{% endhint %}
173+
169174
### `POST /api/agent/heartbeat`
170175

171176
Called after every poll. Sends status and metrics for all managed pipelines.
@@ -228,6 +233,8 @@ Each Vector process receives:
228233
- `VECTOR_LOG=<logLevel>` -- controls Vector's log verbosity
229234
- All resolved secrets as environment variables with `VF_SECRET_` prefix (e.g., `VF_SECRET_AWS_KEY=value`)
230235

236+
Pipeline YAML references secrets as `${VF_SECRET_NAME}` placeholders. Vector's built-in environment variable interpolation resolves these at startup, so secret values never appear in configuration files on disk.
237+
231238
### Metrics sidecar
232239

233240
The agent automatically generates a sidecar config for each pipeline that adds:

docs/public/reference/database.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,6 +125,8 @@ Represents an enrolled agent node.
125125
| `vectorVersion` | `String?` | Reported Vector binary version |
126126
| `os` | `String?` | Operating system and architecture (e.g., `linux/amd64`) |
127127
| `deploymentMode` | `DeploymentMode` | `STANDALONE`, `DOCKER`, or `UNKNOWN` |
128+
| `maintenanceMode` | `Boolean` | Whether the node is in maintenance mode (pipelines are stopped) |
129+
| `maintenanceModeAt` | `DateTime?` | When the node entered maintenance mode (null if not in maintenance) |
128130
| `pendingAction` | `Json?` | Server-initiated action (e.g., self-update command) |
129131
| `createdAt` | `DateTime` | Registration timestamp |
130132

@@ -142,6 +144,9 @@ Logical grouping that contains nodes, pipelines, secrets, and certificates.
142144
| `enrollmentTokenHint` | `String?` | First few characters of the token for display |
143145
| `secretBackend` | `SecretBackend` | Secret storage: `BUILTIN`, `VAULT`, `AWS_SM`, `EXEC` |
144146
| `secretBackendConfig` | `Json?` | Configuration for external secret backends |
147+
| `gitRepoUrl` | `String?` | HTTPS URL of the Git repository for pipeline audit trail |
148+
| `gitBranch` | `String?` | Git branch to commit pipeline YAML to (default: `main`) |
149+
| `gitToken` | `String?` | Encrypted access token for Git repository authentication |
145150
| `createdAt` | `DateTime` | Creation timestamp |
146151

147152
### PipelineVersion

docs/public/user-guide/fleet.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,39 @@ Docker-based agents are updated by pulling the latest image. The **Update** butt
104104

105105
Below the node list, the **Pipeline Deployment Matrix** shows a grid of all deployed pipelines across all nodes in the environment. This lets you see at a glance which pipelines are running on which nodes and their current status.
106106

107+
## Maintenance mode
108+
109+
Maintenance mode lets you temporarily stop all pipelines on a node without removing it from the fleet. This is useful for host upgrades, kernel patches, disk maintenance, or any situation where you need the node idle but still connected.
110+
111+
### Entering maintenance mode
112+
113+
You can toggle maintenance mode from two places:
114+
115+
- **Fleet list** -- Click the **Maintenance** button in the node's row.
116+
- **Node detail page** -- Click the **Enter Maintenance** button in the header, or the **Exit Maintenance** button in the orange banner.
117+
118+
Both locations show a confirmation dialog before entering maintenance mode. The dialog warns that all running pipelines on the node will be stopped.
119+
120+
### What happens in maintenance mode
121+
122+
When maintenance mode is enabled on a node:
123+
124+
1. The node's status badge changes to an orange **Maintenance** indicator with a wrench icon.
125+
2. On the next poll, the agent receives an empty pipeline list from the server, causing all running pipelines to stop gracefully.
126+
3. The agent continues sending heartbeats, so the node remains visible and manageable in the fleet UI.
127+
128+
{% hint style="info" %}
129+
Maintenance mode is per-node. Other nodes in the same environment continue running their pipelines normally.
130+
{% endhint %}
131+
132+
### Exiting maintenance mode
133+
134+
Click **Exit Maintenance** from the fleet list or the node detail page. No confirmation is required. On the next poll cycle, the agent receives its full pipeline configuration again and automatically restarts all pipelines.
135+
136+
{% hint style="warning" %}
137+
Toggling maintenance mode requires the **Admin** role on the team.
138+
{% endhint %}
139+
107140
## Node management
108141

109142
From the node detail page you can:

docs/public/user-guide/pipeline-editor.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ The panel shows:
108108
- **Enabled toggle** -- Disable a component to exclude it from the generated configuration without removing it from the canvas.
109109
- **Type** -- The Vector component type (read-only).
110110
- **Configuration form** -- Auto-generated form fields based on the component's configuration schema. Required fields are marked, and each field has contextual help.
111+
- **Secret picker** -- Sensitive fields (passwords, API keys, tokens) display a secret picker instead of a text input. You must select an existing secret or create a new one inline -- plaintext values cannot be entered directly into sensitive fields. See [Security](../operations/security.md#sensitive-fields) for details.
111112

112113
### VRL editor for transforms
113114

0 commit comments

Comments
 (0)