Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
be7cc4f
feat: add SharedComponent model and PipelineNode link fields
TerrifiedBug Mar 10, 2026
b5574d3
feat: add shared component router with CRUD and link management
TerrifiedBug Mar 10, 2026
64b3360
feat: preserve shared component links and add stale detection to pipe…
TerrifiedBug Mar 10, 2026
8d960a7
feat: wire shared component data through flow store and pipeline page
TerrifiedBug Mar 10, 2026
0f1d05f
feat: add purple shared component visual treatment to pipeline nodes
TerrifiedBug Mar 10, 2026
a68b160
feat: add Shared tab to component palette with drag-to-add
TerrifiedBug Mar 10, 2026
758ebb4
feat: add shared component display, update banner, and save-as-shared…
TerrifiedBug Mar 10, 2026
134760e
feat: create Library page with templates and shared components
TerrifiedBug Mar 10, 2026
e64164b
feat: show stale shared component badge on pipeline list
TerrifiedBug Mar 10, 2026
e1f7e58
feat: handle shared component links in clone, discard, and template e…
TerrifiedBug Mar 10, 2026
c19dab8
fix: resolve useMemo dependency warning in component palette
TerrifiedBug Mar 10, 2026
d85b4c2
feat: add "Open in Library" link and kind filter for shared components
TerrifiedBug Mar 10, 2026
ce324cc
fix: address security and correctness issues in shared component router
TerrifiedBug Mar 10, 2026
1afee24
docs: add shared components documentation and update related pages
TerrifiedBug Mar 10, 2026
811ecb5
merge: resolve conflicts with main (displayName feature)
TerrifiedBug Mar 10, 2026
5fe1ebe
fix: validate environment membership in createFromNode and wrap updat…
TerrifiedBug Mar 10, 2026
e4a7cb4
fix: validate shared component environment in saveGraph and wrap crea…
TerrifiedBug Mar 10, 2026
1d60ae5
fix: sync Zustand store after shared component mutations and fix undo…
TerrifiedBug Mar 10, 2026
7b7e356
fix: sync flow store after Save as Shared Component
TerrifiedBug Mar 10, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/public/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
* [Fleet Management](user-guide/fleet.md)
* [Alerts](user-guide/alerts.md)
* [Templates](user-guide/templates.md)
* [Shared Components](user-guide/shared-components.md)

## Operations

Expand Down
24 changes: 16 additions & 8 deletions docs/public/user-guide/pipeline-editor.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,19 @@ The editor is divided into four main areas:

## Component palette

The left sidebar lists every available Vector component, grouped into three sections:
The left sidebar has two tabs:

- **Sources** -- Components that ingest data into the pipeline.
- **Transforms** -- Components that process, filter, or reshape data in flight.
- **Sinks** -- Components that send data to downstream destinations.
- **Catalog** -- Lists every available Vector component, grouped by kind (Sources, Transforms, Sinks). Each section can be collapsed and is further organized by category (e.g. "Cloud Platform", "Aggregating", "Messaging").
- **Shared** -- Lists [shared components](shared-components.md) available in the current environment. Filter by kind using the All / Source / Transform / Sink buttons.

Each section can be collapsed. When a section contains many components, they are further organized by category (e.g. "Cloud Platform", "Aggregating", "Messaging").

Use the **search bar** at the top of the palette to filter components by name, type, description, or category.
Use the **search bar** at the top of the palette to filter components by name, type, description, or category. The search applies to whichever tab is active.

### Adding a component

Drag a component from the palette and drop it onto the canvas. A new node appears at the drop position, pre-configured with sensible defaults. You can also right-click the canvas to paste previously copied nodes.

Dragging a shared component from the **Shared** tab creates a **linked node** -- the node's configuration is pre-filled from the shared component and pinned to its current version. See [Shared Components](shared-components.md) for details on linked node behavior.

## Canvas

The canvas is where your pipeline takes visual shape. Each component is represented as a **node**, and data flow between components is represented as **edges** (connections).
Expand All @@ -47,7 +46,7 @@ The canvas is where your pipeline takes visual shape. Each component is represen

### Context menus

- **Right-click a node** -- Opens a menu with Copy, Paste, Duplicate, and Delete actions.
- **Right-click a node** -- Opens a menu with Copy, Paste, Duplicate, Delete, and **Save as Shared Component** actions. The shared component option extracts the node's configuration into a reusable [shared component](shared-components.md) and links the node to it.
- **Right-click an edge** -- Opens a menu with a Delete connection action.

## Node types
Expand Down Expand Up @@ -116,6 +115,15 @@ The **Config** tab shows:
- **Configuration form** -- Auto-generated form fields based on the component's configuration schema. Required fields are marked, and each field has contextual help.
- **Secret picker** -- Sensitive fields (passwords, API keys, tokens) display a secret picker instead of a text input. You must select an existing secret or create a new one inline -- plaintext values cannot be entered directly into sensitive fields. See [Security](../operations/security.md#sensitive-fields) for details.

### Linked shared components

When a node is linked to a [shared component](shared-components.md), the detail panel shows additional elements:

- A **purple banner** with the shared component name and an "Open in Library" link.
- **Read-only configuration** -- All config fields are disabled. Edits must be made on the shared component's Library page.
- An **Unlink** button to convert the node back to a regular editable component (the current config snapshot is preserved).
- When the shared component has been updated since the node was last synced, an **amber update banner** appears with an "Accept update" button to pull in the latest configuration.

### VRL editor for transforms

For **remap**, **filter**, and **route** transforms, the detail panel includes an integrated VRL editor (powered by Monaco) instead of a plain text field. The VRL editor provides:
Expand Down
1 change: 1 addition & 0 deletions docs/public/user-guide/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ A pipeline moves through several states during its lifecycle:
- **Stopped** -- The pipeline is deployed but all agent nodes have stopped processing it.
- **Crashed** -- One or more agent nodes report that the pipeline has crashed. Check the pipeline logs for details.
- **Pending deploy** -- Shown as an additional badge when the saved configuration differs from what is currently deployed. Deploy the pipeline to push the latest changes.
- **Updates available** -- Shown as an amber badge when the pipeline contains nodes linked to [shared components](shared-components.md) that have been updated. Hover over the badge to see which shared components have pending updates. Open the pipeline editor to review and accept the changes.
- **Pending Approval** -- Shown when an editor has submitted a deploy request that is waiting for admin approval. See [Deploy approval workflows](#deploy-approval-workflows).

## Creating a pipeline
Expand Down
165 changes: 165 additions & 0 deletions docs/public/user-guide/shared-components.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
# Shared Components

Shared components are reusable, environment-scoped pipeline component configurations that can be linked into multiple pipelines. Edit a shared component in one place and every linked pipeline sees the update -- no more hunting through dozens of pipelines to change an endpoint.

## How shared components work

A shared component stores a canonical configuration for a specific component type (e.g., an Elasticsearch sink with your production cluster settings). When you link a shared component into a pipeline, the pipeline node receives a snapshot of that configuration and pins to a specific version.

When someone edits the shared component, its version number increments. Linked pipelines don't change automatically -- instead, they surface an **Update available** indicator at three levels:

- **Pipeline list** -- An amber "Updates available" badge on the pipeline row.
- **Canvas** -- The linked node's footer changes to "Update available" with an amber dot.
- **Detail panel** -- A banner with an "Accept update" button.

Pipeline owners review and explicitly accept updates before redeploying. This review-first model prevents surprise breakage across production pipelines.

{% hint style="info" %}
Shared components are scoped to a single environment. A production Kafka config is separate from a staging one. If you need the same component in multiple environments, create a shared component in each.
{% endhint %}

## Finding shared components

Shared components are managed in the **Library** page, accessible from the sidebar. The Library has two sections:

- **Templates** -- Reusable pipeline blueprints (see [Templates](templates.md)).
- **Shared Components** -- Reusable component configurations (this page).

### Shared components list

The list shows all shared components in the currently selected environment:

| Column | Description |
|--------|-------------|
| **Name** | The shared component name. Click to open the detail page. |
| **Type** | The Vector component type (e.g., `elasticsearch`, `kafka`). |
| **Kind** | A badge indicating Source, Transform, or Sink. |
| **Linked Pipelines** | Number of distinct pipelines using this shared component. |
| **Version** | Current version number (increments on each config change). |
| **Last Updated** | When the configuration was last modified. |

## Creating a shared component

There are two ways to create a shared component.

### From the Library page

{% stepper %}
{% step %}
### Open the Library
Navigate to **Library > Shared Components** in the sidebar and click **New Shared Component**.
{% endstep %}
{% step %}
### Choose a component type
Select the component type from the catalog grid (e.g., Elasticsearch, Kafka, S3).
{% endstep %}
{% step %}
### Configure
Enter a **Name** and optional **Description**, then fill in the component configuration using the form editor. Click **Create** to save.
{% endstep %}
{% endstepper %}

### From the pipeline editor

{% stepper %}
{% step %}
### Right-click a node
In the pipeline editor, right-click any configured node on the canvas.
{% endstep %}
{% step %}
### Select Save as Shared Component
Choose **Save as Shared Component** from the context menu. A dialog opens.
{% endstep %}
{% step %}
### Name and save
Enter a **Name** and optional **Description**, then click **Save**. The node's current configuration becomes the shared component, and the node is automatically linked to it.
{% endstep %}
{% endstepper %}

## Using shared components in pipelines

### Adding from the component palette

The component palette (left sidebar in the pipeline editor) has two tabs:

- **Catalog** -- The standard list of all Vector component types.
- **Shared** -- Shared components available in the current environment.

The Shared tab includes kind filter buttons (All, Source, Transform, Sink) and supports search by name or component type. Drag a shared component from the list onto the canvas to create a linked node with the shared configuration pre-filled.

### Linked node appearance

Linked nodes are visually distinct on the canvas:

- **Purple accent border** -- A purple border and subtle glow distinguish linked nodes from regular ones.
- **Footer** -- Shows "Shared" with a link icon.
- **Stale indicator** -- When an update is available, the footer changes to "Update available" in amber, and a small amber dot appears on the node corner.

### Detail panel for linked nodes

When you select a linked node, the detail panel shows:

- A **purple banner** with the shared component name and an "Open in Library" link.
- **Read-only configuration** -- Config fields are disabled because the configuration is managed centrally. To edit, go to the Library page.
- An **Unlink** button in the header to convert back to a regular editable node.

{% hint style="warning" %}
Unlinking a node preserves its current configuration but breaks the connection to the shared component. Future updates to the shared component will not reach this node.
{% endhint %}

## Editing a shared component

{% stepper %}
{% step %}
### Open the detail page
Navigate to **Library > Shared Components** and click the shared component you want to edit.
{% endstep %}
{% step %}
### Modify the configuration
Update the configuration using the form editor. You can also edit the name and description.
{% endstep %}
{% step %}
### Save
Click **Save**. The version number increments automatically, and all linked pipelines will see an "Update available" indicator.
{% endstep %}
{% endstepper %}

The detail page also shows a **Linked Pipelines** section listing every pipeline that uses this shared component, with a badge indicating whether each is up to date or has a pending update.

## Accepting updates

When a shared component is updated, linked pipeline nodes become stale. Pipeline owners decide when to accept the new configuration.

### Single node

Select the stale node in the pipeline editor. The detail panel shows an amber banner with an **Accept update** button. Click it to pull in the latest configuration and update the pinned version.

### All nodes in a pipeline

If a pipeline has multiple stale linked nodes, you can accept all updates at once from the pipeline's toolbar or through the API.

{% hint style="info" %}
Accepting an update modifies the pipeline's saved configuration but does not automatically redeploy. You still need to deploy the pipeline to push the changes to agents.
{% endhint %}

## Deleting a shared component

On the shared component detail page, scroll to the **Danger Zone** section and click **Delete**. A confirmation dialog explains what will happen:

- All linked pipeline nodes are **unlinked** -- their configurations remain intact, but they become regular standalone nodes.
- The shared component is permanently removed from the Library.

{% hint style="danger" %}
Deleting a shared component cannot be undone. Linked nodes keep their last-synced configuration but lose the ability to receive future updates.
{% endhint %}

## Edge cases

| Scenario | Behavior |
|----------|----------|
| **Cloning a pipeline** | Linked nodes in the clone remain linked to the same shared components. |
| **Promoting a pipeline** | Shared component links are stripped during cross-environment promotion. Nodes retain their config snapshots as regular standalone nodes. |
| **Saving as template** | Templates are portable across environments, so shared component links are stripped. Template nodes store config snapshots only. |
| **Discarding changes** | Reverting to a previous version restores the shared component links as they were at that version. |
| **Copy/paste nodes** | Pasted nodes retain their link to the same shared component. |
| **GitOps / YAML export** | Generated YAML uses the resolved config snapshot. Importing from YAML creates regular unlinked nodes. |
7 changes: 4 additions & 3 deletions docs/public/user-guide/templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Templates are reusable pipeline blueprints that capture a complete pipeline grap

## Template library

The **Templates** page displays all templates available to your team as a card grid. Each card shows:
Templates are found in the **Library** page (accessible from the sidebar), under the **Templates** tab. The page displays all templates available to your team as a card grid. Each card shows:

- **Name** -- The template name.
- **Category** -- A label such as Logging, Metrics, Archival, Streaming, or a custom category you define.
Expand Down Expand Up @@ -44,8 +44,8 @@ Click **Save Template**. The template now appears in the template library for al

{% stepper %}
{% step %}
### Open the Templates page
Navigate to **Templates** in the sidebar. Make sure you have an environment selected in the header.
### Open the Library
Navigate to **Library > Templates** in the sidebar. Make sure you have an environment selected in the header.
{% endstep %}
{% step %}
### Choose a template
Expand All @@ -72,6 +72,7 @@ Templates are designed to be portable across environments, so they intentionally
- **Secrets** -- Secret values (API keys, passwords, tokens) are never stored in templates. You must configure secrets in the target environment after creating a pipeline from a template.
- **Certificates** -- TLS certificates are environment-specific and are not included.
- **Environment bindings** -- Templates are not tied to any specific environment. They can be used in any environment within the team.
- **Shared component links** -- Nodes that are linked to [shared components](shared-components.md) have their links stripped when saved as a template. The node's configuration snapshot is preserved, but the template node becomes a standalone component.
- **Deployment state** -- Pipelines created from templates start as drafts. You deploy them when ready.

{% hint style="warning" %}
Expand Down
25 changes: 25 additions & 0 deletions prisma/schema.prisma
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ model Environment {
serviceAccounts ServiceAccount[]
deployRequests DeployRequest[]
teamDefaults Team[] @relation("teamDefault")
sharedComponents SharedComponent[]
createdAt DateTime @default(now())
}

Expand Down Expand Up @@ -370,6 +371,11 @@ model PipelineNode {
positionX Float
positionY Float
disabled Boolean @default(false)
sharedComponentId String?
sharedComponent SharedComponent? @relation(fields: [sharedComponentId], references: [id], onDelete: SetNull)
sharedComponentVersion Int?

@@index([sharedComponentId])
}

enum ComponentKind {
Expand All @@ -387,6 +393,25 @@ model PipelineEdge {
sourcePort String?
}

model SharedComponent {
id String @id @default(cuid())
name String
description String?
componentType String
kind ComponentKind
config Json
version Int @default(1)
environmentId String
environment Environment @relation(fields: [environmentId], references: [id])
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt

linkedNodes PipelineNode[]

@@unique([environmentId, name])
@@index([environmentId])
}

model PipelineVersion {
id String @id @default(cuid())
pipelineId String
Expand Down
47 changes: 47 additions & 0 deletions src/app/(dashboard)/library/layout.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
"use client";

import { usePathname, useRouter } from "next/navigation";
import { cn } from "@/lib/utils";
import { FileText, Link2 } from "lucide-react";

const libraryNavItems = [
{ title: "Templates", href: "/library/templates", icon: FileText },
{ title: "Shared Components", href: "/library/shared-components", icon: Link2 },
];

export default function LibraryLayout({ children }: { children: React.ReactNode }) {
const pathname = usePathname();
const router = useRouter();

return (
<div className="flex h-full">
<aside className="flex w-56 shrink-0 flex-col border-r bg-background">
<div className="p-4">
<h2 className="text-lg font-semibold">Library</h2>
<p className="text-xs text-muted-foreground">Reusable templates and components</p>
</div>
<nav className="flex-1 space-y-1 px-3 pb-4">
{libraryNavItems.map((item) => {
const isActive = pathname === item.href || pathname.startsWith(item.href + "/");
return (
<button
key={item.href}
onClick={() => router.push(item.href)}
className={cn(
"flex w-full items-center gap-2 rounded-md px-3 py-2 text-sm transition-colors",
isActive
? "bg-accent font-medium text-accent-foreground"
: "text-muted-foreground hover:bg-accent hover:text-accent-foreground"
)}
>
<item.icon className="h-4 w-4" />
{item.title}
</button>
);
})}
</nav>
</aside>
<main className="flex-1 overflow-auto">{children}</main>
</div>
);
}
5 changes: 5 additions & 0 deletions src/app/(dashboard)/library/page.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { redirect } from "next/navigation";

export default function LibraryPage() {
redirect("/library/templates");
}
Loading
Loading