Skip to content

feat: data classification tags for pipelines#38

Merged
TerrifiedBug merged 9 commits intomainfrom
feat/data-tags
Mar 7, 2026
Merged

feat: data classification tags for pipelines#38
TerrifiedBug merged 9 commits intomainfrom
feat/data-tags

Conversation

@TerrifiedBug
Copy link
Copy Markdown
Owner

Summary

  • Add tags (Json) field to Pipeline and availableTags (Json) field to Team in the Prisma schema, with a corresponding database migration
  • Add team.updateAvailableTags (ADMIN) and team.getAvailableTags (VIEWER) tRPC procedures for managing team-level tag definitions
  • Update pipeline.update to accept and validate tags against the team's available tags; include tags in pipeline.list response
  • Add Classification Tags multi-select to the Pipeline Settings popover (populated from the team's available tags)
  • Show color-coded tag badges (PII=red, PHI=orange, PCI-DSS=purple, Internal=blue, Public=green) next to pipeline names in the pipeline list
  • Add Data Classification Tags management card to Team Settings for admins to add/remove available tags
  • Document the feature in docs/public/user-guide/pipelines.md

Test plan

  • Run the Prisma migration and verify tags column on Pipeline and availableTags column on Team are created
  • As an ADMIN, go to Team Settings and add classification tags (e.g., PII, Internal, PCI-DSS)
  • As an ADMIN, remove a tag from the team's available list
  • Open a pipeline in the editor, click the gear icon, and verify the Classification Tags section appears with the team's available tags
  • Add and remove tags from a pipeline; verify they persist after page reload
  • Verify tag badges appear next to pipeline names in the pipeline list with correct colors
  • Attempt to assign a tag not in the team's available list via API and verify it is rejected
  • As a VIEWER, verify you cannot modify team tags (updateAvailableTags requires ADMIN)
  • Verify TypeScript compilation passes with no errors

@github-actions github-actions bot added documentation Improvements or additions to documentation feature labels Mar 7, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 7, 2026

Greptile Summary

This PR adds a data classification tagging system for pipelines — team admins define available tags (PII, PHI, PCI-DSS, etc.), editors apply them to individual pipelines, and color-coded badges appear in the pipeline list for quick visual identification. All previously raised review concerns have been addressed in this revision.

Key changes:

  • Pipeline.tags (Json[]) and Team.availableTags (Json[]) fields added via additive, backward-compatible migration
  • team.updateAvailableTags (ADMIN + withAudit) and team.getAvailableTags (VIEWER) tRPC procedures, both correctly gated with withTeamAccess
  • pipeline.update validates only newly-added tags against the team's available list (diff-based check), resolving the "stale tag blocks removal" correctness bug from the prior review
  • Frontend uses full optimistic-update patterns (onMutate snapshots + onError rollback) in both pipeline-settings.tsx and settings/page.tsx, eliminating the stale-cache races and lost-input-on-failure issues flagged previously
  • Classification Tags section in PipelineSettings now renders when availableTags.length > 0 || currentTags.length > 0, keeping orphaned tags removable after an admin clears the team list
  • Import consolidation in pipeline-settings.tsx removes previously flagged duplicate bindings; Trash2 and Plus retained for SliSettings
  • Standalone fix in detail-panel.tsx: LiveTailPanel now receives storeKey (flow-store value) instead of the stale componentKey reference

Confidence Score: 5/5

  • Safe to merge — all prior review concerns have been properly addressed and no new correctness or security issues were found.
  • All eight previously flagged issues (duplicate imports, stale-state races, lost input on failure, tag removal blocked by stale tag, validation bypass on null teamId, orphaned-tag UI, missing NOT_FOUND, missing duplicate guard) have been correctly resolved. The new tRPC procedures follow established middleware ordering and audit logging conventions. The migration is additive and backward-compatible. No new bugs introduced.
  • No files require special attention.

Important Files Changed

Filename Overview
src/server/routers/pipeline.ts Tag validation now correctly diffs only newly-added tags against availableTags, explicitly throws NOT_FOUND when teamId is null, and gates the whole block on tags !== undefined. The pipeline.get endpoint now includes environment.teamId so the frontend can resolve the team. Duplicate-tag guard via .refine() added to input schema.
src/server/routers/team.ts Two new procedures added — updateAvailableTags (ADMIN) and getAvailableTags (VIEWER) — both correctly gated with withTeamAccess, the mutation additionally wrapped with withAudit. Duplicate-tag guard via .refine() present. Input validated with min/max string length.
src/components/flow/pipeline-settings.tsx Imports consolidated (no duplicates). Optimistic updates via onMutate/onError/onSettled prevent stale-state races between mutation success and refetch. Classification Tags section now renders when either availableTags or currentTags is non-empty, so orphaned tags remain removable. Trash2 and Plus retained in import for SliSettings.
src/app/(dashboard)/settings/page.tsx Tag management uses full optimistic-update pattern (onMutate clears input and updates cache, onError restores previous cache and input value). availableTags sourced from the optimistically-updated cache so rapid Add/Remove operations compose correctly without stale-state races.
src/app/(dashboard)/pipelines/page.tsx Tag badges rendered next to pipeline names using color-coded CSS classes. SLI health badge simplified to a small dot with tooltip. Pipeline tags cast from Json to string[] consistently with the rest of the codebase.
src/components/flow/detail-panel.tsx Single-line fix: LiveTailPanel now receives storeKey (the authoritative flow-store value) instead of a now-absent componentKey variable, correcting the live-tail component filter.
prisma/schema.prisma Two new nullable Json fields with empty-array defaults: Pipeline.tags and Team.availableTags. Matches the migration SQL exactly.
prisma/migrations/20260308000000_add_data_classification_tags/migration.sql Additive migration only — two ALTER TABLE … ADD COLUMN … JSONB DEFAULT '[]' statements. Backward-compatible; no data loss risk.

Sequence Diagram

sequenceDiagram
    participant Admin as Admin (Settings UI)
    participant Editor as Editor (Pipeline Settings)
    participant TRPCTeam as tRPC team router
    participant TRPCPipeline as tRPC pipeline router
    participant DB as PostgreSQL

    Admin->>TRPCTeam: updateAvailableTags({ teamId, tags })<br/>[ADMIN + withAudit]
    TRPCTeam->>DB: UPDATE Team SET availableTags = tags
    DB-->>TRPCTeam: updated Team
    TRPCTeam-->>Admin: success

    Editor->>TRPCTeam: getAvailableTags({ teamId })<br/>[VIEWER]
    TRPCTeam->>DB: SELECT availableTags FROM Team
    DB-->>TRPCTeam: availableTags[]
    TRPCTeam-->>Editor: string[]

    Editor->>TRPCPipeline: pipeline.update({ id, tags })<br/>[EDITOR + withAudit]
    TRPCPipeline->>DB: SELECT id, tags, environment.teamId FROM Pipeline
    DB-->>TRPCPipeline: existing pipeline
    TRPCPipeline->>DB: SELECT availableTags FROM Team
    DB-->>TRPCPipeline: availableTags[]
    Note over TRPCPipeline: Validate only newly-added tags<br/>against availableTags
    TRPCPipeline->>DB: UPDATE Pipeline SET tags = newTags
    DB-->>TRPCPipeline: updated Pipeline
    TRPCPipeline-->>Editor: success
Loading

Last reviewed commit: 5041b2f

Add tags (Json) to Pipeline model and availableTags (Json) to Team model.
Add team.updateAvailableTags and team.getAvailableTags tRPC procedures.
Update pipeline.update to accept and validate tags against team's available tags.
Include tags in pipeline.list response.
Pipeline list: show color-coded tag badges next to pipeline names.
Pipeline settings: add Classification Tags multi-select dropdown
  populated from the team's available tags.
Team settings: add Data Classification Tags card for admins to
  define/remove available tags per team.
Pipeline get endpoint: include environment.teamId for tag resolution.
Document how admins define available tags per team, how editors apply
tags to pipelines, and the color-coding scheme for tag badges.
- Reject duplicate tags in updateAvailableTags via Zod refine
- Add explicit NOT_FOUND when team lookup returns null during tag validation
- Move setNewTag("") into onSuccess to preserve input on mutation failure
When updating pipeline tags, only validate newly added tags against the
team's available tags list. Previously, all submitted tags were validated,
which blocked tag removal when a stale (deleted) tag was still present
on the pipeline.
- Remove duplicate useTRPC, TanStack Query, and toast imports in
  pipeline-settings.tsx
- Restore Trash2 and Plus icon imports for SliSettings sub-component
- Add missing slis relation to Pipeline model in Prisma schema
- Add missing PipelineHealthBadge component to pipelines list page
- Fix componentKey reference (should be storeKey) in detail-panel.tsx
@TerrifiedBug TerrifiedBug merged commit 7459b2e into main Mar 7, 2026
12 checks passed
@TerrifiedBug TerrifiedBug deleted the feat/data-tags branch March 7, 2026 15:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant