Support hybrid datastore for Organization Units #886
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds comprehensive support for hybrid datastore functionality for Organization Units and significantly expands the export API to support multiple resource types beyond just applications. The changes enable immutable resource management for Identity Providers, Notification Senders, User Schemas, and Organization Units through file-based YAML configurations.
Key Changes:
- Hybrid datastore support for Organization Units (combining file-based and database stores)
- Extended export functionality for Identity Providers, Notification Senders, and User Schemas
- File-based store implementations for IDP, Notification Senders, User Schemas, and OUs
- Immutable resource mode checks in service layers to prevent modifications
- Enhanced parameterization for IDP properties in export templates
Reviewed changes
Copilot reviewed 48 out of 48 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
backend/internal/ou/hybrid_store.go |
Implements hybrid store combining file-based and database stores for OUs |
backend/internal/ou/file_based_store.go |
File-based storage implementation for Organization Units |
backend/internal/ou/init.go |
OU initialization with hybrid store support and topological sorting |
backend/internal/idp/file_based_store.go |
File-based storage for Identity Providers |
backend/internal/idp/service.go |
Immutable mode checks for IDP CRUD operations |
backend/internal/notification/file_based_store.go |
File-based storage for Notification Senders |
backend/internal/notification/init.go |
Notification sender initialization with file-based support |
backend/internal/userschema/file_based_store.go |
File-based storage for User Schemas |
backend/internal/userschema/init.go |
User schema initialization with file-based support |
backend/internal/system/export/service.go |
Extended export service supporting multiple resource types |
backend/internal/system/export/parameterizer.go |
Dynamic property parameterization for IDP exports |
backend/internal/system/export/template_rules.go |
Template rules for IDP property parameterization |
backend/internal/system/export/model.go |
Added fields for IDPs, notification senders, and user schemas |
backend/internal/system/file_based_runtime/manager.go |
Returns empty array instead of error for non-existent directories |
tests/integration/export/exportapi_test.go |
Comprehensive integration tests for IDP export functionality |
docs/guides/immutable-configurations/*.md |
Updated documentation for IDP support |
senthalan
reviewed
Dec 4, 2025
|
|
||
| // GetOrganizationUnitList retrieves root organization units from both stores with pagination. | ||
| // File-based OUs are returned first, followed by database OUs. | ||
| func (h *ouHybridStore) GetOrganizationUnitList(limit, offset int) ([]OrganizationUnitBasic, error) { |
Contributor
There was a problem hiding this comment.
Handling pagination is going to be complex
Comment on lines
+135
to
+159
| _, err := h.fileStore.GetOrganizationUnit(ou.ID) | ||
| if err == nil { | ||
| return errors.New("cannot create organization unit: ID conflicts with immutable file-based OU") | ||
| } | ||
| if err != ErrOrganizationUnitNotFound { | ||
| return fmt.Errorf("error checking file store: %w", err) | ||
| } | ||
|
|
||
| // Check handle conflict in file store | ||
| hasConflict, err := h.fileStore.CheckOrganizationUnitHandleConflict(ou.Handle, ou.Parent) | ||
| if err != nil { | ||
| return fmt.Errorf("error checking handle conflict in file store: %w", err) | ||
| } | ||
| if hasConflict { | ||
| return errors.New("cannot create organization unit: handle conflicts with immutable file-based OU") | ||
| } | ||
|
|
||
| // Check name conflict in file store | ||
| hasConflict, err = h.fileStore.CheckOrganizationUnitNameConflict(ou.Name, ou.Parent) | ||
| if err != nil { | ||
| return fmt.Errorf("error checking name conflict in file store: %w", err) | ||
| } | ||
| if hasConflict { | ||
| return errors.New("cannot create organization unit: name conflicts with immutable file-based OU") | ||
| } |
Contributor
There was a problem hiding this comment.
shouldn't this part of the service layer?
Service layer will call hybridStore.CheckOrganizationUnitNameConflict and hybridStore will check from both stores
| // Check if exists in file store (immutable) | ||
| _, err := h.fileStore.GetOrganizationUnit(ou.ID) | ||
| if err == nil { | ||
| return errors.New("cannot update organization unit: exists in immutable file-based store") |
Contributor
There was a problem hiding this comment.
this means when listing the resource or getting the resource, a flag needs to be sent to indicate whether the resource is readOnly. Else handling the UX in developer portal is going to be problem.
In highlevel, we need think though this hybrid pattern a bit more.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Support hybrid datastore for Organization Units