[DSIP-105][Api-server] Sensitive Variable Type for Secure Secret Handling

[det101]

Search before asking

• I had searched in the DSIP and found no similar DSIP.

Motivation

Currently, variables in DolphinScheduler do not distinguish between sensitive and non-sensitive information. Real-world use cases require securely storing secrets such as passwords or API keys. Without dedicated support, secrets may inadvertently appear in logs or the UI, resulting in potential security risks.

After community feedback (thanks @ruanwenjun), instead of introducing a new Sensitive Variable type, the improved design will add a sensitive boolean field to the existing Property class to enhance security for such scenarios.

Design Detail

• Add a sensitive boolean field to the existing Property class (located in dolphinscheduler-task-plugin/dolphinscheduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/model/Property.java).
• The sensitive field marks whether a variable contains sensitive information (e.g., passwords, API keys, tokens).
• Ensure sensitive content is encrypted before storage, using the mechanism already implemented for data sources.
• Prevent display of sensitive values in logs and the UI (display as ****** or similar masking).
• Limit exposure of sensitive data to only operations where absolutely necessary (e.g., actual task execution with proper authorization).
  -Update API endpoints to support CRUD operations for variables with the sensitive field.
  -Apply changes to variable management interfaces and documentation.

Example: Enhanced Property Class

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Property implements Serializable {
    
    private static final long serialVersionUID = -4045513703397452451L;
    
    private String prop;        // variable name
    private Direct direct;      // IN/OUT direction
    private DataType type;      // data type
    private String value;       // variable value
    
    // New field to mark sensitive variables
    @Builder.Default
    private boolean sensitive = false;
}

Implementation Points

• Database Storage: The sensitive field will be part of the JSON serialization of Property objects (stored in global_params or task_params columns).

• Encryption: When sensitive = true, encrypt the value field before persisting to database, decrypt only when needed.

• UI Masking: Display ****** for sensitive variable values in all UI views.

• API Response: Mask sensitive values in GET responses, return actual values only in execution contexts.

• Logging: Filter out sensitive variable values from all log outputs.

Design Detail

Compatibility, Deprecation, and Migration Plan

• No compatibility issues for existing variables.
• Migration for sensitive values from existing common variables can be performed by users manually.
• No deprecations planned at this stage.

Test Plan

• Add unit and integration tests for all new logic paths.
• Validate that sensitive data does not appear in exported logs, the UI, or variable listings.
• Simulate using API endpoints to get, set, and update Sensitive Variables.

Code of Conduct

• I agree to follow this project's Code of Conduct

[alisha-1000]

Hi @det101, this looks like a valuable feature, especially from a security standpoint 👍
Before starting implementation, do maintainers have a preferred design for how sensitive variables should behave (e.g. UI masking, logging behavior, export/import handling)?

Happy to work on this once the expected approach is clarified.

[SbloodyS]

Please create an DSIP for this. Feature like this belong to DSIP. And you should provide more specific design detail in it. @det101

[alisha-1000]

Thanks for the guidance @SbloodyS
That makes sense — this definitely fits the DSIP process.

@det101, could you please create a DSIP for this feature with the proposed design details?
It would be great if the DSIP could cover things like:

expected behavior of sensitive variables (UI masking, logging, export/import),

reuse of existing encryption/decryption mechanisms,

backward compatibility and migration (if any).

Once the DSIP is up, we can continue the discussion and move toward implementation.

[ruanwenjun]

It's better to add sensitive switch at the currently variable, rather than add a new type, otherwise you need to add a lot of type here.

[det101]

It's better to add sensitive switch at the currently variable, rather than add a new type, otherwise you need to add a lot of type here.

@ruanwenjun You're absolutely right that adding a sensitive switch/flag to the existing Property class is a much better design than introducing a new variable type. Creating a dedicated "Sensitive Variable" type would indeed lead to type proliferation and maintenance challenges.