-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Open
Description
Is your feature request related to a problem? Please describe.
The S3 output plugin doesn't currently support Server-Side Encryption (SSE) headers. If you need SSE with a specific KMS key, you're stuck relying on bucket-level defaults, which doesn't always work for our use case.
We need to:
- Use SSE-KMS with our own customer-managed keys
- Control encryption at the Fluent Bit level instead of depending on bucket policies
Describe the solution you'd like
Add two new config options to the S3 output plugin:
-
sse- Server-side encryption type. Accepted values:AES256- S3-managed keys (SSE-S3)aws:kms- AWS KMS-managed keys (SSE-KMS)aws:kms:dsse- Dual-layer server-side encryption with KMS (DSSE-KMS)
-
sse_kms_key_id- AWS key ARN for SSE-KMS/DSSE-KMS encryption (optional). If not specified when usingaws:kmsoraws:kms:dsse, the default AWS-managed KMS key for S3 is used.
Example configuration:
[OUTPUT]
Name s3
Match *
bucket my-bucket
region eu-west-1
sse aws:kms
sse_kms_key_id arn:aws:kms:us-west-2:123456789012:key/my-key-idThis adds the following headers to S3 uploads:
x-amz-server-side-encryption: encryption typex-amz-server-side-encryption-aws-kms-key-id: KMS key ARN (only for aws:kms or aws:kms:dsse)
Describe alternatives you've considered
- S3 bucket default encryption - Works but we can't control it from Fluent Bit and it's not flexible enough for our setup
- Post-processing with Lambda - Tried this but re-encrypting after upload is slow and leaves data unencrypted temporarily
Additional context
- AWS S3 SSE documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html
- Related PR: out_s3: Add SSE support + validation tests #11410
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels