Skip to content

s3 output: Support SSE headers #11412

@GabrielYamin

Description

@GabrielYamin

Is your feature request related to a problem? Please describe.

The S3 output plugin doesn't currently support Server-Side Encryption (SSE) headers. If you need SSE with a specific KMS key, you're stuck relying on bucket-level defaults, which doesn't always work for our use case.

We need to:

  • Use SSE-KMS with our own customer-managed keys
  • Control encryption at the Fluent Bit level instead of depending on bucket policies

Describe the solution you'd like

Add two new config options to the S3 output plugin:

  1. sse - Server-side encryption type. Accepted values:

    • AES256 - S3-managed keys (SSE-S3)
    • aws:kms - AWS KMS-managed keys (SSE-KMS)
    • aws:kms:dsse - Dual-layer server-side encryption with KMS (DSSE-KMS)
  2. sse_kms_key_id - AWS key ARN for SSE-KMS/DSSE-KMS encryption (optional). If not specified when using aws:kms or aws:kms:dsse, the default AWS-managed KMS key for S3 is used.

Example configuration:

[OUTPUT]
    Name s3
    Match *
    bucket my-bucket
    region eu-west-1
    sse aws:kms
    sse_kms_key_id arn:aws:kms:us-west-2:123456789012:key/my-key-id

This adds the following headers to S3 uploads:

  • x-amz-server-side-encryption: encryption type
  • x-amz-server-side-encryption-aws-kms-key-id: KMS key ARN (only for aws:kms or aws:kms:dsse)

Describe alternatives you've considered

  1. S3 bucket default encryption - Works but we can't control it from Fluent Bit and it's not flexible enough for our setup
  2. Post-processing with Lambda - Tried this but re-encrypting after upload is slow and leaves data unencrypted temporarily

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions