From 66dde701488e9c236289e0ee29634dfad6e5d45d Mon Sep 17 00:00:00 2001 From: Gabriel Yamin Date: Wed, 28 Jan 2026 16:47:12 +0200 Subject: [PATCH 1/3] docs: s3: SSE configuration docs Signed-off-by: Gabriel Yamin --- pipeline/outputs/s3.md | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/pipeline/outputs/s3.md b/pipeline/outputs/s3.md index e325660a2..48993866d 100644 --- a/pipeline/outputs/s3.md +++ b/pipeline/outputs/s3.md @@ -80,6 +80,8 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor | `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` | | `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` | | `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` | +| `sse` | Server-side encryption for S3 objects. Set to `AES256` for S3-managed keys (SSE-S3), `aws:kms` for AWS KMS-managed keys (SSE-KMS), or `aws:kms:dsse` for dual-layer server-side encryption with KMS (DSSE-KMS). | _none_ | +| `sse_kms_key_id` | AWS KMS key ID (or key ARN) for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ | | `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` | | `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` | | `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` | @@ -574,8 +576,48 @@ pipeline: {% endtab %} {% endtabs %} +An example using SSE-KMS encryption: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + + outputs: + - name: s3 + match: '*' + bucket: your-bucket + region: us-east-1 + store_dir: /home/ec2-user/buffer + total_file_size: 50M + upload_timeout: 10m + sse: aws:kms + sse_kms_key_id: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012 +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[OUTPUT] + Name s3 + Match * + bucket your-bucket + region us-east-1 + store_dir /home/ec2-user/buffer + total_file_size 50M + upload_timeout 10m + sse aws:kms + sse_kms_key_id arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012 +``` + +{% endtab %} +{% endtabs %} + ## AWS for Fluent Bit + Amazon distributes a container image with Fluent Bit and plugins. ### GitHub From 0ac085eb8cf386dfb15771d5760c66e28a0828fb Mon Sep 17 00:00:00 2001 From: Gabriel Yamin <43830000+GabrielYamin@users.noreply.github.com> Date: Thu, 29 Jan 2026 09:07:47 +0200 Subject: [PATCH 2/3] update key_id to say key ARN only (according to docs) Signed-off-by: Gabriel Yamin <43830000+GabrielYamin@users.noreply.github.com> --- pipeline/outputs/s3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/outputs/s3.md b/pipeline/outputs/s3.md index 48993866d..37ecfcf79 100644 --- a/pipeline/outputs/s3.md +++ b/pipeline/outputs/s3.md @@ -81,7 +81,7 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor | `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` | | `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` | | `sse` | Server-side encryption for S3 objects. Set to `AES256` for S3-managed keys (SSE-S3), `aws:kms` for AWS KMS-managed keys (SSE-KMS), or `aws:kms:dsse` for dual-layer server-side encryption with KMS (DSSE-KMS). | _none_ | -| `sse_kms_key_id` | AWS KMS key ID (or key ARN) for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ | +| `sse_kms_key_id` | AWS KMS key ARN for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ | | `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` | | `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` | | `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` | From 47859f704c3ad995c202919bfe53af4cb8089817 Mon Sep 17 00:00:00 2001 From: Gabriel Yamin <43830000+GabrielYamin@users.noreply.github.com> Date: Thu, 29 Jan 2026 17:21:51 +0200 Subject: [PATCH 3/3] Update SSE and KMS key ID descriptions in s3.md Signed-off-by: Gabriel Yamin <43830000+GabrielYamin@users.noreply.github.com> --- pipeline/outputs/s3.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pipeline/outputs/s3.md b/pipeline/outputs/s3.md index 37ecfcf79..81c8c0546 100644 --- a/pipeline/outputs/s3.md +++ b/pipeline/outputs/s3.md @@ -80,8 +80,8 @@ The [Prometheus success/retry/error metrics values](../../administration/monitor | `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` | | `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` | | `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` | -| `sse` | Server-side encryption for S3 objects. Set to `AES256` for S3-managed keys (SSE-S3), `aws:kms` for AWS KMS-managed keys (SSE-KMS), or `aws:kms:dsse` for dual-layer server-side encryption with KMS (DSSE-KMS). | _none_ | -| `sse_kms_key_id` | AWS KMS key ARN for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ | +| `sse` | Server-side encryption (SSE) for S3 objects. Set to `AES256` for S3-managed keys (SSE-S3), `aws:kms` for AWS Key Management Service (SSE-KMS), or `aws:kms:dsse` for dual-layer server-side encryption with KMS (DSSE-KMS). | _none_ | +| `sse_kms_key_id` | AWS KMS key Resource Name (ARN) for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ | | `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` | | `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` | | `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` |