output: azure_blob: compression with zstd update#2309
output: azure_blob: compression with zstd update#2309nberlee wants to merge 1 commit intofluent:masterfrom
Conversation
WalkthroughDocumentation updates to the Azure Blob output: Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~3 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: defaults Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pipeline/outputs/azure_blob.md(1 hunks)
🔇 Additional comments (1)
pipeline/outputs/azure_blob.md (1)
29-30: Well-documented compression options and inheritance behavior.The documentation clearly explains:
- Both
compress(transport-level) andcompress_blob(blob-level) compression support, with zstd now available- How
compress_blobinherits the codec fromcompresswhen set, establishing clear precedence- The incompatibility constraint with
blob_type = appendblobThese clarifications significantly improve the documentation. Once the underlying code merges in fluent-bit/fluent-bit#11202, verify that the documented behavior matches the implementation.
Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
| | `compress` | Sets payload compression in network transfer. Supported value: `gzip` | _none_ | | ||
| | `compress_blob` | Enables GZIP compression in the final `blockblob` file. This option isn't compatible when `blob_type` = `appendblob`. | _none_ | | ||
| | `compress` | Sets payload compression in network transfer. Supported values: `gzip`, `zstd`. | _none_ | | ||
| | `compress_blob` | Enables compression in the final `blockblob` file. When enabled without `compress`, it uses GZIP; if `compress` is also set, it inherits that codec. This option isn't compatible when `blob_type` = `appendblob`. | _none_ | |
There was a problem hiding this comment.
What does isn't compatible mean here? Does it error out or silently disable or something else?
There was a problem hiding this comment.
This wasn't changed, fluent-bit will error out as the two are not compatible, so fluent-bot will not be able to know what to do.
see https://github.com/fluent/fluent-bit/blob/master/plugins/out_azure_blob/azure_blob_conf.c#L669
There was a problem hiding this comment.
Sure, I don't mean anything about change I mean it's not clear what it means so can you update with a bit more detail. If you configure this incorrectly does it trigger an error and fail or carry on but silently ignore.
|
this looks fine to me, writing-wise. I don't know why the vale test is being picky, it's not supposed to fail out on a spelling issue. Pat's question remains though so I won't stamp it at this time. |
|
@patrick-stephens can you suggest a change that would make sense for @nberlee to apply? |
For me I just want to know what happens with the misconfiguration, does it fail explicitly or silently ignore or do something else? I know it's not something changed but as part of the update we should try to improve and capture that. |
Document the new zstd option for Azure Blob network compression and explain how compress_blob inherits its codec, clarifying precedence between transport and blob-level compression.
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.