"storage.backlog.shutdown_flush" is undocumented

[bsmith5uoe]

Bug Report

Describe the bug
If the option storage.backlog.shutdown_flush is not enabled, then fluent-bit can lose data. This option and what it does is undocumented (what is does is cause sb_segregate_chunks to be called after the grace period which moves checks to the backlog).

To Reproduce
I have fluent-bit configured with a http output. I turned off the HTTP server, so that fluent-bit is retrying. I then restarted fluent-bit. With the option not enabled, after the grace period, all the retrying HTTP requests are discarded. With this option enabled, the requests are instead added to the backlog and reloaded on restart.

Expected behavior
I expected this behaviour to be described in documentation such as the "Buffering and storage" page. I also expected this option to be enabled by default!

[patrick-stephens]

@eschabell did you resolve this in docs? I'm going to transfer it to the docs repo I think as it is not an implementation problem.

[eschabell]

Adding this to my todo list, thanks @bsmith5uoe for the issue.