From 491c54468e58d6b5a776bad5a911d1c542a9ff14 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Fri, 31 Oct 2025 16:01:57 -0500 Subject: [PATCH 1/4] bin: run additional validation for --dry-run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There are certain kinds of invalid configurations that pass the `--dry-run` check, but fail to start at run time. This commit addresses the issue by re-using some of the hot reload validation logic. For example, this config is trivially invalid, passes `--dry-run`, and fails at runtime: ``` pipeline: inputs: - name: dummy tag: test invalid_property_that_does_not_exist: some_value outputs: - name: stdout match: '*' ``` ``` zsh ❮ fluent-bit --dry-run -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.1.1 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ configuration test is successful zsh ❮ fluent-bit -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.1.1 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ [2025/10/31 16:05:55.268532000] [ info] [fluent bit] version=4.1.1, commit=, pid=21037 [2025/10/31 16:05:55.268808000] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128 [2025/10/31 16:05:55.269140000] [ info] [simd ] disabled [2025/10/31 16:05:55.269147000] [ info] [cmetrics] version=1.0.5 [2025/10/31 16:05:55.269407000] [ info] [ctraces ] version=0.6.6 [2025/10/31 16:05:55.269476000] [error] [config] dummy: unknown configuration property 'invalid_property_that_does_not_exist'. The following properties are allowed: samples, dummy, metadata, rate, interval_sec, interval_nsec, copies, start_time_sec, start_time_nsec, fixed_timestamp, flush_on_startup, and test_hang_on_exit. [2025/10/31 16:05:55.269486000] [ help] try the command: fluent-bit -i dummy -h [2025/10/31 16:05:55.269515000] [error] [engine] input initialization failed ``` With this commit, we can see that the additional validation from hot reload catches the error right away: ``` zsh ❯ bin/fluent-bit --dry-run -c ~/tmp/fbconfig-bad-property.yaml Fluent Bit v4.2.0 * Copyright (C) 2015-2025 The Fluent Bit Authors * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd * https://fluentbit.io ______ _ _ ______ _ _ ___ __ | ___| | | | | ___ (_) | / | / | | |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __/ /| | `| | | _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| | | | | | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /\___ |__| |_ \_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ |_(_)___/ [2025/10/31 16:07:50.402568000] [error] [config] dummy: unknown configuration property 'invalid_property_that_does_not_exist'. The following properties are allowed: samples, dummy, metadata, rate, interval_sec, interval_nsec, copies, start_time_sec, start_time_nsec, fixed_timestamp, flush_on_startup, and test_hang_on_exit. [2025/10/31 16:07:50.402792000] [ help] try the command: bin/fluent-bit -i dummy -h [2025/10/31 16:07:50.402800000] [error] [reload] check properties for input plugins is failed ``` (The logs of course now say `[reload]`, which is a little misleading... we can clean that up, if desired). Signed-off-by: Andrew Hayworth --- src/fluent-bit.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/fluent-bit.c b/src/fluent-bit.c index 252fbf98a60..06b14940a4c 100644 --- a/src/fluent-bit.c +++ b/src/fluent-bit.c @@ -1437,6 +1437,11 @@ static int flb_main_run(int argc, char **argv) #endif if (config->dry_run == FLB_TRUE) { + ret = flb_reload_property_check_all(config); + if (ret != 0) { + exit(EXIT_FAILURE); + } + fprintf(stderr, "configuration test is successful\n"); flb_init_env(); flb_cf_destroy(cf_opts); From 5906c29167ed2a88dc5bf3d6405cef4ac0e0bc08 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 3 Nov 2025 10:21:43 -0600 Subject: [PATCH 2/4] bin: ensure we clean up after error and successful dry-run We need to perform the same partial initialization and releasing of resources whether or not the dry-run was successful, so this commit ensures that we call those functions before exiting either way. Signed-off-by: Andrew Hayworth --- src/fluent-bit.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/fluent-bit.c b/src/fluent-bit.c index 06b14940a4c..7fef20c3b52 100644 --- a/src/fluent-bit.c +++ b/src/fluent-bit.c @@ -1438,14 +1438,16 @@ static int flb_main_run(int argc, char **argv) if (config->dry_run == FLB_TRUE) { ret = flb_reload_property_check_all(config); - if (ret != 0) { - exit(EXIT_FAILURE); - } - fprintf(stderr, "configuration test is successful\n"); + /* At this point config test is done, so clean up after ourselves */ flb_init_env(); flb_cf_destroy(cf_opts); flb_destroy(ctx); + + if (ret != 0) { + exit(EXIT_FAILURE); + } + fprintf(stderr, "configuration test is successful\n"); exit(EXIT_SUCCESS); } From 13234816e09b686a70393d3d746cb8b23c90b942 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 10 Nov 2025 09:34:08 -0600 Subject: [PATCH 3/4] tests: runtime_shell: test additional dry-run validation This adds a runtime_shell test that runs fluent-bit in dry-run mode and ensures we are running the additional validation recently added. Specifically, we test that configuration property validation is happening, which is borrowed from the hot reload code and was not previously used for dry-run validation. Signed-off-by: Andrew Hayworth --- .../runtime_shell/dry_run_invalid_property.sh | 64 +++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 tests/runtime_shell/dry_run_invalid_property.sh diff --git a/tests/runtime_shell/dry_run_invalid_property.sh b/tests/runtime_shell/dry_run_invalid_property.sh new file mode 100755 index 00000000000..ee48c8693b8 --- /dev/null +++ b/tests/runtime_shell/dry_run_invalid_property.sh @@ -0,0 +1,64 @@ +#!/bin/sh + +# Setup environment if not already set +if [ -z "$FLB_BIN" ]; then + FLB_ROOT=${FLB_ROOT:-$(cd $(dirname $0)/../.. && pwd)} + FLB_BIN=${FLB_BIN:-$FLB_ROOT/build/bin/fluent-bit} +fi + +echo "Using Fluent Bit at: $FLB_BIN" + +# Create a temporary YAML config file with an invalid property +cat > /tmp/dry_run_invalid_property.yaml << EOL +service: + log_level: debug + flush: 1 +pipeline: + inputs: + - name: dummy + tag: test + invalid_property_that_does_not_exist: some_value + outputs: + - name: stdout + match: '*' +EOL + +echo "Running Fluent Bit with --dry-run and invalid property config..." +echo "YAML Config:" +cat /tmp/dry_run_invalid_property.yaml + +# Redirect stdout and stderr to a file for analysis +OUTPUT_FILE="/tmp/dry_run_invalid_property_output.txt" +$FLB_BIN --dry-run -c /tmp/dry_run_invalid_property.yaml > $OUTPUT_FILE 2>&1 + +# Check exit code - we expect it to fail +EXIT_CODE=$? +echo "Fluent Bit --dry-run exited with code: $EXIT_CODE" + +# Show the output +echo "Output file content:" +cat $OUTPUT_FILE + +# Check if the output contains an error about the unknown configuration property +UNKNOWN_PROPERTY=$(grep -c "unknown configuration property 'invalid_property_that_does_not_exist'" $OUTPUT_FILE || true) +RELOAD_ERROR=$(grep -c "check properties for input plugins is failed" $OUTPUT_FILE || true) + +# Clean up +echo "Cleaning up..." +rm -f /tmp/dry_run_invalid_property.yaml +rm -f $OUTPUT_FILE + +# Check results - we expect: +# 1. Fluent Bit to fail (non-zero exit code) +# 2. Error message about unknown configuration property +# 3. Error message from reload validation +if [ "$EXIT_CODE" -ne 0 ] && [ "$UNKNOWN_PROPERTY" -gt 0 ] && [ "$RELOAD_ERROR" -gt 0 ]; then + echo "Test passed: Fluent Bit --dry-run correctly detected invalid property and failed" + exit 0 +else + echo "Test failed: Fluent Bit --dry-run should detect invalid properties and fail" + echo "Exit code: $EXIT_CODE (expected non-zero)" + echo "Unknown property message count: $UNKNOWN_PROPERTY (expected > 0)" + echo "Reload error message count: $RELOAD_ERROR (expected > 0)" + exit 1 +fi From 066a00b24bb568f66a580fb86e7522115013a2d3 Mon Sep 17 00:00:00 2001 From: Andrew Hayworth Date: Mon, 10 Nov 2025 10:20:10 -0600 Subject: [PATCH 4/4] tests: runtime_shell: actually run dry_run_invalid_property.sh This was an oversight on my part - we actually need to list this as a test for it to be run. Signed-off-by: Andrew Hayworth --- tests/runtime_shell/CMakeLists.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/runtime_shell/CMakeLists.txt b/tests/runtime_shell/CMakeLists.txt index 30ae87a6b96..11ca3f1c2c9 100644 --- a/tests/runtime_shell/CMakeLists.txt +++ b/tests/runtime_shell/CMakeLists.txt @@ -5,6 +5,7 @@ configure_file( set(UNIT_TESTS_SH custom_calyptia.sh + dry_run_invalid_property.sh in_dummy_expect.sh in_tail_expect.sh in_http_tls_expect.sh