chore(ci): reorganize integration test files #24108

pront · 2025-10-29T20:48:07Z

Summary

No functional changes.

Vector configuration

How did you test this PR?

Change Type

Bug fix
New feature
Non-functional (chore, refactoring, docs)
Performance

Is this a breaking change?

Yes
No

Does this PR include user facing changes?

Yes. Please add a changelog fragment based on our guidelines.
No. A maintainer will apply the no-changelog label to this PR.

References

Notes

Please read our Vector contributor resources.
Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
Some CI checks run only after we manually approve them.
- We recommend adding a pre-push hook, please see this template.
- Alternatively, we recommend running the following locally before pushing to the remote branch:
  - make fmt
  - make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
  - make test
After a review is requested, please avoid force pushes to help us review incrementally.
- Feel free to push as many commits as you want. They will be squashed into one before merging.
- For example, you can run git merge origin master and git push.
If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

datadog-vectordotdev · 2025-10-30T13:37:47Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: ee13f9d | Docs | Was this helpful? Give us feedback!}

tests/integration/nats/config/compose.yaml

* chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(ci): reorganize integration test files (#24108) * chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com> * chore(vdev): move all utils in a new utils folder (#24143) * chore(vdev): move all utils in a new utils folder * move remaining files to utils dir * move remaining files to utils dir * fmt * chore(vdev): make modules visible to rustfmt (#24162) * Remove mod inside cli_subcommands macro * cargo fmt * chore(vdev): apply vdev rust check fixes * Link to rustfmt issue * fix(vdev): release prepare vrl version pinning (#24158) * Add --dry-run to release prepare * Add error handling and checks to pin_vrl_version * Add wrapper to toml * Remove wrapper, parse as Table instead * Fix vrl pinning logic * Enable preserve_order feature in toml crate * Use dependency instead of whole toml * Fix dry run docs * Fix dry run wording * refactor to use toml_edit * Add update_vrl_to_version to add unit test * Use indoc in prepare.rs * Remove preserve_order feature * chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178) * chore(ci): temporarily remove homebrew publish step from publish workflow (#24185) This temporarily removes the publish-homebrew job from the publish workflow to address issue #24139. This is step 1 of the plan to fix the homebrew publishing process. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com> * fix(blackhole sink): disable rate limiting for periodic stats messages (#24190) * fix(blackhole sink): disable rate limiting for periodic stats messages The blackhole sink's informational "Collected events" messages were being rate-limited since rate limiting was enabled by default in #24024. This is undesirable because: 1. These are deliberately scheduled periodic messages (controlled by the `print_interval_secs` config option), not error conditions that could flood the logs 2. Users explicitly configure the frequency - rate limiting defeats that explicit configuration and breaks user expectations 3. The interval timer already prevents log flooding, making additional rate limiting redundant 4. The blackhole sink is used for debugging/testing, where predictable output is essential This fix adds `internal_log_rate_limit = false` to both info! calls, similar to how the console sink disables rate limiting for its critical operational messages. Fixes #24188 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add changelog fragment for blackhole sink rate limiting fix --------- Co-authored-by: Claude <noreply@anthropic.com> * chore(internal logs): Disable rate limiting for critical internal error logs (#24192) * Improve debug log in aws_ecs_metrics * Add internal_log_rate_limit = false to config reload errors * Disable rate limiting for lua build failure * Disable rate limiting for amqp build failure * Add internal_log_rate_limit = false to important failures during reload * Disable log rate limit for repeated events in process.rs * Update src/config/watcher.rs * Fix message format * fix(tracing): prevent panic for traces without standard fields (#24191) * fix(tracing): prevent panic for traces without std fields * forbid unwrap in this lib * clippy fix * improve events_with_custom_fields_no_message_dont_panic, advance to next window * chore(tracing): do not rate limit utlization report (#24202) * chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203) * chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError * update topology_doesnt_reload_new_data_dir test * chore(dev): cargo fmt * strengthen tests, since now we have a better interface * chore(releasing): Add 0.51.0 known issues (#24211) * Format fixes * Add 0.51.0 known issues * Add dots * Reword log panic known issue * Reword VRL missing fns * Backtick versions * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Fix cue docs fmt --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Bump vector version to 0.51.1 * Bump VRL to 0.28.1 * Generate 0.51.1 cue * Add vrl_changelog * Remove whitespace * chore(releasing): Updated distribution/install.sh vector version to 0.51.1 * chore(releasing): Add 0.51.1 to versions.cue * chore(releasing): Created release md file * Add description * Reorder items and fix format * Finally fix styling * Bump date --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>

* chore(ci): reorganize integration test files (#24108) * chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com> * chore(vdev): move all utils in a new utils folder (#24143) * chore(vdev): move all utils in a new utils folder * move remaining files to utils dir * move remaining files to utils dir * fmt * chore(vdev): make modules visible to rustfmt (#24162) * Remove mod inside cli_subcommands macro * cargo fmt * chore(vdev): apply vdev rust check fixes * Link to rustfmt issue * fix(vdev): release prepare vrl version pinning (#24158) * Add --dry-run to release prepare * Add error handling and checks to pin_vrl_version * Add wrapper to toml * Remove wrapper, parse as Table instead * Fix vrl pinning logic * Enable preserve_order feature in toml crate * Use dependency instead of whole toml * Fix dry run docs * Fix dry run wording * refactor to use toml_edit * Add update_vrl_to_version to add unit test * Use indoc in prepare.rs * Remove preserve_order feature * chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178) * chore(ci): temporarily remove homebrew publish step from publish workflow (#24185) This temporarily removes the publish-homebrew job from the publish workflow to address issue #24139. This is step 1 of the plan to fix the homebrew publishing process. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com> * fix(blackhole sink): disable rate limiting for periodic stats messages (#24190) * fix(blackhole sink): disable rate limiting for periodic stats messages The blackhole sink's informational "Collected events" messages were being rate-limited since rate limiting was enabled by default in #24024. This is undesirable because: 1. These are deliberately scheduled periodic messages (controlled by the `print_interval_secs` config option), not error conditions that could flood the logs 2. Users explicitly configure the frequency - rate limiting defeats that explicit configuration and breaks user expectations 3. The interval timer already prevents log flooding, making additional rate limiting redundant 4. The blackhole sink is used for debugging/testing, where predictable output is essential This fix adds `internal_log_rate_limit = false` to both info! calls, similar to how the console sink disables rate limiting for its critical operational messages. Fixes #24188 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add changelog fragment for blackhole sink rate limiting fix --------- Co-authored-by: Claude <noreply@anthropic.com> * chore(internal logs): Disable rate limiting for critical internal error logs (#24192) * Improve debug log in aws_ecs_metrics * Add internal_log_rate_limit = false to config reload errors * Disable rate limiting for lua build failure * Disable rate limiting for amqp build failure * Add internal_log_rate_limit = false to important failures during reload * Disable log rate limit for repeated events in process.rs * Update src/config/watcher.rs * Fix message format * fix(tracing): prevent panic for traces without standard fields (#24191) * fix(tracing): prevent panic for traces without std fields * forbid unwrap in this lib * clippy fix * improve events_with_custom_fields_no_message_dont_panic, advance to next window * chore(tracing): do not rate limit utlization report (#24202) * chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203) * chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError * update topology_doesnt_reload_new_data_dir test * chore(dev): cargo fmt * strengthen tests, since now we have a better interface * chore(releasing): Add 0.51.0 known issues (#24211) * Format fixes * Add 0.51.0 known issues * Add dots * Reword log panic known issue * Reword VRL missing fns * Backtick versions * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Fix cue docs fmt --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Bump vector version to 0.51.1 * Bump VRL to 0.28.1 * Generate 0.51.1 cue * Add vrl_changelog * Remove whitespace * chore(releasing): Updated distribution/install.sh vector version to 0.51.1 * chore(releasing): Add 0.51.1 to versions.cue * chore(releasing): Created release md file * Add description * Reorder items and fix format * Finally fix styling * Bump date --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> chore(releasing): pull in `internal_logs` fix into 0.51.1 (#24225) * fix(internal_logs source): remove rate limit (#24218) * fix(internal_logs source): remove rate limit * add changelog * Debug commit * Fix validated - Revert "Debug commit" This reverts commit c7b9ec9. * add unit test * fix check-events * Pull in internal_logs source fix * Add internal_logs known issue * Bump date * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* chore(releasing): prepare v0.51.1 release (#24214) * chore(ci): reorganize integration test files (#24108) * chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com> * chore(vdev): move all utils in a new utils folder (#24143) * chore(vdev): move all utils in a new utils folder * move remaining files to utils dir * move remaining files to utils dir * fmt * chore(vdev): make modules visible to rustfmt (#24162) * Remove mod inside cli_subcommands macro * cargo fmt * chore(vdev): apply vdev rust check fixes * Link to rustfmt issue * fix(vdev): release prepare vrl version pinning (#24158) * Add --dry-run to release prepare * Add error handling and checks to pin_vrl_version * Add wrapper to toml * Remove wrapper, parse as Table instead * Fix vrl pinning logic * Enable preserve_order feature in toml crate * Use dependency instead of whole toml * Fix dry run docs * Fix dry run wording * refactor to use toml_edit * Add update_vrl_to_version to add unit test * Use indoc in prepare.rs * Remove preserve_order feature * chore(deps): update VRL to add missing stdlib fns from 0.28 (#24178) * chore(ci): temporarily remove homebrew publish step from publish workflow (#24185) This temporarily removes the publish-homebrew job from the publish workflow to address issue #24139. This is step 1 of the plan to fix the homebrew publishing process. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com> * fix(blackhole sink): disable rate limiting for periodic stats messages (#24190) * fix(blackhole sink): disable rate limiting for periodic stats messages The blackhole sink's informational "Collected events" messages were being rate-limited since rate limiting was enabled by default in #24024. This is undesirable because: 1. These are deliberately scheduled periodic messages (controlled by the `print_interval_secs` config option), not error conditions that could flood the logs 2. Users explicitly configure the frequency - rate limiting defeats that explicit configuration and breaks user expectations 3. The interval timer already prevents log flooding, making additional rate limiting redundant 4. The blackhole sink is used for debugging/testing, where predictable output is essential This fix adds `internal_log_rate_limit = false` to both info! calls, similar to how the console sink disables rate limiting for its critical operational messages. Fixes #24188 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add changelog fragment for blackhole sink rate limiting fix --------- Co-authored-by: Claude <noreply@anthropic.com> * chore(internal logs): Disable rate limiting for critical internal error logs (#24192) * Improve debug log in aws_ecs_metrics * Add internal_log_rate_limit = false to config reload errors * Disable rate limiting for lua build failure * Disable rate limiting for amqp build failure * Add internal_log_rate_limit = false to important failures during reload * Disable log rate limit for repeated events in process.rs * Update src/config/watcher.rs * Fix message format * fix(tracing): prevent panic for traces without standard fields (#24191) * fix(tracing): prevent panic for traces without std fields * forbid unwrap in this lib * clippy fix * improve events_with_custom_fields_no_message_dont_panic, advance to next window * chore(tracing): do not rate limit utlization report (#24202) * chore(internal metrics): move config_reload_* metrics to VectorReload* (#24203) * chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError * update topology_doesnt_reload_new_data_dir test * chore(dev): cargo fmt * strengthen tests, since now we have a better interface * chore(releasing): Add 0.51.0 known issues (#24211) * Format fixes * Add 0.51.0 known issues * Add dots * Reword log panic known issue * Reword VRL missing fns * Backtick versions * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Fix cue docs fmt --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Bump vector version to 0.51.1 * Bump VRL to 0.28.1 * Generate 0.51.1 cue * Add vrl_changelog * Remove whitespace * chore(releasing): Updated distribution/install.sh vector version to 0.51.1 * chore(releasing): Add 0.51.1 to versions.cue * chore(releasing): Created release md file * Add description * Reorder items and fix format * Finally fix styling * Bump date --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> chore(releasing): pull in `internal_logs` fix into 0.51.1 (#24225) * fix(internal_logs source): remove rate limit (#24218) * fix(internal_logs source): remove rate limit * add changelog * Debug commit * Fix validated - Revert "Debug commit" This reverts commit c7b9ec9. * add unit test * fix check-events * Pull in internal_logs source fix * Add internal_logs known issue * Bump date * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Delete changelog entries --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

* chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com>

* chore(releasing): prepare v0.51.1 release (vectordotdev#24214) * chore(ci): reorganize integration test files (vectordotdev#24108) * chore(ci): reorganize integration test files * revert nats data move * fix paths to data * fix more paths * amqp fix * Fix logstash int tests shared files paths * Rename scripts/integration -> tests/integration * scripts/integration -> tests/integration in source files * Fix scripts->tests * Fix gcp paths * Fix http-client pem path * move nats data --------- Co-authored-by: Thomas <thomas.schneider@datadoghq.com> * chore(vdev): move all utils in a new utils folder (vectordotdev#24143) * chore(vdev): move all utils in a new utils folder * move remaining files to utils dir * move remaining files to utils dir * fmt * chore(vdev): make modules visible to rustfmt (vectordotdev#24162) * Remove mod inside cli_subcommands macro * cargo fmt * chore(vdev): apply vdev rust check fixes * Link to rustfmt issue * fix(vdev): release prepare vrl version pinning (vectordotdev#24158) * Add --dry-run to release prepare * Add error handling and checks to pin_vrl_version * Add wrapper to toml * Remove wrapper, parse as Table instead * Fix vrl pinning logic * Enable preserve_order feature in toml crate * Use dependency instead of whole toml * Fix dry run docs * Fix dry run wording * refactor to use toml_edit * Add update_vrl_to_version to add unit test * Use indoc in prepare.rs * Remove preserve_order feature * chore(deps): update VRL to add missing stdlib fns from 0.28 (vectordotdev#24178) * chore(ci): temporarily remove homebrew publish step from publish workflow (vectordotdev#24185) This temporarily removes the publish-homebrew job from the publish workflow to address issue vectordotdev#24139. This is step 1 of the plan to fix the homebrew publishing process. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com> * fix(blackhole sink): disable rate limiting for periodic stats messages (vectordotdev#24190) * fix(blackhole sink): disable rate limiting for periodic stats messages The blackhole sink's informational "Collected events" messages were being rate-limited since rate limiting was enabled by default in vectordotdev#24024. This is undesirable because: 1. These are deliberately scheduled periodic messages (controlled by the `print_interval_secs` config option), not error conditions that could flood the logs 2. Users explicitly configure the frequency - rate limiting defeats that explicit configuration and breaks user expectations 3. The interval timer already prevents log flooding, making additional rate limiting redundant 4. The blackhole sink is used for debugging/testing, where predictable output is essential This fix adds `internal_log_rate_limit = false` to both info! calls, similar to how the console sink disables rate limiting for its critical operational messages. Fixes vectordotdev#24188 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add changelog fragment for blackhole sink rate limiting fix --------- Co-authored-by: Claude <noreply@anthropic.com> * chore(internal logs): Disable rate limiting for critical internal error logs (vectordotdev#24192) * Improve debug log in aws_ecs_metrics * Add internal_log_rate_limit = false to config reload errors * Disable rate limiting for lua build failure * Disable rate limiting for amqp build failure * Add internal_log_rate_limit = false to important failures during reload * Disable log rate limit for repeated events in process.rs * Update src/config/watcher.rs * Fix message format * fix(tracing): prevent panic for traces without standard fields (vectordotdev#24191) * fix(tracing): prevent panic for traces without std fields * forbid unwrap in this lib * clippy fix * improve events_with_custom_fields_no_message_dont_panic, advance to next window * chore(tracing): do not rate limit utlization report (vectordotdev#24202) * chore(internal metrics): move config_reload_* metrics to VectorReload* (vectordotdev#24203) * chore(internal metrics): move config_reload_* metrics to VectorReloaded and VectorReloadedError * update topology_doesnt_reload_new_data_dir test * chore(dev): cargo fmt * strengthen tests, since now we have a better interface * chore(releasing): Add 0.51.0 known issues (vectordotdev#24211) * Format fixes * Add 0.51.0 known issues * Add dots * Reword log panic known issue * Reword VRL missing fns * Backtick versions * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Fix cue docs fmt --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Bump vector version to 0.51.1 * Bump VRL to 0.28.1 * Generate 0.51.1 cue * Add vrl_changelog * Remove whitespace * chore(releasing): Updated distribution/install.sh vector version to 0.51.1 * chore(releasing): Add 0.51.1 to versions.cue * chore(releasing): Created release md file * Add description * Reorder items and fix format * Finally fix styling * Bump date --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> chore(releasing): pull in `internal_logs` fix into 0.51.1 (vectordotdev#24225) * fix(internal_logs source): remove rate limit (vectordotdev#24218) * fix(internal_logs source): remove rate limit * add changelog * Debug commit * Fix validated - Revert "Debug commit" This reverts commit c7b9ec9. * add unit test * fix check-events * Pull in internal_logs source fix * Add internal_logs known issue * Bump date * Update website/cue/reference/releases/0.51.0.cue Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com> * Delete changelog entries --------- Co-authored-by: Pavlos Rontidis <pavlos.rontidis@gmail.com>

github-actions bot added domain: ci Anything related to Vector's CI environment domain: vdev Anything related to the vdev tooling labels Oct 29, 2025

pront added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label Oct 29, 2025

pront marked this pull request as ready for review October 29, 2025 20:50

pront requested a review from a team as a code owner October 29, 2025 20:50

pront enabled auto-merge October 29, 2025 21:10

github-actions bot added the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Oct 30, 2025

github-actions bot removed the domain: core Anything related to core crates i.e. vector-core, core-common, etc label Oct 30, 2025

chore(ci): reorganize integration test files

7e3c370

pront force-pushed the pront-reorg-int-tests branch from fbb6422 to 7e3c370 Compare October 30, 2025 14:36

thomasqueirozb approved these changes Oct 30, 2025

View reviewed changes

revert nats data move

1512994