feat(metrics): integrate vmagent for metrics collection in fuzzy tests #1713
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nd Handlers fuzzy tests
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
- Updated fuzzy load test workflows to use short commit hashes for naming. - Improved logging in run-vmagent.sh to display process IDs for better tracking. - Enhanced setup-vmagent.sh to safely extract the vmagent binary and verify its presence, including a fallback mechanism for different archive structures.
|
This pull request has been automatically marked as stale. If this pull request is still relevant, please leave any comment (for example, "bump"), and we'll keep it open. We are sorry that we haven't been able to prioritize reviewing it yet. Your contribution is very much appreciated. |
|
"This pull request has been automatically closed because it has been inactive for more than 7 days. Please reopen and see this PR through its review if it is essential." |
- Adjusted timeout settings for KV Store and KV Store with Handlers tests from 60 to 180 seconds. - Cleaned up comments and removed unnecessary echo statements in the workflow scripts. - Enhanced the handling of Docker image pulling to avoid failure on pull errors. - Updated summary generation to remove profiling-specific details, making it more concise.
- Changed the global scrape interval from 5s to 15s. - Updated job-specific scrape intervals from 5s to 15s for consistency.
- Updated global and job-specific scrape intervals from 15s to 30s in VM agent configuration. - Increased timeout setting for fuzzy tests from 180 to 360 seconds.
- Changed default test duration from 5 minutes to 45 minutes in the fuzzy load test workflow. - Removed outdated TODO comments regarding workflow configuration. - Cleaned up the fuzzy test YAML by removing unnecessary final handlers verification steps.
- Added execute permissions for setup, run, and cleanup scripts in the fuzzy load test workflow to ensure proper execution during the CI process.
fbozic
approved these changes
Jan 20, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[CI/CD] Add Victoria Metrics scraping to fuzzy load test workflow
Description
This PR adds Victoria Metrics (vmagent) integration to the fuzzy load test workflow, enabling automatic metrics collection from
merodprocesses during long-running performance tests. The vmagent lifecycle and dynamic scrape configuration are handled via reusable scripts to reduce duplication and improve maintainability.Dashboard : https://grafana.apps.dev.p2p.aws.calimero.network/d/merod-gha/merod-gha?orgId=1&from=2025-12-30T14:40:43.543Z&to=2025-12-30T15:16:06.991Z&timezone=browser&var-execution_platform=gha&var-execution_environment=vm&var-merod_name=kv-store-287a1be&var-rate_interval=5m&var-percentile=0.99&var-context_id=$__all
Key changes
New scripts (vmagent lifecycle management):
scripts/setup-vmagent.sh: Prepares vmagent (download/extract/configure) and emits outputs consumed by the workflowscripts/run-vmagent.sh: Starts vmagent and periodically refreshes scrape configuration asmerodprocesses come/goscripts/cleanup-vmagent.sh: Gracefully shuts down vmagent and cleans up temp filesWorkflow updates (
.github/workflows/fuzzy-load-test.yml):master(scoped paths) and on pull_request (scoped to workflow + fuzzy-test configs), plus manualworkflow_dispatchMetrics collection
merodprocesses listening on ports 2420–2499Notes
enable_profiling) and profiling artifact uploads; vmagent metrics collection runs alongside the fuzzy tests.Test plan
Documentation update
Note
Adds automated metrics scraping to long-running fuzzy tests via
vmagent, with reusable scripts and workflow integration for bothkv-storeandkv-store-with-handlers.scripts/setup-vmagent.sh,scripts/run-vmagent.sh,scripts/cleanup-vmagent.shto download/start/refresh/reload/stopvmagentand manage auth/bearer token.github/workflows/fuzzy-load-test.yml): setupvmagent, run tests with collection, cleanup, and uploadvmagentlogs/config as artifacts; simplify image pull; remove profiling-only decorations in summaryfuzzy-kv-node,fuzzy-handlers-node), periodic refresh, labeled with commit/branch/run metadata180; remove end-of-test stats/assert blocks from handlers test; minor whitespace/formatting cleanupWritten by Cursor Bugbot for commit 5e7c7a4. This will update automatically on new commits. Configure here.