Fix VM and volume metrics listing regressions #12284

winterhazel · 2025-12-18T03:56:56Z

Description

This PR fixes a regression in commit c8d44d9 that prevents non-admin accounts from listing the metrics of VMs and volumes that belong to a project (see #12237).

Previously, the following example condition was used in the listVirtualMachinesMetrics API in order to find VMs that could have their metrics listed by a user:

vm_instance.type='User' AND user_vm.id=88 AND vm_instance.removed IS NULL

In 4.20.2 and 4.22.0, the condition also restricts the result to VMs that belong to the calling account; hence, VMs that belong to a project are not returned:

WHERE vm_instance.type='User' AND vm_instance.account_id=6 AND user_vm.id=88 AND vm_instance.removed IS NULL

With this patch, the ID of projects that the user has access to is also included in the condition:

WHERE vm_instance.type='User' AND vm_instance.account_id IN (11,6)  AND user_vm.id=88 AND vm_instance.removed IS NULL

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

I manually verified that, using both the id and ids parameters:

Root admins can access the metrics of VMs/volumes belonging to other users and projects
Domain admins can access the metrics of VMs/volumes belonging to users and projects of their domain
Users can access the metrics of VMs/volumes belonging to them and to projects they have access to

winterhazel · 2025-12-18T03:57:19Z

@blueorangutan package

blueorangutan · 2025-12-18T03:58:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-12-18T04:07:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.23%. Comparing base (ba26d95) to head (67c5241).
⚠️ Report is 3 commits behind head on 4.20.

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #12284      +/-   ##
============================================
+ Coverage     16.22%   16.23%   +0.01%     
- Complexity    13358    13379      +21     
============================================
  Files          5657     5657              
  Lines        498692   498865     +173     
  Branches      60530    60545      +15     
============================================
+ Hits          80932    81011      +79     
- Misses       408738   408820      +82     
- Partials       9022     9034      +12

Flag	Coverage Δ
uitests	`4.00% <ø> (ø)`
unittests	`17.09% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-12-18T05:17:47Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16080

kiranchavala

LGTM

Created a user in a domain
Login, as the user and Create a project
Deploy a vm in the project and check the metrics tab

Before fix

the API listVirtualMachinesUsageHistory has no response

After fix

the API listVirtualMachinesUsageHistory has response

DaanHoogland

clgtm, one remark/question though; two bits of code are repeated with a similar statement in between, Can we create a method for it?

winterhazel · 2025-12-18T12:53:31Z

@kiranchavala thanks for testing :]

@DaanHoogland yup, I will do some refactoring (and some extra tests to be sure) before marking it as ready.

…gnored when passing multiple IDs

winterhazel · 2025-12-18T20:11:14Z

@blueorangutan package

blueorangutan · 2025-12-18T20:12:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:12:58Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16090

winterhazel · 2025-12-18T20:17:56Z

@DaanHoogland I extracted the code that builds the parameters to a method. Tests are also ok.

I noticed that there was another regression in which admins were not able to list the metrics of VMs/volumes belonging to projects via the ids parameter. It should be fixed now too.

Example

<before>
(admin) 🐢 > list virtualmachinesusagehistory id=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
{
  "count": 1,
  "virtualmachine": [
    {
      "displayname": "k",
      "id": "69ecb2a6-dc91-46b9-a5d9-06e8222c5084",
      "name": "i-2-88-VM",
      "stats": [
        {
          "cpuused": "0%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:21:54+0000"
        },
        {
          "cpuused": "20.77%",
          "diskiopstotal": 835,
          "diskioread": 640,
          "diskiowrite": 195,
          "diskkbsread": 10876,
          "diskkbswrite": 1612,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 2,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:22:54+0000"
        },
        {
          "cpuused": "13.52%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:23:54+0000"
        }
      ]
    }
  ]
}

(admin) 🐢 > list virtualmachinesusagehistory ids=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
<empty>

<with the patch>
(admin) 🐢 > list virtualmachinesusagehistory ids=69ecb2a6-dc91-46b9-a5d9-06e8222c5084
{
  "count": 1,
  "virtualmachine": [
    {
      "displayname": "k",
      "id": "69ecb2a6-dc91-46b9-a5d9-06e8222c5084",
      "name": "i-2-88-VM",
      "stats": [
        {
          "cpuused": "0%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:21:54+0000"
        },
        {
          "cpuused": "20.77%",
          "diskiopstotal": 835,
          "diskioread": 640,
          "diskiowrite": 195,
          "diskkbsread": 10876,
          "diskkbswrite": 1612,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 2,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:22:54+0000"
        },
        {
          "cpuused": "13.52%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:23:54+0000"
        },
        {
          "cpuused": "60.96%",
          "diskiopstotal": 1116,
          "diskioread": 902,
          "diskiowrite": 214,
          "diskkbsread": 67400,
          "diskkbswrite": 2040,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:24:55+0000"
        },
        {
          "cpuused": "40.05%",
          "diskiopstotal": 277,
          "diskioread": 110,
          "diskiowrite": 167,
          "diskkbsread": 13304,
          "diskkbswrite": 38524,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:25:56+0000"
        },
        {
          "cpuused": "26.1%",
          "diskiopstotal": 3,
          "diskioread": 0,
          "diskiowrite": 3,
          "diskkbsread": 0,
          "diskkbswrite": 16,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:26:58+0000"
        },
        {
          "cpuused": "25.4%",
          "diskiopstotal": 1,
          "diskioread": 0,
          "diskiowrite": 1,
          "diskkbsread": 0,
          "diskkbswrite": 4,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:27:59+0000"
        },
        {
          "cpuused": "24.83%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:29:02+0000"
        },
        {
          "cpuused": "24.91%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:30:04+0000"
        },
        {
          "cpuused": "25.87%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:31:09+0000"
        },
        {
          "cpuused": "25.24%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:32:11+0000"
        },
        {
          "cpuused": "20.79%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:33:12+0000"
        },
        {
          "cpuused": "6.03%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:34:13+0000"
        },
        {
          "cpuused": "5.77%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:35:13+0000"
        },
        {
          "cpuused": "5.78%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:36:13+0000"
        },
        {
          "cpuused": "7.01%",
          "diskiopstotal": 158,
          "diskioread": 103,
          "diskiowrite": 55,
          "diskkbsread": 412,
          "diskkbswrite": 472,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:37:13+0000"
        },
        {
          "cpuused": "8.05%",
          "diskiopstotal": 3,
          "diskioread": 0,
          "diskiowrite": 3,
          "diskkbsread": 0,
          "diskkbswrite": 16,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:38:14+0000"
        },
        {
          "cpuused": "8.79%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:39:14+0000"
        },
        {
          "cpuused": "6.4%",
          "diskiopstotal": 0,
          "diskioread": 0,
          "diskiowrite": 0,
          "diskkbsread": 0,
          "diskkbswrite": 0,
          "memoryintfreekbs": 353728,
          "memorykbs": 524288,
          "memorytargetkbs": 524288,
          "networkkbsread": 0,
          "networkkbswrite": 0,
          "networkread": "0.00 MiB",
          "networkwrite": "0.00 MiB",
          "timestamp": "2025-12-18T19:40:14+0000"
        }
      ]
    }
  ]
}

Also, passing the ids parameter to listVolumesUsageHistory was throwing an exception. I took the opportunity to fix this here as well.

winterhazel · 2025-12-18T20:29:00Z

@blueorangutan package

blueorangutan · 2025-12-18T20:30:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:31:03Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16091

winterhazel · 2025-12-18T20:49:55Z

@blueorangutan package

blueorangutan · 2025-12-18T20:52:04Z

@winterhazel a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-12-18T20:52:58Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16092

DaanHoogland · 2025-12-19T08:08:06Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 16092

we hit the github API limit @winterhazel , don’t worry about this.

plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java

blueorangutan · 2025-12-19T10:43:04Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 16103

DaanHoogland · 2025-12-19T16:47:15Z

@blueorangutan test

blueorangutan · 2025-12-19T16:48:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-12-20T07:15:58Z

[SF] Trillian test result (tid-15033)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 49197 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr12284-t15033-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

* 4.22: Update templateConfig.sh to not break with directorys with space on t… (#10898) Fix VM and volume metrics listing regressions (#12284) packaging: use latest cmk release link directly (#11429) api:rename RegisterCmd.java => RegisterUserKeyCmd.java (#12259) Prioritize copying templates from other secondary storages instead of downloading them (#10363) Show time correctly in the backup schedule UI (#12012) kvm: use preallocation option for fat disk resize (#11986) Python exception processing static routes fixed (#11967) KVM memballooning requires free page reporting and autodeflate (#11932) api: create/register/upload template with empty template tag (#12234)

Fix VM and volume metrics listing regression

c7989c0

kiranchavala approved these changes Dec 18, 2025

View reviewed changes

DaanHoogland approved these changes Dec 18, 2025

View reviewed changes

winterhazel added 2 commits December 18, 2025 17:03

Extract creation of search parameters & fix project resources being i…

c56594b

…gnored when passing multiple IDs

Remove unused import

fbf7551

winterhazel changed the title ~~Fix VM and volume metrics listing regression~~ Fix VM and volume metrics listing regressions Dec 18, 2025

winterhazel marked this pull request as ready for review December 18, 2025 20:11

winterhazel added type:regression component:metrics labels Dec 18, 2025

Remove another unused import

67c5241

weizhouapache reviewed Dec 19, 2025

View reviewed changes

plugins/metrics/src/main/java/org/apache/cloudstack/metrics/MetricsServiceImpl.java Show resolved Hide resolved

DaanHoogland merged commit 061ce9b into apache:4.20 Dec 20, 2025
26 checks passed

Fix VM and volume metrics listing regressions #12284

Fix VM and volume metrics listing regressions #12284

Uh oh!

Conversation

winterhazel commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

codecov bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

kiranchavala left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

winterhazel commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

blueorangutan commented Dec 18, 2025

Uh oh!

DaanHoogland commented Dec 19, 2025

Uh oh!

Uh oh!

blueorangutan commented Dec 19, 2025

Uh oh!

DaanHoogland commented Dec 19, 2025

Uh oh!

blueorangutan commented Dec 19, 2025

Uh oh!

blueorangutan commented Dec 20, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

winterhazel commented Dec 18, 2025 •

edited

Loading

codecov bot commented Dec 18, 2025 •

edited

Loading