Skip to content

OCPBUGS-74232: volume-data-source-validator should run on master nodes#659

Open
RomanBednar wants to merge 1 commit intoopenshift:mainfrom
RomanBednar:OCPBUGS-74232
Open

OCPBUGS-74232: volume-data-source-validator should run on master nodes#659
RomanBednar wants to merge 1 commit intoopenshift:mainfrom
RomanBednar:OCPBUGS-74232

Conversation

@RomanBednar
Copy link
Contributor

The volume-data-source-validator deployment was missing nodeSelector and tolerations required to ensure it runs on master/control plane nodes. This is inconsistent with other control plane components in the same namespace (openshift-cluster-storage-operator) such as vsphere-problem-detector and the main cluster-storage-operator.

Added:

  • nodeSelector with node-role.kubernetes.io/master label
  • CriticalAddonsOnly toleration
  • node-role.kubernetes.io/master toleration with NoSchedule effect

This ensures the validator pod:

  • Only schedules on control plane nodes
  • Tolerates the NoSchedule taint on master nodes
  • Maintains consistency with other critical control plane components

cc @openshift/storage

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jan 22, 2026
@openshift-ci-robot
Copy link
Contributor

@RomanBednar: This pull request references Jira Issue OCPBUGS-74232, which is invalid:

  • expected the bug to target the "4.22.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

The volume-data-source-validator deployment was missing nodeSelector and tolerations required to ensure it runs on master/control plane nodes. This is inconsistent with other control plane components in the same namespace (openshift-cluster-storage-operator) such as vsphere-problem-detector and the main cluster-storage-operator.

Added:

  • nodeSelector with node-role.kubernetes.io/master label
  • CriticalAddonsOnly toleration
  • node-role.kubernetes.io/master toleration with NoSchedule effect

This ensures the validator pod:

  • Only schedules on control plane nodes
  • Tolerates the NoSchedule taint on master nodes
  • Maintains consistency with other critical control plane components

cc @openshift/storage

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested review from jsafrane and tsmetana January 22, 2026 10:16
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jan 22, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: RomanBednar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 22, 2026
@RomanBednar
Copy link
Contributor Author

/jira refresh
/cherry-pick release-4.21
/cherry-pick release-4.20

@openshift-cherrypick-robot

@RomanBednar: once the present PR merges, I will cherry-pick it on top of release-4.20, release-4.21 in new PRs and assign them to you.

Details

In response to this:

/jira refresh
/cherry-pick release-4.21
/cherry-pick release-4.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Jan 22, 2026
@openshift-ci-robot
Copy link
Contributor

@RomanBednar: This pull request references Jira Issue OCPBUGS-74232, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (wduan@redhat.com), skipping review request.

Details

In response to this:

/jira refresh
/cherry-pick release-4.21
/cherry-pick release-4.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Jan 22, 2026
@RomanBednar
Copy link
Contributor Author

/retest-required

@duanwei33
Copy link

/verified by @duanwei33

$ curl -sL https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-storage-operator/659/pull-ci-openshift-cluster-storage-operator-main-e2e-gcp-csi/2014281016293724160/artifacts/e2e-gcp-csi/gather-extra/artifacts/pods.json | jq '.items[] | select(.metadata.labels | tostring | contains("volume-data-source-validator")) | {podName: .metadata.name, nodeName: .spec.nodeName, nodeSelector: .spec.nodeSelector}'
{
  "podName": "volume-data-source-validator-7fbd4f4889-49rqs",
  "nodeName": "ci-op-fy71pm30-c31b1-4mtgp-master-0",
  "nodeSelector": {
    "node-role.kubernetes.io/master": ""
  }
}

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 2, 2026
@duanwei33
Copy link

/retest-required

@openshift-ci-robot
Copy link
Contributor

@duanwei33: This PR has been marked as verified by @duanwei33.

Details

In response to this:

/verified by @duanwei33

$ curl -sL https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-storage-operator/659/pull-ci-openshift-cluster-storage-operator-main-e2e-gcp-csi/2014281016293724160/artifacts/e2e-gcp-csi/gather-extra/artifacts/pods.json | jq '.items[] | select(.metadata.labels | tostring | contains("volume-data-source-validator")) | {podName: .metadata.name, nodeName: .spec.nodeName, nodeSelector: .spec.nodeSelector}'
{
 "podName": "volume-data-source-validator-7fbd4f4889-49rqs",
 "nodeName": "ci-op-fy71pm30-c31b1-4mtgp-master-0",
 "nodeSelector": {
   "node-role.kubernetes.io/master": ""
 }
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@duanwei33
Copy link

Wait, after checking the failed hypershift job, the volume-data-source-validator is deployed to the guest cluster and the guest cluster only has worker nodes — so the storage CO is degraded.

  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: "2026-03-02T05:16:33Z"
      message: '0/3 nodes are available: 3 node(s) didn''t match Pod''s node affinity/selector.
        no new claims to deallocate, preemption: 0/3 nodes are available: 3 Preemption
        is not helpful for scheduling.'
      observedGeneration: 1
      reason: Unschedulable
      status: "False"
      type: PodScheduled
    phase: Pending

I think we need to add this like the csidriveroperators.

@duanwei33
Copy link

/verified cancel

@openshift-ci-robot
Copy link
Contributor

@duanwei33: The /verified command must be used with one of the following actions: by, later, remove, or bypass. See https://docs.ci.openshift.org/docs/architecture/jira/#premerge-verification for more information.

Details

In response to this:

/verified cancel

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@duanwei33
Copy link

/verified remove

@openshift-ci-robot
Copy link
Contributor

@duanwei33: /verified bypass and /verified remove do not support arguments. See https://docs.ci.openshift.org/docs/architecture/jira/#premerge-verification for more information.

Details

In response to this:

/verified remove

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@duanwei33
Copy link

/verified remove

@openshift-ci-robot openshift-ci-robot removed the verified Signifies that the PR passed pre-merge verification criteria label Mar 2, 2026
@openshift-ci-robot
Copy link
Contributor

@duanwei33: The verified label has been removed.

Details

In response to this:

/verified remove

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

…hift scheduling

Standalone: add nodeSelector and tolerations to the validator deployment so it
schedules on master/control plane nodes, consistent with other components in
openshift-cluster-storage-operator (e.g. vsphere-problem-detector).

HyperShift: guest clusters have no master nodes, so the same nodeSelector left
the pod Pending. Add WithControlPlaneTopologyHook so when
Infrastructure.Status.ControlPlaneTopology is External the nodeSelector is
cleared and the validator can schedule on guest worker nodes.
@coderabbitai
Copy link

coderabbitai bot commented Mar 18, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 97e37723-34a2-4fa8-ba7d-257ff737d210

📥 Commits

Reviewing files that changed from the base of the PR and between f7b0b15 and d9b6091.

📒 Files selected for processing (2)
  • assets/volumedatasourcevalidator/04_deployment.yaml
  • pkg/operator/volumedatasourcevalidator/controller.go

Walkthrough

The volume data source validator is updated to support control plane topology awareness by adding scheduling constraints to the deployment manifest and integrating infrastructure configuration informers and control plane topology hooks into the deployment controller.

Changes

Cohort / File(s) Summary
Volume Data Source Validator Scheduling
assets/volumedatasourcevalidator/04_deployment.yaml
Added nodeSelector and tolerations to Pod spec to restrict validator pod scheduling to master nodes with CriticalAddonsOnly and NoSchedule taints.
Validator Controller Configuration
pkg/operator/volumedatasourcevalidator/controller.go
Integrated infrastructure config informer and control plane topology deployment hook to enable topology-aware scheduling logic.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
📝 Coding Plan
  • Generate coding plan for human review comments

Comment @coderabbitai help to get the list of available commands and usage tips.

Tip

You can get early access to new features in CodeRabbit.

Enable the early_access setting to enable early access features such as new models, tools, and more.

@RomanBednar
Copy link
Contributor Author

@duanwei33 I think we need WithControlPlaneTopologyHook to remove the node selector on hypershift - this is what problem detector is using too.

I'm assuming it is ok to run the validator in guest cluster (as before) because VolumePopulator objects should live in the guest cluster too I think because on hypershift populator authors would not have access to management cluster.

The validator could run in management cluster instead. However, that would require much more refactoring to a structure similar to CSI Operators, as you suggested.

@duanwei33
Copy link

@RomanBednar Thanks for the explanation, I see the WithControlPlaneTopologyHook implementation now. This works on both standard and hypershift clusters, and that's fine from my side.

@duanwei33
Copy link

/retest

@duanwei33
Copy link

/verified by @duanwei33

Just re-triggered the hypershift-aws-e2e-external job (one test case looks like a flake).

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Mar 19, 2026
@openshift-ci-robot
Copy link
Contributor

@duanwei33: This PR has been marked as verified by @duanwei33.

Details

In response to this:

/verified by @duanwei33

Just re-triggered the hypershift-aws-e2e-external job (one test case looks like a flake).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@RomanBednar
Copy link
Contributor Author

/retest-required

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Mar 19, 2026

@RomanBednar: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants