Skip to content

Conversation

@Tai-RedHat
Copy link

@Tai-RedHat Tai-RedHat commented Dec 16, 2025

link to: https://issues.redhat.com/browse/MON-4439

local test result:

% oc get co/monitoring
NAME         VERSION                              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
monitoring   4.21.0-0.nightly-2025-12-15-125449   True        False         False      33m

% ./openshift-tests run-test "[sig-instrumentation] Prometheus [apigroup:image.openshift.io] when installed on the cluster should use endpoint slices for service discovery [Suite:openshift/conformance/parallel]"
...
  Dec 16 14:25:32.266: INFO: The following jobs are still using Endpoints for service discovery instead of EndpointSlices:
   - serviceMonitor/openshift-apiserver-operator/openshift-apiserver-operator/0
   - serviceMonitor/openshift-apiserver/openshift-apiserver/0
   - serviceMonitor/openshift-apiserver/openshift-apiserver-operator-check-endpoints/0
   - serviceMonitor/openshift-authentication-operator/authentication-operator/0
   - serviceMonitor/openshift-authentication/oauth-openshift/0
   - serviceMonitor/openshift-catalogd/catalogd-metrics-monitor/0
   - serviceMonitor/openshift-cloud-credential-operator/cloud-credential-operator/0
   - serviceMonitor/openshift-cluster-csi-drivers/aws-ebs-csi-driver-controller-monitor/0
   - serviceMonitor/openshift-cluster-csi-drivers/aws-ebs-csi-driver-controller-monitor/1
   - serviceMonitor/openshift-cluster-csi-drivers/aws-ebs-csi-driver-controller-monitor/2
   - serviceMonitor/openshift-cluster-csi-drivers/aws-ebs-csi-driver-controller-monitor/3
   - serviceMonitor/openshift-cluster-csi-drivers/aws-ebs-csi-driver-controller-monitor/4
   - serviceMonitor/openshift-cluster-machine-approver/cluster-machine-approver/0
   - serviceMonitor/openshift-cluster-node-tuning-operator/node-tuning-operator/0
   - serviceMonitor/openshift-cluster-samples-operator/cluster-samples-operator/0
   - serviceMonitor/openshift-cluster-storage-operator/cluster-storage-operator/0
   - serviceMonitor/openshift-cluster-version/cluster-version-operator/0
   - serviceMonitor/openshift-config-operator/config-operator/0
   - serviceMonitor/openshift-console-operator/console-operator/0
   - serviceMonitor/openshift-console/console/0
   - serviceMonitor/openshift-controller-manager-operator/openshift-controller-manager-operator/0
   - serviceMonitor/openshift-controller-manager/openshift-controller-manager/0
   - serviceMonitor/openshift-dns-operator/dns-operator/0
   - serviceMonitor/openshift-dns/dns-default/0
   - serviceMonitor/openshift-etcd-operator/etcd/0
   - serviceMonitor/openshift-etcd-operator/etcd-operator/0
   - serviceMonitor/openshift-image-registry/image-registry/0
   - serviceMonitor/openshift-image-registry/image-registry-operator/0
   - serviceMonitor/openshift-ingress-operator/ingress-operator/0
   - serviceMonitor/openshift-ingress/router-default/0
   - serviceMonitor/openshift-insights/insights-operator/0
   - serviceMonitor/openshift-kube-apiserver-operator/kube-apiserver-operator/0
   - serviceMonitor/openshift-kube-apiserver/kube-apiserver/0
   - serviceMonitor/openshift-kube-controller-manager-operator/kube-controller-manager-operator/0
   - serviceMonitor/openshift-kube-controller-manager/kube-controller-manager/0
   - serviceMonitor/openshift-kube-scheduler-operator/kube-scheduler-operator/0
   - serviceMonitor/openshift-kube-scheduler/kube-scheduler/0
   - serviceMonitor/openshift-kube-scheduler/kube-scheduler/1
   - serviceMonitor/openshift-machine-api/cluster-autoscaler-operator/0
   - serviceMonitor/openshift-machine-api/machine-api-controllers/0
   - serviceMonitor/openshift-machine-api/machine-api-controllers/1
   - serviceMonitor/openshift-machine-api/machine-api-controllers/2
   - serviceMonitor/openshift-machine-api/machine-api-operator/0
   - serviceMonitor/openshift-machine-config-operator/machine-config-controller/0
   - serviceMonitor/openshift-machine-config-operator/machine-config-daemon/0
   - serviceMonitor/openshift-machine-config-operator/machine-config-operator/0
   - serviceMonitor/openshift-marketplace/marketplace-operator/0
   - serviceMonitor/openshift-monitoring/cluster-monitoring-operator/0
   - serviceMonitor/openshift-monitoring/kube-state-metrics/0
   - serviceMonitor/openshift-monitoring/kube-state-metrics/1
   - serviceMonitor/openshift-monitoring/metrics-server/0
   - serviceMonitor/openshift-monitoring/openshift-state-metrics/0
   - serviceMonitor/openshift-monitoring/openshift-state-metrics/1
   - serviceMonitor/openshift-monitoring/prometheus-operator/0
   - serviceMonitor/openshift-monitoring/promtail-monitor/0
   - serviceMonitor/openshift-monitoring/thanos-querier/0
   - serviceMonitor/openshift-multus/monitor-multus-admission-controller/0
   - serviceMonitor/openshift-multus/monitor-network/0
   - serviceMonitor/openshift-network-diagnostics/network-check-source/0
   - serviceMonitor/openshift-network-operator/network-operator/0
   - serviceMonitor/openshift-oauth-apiserver/openshift-oauth-apiserver/0
   - serviceMonitor/openshift-operator-controller/operator-controller-metrics-monitor/0
   - serviceMonitor/openshift-operator-lifecycle-manager/catalog-operator/0
   - serviceMonitor/openshift-operator-lifecycle-manager/olm-operator/0
   - serviceMonitor/openshift-operator-lifecycle-manager/package-server-manager-metrics/0
   - serviceMonitor/openshift-ovn-kubernetes/monitor-ovn-control-plane-metrics/0
   - serviceMonitor/openshift-ovn-kubernetes/monitor-ovn-node/0
   - serviceMonitor/openshift-ovn-kubernetes/monitor-ovn-node/1
   - serviceMonitor/openshift-route-controller-manager/openshift-route-controller-manager/0
   - serviceMonitor/openshift-service-ca-operator/service-ca-operator/0
...
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped

@openshift-ci-robot
Copy link

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Dec 16, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Tai-RedHat
Once this PR has been reviewed and has the lgtm label, please assign jan--f for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot
Copy link

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Dec 16, 2025

@Tai-RedHat: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-metal-ipi-ovn-ipv6 f7237d0 link true /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-trt
Copy link

openshift-trt bot commented Dec 16, 2025

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: f7237d0

  • "[sig-instrumentation] Prometheus [apigroup:image.openshift.io] when installed on the cluster should use endpoint slices for service discovery [Suite:openshift/conformance/parallel]" [Total: 5, Pass: 5, Fail: 0, Flake: 0]

Copy link
Contributor

@machine424 machine424 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I have some prepositions

})

g.It("should use endpoint slices for service discovery", func() {
configContents, err := helper.GetURLWithToken(helper.MustJoinUrlPath(prometheusURL, "/api/v1/status/config"), bearerToken)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about using /api/v1/targets instead?
as used in

contents, err := helper.GetURLWithToken(helper.MustJoinUrlPath(prometheusURL, "api/v1/targets"), bearerToken)
and other places.

we can then check whether __meta_kubernetes_endpoints_name (https://prometheus.io/docs/prometheus/latest/configuration/configuration/#endpoints) is part of data.activeTargets[*].discoveredLabels

for unmarshalling there is

type prometheusTargets struct {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(we can "export" that promTargets function if needed)

}
if len(jobsUsingEndpoints) > 0 {
formattedList := "\n - " + strings.Join(jobsUsingEndpoints, "\n - ")
// TODO(MON-4439): This should be a hard failure once all components have migrated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can hardcode the targets to exclude somewhere and skip the check for them.
And of course have a ticket for them to migrate them later.

}
})

g.It("should use endpoint slices for service discovery", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
g.It("should use endpoint slices for service discovery", func() {
g.It("should only discover targets via endpointslice", func() {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants