Error when setting logVerbose=1: Goldilocks controller fails with RBAC error for Prometheus resource

[jo-schulz]

What happened?

According to the documentation, the logVerbose configuration option can be set to values between 1 and 10.
When setting logVerbose=1, the controller fails at runtime and logs the following error:

"An error occured retrieving the top level controller for this pod" err="prometheuses.monitoring.coreos.com is forbidden: User "system:serviceaccount:NAMESPACE:goldilocks-controller" cannot list resource "prometheuses" in API group "monitoring.coreos.com" in the namespace "NAMESPACE"" prometheus-cluster-prometheus-0="NAMESPACE"

The error indicates that the controller attempts to list prometheuses.monitoring.coreos.com, but the service account does not have the required RBAC permissions.

This behavior only occurs when logVerbose is set to 1.
When logVerbose is set to the default value (2) or higher values such as 3, the controller runs without errors using the same default RBAC configuration.

What did you expect to happen?

• Setting logVerbose to any documented value (1–10) should not cause the controller to fail
• Log verbosity should only affect logging output and should not introduce additional functional behavior or RBAC requirements
• If additional RBAC permissions are required for certain verbosity levels, this should be clearly documented
• As a user following the documentation, there is currently no indication that setting logVerbose=1 requires extra role or cluster role permissions, making this behavior unexpected and difficult to diagnose

How can we reproduce this?

1. Deploy the controller with the RBAC configuration recommended by the documentation
2. Configure the controller with logVerbose=1
3. Restart or redeploy the controller
4. Observe the controller logs and see the RBAC error related to listing prometheuses.monitoring.coreos.com
5. Change logVerbose back to 2 (default) or 3
6. Restart the controller again and observe that the error disappears and the controller works correctly without any RBAC changes

Version

v4.14.1 / helm version 10.2.0

Search

• I did search for other open and closed issues before opening this.

Code of Conduct

• I agree to follow this project's Code of Conduct

Additional context

No response

[sudermanjr]

Can you please share the full output of a log using --previous? That error is non-exiting in most cases, and there is likely another reason for the shutdown. Also please share the output of kubectl describe po

This could also be a timing issue, and have nothing to do with the log verbosity. Dialing in the actual reason for the pod shutdown (via kubectl describe and entire log output) will be necessary.

[sudermanjr]

Similar issue:

#396