🌱 OPRUN-4261 Migrate e2e tests to Godog BDD framework #2365

pedjak · 2025-11-28T15:11:56Z

Description

Replace traditional Go e2e tests with Godog (Cucumber for Go) to improve test readability and maintainability through behavior-driven development.

Benefits:

Living Documentation: Test scenarios serve as up-to-date documentation of system behavior
Better Collaboration: Product owners can read and validate test scenarios
Reduced Duplication: Reusable step definitions eliminate code repetition
Improved Maintainability: Changes to common patterns happen in one place
Clearer Intent: Gherkin syntax makes test purpose immediately obvious
Easier Debugging: Clear separation between what is tested (features) and how (steps)
Concurrent Execution: Set the ground work for running tests in parallel

Changes:

Convert existing test scenarios to Gherkin feature files
Implement reusable step definitions in steps/steps.go
Add scenario hooks for setup/teardown and feature gate detection
Provide comprehensive documentation in test/e2e/README.md
Remove legacy test files (cluster_extension_install_test.go, etc.)
Added detailed README covering:
- Architecture and design patterns
- How to write new tests
- Running tests with various options
- Best practices and troubleshooting
Go test driving code reduced by ~1000 lines

Migration Notes

No changes to test behavior or coverage - this is purely a refactoring
All existing test scenarios are preserved with equivalent Gherkin implementations
Test execution remains the same via make test-e2e

Assisted-By: Claude noreply@anthropic.com

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

netlify · 2025-11-28T15:12:02Z

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	`bf10225`
🔍 Latest deploy log	https://app.netlify.com/projects/olmv1/deploys/6941be61c0b20b00083c1700
😎 Deploy Preview	https://deploy-preview-2365--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This PR migrates the e2e test suite from traditional Go testing framework to Godog (BDD/Cucumber framework), enabling behavior-driven development with Gherkin feature files. The migration maintains test coverage while reorganizing tests into feature files with step definitions.

Key Changes

Replaced traditional Go test functions with Godog scenarios and step definitions
Added Gherkin .feature files describing test behavior in a more readable format
Introduced new test infrastructure (steps.go, hooks.go) to support BDD testing

Reviewed changes

Copilot reviewed 19 out of 21 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/e2e/features_test.go	New test entry point initializing Godog suite with scenario and suite initializers
test/e2e/features/steps/steps.go	Implements step definitions mapping Gherkin steps to Go functions
test/e2e/features/steps/hooks.go	Provides scenario lifecycle hooks and feature gate detection
test/e2e/features/*.feature	Gherkin feature files defining test scenarios (install, update, recover, metrics)
test/e2e/features/steps/testdata/*.yaml	YAML templates for test resources (catalogs, RBAC)
test/e2e/*_test.go	Removed traditional test files migrated to feature files
test/e2e/network_policy_test.go	Added client initialization and helper function
go.mod, go.sum	Added Cucumber/Godog dependencies

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/steps/steps.go

test/e2e/features/steps/hooks.go

test/e2e/features/steps/steps.go

test/e2e/features/steps/testdata/rbac-template.yaml

test/e2e/features_test.go

codecov · 2025-11-28T16:08:10Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.05%. Comparing base (39718ba) to head (bf10225).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2365      +/-   ##
==========================================
- Coverage   73.07%   73.05%   -0.03%     
==========================================
  Files         100      100              
  Lines        7641     7641              
==========================================
- Hits         5584     5582       -2     
- Misses       1622     1623       +1     
- Partials      435      436       +1

Flag	Coverage Δ
e2e	`43.81% <ø> (-0.92%)`	⬇️
experimental-e2e	`48.77% <ø> (-0.58%)`	⬇️
unit	`57.11% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 20 out of 22 changed files in this pull request and generated 9 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-01T18:12:19Z

test/e2e/steps/steps.go

+		if stdErr := string(func() *exec.ExitError {
+			target := &exec.ExitError{}
+			_ = errors.As(err, &target)
+			return target
+		}().Stderr); !strings.Contains(stdErr, errMsg) {


Potential nil pointer dereference when extracting stderr from error. If the error is not of type *exec.ExitError, this will panic. Consider adding a nil check:

waitFor(ctx, func() bool { _, err := kubectlWithInput(yamlContent, "apply", "-f", "-") if err == nil { return false } var exitErr *exec.ExitError if errors.As(err, &exitErr) && strings.Contains(string(exitErr.Stderr), errMsg) { return true } return false })

Suggested change

if stdErr := string(func() *exec.ExitError {

target := &exec.ExitError{}

_ = errors.As(err, &target)

return target

}().Stderr); !strings.Contains(stdErr, errMsg) {

var exitErr *exec.ExitError

if errors.As(err, &exitErr) {

stdErr := string(exitErr.Stderr)

if !strings.Contains(stdErr, errMsg) {

return false

}

return true

}

return false

test/e2e/features_test.go

test/e2e/steps/hooks.go

Copilot · 2025-12-01T18:12:21Z

test/e2e/steps/hooks.go

+		if _, err := kubectl("delete", r.kind, r.name, "-n", sc.namespace); err != nil {
+			logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(err.(*exec.ExitError).Stderr))


Potential nil pointer dereference when asserting error type. If the error is not of type *exec.ExitError, this will panic when accessing .Stderr. Consider checking if the type assertion succeeded:

if _, err := kubectl("delete", r.kind, r.name, "-n", sc.namespace); err != nil { var exitErr *exec.ExitError stderr := "" if errors.As(err, &exitErr) { stderr = string(exitErr.Stderr) } logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", stderr) }

test/e2e/features_test.go

test/e2e/steps/hooks.go

Copilot · 2025-12-01T18:12:22Z

test/e2e/steps/hooks.go

+}
+
+func scenarioCtx(ctx context.Context) *scenarioContext {
+	return ctx.Value(scenarioContextKey).(*scenarioContext)


Potential nil pointer dereference. If ctx.Value(scenarioContextKey) returns nil, this will panic. Consider adding a nil check and returning a helpful error:

func scenarioCtx(ctx context.Context) *scenarioContext { val := ctx.Value(scenarioContextKey) if val == nil { panic("scenario context not found in context") } sc, ok := val.(*scenarioContext) if !ok { panic("scenario context has wrong type") } return sc }

Suggested change

return ctx.Value(scenarioContextKey).(*scenarioContext)

val := ctx.Value(scenarioContextKey)

if val == nil {

panic("scenario context not found in context")

}

sc, ok := val.(*scenarioContext)

if !ok {

panic("scenario context has wrong type")

}

return sc

Makefile

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/e2e/features_test.go

test/e2e/steps/steps.go

test/e2e/features/update.feature

Copilot · 2025-12-01T18:24:38Z

test/e2e/steps/hooks.go

+			logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(func() *exec.ExitError {
+				target := &exec.ExitError{}
+				_ = errors.As(err, &target)
+				return target
+			}().Stderr))


Potential nil pointer dereference. If errors.As(err, &target) returns false, target will be nil and accessing target.Stderr will cause a panic.

Suggested fix:

var stderrStr string var exitErr *exec.ExitError if errors.As(err, &exitErr) { stderrStr = string(exitErr.Stderr) } logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", stderrStr)

Suggested change

logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(func() *exec.ExitError {

target := &exec.ExitError{}

_ = errors.As(err, &target)

return target

}().Stderr))

var stderrStr string

var exitErr *exec.ExitError

if errors.As(err, &exitErr) {

stderrStr = string(exitErr.Stderr)

}

logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", stderrStr)

joelanford

Looks like a really nice improvement overall. Just some minor comments/questions.

test/e2e/features/install.feature

joelanford · 2025-12-01T18:56:55Z

test/e2e/features/recover.feature

+                - "sleep"
+                args:
+                - "1000"
+                image: busybox:1.36


Is this the same image we've always used?

I recall there being issues in the past with rate limiting from Docker Hub.

Yes, that is the one.

I think we were hitting rate-limiting because we had the image untagged/set to latest, so it would pull every time no matter what. This is tagged so it should be fine.

true - we may want to copy it over to quay.io/operator-framework or something (assuming there's no licensing issue with that) to avoid those issues (not for this PR tho...)

We use registry.k8s.io/e2e-test-images/busybox:1.36.1-1 for downstream testing, it ought to be compatible.

We use registry.k8s.io/e2e-test-images/busybox:1.36.1-1 for downstream testing, it ought to be compatible.

I did not modify this part at all in this PR, see https://github.com/operator-framework/operator-controller/blob/main/test/e2e/cluster_extension_install_test.go#L722

joelanford · 2025-12-01T18:59:39Z

test/e2e/features/update.feature

+      """
+    Then ClusterExtension reports Progressing as True with Reason Retrying:
+      """
+      error upgrading from currently installed version "1.0.0": no bundles found for package "test" matching version "1.2.0"


Unrelated to this PR, but something we need to fix separately. This message makes it sound like 1.2.0 just doesn't exist. But it does. It just isn't a successor of the currently installed version.

Created #2385

joelanford · 2025-12-01T19:04:08Z

test/e2e/features_test.go

+	"github.com/spf13/pflag"
+	ctrl "sigs.k8s.io/controller-runtime"
+	//ctrllog "sigs.k8s.io/controller-runtime/pkg/log"
+	"sigs.k8s.io/controller-runtime/pkg/log/zap"


Avoid new zap dependency? I think we're using klog in our main.go's. Can we use that here too?

joelanford · 2025-12-01T19:12:58Z

test/e2e/steps/hooks.go

+
+func ScenarioCleanup(ctx context.Context, _ *godog.Scenario, err error) (context.Context, error) {
+	sc := scenarioCtx(ctx)
+	for _, p := range sc.backGroundCmds {


Kill and wait processes concurrently? Or does order matter?

Actually I think we do not need to wait at all.

joelanford · 2025-12-01T19:14:00Z

test/e2e/steps/hooks.go

+	}
+	forDeletion = append(forDeletion, sc.addedResources...)
+	forDeletion = append(forDeletion, resource{name: sc.namespace, kind: "namespace"})
+	for _, r := range forDeletion {


Same here: Can we delete objects concurrently?

We could try, but not sure what we are gonna gain with it?

Deletion can sometimes take a little while if finalizers need to be processed. I assume we want foreground deletion so that we can be sure cleanup is complete before we move on. Seems like it could speed up cleanup considerably in those cases.

Sure, we could do it.

joelanford · 2025-12-01T19:33:12Z

test/e2e/features/install.feature

+      error for resolved bundle "single-namespace-operator.1.0.0" with version "1.0.0":
+      invalid ClusterExtension configuration: invalid configuration: required field "watchNamespace" is missing
+      """
+    When ClusterExtension is updated


I wonder if we could strategic merge patch instead and have a smaller yaml to make it more clear what's changing?

That would be an improvement.

I wonder if we could strategic merge patch instead and have a smaller yaml to make it more clear what's changing?

We could ofcourse, we craft the grammar and the semantic: how would like to look like? If we would writing user docs, how this should be read by users?

I guess the underlying user action would be something like kubectl patch - so maybe And user patches ClusterExtension with kubectl or something like that?

I would keep it declarative as much as possible, hence mentioning kubectl would not be good.

tmshort · 2025-12-01T20:22:18Z

test/e2e/features/install.feature

+    And resource is applied
+      """
+      apiVersion: v1
+      kind: Namespace
+      metadata:
+        name: single-namespace-operator-target
+      """
+    And ClusterExtension is applied


Just nothing the inconsistency here:

resource is applied
vs

ClusterExtension is applied

The first one is generic, the second one is indicating a very specific resource. What's the reasoning behind this?

The first one is generic, the second one is indicating a very specific resource. What's the reasoning behind this?

Improved readability/focus on what matters. The fact that we use the same go code under the hood for both step is not important here - the reader should immediately understand what a step is about. Hence, we could even replace the generic step "resource is applied" with something like ([[:alnum:]]+) is applied or even ([[:alnum:]]+) is available so that we better document what is going on. Also, in this particular case, we could even create very concrete step namespace ([[:alnum:]]+) is available that is going to assure that the given namespace is created if not exists already.

tmshort · 2025-12-01T20:23:06Z

test/e2e/features/install.feature

+      error for resolved bundle "single-namespace-operator.1.0.0" with version "1.0.0":
+      invalid ClusterExtension configuration: invalid configuration: required field "watchNamespace" is missing
+      """
+    When ClusterExtension is updated


That would be an improvement.

test/e2e/features_test.go

tmshort · 2025-12-01T20:33:18Z

test/e2e/steps/hooks.go

+	forDeletion = append(forDeletion, sc.addedResources...)
+	forDeletion = append(forDeletion, resource{name: sc.namespace, kind: "namespace"})
+	for _, r := range forDeletion {
+		if _, err := kubectl("delete", r.kind, r.name, "-n", sc.namespace); err != nil {


There's no distinction here between namespace-scoped resources (e.g. SAs), and cluster-scoped resources (e.g. CE). 'kubectl' may not complain about the -n argument for cluster-scoped resources, but it's basically wrong.

test/e2e/steps/steps.go

tmshort · 2025-12-01T21:00:06Z

test/e2e/steps/steps.go

+	result := strings.ReplaceAll(content, "$TEST_NAMESPACE", sc.namespace)
+	result = strings.ReplaceAll(result, "$NAME", sc.clusterExtensionName)


I could see us wanting to expand this list, or even to make this a bit more dynamic. But probably not now.

Also noting an inconsistency in substitution mechanisms. Here it's $VAR, where as at: https://github.com/operator-framework/operator-controller/pull/2365/files#diff-37528e433a53ab946ef66fda327001b3a125c05c7ac9dfd2b49529fbfdc50cd3R378 it's {var}

Also noting an inconsistency in substitution mechanisms. Here it's $VAR, where as at: https://github.com/operator-framework/operator-controller/pull/2365/files#diff-37528e433a53ab946ef66fda327001b3a125c05c7ac9dfd2b49529fbfdc50cd3R378 it's {var}

IMO, bash-style like variables are understandable/known for a wider audience. We could use those in testdata templates as well.

I think both are known, and in BASH, the recommendation is to use ${VAR} vs $VAR, so going with {VAR} is not that much different.

tmshort · 2025-12-01T21:01:36Z

test/e2e/steps/steps.go

+)
+
+func kubectl(args ...string) (string, error) {
+	cmd := exec.Command("kubectl", args...)


Since we run these tests in other environments, we might need to consider the use of oc as a substitute for kubectl. Here and elsewhere.

added --k8s.cli command arg that can tweak that.

test/e2e/steps/steps.go

jianzhangbjz · 2025-12-02T08:21:46Z

Hi team, what’s the reason for choosing Cucumber instead of Ginkgo? What’s the background behind this decision? Thanks! I’m asking because the OpenShift QE team used the Cucumber framework for E2E testing before 2020, but it took the team almost a year to fully migrate to Ginkgo. The main reasons for choosing Ginkgo include:

Go-Native design, which fits better with Go-based projects.
Expressive BDD-Style Syntax that is cleaner and easier to maintain.
More powerful and stable test organization compared to Cucumber.
Rich Assertion Library (Gomega) that simplifies writing test expectations.
Built-in Parallelization for faster test execution.
Better for Unit and Integration Tests
Widely adopted in the Kubernetes ecosystem, making it a de-facto standard.
Progressive enhancement, with active development and community support.

pedjak · 2025-12-02T09:36:50Z

Hi team, what’s the reason for choosing Cucumber instead of Ginkgo?

@jianzhangbjz thanks for reaching out. I think I have summarized the motivation in the PR description, let me know if some of the points you raised are unanswered. IMO Ginko for e2e is still less readable for non-devs (and for devs as well)

Copilot

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-15T20:49:35Z

test/e2e/README.md

+```go
+func BundleInstalled(ctx context.Context, name, version string) error {
+    sc := scenarioCtx(ctx)
+    waitFor(ctx, func () bool {


Typo in function signature - there's an extra space in 'func () bool'. Should be 'func() bool' without the space.

Suggested change

waitFor(ctx, func () bool {

waitFor(ctx, func() bool {

Copilot · 2025-12-15T20:49:36Z

test/e2e/README.md

+All asynchronous operations use `waitFor` with consistent timeout (300s) and tick (1s):
+
+```go
+waitFor(ctx, func () bool {


Same typo as above - extra space in 'func () bool'.

Suggested change

waitFor(ctx, func () bool {

waitFor(ctx, func() bool {

Copilot · 2025-12-15T20:49:36Z

test/e2e/steps/steps.go

+		return fmt.Errorf("failed to apply RBAC configuration: %v: %s", err, string(func() *exec.ExitError {
+			target := &exec.ExitError{}
+			_ = errors.As(err, &target)
+			return target
+		}().Stderr))


Another instance of the same nil pointer dereference pattern with exec.ExitError type assertion on line 484.

Suggested change

return fmt.Errorf("failed to apply RBAC configuration: %v: %s", err, string(func() *exec.ExitError {

target := &exec.ExitError{}

_ = errors.As(err, &target)

return target

}().Stderr))

var exitErr *exec.ExitError

stderr := ""

if errors.As(err, &exitErr) && exitErr != nil {

stderr = string(exitErr.Stderr)

}

return fmt.Errorf("failed to apply RBAC configuration: %v: %s", err, stderr)

Copilot · 2025-12-15T20:49:36Z

test/e2e/steps/hooks.go

+				logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(func() *exec.ExitError {
+					target := &exec.ExitError{}
+					_ = errors.As(err, &target)
+					return target
+				}().Stderr))


Another nil pointer dereference with the exec.ExitError type assertion on line 154.

Suggested change

logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(func() *exec.ExitError {

target := &exec.ExitError{}

_ = errors.As(err, &target)

return target

}().Stderr))

logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", func() string {

var exitErr *exec.ExitError

if errors.As(err, &exitErr) && exitErr != nil {

return string(exitErr.Stderr)

}

return ""

}())

tmshort

I think it's generally fine, just a few small nits.

I think there's a semantic issue with "When", "Then" and "And", which doesn't matter in terms of the tests working, but instead how the tests are understood by humans.

It appeared to me that "When" represents a case where the tests does something (i.e. a test action), whereas "Then" and "And" represents a condition to be tested for.

This wasn't always consistent, as some actions seems to be under Then/Ands.

test/e2e/steps/steps.go

tmshort · 2025-12-15T20:57:43Z

test/e2e/steps/steps.go

+	return resp, nil
+}
+
+func SendMetricsRequest(ctx context.Context, serviceAccount string, endpoint string, controllerName string) error {


Note that his is using port-forwarding to scrape metrics, rather than a pod in the cluster (as the original test does).

Correct, the advantage is that we do not need to pull an additional image + deploy another container.

Yup, it's simply noting the change to the test process.

tmshort · 2025-12-15T20:59:46Z

test/e2e/network_policy_test.go

+func getComponentNamespace(t *testing.T, client, selector string) string {
+	cmd := exec.Command(client, "get", "pods", "--all-namespaces", "--selector="+selector, "--output=jsonpath={.items[0].metadata.namespace}") //nolint:gosec // just gathering pods for a given selector
+	output, err := cmd.CombinedOutput()
+	require.NoError(t, err, "Error determining namespace: %s", string(output))
+
+	namespace := string(bytes.TrimSpace(output))
+	if namespace == "" {
+		t.Fatal("No namespace found for selector " + selector)
+	}
+	return namespace
+}
+
+func init() {
+	cfg = ctrl.GetConfigOrDie()
+
+	var err error
+	utilruntime.Must(apiextensionsv1.AddToScheme(scheme.Scheme))
+	c, err = client.New(cfg, client.Options{Scheme: scheme.Scheme})
+	utilruntime.Must(err)
+}


Since it's just moving the function from a deleted file, that should be fine.

tmshort · 2025-12-15T21:00:56Z

test/e2e/features/status.feature

+    When resource "deployment/test-operator" reports as ready
+    Then ClusterExtension is available


But When seems to be more of an action the test takes, rather than the checking of a condition.

pedjak · 2025-12-15T21:10:38Z

This wasn't always consistent, as some actions seems to be under Then/Ands.

If there are multiple actions then scenario are structured like:

When action1
And action2
And action3

Similar for where there are multiple checks:

Then check1
And check2
And check3

tmshort · 2025-12-15T21:17:55Z

This wasn't always consistent, as some actions seems to be under Then/Ands.

If there are multiple actions then scenario are structured like:
When action1
And action2
And action3
Similar for where there are multiple checks:
Then check1
And check2
And check3

There still seem to be inconsistencies, however, but I will have to go through them since the latest changes,

Copilot

Pull request overview

Copilot reviewed 25 out of 26 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-15T21:22:18Z

test/e2e/steps/steps.go

+	}
+}
+
+func k8scli(args ...string) (string, error) {


The function name k8scli uses lowercase for CLI, which is inconsistent with typical Go naming conventions. Consider renaming to k8sCLI or k8sClient for better clarity and consistency.

Copilot · 2025-12-15T21:22:18Z

test/e2e/steps/steps.go

+		if err := portForwardCmd.Process.Kill(); err != nil {
+			return err
+		}
+		if _, err := portForwardCmd.Process.Wait(); err != nil {
+			return err
+		}


After killing the port-forward process, the code waits for it to exit. However, if Kill fails, the code still calls Wait on a process that might not exist or was not killed successfully, potentially causing a hang or panic. Check the error from Kill and handle it appropriately before calling Wait.

Copilot · 2025-12-15T21:22:18Z

test/e2e/README.md

+  Background:
+    Given OLM is available
+    And "test" catalog serves bundles
+    And Service account "olm-sa" with needed permissions is available in test namespace


In Kubernetes context, "ServiceAccount" should be written as one word (PascalCase) to match the API resource type name.

Suggested change

And Service account "olm-sa" with needed permissions is available in test namespace

And ServiceAccount "olm-sa" with needed permissions is available in test namespace

Copilot · 2025-12-15T21:22:19Z

test/e2e/steps/steps.go

+	}
+	podNameCmd := []string{"get", "pod", "-n", olmNamespace, "-o", "jsonpath={.items}"}
+	for k, v := range service.Spec.Selector {
+		podNameCmd = append(podNameCmd, fmt.Sprintf("--selector=%s=%s", k, v))


The nolint:gosec comment disables security scanning but the justification is incomplete. The command builds a pod selector from service.Spec.Selector which comes from a service fetched from the cluster. While this is generally safe, a more specific justification would be helpful, such as "selector values are validated by Kubernetes API".

Suggested change

podNameCmd = append(podNameCmd, fmt.Sprintf("--selector=%s=%s", k, v))

podNameCmd = append(podNameCmd, fmt.Sprintf("--selector=%s=%s", k, v)) //nolint:gosec // selector values are validated by Kubernetes API, so this is safe

tmshort · 2025-12-16T15:52:21Z

The code looks ok, but when I run this locally on Fedora 42 x86_64 via make test-e2e, I'm not seeing any output. Seems to be hanging. I'm also seeing two zombied kubectl processes.

3366938 pts/13   S+     0:00  |   \_ make test-e2e
3393333 pts/13   Sl+    0:00  |       \_ go test -count=1 -v ./test/e2e/...
3393342 pts/13   Sl+    0:05  |           \_ /home/tshort/sdk/go1.25.3/bin/go test -count=1 -v ./test/e2e/...
3406425 pts/13   Sl+    0:02  |               \_ /tmp/go-build554489889/b001/e2e.test -test.paniconexit0 -test.timeout=10m0s -test.count=1 -test.v=true
3409322 pts/13   Z+     0:00  |                   \_ [kubectl] <defunct>
3411033 pts/13   Z+     0:00  |                   \_ [kubectl] <defunct>

...and it finally finished, but the output comes out all at once, which isn't acceptable. We should really be able to see the progress of the test as it runs.

Ah, it was terminated via SIGQUIT for taking too long...

SIGQUIT: quit
PC=0x48b6c1 m=0 sigcode=0

...

*** Test killed with quit: ran too long (11m0s).
FAIL    github.com/operator-framework/operator-controller/test/e2e      660.022s
?       github.com/operator-framework/operator-controller/test/e2e/steps        [no test files]
FAIL
make: *** [Makefile:218: e2e] Error 1

I rand this twice... I'd like someone else to run this locally to ensure it's not just me.

pedjak · 2025-12-16T19:02:57Z

The code looks ok, but when I run this locally on Fedora 42 x86_64 via make test-e2e, I'm not seeing any output. Seems to be hanging. I'm also seeing two zombied kubectl processes

I can confirm you the same - I though first it was maybe because make buffers the output, but I fixed by slightly changing how we invoke tests to:

go test -count=1 -v ./test/e2e/features_test.go

pushed the change.

Copilot

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-16T19:08:40Z

test/e2e/steps/hooks.go

+		if c.Name == "manager" {
+			for _, arg := range c.Args {
+				if matches := featureGatePattern.FindStringSubmatch(arg); matches != nil {
+					v, _ := strconv.ParseBool(matches[2])


The error from strconv.ParseBool is ignored, which could lead to silent failures when parsing feature gate values. If parsing fails, the feature gate will remain at its default value without any indication of an error.

Suggested change

v, _ := strconv.ParseBool(matches[2])

v, err := strconv.ParseBool(matches[2])

if err != nil {

panic(fmt.Errorf("invalid boolean value for feature gate %q: %v", matches[1], err))

}

Copilot · 2025-12-16T19:08:41Z

test/e2e/steps/hooks.go

+	if err := json.Unmarshal([]byte(raw), &dl); err != nil {
+		return


The error from json.Unmarshal is silently ignored with a return statement. If unmarshaling fails, the function continues with an empty deployment list, potentially missing the OLM deployment. Consider logging the error or handling it more explicitly.

Copilot · 2025-12-16T19:08:41Z

test/e2e/steps/steps.go

+		if err := portForwardCmd.Process.Kill(); err != nil {
+			return err


The error returned from Process.Kill is not checked. If killing the process fails, the function continues to call Process.Wait on line 595, which could cause unexpected behavior or panics if the process handle is invalid.

Copilot · 2025-12-16T19:08:41Z

test/e2e/network_policy_test.go

+var (
+	cfg *rest.Config
+	c   client.Client
+)


The global variables cfg and c are declared here but were previously declared in e2e_suite_test.go (which is being removed). This creates duplicate declarations across the package. These variables should be consolidated or their scope reconsidered to avoid confusion.

Copilot · 2025-12-16T19:08:42Z

test/e2e/steps/steps.go

+	}
+	sc.metricsResponse = make(map[string]string)
+	for _, p := range pods {
+		portForwardCmd := exec.Command(k8sCli, "port-forward", "-n", p.Namespace, fmt.Sprintf("pod/%s", p.Name), fmt.Sprintf("8443:%d", metricsPort)) //nolint:gosec // perfectly safe to start port-forwarder for provided controller name


The gosec warning is suppressed with a misleading comment. The command uses fmt.Sprintf to construct arguments including user-provided values (p.Name, metricsPort), which could be exploited if these values are not properly validated. The port comes from service.Spec.Ports which should be safe, but the pod name originates from the cluster and should be validated.

Replace traditional Go e2e tests with Godog (Cucumber for Go) to improve test readability and maintainability through behavior-driven development. Changes: - Convert existing test scenarios to Gherkin feature files - Implement reusable step definitions in steps/steps.go - Add scenario hooks for setup/teardown and feature gate detection - Provide comprehensive documentation in test/e2e/README.md - Remove legacy test files (cluster_extension_install_test.go, etc.) Benefits: - Human-readable test scenarios serve as living documentation - Better separation between test specification and implementation - Easier collaboration between technical and non-technical stakeholders - Reduced code duplication through reusable step definitions Assisted-By: "Claude <noreply@anthropic.com>"

Copilot

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-16T20:26:39Z

test/e2e/network_policy_test.go

+var (
+	cfg *rest.Config
+	c   client.Client
+)


The global variables cfg and c are redeclared in this file, shadowing the same variables that were previously declared in e2e_suite_test.go. Since e2e_suite_test.go has been removed, these variables are now initialized twice (once here and once in network_policy_test.go). Consider consolidating the initialization into a shared test setup file or package-level init function to avoid duplication and potential confusion.

Suggested change

var (

cfg *rest.Config

c client.Client

)

// cfg and c are initialized in the shared test setup file (e.g., suite_test.go)

Copilot · 2025-12-16T20:26:40Z

test/e2e/steps/hooks.go

+	namespace            string
+	clusterExtensionName string
+	removedResources     []unstructured.Unstructured
+	backGroundCmds       []*exec.Cmd


The field name backGroundCmds uses inconsistent casing - it should be either backgroundCmds (camelCase) or BackgroundCmds (if exported). The uppercase 'G' in the middle is non-standard Go naming convention.

Suggested change

backGroundCmds []*exec.Cmd

backgroundCmds []*exec.Cmd

tmshort · 2025-12-16T20:48:24Z

It's working in my local repo now.
I think we want to create a Jira tickets for follow-on work:

restoring color output
getting component namespaces from BeforeSuite()
anything else we want to clean up
any updates needed for downstreaming

tmshort · 2025-12-16T20:48:44Z

/lgtm

Copilot AI review requested due to automatic review settings November 28, 2025 15:11

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 28, 2025

Copilot started reviewing on behalf of pedjak November 28, 2025 15:12 View session

Copilot finished reviewing on behalf of pedjak November 28, 2025 15:13

Copilot AI reviewed Nov 28, 2025

View reviewed changes

pedjak force-pushed the e2e-spec-by-example-godog branch from 730ba01 to 0482745 Compare November 28, 2025 15:53

Copilot AI review requested due to automatic review settings December 1, 2025 18:01

pedjak force-pushed the e2e-spec-by-example-godog branch from 0482745 to 7038e17 Compare December 1, 2025 18:01

pedjak changed the title ~~wip: migrate e2e tests to godog framework (aka BDD/Cucumber)~~ Migrate e2e tests to Godog BDD framework Dec 1, 2025

Copilot started reviewing on behalf of pedjak December 1, 2025 18:03 View session

Copilot finished reviewing on behalf of pedjak December 1, 2025 18:04

Copilot AI reviewed Dec 1, 2025

View reviewed changes

pedjak changed the title ~~Migrate e2e tests to Godog BDD framework~~ 🌱 Migrate e2e tests to Godog BDD framework Dec 1, 2025

pedjak force-pushed the e2e-spec-by-example-godog branch from 7038e17 to 844eaa1 Compare December 1, 2025 18:14

pedjak marked this pull request as ready for review December 1, 2025 18:14

pedjak requested a review from a team as a code owner December 1, 2025 18:14

Copilot AI review requested due to automatic review settings December 1, 2025 18:14

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 1, 2025

openshift-ci bot requested review from oceanc80 and perdasilva December 1, 2025 18:15

Copilot started reviewing on behalf of pedjak December 1, 2025 18:15 View session

Copilot finished reviewing on behalf of pedjak December 1, 2025 18:16

pedjak force-pushed the e2e-spec-by-example-godog branch from 844eaa1 to 63e2440 Compare December 1, 2025 18:20

Copilot AI reviewed Dec 1, 2025

View reviewed changes

joelanford reviewed Dec 1, 2025

View reviewed changes

tmshort reviewed Dec 1, 2025

View reviewed changes

Copilot started reviewing on behalf of pedjak December 15, 2025 20:37 View session

Copilot AI reviewed Dec 15, 2025

View reviewed changes

tmshort reviewed Dec 15, 2025

View reviewed changes

pedjak force-pushed the e2e-spec-by-example-godog branch from fffd715 to a7f8e25 Compare December 15, 2025 21:05

Copilot AI review requested due to automatic review settings December 15, 2025 21:11

pedjak force-pushed the e2e-spec-by-example-godog branch from a7f8e25 to 8dac36a Compare December 15, 2025 21:11

Copilot started reviewing on behalf of pedjak December 15, 2025 21:11 View session

Copilot AI reviewed Dec 15, 2025

View reviewed changes

pedjak requested a review from tmshort December 16, 2025 08:15

pedjak force-pushed the e2e-spec-by-example-godog branch from 8dac36a to 1121604 Compare December 16, 2025 13:50

Copilot AI review requested due to automatic review settings December 16, 2025 18:59

pedjak force-pushed the e2e-spec-by-example-godog branch from 1121604 to c17e8d4 Compare December 16, 2025 18:59

Copilot started reviewing on behalf of pedjak December 16, 2025 18:59 View session

Copilot AI reviewed Dec 16, 2025

View reviewed changes

pedjak force-pushed the e2e-spec-by-example-godog branch from c17e8d4 to f07ab4f Compare December 16, 2025 19:41

Copilot AI review requested due to automatic review settings December 16, 2025 20:17

pedjak force-pushed the e2e-spec-by-example-godog branch from f07ab4f to bf10225 Compare December 16, 2025 20:17

Copilot started reviewing on behalf of pedjak December 16, 2025 20:18 View session

Copilot AI reviewed Dec 16, 2025

View reviewed changes

pedjak changed the title ~~🌱 Migrate e2e tests to Godog BDD framework~~ 🌱 OPRUN-4261 Migrate e2e tests to Godog BDD framework Dec 16, 2025

openshift-ci bot assigned tmshort Dec 16, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Dec 16, 2025

openshift-merge-bot bot merged commit aaffdeb into operator-framework:main Dec 16, 2025
33 checks passed

-		if stdErr := string(func() *exec.ExitError {
-			target := &exec.ExitError{}
-			_ = errors.As(err, &target)
-			return target
-		}().Stderr); !strings.Contains(stdErr, errMsg) {
+		var exitErr *exec.ExitError
+		if errors.As(err, &exitErr) {
+			stdErr := string(exitErr.Stderr)
+			if !strings.Contains(stdErr, errMsg) {
+				return false
+			}
+			return true
+		}
+		return false

		if _, err := kubectl("delete", r.kind, r.name, "-n", sc.namespace); err != nil {
		logger.Info("Error deleting resource", "name", r.name, "namespace", sc.namespace, "stderr", string(err.(*exec.ExitError).Stderr))

-	return ctx.Value(scenarioContextKey).(*scenarioContext)
+	val := ctx.Value(scenarioContextKey)
+	if val == nil {
+		panic("scenario context not found in context")
+	}
+	sc, ok := val.(*scenarioContext)
+	if !ok {
+		panic("scenario context has wrong type")
+	}
+	return sc

		result := strings.ReplaceAll(content, "$TEST_NAMESPACE", sc.namespace)
		result = strings.ReplaceAll(result, "$NAME", sc.clusterExtensionName)

		When resource "deployment/test-operator" reports as ready
		Then ClusterExtension is available

	And Service account "olm-sa" with needed permissions is available in test namespace
	And ServiceAccount "olm-sa" with needed permissions is available in test namespace

	podNameCmd = append(podNameCmd, fmt.Sprintf("--selector=%s=%s", k, v))
	podNameCmd = append(podNameCmd, fmt.Sprintf("--selector=%s=%s", k, v)) //nolint:gosec // selector values are validated by Kubernetes API, so this is safe

-					v, _ := strconv.ParseBool(matches[2])
+					v, err := strconv.ParseBool(matches[2])
+					if err != nil {
+						panic(fmt.Errorf("invalid boolean value for feature gate %q: %v", matches[1], err))
+					}

		if err := json.Unmarshal([]byte(raw), &dl); err != nil {
		return

		if err := portForwardCmd.Process.Kill(); err != nil {
		return err

🌱 OPRUN-4261 Migrate e2e tests to Godog BDD framework #2365

🌱 OPRUN-4261 Migrate e2e tests to Godog BDD framework #2365

Uh oh!

Conversation

pedjak commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Migration Notes

Reviewer Checklist

Uh oh!

netlify bot commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for olmv1 ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

joelanford left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pedjak commented Nov 28, 2025 •

edited

Loading

netlify bot commented Nov 28, 2025 •

edited

Loading

codecov bot commented Nov 28, 2025 •

edited

Loading