test: enable ui e2e tests in host, member, and registration-service repos #1238
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WalkthroughA conditional check that previously gated UI E2E tests behind a REPO_NAME filter is removed from the test-e2e target in make/test.mk. The tests now run unconditionally on CI, with simplified comments reflecting this change. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
make/test.mk(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: rsoaresd
Repo: codeready-toolchain/toolchain-e2e PR: 1232
File: make/devsandbox-dashboard.mk:57-57
Timestamp: 2025-12-11T16:29:34.403Z
Learning: In make/devsandbox-dashboard.mk, the test-devsandbox-dashboard-e2e-local target intentionally allows PUBLISH_UI=true and DEPLOY_UI=true to enable publishing and deploying when running locally (outside a container), while test-devsandbox-dashboard-in-container sets PUBLISH_UI=false because image pushing is not available from inside the container.
📚 Learning: 2025-12-11T16:29:34.403Z
Learnt from: rsoaresd
Repo: codeready-toolchain/toolchain-e2e PR: 1232
File: make/devsandbox-dashboard.mk:57-57
Timestamp: 2025-12-11T16:29:34.403Z
Learning: In make/devsandbox-dashboard.mk, the test-devsandbox-dashboard-e2e-local target intentionally allows PUBLISH_UI=true and DEPLOY_UI=true to enable publishing and deploying when running locally (outside a container), while test-devsandbox-dashboard-in-container sets PUBLISH_UI=false because image pushing is not available from inside the container.
Applied to files:
make/test.mk
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Unit Tests
- GitHub Check: GolangCI Lint
- GitHub Check: Build & push operator bundles & dashboard image for e2e tests
Comment on lines
48
to
53
| ifeq ($(CI),true) | ||
| # if we are running on CI, we want to run the ui e2e tests in the toolchain-e2e presubmit and periodic CI jobs | ||
| # if REPO_NAME is not set, it means that the e2e tests were triggered by the periodic CI job | ||
| ifeq ($(filter-out toolchain-e2e,$(REPO_NAME)),) | ||
| # if we are running on CI, we want to run the ui e2e tests | ||
| $(MAKE) test-devsandbox-dashboard-e2e | ||
| @echo "UI E2E tests successfully finished" | ||
| @echo "To clean the Developer Sandbox Dashboard run 'make clean-devsandbox-dashboard'" | ||
| endif | ||
| endif |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Examine UI E2E test targets and REPO_NAME conditionals in make/test.mk
echo "=== Lines 48-53 from make/test.mk ==="
sed -n '48,53p' make/test.mk
echo ""
echo "=== Search for test-devsandbox-dashboard-e2e target definition ==="
rg -A 15 "test-devsandbox-dashboard-e2e:" --type=make
echo ""
echo "=== Check for REPO_NAME references in make/test.mk ==="
rg "REPO_NAME" make/test.mk -B 2 -A 2
echo ""
echo "=== Search for any REPO_NAME conditionals in test-related make files ==="
rg -A 3 "ifeq.*REPO_NAME" make/ --type=make | head -40
echo ""
echo "=== Check what CI environment variables and conditions are used ==="
rg "CI.*true" make/test.mk -B 2 -A 5Repository: codeready-toolchain/toolchain-e2e
Length of output: 2263
🏁 Script executed:
#!/bin/bash
# Check the dependencies and configuration of the UI E2E test targets
echo "=== Full test-devsandbox-dashboard-e2e target definition ==="
rg -B 5 -A 20 "^test-devsandbox-dashboard-e2e:" make/devsandbox-dashboard.mk
echo ""
echo "=== Get-and-publish-devsandbox-dashboard target ==="
rg -B 2 -A 15 "^get-and-publish-devsandbox-dashboard:" make/devsandbox-dashboard.mk
echo ""
echo "=== e2e-run-devsandbox-dashboard target ==="
rg -B 2 -A 10 "^e2e-run-devsandbox-dashboard:" make/devsandbox-dashboard.mk
echo ""
echo "=== Check if there are any REPO_NAME/repo-specific conditions in devsandbox-dashboard.mk ==="
rg "REPO_NAME|ifeq.*REPO|ifdef.*REPO" make/devsandbox-dashboard.mk || echo "No REPO_NAME conditionals found"
echo ""
echo "=== Check if get-and-publish target has any repo filtering ==="
rg -B 3 -A 15 "get-and-publish" make/devsandbox-dashboard.mk | head -50Repository: codeready-toolchain/toolchain-e2e
Length of output: 6018
🏁 Script executed:
#!/bin/bash
# Check PUBLISH_UI and DEPLOY_UI settings and CI context
echo "=== Check how test-devsandbox-dashboard-e2e is called with PUBLISH_UI/DEPLOY_UI ==="
rg "PUBLISH_UI|DEPLOY_UI" make/devsandbox-dashboard.mk | head -20
echo ""
echo "=== Check if these variables are set conditionally anywhere in make/test.mk ==="
rg "PUBLISH_UI|DEPLOY_UI" make/test.mk
echo ""
echo "=== Check the manage-devsandbox-dashboard.sh script to understand parameter handling ==="
head -60 scripts/ci/manage-devsandbox-dashboard.sh
echo ""
echo "=== Check if there are any CI-specific conditions setting PUBLISH_UI/DEPLOY_UI ==="
rg -B 5 "test-devsandbox-dashboard-e2e" make/ --type=make | grep -E "PUBLISH_UI|DEPLOY_UI|ifeq|ifneq"Repository: codeready-toolchain/toolchain-e2e
Length of output: 3285
🏁 Script executed:
#!/bin/bash
# Check git history to understand if REPO_NAME conditional was actually removed
echo "=== Check recent git history for make/test.mk ==="
git log --oneline -10 -- make/test.mk
echo ""
echo "=== Show the diff for make/test.mk to see what changed ==="
git diff HEAD~1 -- make/test.mk 2>/dev/null || echo "No previous commit available"
echo ""
echo "=== Check if there are any repo-specific test targets elsewhere ==="
rg "test-.*-e2e" make/ --type=make | grep -v "devsandbox" | head -20
echo ""
echo "=== Verify the default values for PUBLISH_UI and DEPLOY_UI across all make files ==="
rg "PUBLISH_UI|DEPLOY_UI" make/ --type=make | grep -E "^\s*[A-Z_]+ \?=" | sort | uniqRepository: codeready-toolchain/toolchain-e2e
Length of output: 586
🏁 Script executed:
#!/bin/bash
# Examine the git commit to see what changed
echo "=== Show the commit details for 'run ui e2e tests' ==="
git show cadd731 --stat
echo ""
echo "=== Show the actual diff of the commit ==="
git show cadd731 -- make/test.mk
echo ""
echo "=== Check if there are CI variables and how they interact ==="
grep -n "^CI" make/test.mk || echo "No CI variable definition in make/test.mk"
echo ""
echo "=== Look for any conditional checks on repository context in devsandbox-dashboard.mk ==="
rg -n "ifeq|ifneq|ifdef|ifndef" make/devsandbox-dashboard.mk | head -30Repository: codeready-toolchain/toolchain-e2e
Length of output: 41603
🏁 Script executed:
#!/bin/bash
# Final verification: check if there are any other branches or history that might show previous versions
echo "=== Check git log for any history before this commit ==="
git log --all --oneline | head -5
echo ""
echo "=== Verify this is truly the first commit in the repository ==="
git rev-list --all --count
echo ""
echo "=== Check if there's any reference to REPO_NAME in other files ==="
rg "REPO_NAME" . --type make --type sh | head -20Repository: codeready-toolchain/toolchain-e2e
Length of output: 1754
🏁 Script executed:
#!/bin/bash
# Check the previous version of make/test.mk to see if REPO_NAME was there before
echo "=== Show the parent commit to see what was there before ==="
git show b1147e8:make/test.mk | sed -n '48,60p' 2>/dev/null || echo "File might not exist in parent commit"
echo ""
echo "=== Check if make/test.mk existed in the parent commit ==="
git show b1147e8 -- make/test.mk 2>&1 | head -20
echo ""
echo "=== Look at the actual change - check what was in b1147e8 for the test-e2e target ==="
git show b1147e8:make/test.mk 2>/dev/null | grep -A 10 "^\.PHONY: test-e2e" || echo "Not found in b1147e8"
echo ""
echo "=== Check the diff between b1147e8 and cadd731 for make/test.mk ==="
git diff b1147e8 cadd731 -- make/test.mk | head -80Repository: codeready-toolchain/toolchain-e2e
Length of output: 5656
Unconditional UI E2E tests now enabled in all CI repositories.
The removal of the REPO_NAME filter allows UI E2E tests to run in all repository contexts (toolchain-e2e, host-operator, member-operator, registration-service) when CI=true. This correctly implements the PR objective.
The underlying test-devsandbox-dashboard-e2e target depends on get-and-publish-devsandbox-dashboard and e2e-run-devsandbox-dashboard, both of which rely on default values PUBLISH_UI=true and DEPLOY_UI=true. Verify that:
- The UI deployment and infrastructure are consistently available across all target repository CI pipelines
- Required environment variables and configurations are properly set in CI for all repositories running these tests
alexeykazakov
approved these changes
Dec 16, 2025
|
xcoulon
approved these changes
Dec 17, 2025
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alexeykazakov, mfrancisc, rsoaresd, xcoulon The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
We need to also enable UI E2E Tests in host, member, and registration-service repos
Issue ticket number and link
SANDBOX-1542
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.