ci(medium): Implement Gemini Multimodal UI Review

[arii]

Description

Implemented the @gemini-ui-review command to provide automated multimodal UI reviews for HRM. The implementation includes a Playwright script for screenshot capture, a Gemini API integration for analysis, and a GitHub Action workflow orchestrated by the central bot command router. Authentication is seamlessly handled through mocked sessions.

Fixes #9576

Change Type: ✨ New feature (non-breaking change adding functionality)

Related Issues

Closes #9576

Original PR Body

Implemented the @gemini-ui-review command to provide automated multimodal UI reviews for HRM. The implementation includes a Playwright script for screenshot capture, a Gemini API integration for analysis, and a GitHub Action workflow orchestrated by the central bot command router. Authentication is seamlessly handled through mocked sessions.

Fixes #9576

---

PR created automatically by Jules for task 9167995831481264948 started by @arii

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[github-actions]

👋 Welcome to HRM!

Thanks for your contribution. This repository uses Gemini AI for automated triage, code review, and generation.

🤖 Gemini Manual Trigger Quick Reference

Command	Action
@gemini-bot	Run AI Code Review (PR only)
@gemini-enrich	Run PR Enrichment (PR only)
@gemini-triage	Run Issue Triage
@gemini-coder <task>	Generate Code
@create-review-issues	Create issues from review (PR only)
@gemini-help	Show this help message
@pr-squash	Squash PR commits (PR only)
@conflict-resolve	Resolve merge conflicts (PR only)

For more details and GitHub CLI examples, see the Manual Trigger Guide.

[github-actions]

ℹ️ Gemini Review Disabled

Automatic review is currently disabled via GEMINI_ENABLE_PR_REVIEW. To run review manually:

Via Comment:

@gemini-bot

Via GitHub Actions:

1. Go to Actions → Reusable Gemini Review
2. Click "Run workflow"
3. Enter PR number: 9577

Via GitHub CLI:

gh workflow run reusable-gemini-review.yml -f pr_number=9577

🤖 Gemini Manual Trigger Guide

To re-enable: Set repository variable GEMINI_ENABLE_PR_REVIEW=true

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[arii]

🤖 AI Technical Audit

Deep Architectural Review: Gemini Multimodal UI Review

Summary

This PR introduces an automated UI review cycle using Gemini 1.5 Pro and Playwright. While the integration into comment-ops.yml is clean, the script implementation has several reliability and maintainability issues, particularly around environment handling and error reporting.

Architectural Impact

• CI Load: Introducing a pnpm run build and a full Playwright environment in a GitHub Action triggered by comments will significantly increase CI runner usage. This should be monitored for cost and queue times.
• Authentication Strategy: Relying on mockLoggedInSession for a production-built app in CI is fragile. It assumes the production build respects the same test-injection hooks as the development/test builds.

File-by-File Analysis

1. .github/workflows/gemini-ui-review.yml

• Problem: The workflow runs pnpm run build every time the command is called. For a UI review, this is incredibly slow.
• Implementation Sample:

  # Instead of building every time, check for an existing deployment URL
  - name: Set Target URL
    run: echo "TARGET_URL=${{ github.event.deployment_status.target_url || 'http://localhost:3000' }}" >> $GITHUB_ENV

2. scripts/ci/run-ui-review.ts

• Problem: Hardcoded waitForTimeout(2000) is a known anti-pattern that leads to flaky CI. It doesn't guarantee the UI is ready for analysis.
• Implementation Sample:

  // Replace waitForTimeout with specific element visibility
  await page.waitForSelector('[data-testid="main-dashboard"]', { state: 'visible' });

• Problem: The script fails silently regarding GitHub comment creation if PR_NUMBER is missing, but still consumes Gemini tokens.

3. tests/playwright/lib/mocks.ts

• Problem: The change from @ts-expect-error to any casting is a regression in type safety.
• Implementation Sample:

  // Better: Define a proper interface for the window extension
  interface TestWindow extends Window {
    __TEST_CONTROLS__?: { dispatch: (action: any) => void };
  }
  const win = window as unknown as TestWindow;

---

ANTI-AI-SLOP DIRECTIVES

1. OVERLY VERBOSE COMMENTS:
   • scripts/ci/run-ui-review.ts: "Mock the session to ensure we are logged in" - The function name mockLoggedInSession already says this.
   • scripts/ci/run-ui-review.ts: "Additional wait for any dynamic content/animations to settle" - This comment justifies a bad practice (static wait).
2. OVER-ENGINEERING:
   • The use of @octokit/rest as a full dependency in package.json just to post a single comment. GitHub Actions already provide gh CLI which can post comments without extra JS dependencies.
3. DUPLICATE HOOKS/TYPES:
   • The script re-implements screenshot logic that likely exists in the Playwright test suite. It should reuse takeScreenshot utilities if available.
4. CODE RATIO:
   • Deletion Candidate: The if (!owner || !repo) block in run-ui-review.ts is redundant because GitHub Actions' environment variables are guaranteed if the script is running in that context.
   • Deletion Candidate: Remove the SPOTIFY_CLIENT_ID mocks in the YAML; they are likely not needed for a simple UI visual check unless the app crashes without them.
5. STALE FEATURES:
   • The PR adds is_ui_review but does not check if older "Visual Regression" jobs in the repo are now redundant or should be merged.

Review automatically published via RepoAuditor.

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[arii]

🤖 AI Technical Audit

🏗️ Architectural Impact

This PR introduces an automated multimodal UI review pipeline using Gemini 1.5 Pro. It successfully integrates Playwright for screenshot capture and GH Actions for orchestration. However, there are significant security and efficiency concerns regarding the build process and local deployment triggers.

🔍 File-by-File Analysis

.github/workflows/gemini-ui-review.yml

• Problem: The workflow runs pnpm run build and starts the app locally every time a deployment URL isn't found. This adds ~5-7 minutes of CI time and consumes significant runners. Furthermore, starting a production build in background (&) without proper process cleanup can lead to orphaned processes.
• Implementation Sample:

# Instead of manual build/start, use a cached build or dedicated preview environment
- name: Run UI Review
  run: |
    if [ "${{ steps.deploy-url.outputs.url }}" == "http://localhost:3000" ]; then
      echo "Error: UI Review requires a deployed preview URL to avoid expensive CI builds."
      exit 1
    fi

scripts/ci/run-ui-review.ts

• Problem: Error handling when parsing Gemini's response is fragile. While responseMimeType: 'application/json' is used, JSON.parse is called directly on result.response.text() without verifying if the AI returned a code block or raw string.
• Implementation Sample:

// Safer parsing
const responseText = result.response.text().replace(/```json|```/g, '').trim();
try {
  const responseJson = JSON.parse(responseText);
  // ...
} catch (e) {
  throw new Error(`Failed to parse Gemini JSON: ${responseText}`);
}

comments_raw.json

• Problem: This file contains a 401 Unauthorized error message and appears to be a leftover debug artifact from a local failed API call.
• Action: Delete this file immediately.

🛡️ Anti-AI-Slop Directives

1. OVERLY VERBOSE COMMENTS: scripts/ci/run-ui-review.ts L11-L15: Comments like "Starting Gemini Multimodal UI Review..." and "Navigating to..." add no value to logs that already output command execution.
2. OVER-ENGINEERING: The workflow tries to dynamically fetch deployment URLs via gh api and jq pipes that are prone to failure if the deployment naming convention changes. It's safer to rely on standard GH environment variables or set the URL as an input.
3. DUPLICATE HOOKS/TYPES: The TestWindow interface in tests/playwright/lib/mocks.ts is redefined locally. This should be moved to types/global.d.ts as it's a common pattern for Playwright testing in this repo.
4. CODE RATIO: Total lines added: ~200.
   • Lines to remove:
     • comments_raw.json (5 lines) - Artifact.
     • scripts/ci/run-ui-review.ts L101-105 (5 lines) - Redundant fs.existsSync checks and repetitive console logs.
5. STALE FEATURES: N/A. This is a net-new command functionality.

🚦 GitHub Checks Correlation

All checks passed, but note that Execute Gemini Review was skipped. This confirms the routing logic in comment-ops.yml is working as intended (only triggering when the specific @gemini-ui-review string is found).

Review automatically published via RepoAuditor.

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[arii]

@gemini-ui-review

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide

[arii]

🤖 AI Technical Audit

PR Review: Gemini Multimodal UI Review (#9577)

Summary

This PR introduces a multimodal UI review command (@gemini-ui-review) using Playwright for visual capture and Gemini 1.5 Pro for analysis. While the integration into the comment-ops.yml router is solid, the implementation has significant TypeScript errors and architectural redundancies that must be addressed.

Anti-AI-Slop Directives

1. OVERLY VERBOSE COMMENTS: The comment at .github/workflows/comment-ops.yml:38 is redundant. It explains what grep -qiE and ^ do, which is standard regex knowledge for CI engineers.
2. OVER-ENGINEERING: The script uses GoogleGenerativeAI with a rigid responseSchema for a single-field response (analysis). It is simpler to request a raw string from Gemini and wrap it in Markdown manually.
3. DUPLICATE HOOKS/TYPES: types/global.d.ts adds TestWindow. This interface is already standard in most Playwright/Cypress setups; check tests/playwright/global.d.ts (likely exists) or use window as any in the single script where it's used to avoid global pollution.
4. CODE RATIO: Total lines added: ~160. 12 lines can be deleted by refactoring the comment-ops.yml router. The repetitive echo "is_command_triggered=true" >> $GITHUB_OUTPUT across every if block is highly inefficient.
5. STALE FEATURES: None. This is a net-new feature.

File-by-File Analysis

.github/workflows/comment-ops.yml

Problem: Extreme duplication in the router logic. Every command now sets is_command_triggered. This should be handled once at the end of the script.
Implementation Sample:

# Remove is_command_triggered from every 'if' block. Add this at the end:
if grep -q "true" <<< "$GITHUB_OUTPUT"; then
  echo "is_command_triggered=true" >> $GITHUB_OUTPUT
fi

scripts/ci/run-ui-review.ts

Problem: Critical Failure. The GITHUB CHECKS show Type Check and Build failures. This script imports mockLoggedInSession from ../../tests/playwright/lib/mocks, which likely contains TSX/JSX or browser-specific types that tsx cannot resolve without a proper tsconfig for the scripts directory. Furthermore, the responseSchema implementation for Gemini 1.5 Pro requires specific versioning in the SDK which might mismatch with package.json.
Implementation Sample:

// Avoid importing heavy test mocks into a CI script if possible.
// Instead, use direct cookie/localStorage injection to minimize dependencies.
await context.addCookies([{ name: 'session-id', value: 'mock', url: targetUrl }]);

types/global.d.ts

Problem: Adding TestWindow globally for a CI script is bad practice. This affects the entire project's IDE intellisense for a feature only used in one script.
Implementation Sample:

// Delete from global.d.ts and cast locally in run-ui-review.ts
(page as any).evaluate(() => window.__TEST_READY__ = true);

Architectural Impact

• CI Latency: Adding npx playwright install --with-deps chromium adds ~45-60s to the comment-ops workflow. This should be cached or moved to a specialized container.
• Security: The workflow uses secrets: inherit for a script that executes external commands. Ensure GEMINI_API_KEY has restricted quota to prevent abuse via PR comments.

GitHub Checks Correlation

• Type Check Failure: Confirmed. The run-ui-review.ts imports from the tests/ directory are causing resolution issues because tsx doesn't inherit the test-specific TS configuration.
• Build Check Failure: Likely caused by the same module resolution issue.

Review automatically published via RepoAuditor.

[github-actions]

🤖 Gemini Review Skipped

Review was skipped for the following reason: Gemini review is globally disabled

🤖 Gemini Manual Trigger Guide