Skip to content

ci: optimize CI workflows and Playwright tests#1264

Open
Huijiro wants to merge 2 commits intomainfrom
ci/optimize-playwright-tests
Open

ci: optimize CI workflows and Playwright tests#1264
Huijiro wants to merge 2 commits intomainfrom
ci/optimize-playwright-tests

Conversation

@Huijiro
Copy link
Copy Markdown
Member

@Huijiro Huijiro commented Mar 24, 2026

  • Add Bun dependency caching to all workflow jobs (~30-60s per job)
  • Add Playwright browser caching with conditional install (~30-45s per job)
  • Reduce Playwright retries from 2 to 1 (10-20% test time savings)
  • Enable full parallelism for framework tests (was sequential)
  • Start all framework demo servers in parallel instead of sequentially
  • Reduce global-setup retries from 60 to 30 for faster failure detection
  • Make Playwright workers configurable via PLAYWRIGHT_WORKERS env var

Expected overall CI time reduction: ~15-20%

Summary by CodeRabbit

  • Chores
    • Added dependency and build artifact caching to CI workflows for faster build times.
    • Enabled parallel test execution and optimized test configuration for improved CI efficiency.
    • Reduced retry attempts in CI tests from 2 to 1 and made worker count configurable.
    • Streamlined test framework demo script execution to run servers concurrently.

- Add Bun dependency caching to all workflow jobs (~30-60s per job)
- Add Playwright browser caching with conditional install (~30-45s per job)
- Reduce Playwright retries from 2 to 1 (10-20% test time savings)
- Enable full parallelism for framework tests (was sequential)
- Start all framework demo servers in parallel instead of sequentially
- Reduce global-setup retries from 60 to 30 for faster failure detection
- Make Playwright workers configurable via PLAYWRIGHT_WORKERS env var

Expected overall CI time reduction: ~15-20%
@agentuity-agent
Copy link
Copy Markdown

agentuity-agent bot commented Mar 24, 2026

The latest Agentuity deployment details.

Project Deployment Preview Updated (UTC)
docs 🔴 Failed (deploy_715b7a2bb4e6547d5d3487ec9782ddef) - 2026-03-24T17:42:57Z

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 24, 2026

📝 Walkthrough

Walkthrough

Updates to testing infrastructure and CI workflows: Bun and Playwright caching added to GitHub workflows; Playwright retry counts reduced from 2 to 1 in CI; worker configuration made dynamic via environment variables; readiness check attempts reduced from 60 to 30; test framework script refactored for concurrent server execution.

Changes

Cohort / File(s) Summary
CI/CD Caching
.github/workflows/build.yaml, .github/workflows/package-smoke-test.yaml
Added caching for Bun dependencies and Playwright browser binaries using actions/cache@v4 with OS-scoped keys; Playwright browser installation is conditional on cache miss.
Playwright Configuration
playwright.config.ts, playwright.frameworks.config.ts
Reduced CI retries from 2 to 1; made worker count configurable via PLAYWRIGHT_WORKERS environment variable; enabled full parallel test execution.
Test Setup & Execution
e2e/global-setup.ts, scripts/test-framework-demos.sh
Reduced readiness check attempts from 60 to 30; refactored test script for concurrent server startup with unified cleanup and single Playwright invocation.
🚥 Pre-merge checks | ✅ 1
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 24, 2026

📦 Canary Packages Published

version: 1.0.56-ef2bd3a

Packages
Package Version URL
@agentuity/react 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-react-1.0.56-ef2bd3a.tgz
@agentuity/queue 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-queue-1.0.56-ef2bd3a.tgz
@agentuity/workbench 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-workbench-1.0.56-ef2bd3a.tgz
@agentuity/task 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-task-1.0.56-ef2bd3a.tgz
@agentuity/core 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-core-1.0.56-ef2bd3a.tgz
@agentuity/runtime 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-runtime-1.0.56-ef2bd3a.tgz
@agentuity/vector 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-vector-1.0.56-ef2bd3a.tgz
@agentuity/coder 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-coder-1.0.56-ef2bd3a.tgz
@agentuity/db 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-db-1.0.56-ef2bd3a.tgz
@agentuity/frontend 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-frontend-1.0.56-ef2bd3a.tgz
@agentuity/opencode 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-opencode-1.0.56-ef2bd3a.tgz
@agentuity/evals 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-evals-1.0.56-ef2bd3a.tgz
@agentuity/email 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-email-1.0.56-ef2bd3a.tgz
@agentuity/cli 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-cli-1.0.56-ef2bd3a.tgz
@agentuity/postgres 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-postgres-1.0.56-ef2bd3a.tgz
@agentuity/claude-code 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-claude-code-1.0.56-ef2bd3a.tgz
@agentuity/drizzle 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-drizzle-1.0.56-ef2bd3a.tgz
@agentuity/sandbox 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-sandbox-1.0.56-ef2bd3a.tgz
@agentuity/keyvalue 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-keyvalue-1.0.56-ef2bd3a.tgz
@agentuity/auth 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-auth-1.0.56-ef2bd3a.tgz
@agentuity/schedule 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schedule-1.0.56-ef2bd3a.tgz
@agentuity/server 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-server-1.0.56-ef2bd3a.tgz
@agentuity/schema 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schema-1.0.56-ef2bd3a.tgz
@agentuity/webhook 1.0.56-ef2bd3a https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-webhook-1.0.56-ef2bd3a.tgz
Install

Add to your package.json:

{
  "dependencies": {
    "@agentuity/react": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-react-1.0.56-ef2bd3a.tgz",
    "@agentuity/queue": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-queue-1.0.56-ef2bd3a.tgz",
    "@agentuity/workbench": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-workbench-1.0.56-ef2bd3a.tgz",
    "@agentuity/task": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-task-1.0.56-ef2bd3a.tgz",
    "@agentuity/core": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-core-1.0.56-ef2bd3a.tgz",
    "@agentuity/runtime": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-runtime-1.0.56-ef2bd3a.tgz",
    "@agentuity/vector": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-vector-1.0.56-ef2bd3a.tgz",
    "@agentuity/coder": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-coder-1.0.56-ef2bd3a.tgz",
    "@agentuity/db": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-db-1.0.56-ef2bd3a.tgz",
    "@agentuity/frontend": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-frontend-1.0.56-ef2bd3a.tgz",
    "@agentuity/opencode": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-opencode-1.0.56-ef2bd3a.tgz",
    "@agentuity/evals": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-evals-1.0.56-ef2bd3a.tgz",
    "@agentuity/email": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-email-1.0.56-ef2bd3a.tgz",
    "@agentuity/cli": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-cli-1.0.56-ef2bd3a.tgz",
    "@agentuity/postgres": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-postgres-1.0.56-ef2bd3a.tgz",
    "@agentuity/claude-code": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-claude-code-1.0.56-ef2bd3a.tgz",
    "@agentuity/drizzle": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-drizzle-1.0.56-ef2bd3a.tgz",
    "@agentuity/sandbox": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-sandbox-1.0.56-ef2bd3a.tgz",
    "@agentuity/keyvalue": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-keyvalue-1.0.56-ef2bd3a.tgz",
    "@agentuity/auth": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-auth-1.0.56-ef2bd3a.tgz",
    "@agentuity/schedule": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schedule-1.0.56-ef2bd3a.tgz",
    "@agentuity/server": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-server-1.0.56-ef2bd3a.tgz",
    "@agentuity/schema": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schema-1.0.56-ef2bd3a.tgz",
    "@agentuity/webhook": "https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-webhook-1.0.56-ef2bd3a.tgz"
  }
}

Or install directly:

bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-react-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-queue-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-workbench-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-task-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-core-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-runtime-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-vector-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-coder-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-db-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-frontend-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-opencode-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-evals-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-email-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-cli-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-postgres-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-claude-code-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-drizzle-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-sandbox-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-keyvalue-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-auth-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schedule-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-server-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-schema-1.0.56-ef2bd3a.tgz
bun add https://agentuity-sdk-objects.t3.storageapi.dev/npm/1.0.56-ef2bd3a/agentuity-webhook-1.0.56-ef2bd3a.tgz

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (5)
e2e/global-setup.ts (2)

104-106: Use StructuredError for error handling.

The code throws a plain Error, but the coding guidelines require using StructuredError from @agentuity/core for error handling. As per coding guidelines: Use StructuredError from @agentuity/core for error handling.

♻️ Refactor to use StructuredError

At the top of the file, add the import:

 import { execSync } from 'node:child_process';
 import { join } from 'node:path';
+import { StructuredError } from '@agentuity/core';

Then update the error:

-	throw new Error(
-		`Dev server did not become ready after ${MAX_RETRIES} attempts (${MAX_RETRIES * RETRY_DELAY}ms)`
-	);
+	throw new StructuredError(
+		'DEV_SERVER_NOT_READY',
+		`Dev server did not become ready after ${MAX_RETRIES} attempts (${MAX_RETRIES * RETRY_DELAY}ms)`,
+		{ maxRetries: MAX_RETRIES, totalTimeout: MAX_RETRIES * RETRY_DELAY }
+	);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@e2e/global-setup.ts` around lines 104 - 106, Replace the plain Error throw
with a StructuredError: add an import for StructuredError from '@agentuity/core'
at the top of the file, then replace the throw new Error(...) at the dev-server
readiness timeout with throw new StructuredError(...) supplying a clear message
(same text), a short machine-friendly code like "DEV_SERVER_TIMEOUT", and
metadata containing MAX_RETRIES and RETRY_DELAY (or total ms) so the error is
structured and contains the same info; keep the existing message text in the
StructuredError's message field.

12-12: LGTM: Reasonable reduction for faster failure detection.

Reducing from 60 to 30 attempts is well-aligned with the PR objectives. Given the 1-second delay between attempts and 5-10 second timeouts per HTTP request, 30 attempts still provides substantial wait time (30+ seconds minimum, potentially much longer if requests timeout). This should be sufficient for typical Vite dev server startup while failing faster when there are real issues.

Optional enhancement: Consider making this configurable via an environment variable (e.g., MAX_SETUP_RETRIES) for consistency with the PLAYWRIGHT_WORKERS approach used elsewhere in this PR.

🔧 Optional: Make MAX_RETRIES configurable
-const MAX_RETRIES = 30;
+const MAX_RETRIES = parseInt(process.env.MAX_SETUP_RETRIES || '30', 10);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@e2e/global-setup.ts` at line 12, Replace the hardcoded MAX_RETRIES constant
with a configurable value sourced from an environment variable (e.g.,
MAX_SETUP_RETRIES) so tests can override retry count; specifically, change the
declaration of MAX_RETRIES to read process.env.MAX_SETUP_RETRIES, parse it as an
integer, fall back to 30 when missing or invalid, and ensure the value is a
positive integer before use (refer to the MAX_RETRIES symbol in global-setup.ts
and the surrounding retry logic).
scripts/test-framework-demos.sh (2)

178-184: Consider using an array for project arguments.

Using an unquoted $PROJECT_ARGS relies on word splitting, which works here but is fragile. Using a Bash array would be more robust and idiomatic.

💡 Optional: Use array for project arguments
-PROJECT_ARGS=""
-for proj in "${PROJECTS[@]}"; do
-	PROJECT_ARGS="$PROJECT_ARGS --project=$proj"
-done
-
-cd "$SDK_ROOT"
-bun run playwright test --config=playwright.frameworks.config.ts $PROJECT_ARGS
+PROJECT_ARGS=()
+for proj in "${PROJECTS[@]}"; do
+	PROJECT_ARGS+=("--project=$proj")
+done
+
+cd "$SDK_ROOT"
+bun run playwright test --config=playwright.frameworks.config.ts "${PROJECT_ARGS[@]}"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/test-framework-demos.sh` around lines 178 - 184, The current script
builds PROJECT_ARGS as a quoted string which relies on word-splitting; change to
a Bash array (e.g., PROJECT_ARGS_ARRAY) and push each "--project=$proj" into it
while iterating over PROJECTS, then invoke the test command with the array
expanded as "${PROJECT_ARGS_ARRAY[@]}" so each project argument is preserved
safely; update references to PROJECT_ARGS to use the new array variable where
the bun run playwright test command is executed.

145-161: Minor: Comment says "parallel" but waiting is sequential.

The comment on line 145 says "Wait for all servers in parallel" but the wait_for_server calls execute sequentially. This works correctly but the comment is misleading. If one server is slow to start, the others won't be checked until it completes.

For true parallel waiting, you could use background processes with wait, though the current sequential approach is simpler and the time difference is likely negligible since all servers start concurrently anyway.

💡 Optional: True parallel waiting
-# Wait for all servers in parallel (print status as they become ready)
-WAIT_FAILURE=0
-
-if [ "$RUN_TANSTACK" = true ]; then
-	wait_for_server "http://localhost:3000" "TanStack web (3000)" 90 || WAIT_FAILURE=1
-	wait_for_server "http://localhost:3500" "TanStack agent (3500)" 30 || WAIT_FAILURE=1
-fi
-
-if [ "$RUN_NEXTJS" = true ]; then
-	wait_for_server "http://localhost:3001" "Next.js web (3001)" 90 || WAIT_FAILURE=1
-	wait_for_server "http://localhost:3501" "Next.js agent (3501)" 30 || WAIT_FAILURE=1
-fi
-
-if [ "$RUN_VITE_RSC" = true ]; then
-	wait_for_server "http://localhost:3002" "Vite RSC web (3002)" 90 || WAIT_FAILURE=1
-	wait_for_server "http://localhost:3502" "Vite RSC agent (3502)" 30 || WAIT_FAILURE=1
-fi
+# Wait for all servers (sequentially per framework, but frameworks started in parallel)
+WAIT_FAILURE=0
+
+if [ "$RUN_TANSTACK" = true ]; then
+	wait_for_server "http://localhost:3000" "TanStack web (3000)" 90 || WAIT_FAILURE=1
+	wait_for_server "http://localhost:3500" "TanStack agent (3500)" 30 || WAIT_FAILURE=1
+fi
+
+if [ "$RUN_NEXTJS" = true ]; then
+	wait_for_server "http://localhost:3001" "Next.js web (3001)" 90 || WAIT_FAILURE=1
+	wait_for_server "http://localhost:3501" "Next.js agent (3501)" 30 || WAIT_FAILURE=1
+fi
+
+if [ "$RUN_VITE_RSC" = true ]; then
+	wait_for_server "http://localhost:3002" "Vite RSC web (3002)" 90 || WAIT_FAILURE=1
+	wait_for_server "http://localhost:3502" "Vite RSC agent (3502)" 30 || WAIT_FAILURE=1
+fi

Or simply fix the comment to accurately describe the behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/test-framework-demos.sh` around lines 145 - 161, The comment claiming
"Wait for all servers in parallel" is incorrect because the script calls
wait_for_server sequentially; either update the comment to accurately state
"Wait for all servers sequentially" or change the logic to run wait_for_server
in background and collect statuses with wait (e.g., launch each selected server
check for RUN_TANSTACK, RUN_NEXTJS, RUN_VITE_RSC with wait_for_server ... & and
capture failures into WAIT_FAILURE, then wait for all jobs), referencing the
existing WAIT_FAILURE variable and the wait_for_server function calls to locate
the code to modify.
.github/workflows/package-smoke-test.yaml (1)

450-461: Consider unconditionally installing Playwright system dependencies, or document the runner image assumption.

When the Playwright browser cache hits, the bunx playwright install --with-deps chromium step is skipped. While the --with-deps flag installs both browser binaries and system dependencies (libnss3, libatk, etc.), the cache only restores binaries. On current GitHub Ubuntu runners, these system dependencies are pre-installed, but this creates an implicit assumption about the runner image.

To make this more robust, either:

  1. Run bunx playwright install-deps chromium unconditionally after a cache hit (minimal overhead if already present)
  2. Add a comment documenting that this approach assumes the runner image includes Playwright system dependencies
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/package-smoke-test.yaml around lines 450 - 461, The
workflow skips the "Install Playwright browsers" step when the
"playwright-cache" cache-hit is true, which restores browser binaries but not
system libraries; update the job so that after the cache restore (step id
playwright-cache) you always run the system deps installer (e.g., run bunx
playwright install-deps chromium unconditionally or remove the if: condition on
the "Install Playwright browsers" step), or alternatively add an explicit inline
comment near the playwright-cache and Install Playwright browsers steps stating
the runner image is expected to already include Playwright system dependencies.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In @.github/workflows/package-smoke-test.yaml:
- Around line 450-461: The workflow skips the "Install Playwright browsers" step
when the "playwright-cache" cache-hit is true, which restores browser binaries
but not system libraries; update the job so that after the cache restore (step
id playwright-cache) you always run the system deps installer (e.g., run bunx
playwright install-deps chromium unconditionally or remove the if: condition on
the "Install Playwright browsers" step), or alternatively add an explicit inline
comment near the playwright-cache and Install Playwright browsers steps stating
the runner image is expected to already include Playwright system dependencies.

In `@e2e/global-setup.ts`:
- Around line 104-106: Replace the plain Error throw with a StructuredError: add
an import for StructuredError from '@agentuity/core' at the top of the file,
then replace the throw new Error(...) at the dev-server readiness timeout with
throw new StructuredError(...) supplying a clear message (same text), a short
machine-friendly code like "DEV_SERVER_TIMEOUT", and metadata containing
MAX_RETRIES and RETRY_DELAY (or total ms) so the error is structured and
contains the same info; keep the existing message text in the StructuredError's
message field.
- Line 12: Replace the hardcoded MAX_RETRIES constant with a configurable value
sourced from an environment variable (e.g., MAX_SETUP_RETRIES) so tests can
override retry count; specifically, change the declaration of MAX_RETRIES to
read process.env.MAX_SETUP_RETRIES, parse it as an integer, fall back to 30 when
missing or invalid, and ensure the value is a positive integer before use (refer
to the MAX_RETRIES symbol in global-setup.ts and the surrounding retry logic).

In `@scripts/test-framework-demos.sh`:
- Around line 178-184: The current script builds PROJECT_ARGS as a quoted string
which relies on word-splitting; change to a Bash array (e.g.,
PROJECT_ARGS_ARRAY) and push each "--project=$proj" into it while iterating over
PROJECTS, then invoke the test command with the array expanded as
"${PROJECT_ARGS_ARRAY[@]}" so each project argument is preserved safely; update
references to PROJECT_ARGS to use the new array variable where the bun run
playwright test command is executed.
- Around line 145-161: The comment claiming "Wait for all servers in parallel"
is incorrect because the script calls wait_for_server sequentially; either
update the comment to accurately state "Wait for all servers sequentially" or
change the logic to run wait_for_server in background and collect statuses with
wait (e.g., launch each selected server check for RUN_TANSTACK, RUN_NEXTJS,
RUN_VITE_RSC with wait_for_server ... & and capture failures into WAIT_FAILURE,
then wait for all jobs), referencing the existing WAIT_FAILURE variable and the
wait_for_server function calls to locate the code to modify.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a1b98f45-3aa7-4c7e-984e-fc612d7b31fe

📥 Commits

Reviewing files that changed from the base of the PR and between 51c2f39 and ec32ce9.

📒 Files selected for processing (6)
  • .github/workflows/build.yaml
  • .github/workflows/package-smoke-test.yaml
  • e2e/global-setup.ts
  • playwright.config.ts
  • playwright.frameworks.config.ts
  • scripts/test-framework-demos.sh
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (15)
  • GitHub Check: Standalone Agent Test
  • GitHub Check: Framework Integration Tests (TanStack & Next.js)
  • GitHub Check: Playwright E2E Smoke Test
  • GitHub Check: Queue CLI Tests
  • GitHub Check: Queue SDK Tests
  • GitHub Check: Storage CLI Tests
  • GitHub Check: SDK Integration Test Suite
  • GitHub Check: Package Installation & Usage Test
  • GitHub Check: Cloud Deployment Tests
  • GitHub Check: Template Integration Tests
  • GitHub Check: Sandbox CLI Tests
  • GitHub Check: Postgres SSL Integration Test
  • GitHub Check: Pack & Upload
  • GitHub Check: Build
  • GitHub Check: Agentuity Deployment
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Use Biome as code formatter with tabs (width 3), single quotes, semicolons, lineWidth 100, and trailingCommas es5

Files:

  • e2e/global-setup.ts
  • playwright.frameworks.config.ts
  • playwright.config.ts
**/*.{ts,tsx}

📄 CodeRabbit inference engine (AGENTS.md)

**/*.{ts,tsx}: Use TypeScript Strict mode with ESNext target and bundler moduleResolution
Use StructuredError from @agentuity/core for error handling

Files:

  • e2e/global-setup.ts
  • playwright.frameworks.config.ts
  • playwright.config.ts
🧠 Learnings (1)
📚 Learning: 2025-12-21T00:31:41.858Z
Learnt from: jhaynie
Repo: agentuity/sdk PR: 274
File: packages/cli/src/cmd/build/vite/server-bundler.ts:12-41
Timestamp: 2025-12-21T00:31:41.858Z
Learning: In Bun runtime, BuildMessage and ResolveMessage are global types and are not exported from the bun module. Do not import { BuildMessage } from 'bun' or similar; these types are available globally and should be used without import. This applies to all TypeScript files that target the Bun runtime within the repository.

Applied to files:

  • e2e/global-setup.ts
  • playwright.frameworks.config.ts
  • playwright.config.ts
🔇 Additional comments (4)
playwright.config.ts (1)

11-12: LGTM!

The CI configuration changes are sensible:

  • Reducing retries from 2 to 1 provides faster failure feedback while still allowing one retry for flaky tests.
  • Using Number(process.env.PLAYWRIGHT_WORKERS) || undefined allows flexible worker configuration while defaulting to Playwright's auto-detection based on available CPUs.
playwright.frameworks.config.ts (1)

10-13: LGTM!

Good optimization enabling full parallelism for framework tests. The configuration is now consistent with playwright.config.ts. Since each framework project targets a different port (3000, 3001, 3002), running them in parallel should not cause conflicts.

.github/workflows/build.yaml (1)

37-43: LGTM!

The Bun dependency caching is correctly configured:

  • Uses the appropriate cache path (~/.bun/install/cache)
  • Cache key includes lockfile hash for proper invalidation
  • Restore keys provide fallback for partial cache hits
.github/workflows/package-smoke-test.yaml (1)

27-33: LGTM!

Bun dependency caching is consistently applied across all jobs that run bun install. The installation-type-test job correctly omits caching since it doesn't install dependencies.

Also applies to: 51-57, 94-100, 126-132, 182-188, 258-264, 290-296, 329-335, 368-374, 405-411, 439-445, 495-501

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant