Rust playwright queue

[kfreitag1]

Adds a new rust implementation of ci-queue with an integration using the playwright CLI.

Relevant checkout-web PR: https://app.graphite.dev/github/pr/shop/world/151706

Key Components

Core Library (ci-queue-core)

• Queue trait system: Generic interface supporting multiple test identifier types
• Static queue: In-memory queue for single-worker scenarios
• Distributed queue: Redis-backed implementation with master election, test reservation, and failure recovery
• Configurable behaviour: Requeue limits, timeouts, and tolerance thresholds

Playwright Integration (ci-queue-playwright)

• Test discovery: Parses playwright test --list JSON output to identify all tests
• Node runner detection: Automatically detects npm, yarn, pnpm, or npx
• Distributed execution: Runs individual tests via playwright test file:line syntax
• CLI interface: Feature-rich command-line tool with configuration options matching minitest-queue
• Compiled as a binary for lower memory footprint and no build overheads during CI

Architecture Highlights

1. Type-safe test identifiers: Generic TestIdentifier trait allows custom test representations while maintaining Redis compatibility
2. Atomic Redis operations: All critical operations use Lua scripts ensuring consistency across workers
3. Timeout-based recovery: Lost tests are automatically reassigned after configurable timeout periods
4. Smart requeueing: Failed tests can be retried with position control to avoid blocking new tests

playwright-queue example usage

  playwright-queue
    --queue redis://redis:6379
    --build $BUILD_ID
    run
    --worker $WORKER_ID
    --max-requeues 2
    --requeue-tolerance 0.05
    --timeout 60

Testing

• Comprehensive integration tests for distributed queue scenarios
• Tests cover single/multiple workers, requeueing, timeouts, and failure recovery

Future Enhancements

Several features to be ported from minitest-queue:

• Report generation and aggregation
• Test duration tracking and limits
• Grind mode for flaky test detection
• Bisect capability for finding problematic test combinations
• Heartbeat monitoring for long-running tests

More fine-grained test splitting between different platforms
Example:

should be ran as 5 separate items in the queue

Compatibility

• Maintains compatibility with existing Redis data structures
• Can interoperate with Ruby/Python implementations using the same build IDs
• Follows established ci-queue patterns and conventions

[ChrisBr]

Thank you for your contribution.

I don't think we will accept any more queue implementations for ci-queue at this point. Python is already unmaintained / broken. Long term we may even deprecate Ruby in favour of Megatest and minitest-distributed.

I recommend to extract a standalone Rust implementation.