Skip to content

feat(tests): blackbox test plugin for property-based testing#255

Draft
JeroenSoeters wants to merge 8 commits intomainfrom
worktree-perf-tests
Draft

feat(tests): blackbox test plugin for property-based testing#255
JeroenSoeters wants to merge 8 commits intomainfrom
worktree-perf-tests

Conversation

@JeroenSoeters
Copy link
Collaborator

@JeroenSoeters JeroenSoeters commented Feb 25, 2026

Summary

  • Add design doc and implementation plan for a blackbox property-based correctness and performance test suite
  • Scaffold test plugin binary (tests/testplugin/) with in-memory cloud state, CRUD operations, and failure injection stubs
  • Phase 1 Tasks 1-3 complete, Task 4 in progress (TDD checkpoint: failing tests written)

What's here

Design & plan docs (docs/plans/):

  • Performance and correctness testing design doc
  • Detailed implementation plan (13 tasks, 5 phases)

Test plugin (tests/testplugin/):

  • Standalone Go module with plugin SDK integration
  • CloudState — thread-safe in-memory cloud simulation (6 tests passing)
  • TestPlugin — full CRUD backed by CloudState (5 tests passing)
  • InjectionState / OperationLog — stubs with failing tests (Task 4 WIP)
  • messages.go — message types for TestController actor communication

What's next

  • Implement InjectionState and OperationLog (finish Task 4)
  • Wire injection into plugin CRUD (Task 5)
  • Custom main.go with TestController Ergo actor (Task 6)
  • End-to-end smoke test (Task 7)
  • Phase 2: test harness, rapid generators, property tests

Tasks 1-3 complete: test plugin module, cloud state store, CRUD operations.
Task 4 in progress: injection/operation log stubs with failing tests.
…ller actor

Phase 1 tasks 4-6: implement InjectionState (thread-safe error/latency
rule matching), OperationLog (append-only operation recording),
TestController Ergo actor for test harness control, wire injection and
logging into plugin CRUD methods, and custom main.go that spawns
TestController alongside standard plugin infrastructure.
Phase 1 task 7: end-to-end smoke test that builds the test plugin
binary, starts an in-process metastructure, waits for plugin
registration, applies a forma with a Test::Generic::Resource, and
verifies the command completes successfully.
Move TestController request/response types from testplugin to a shared
tests/testcontrol module so both the test plugin and test harness can
use them for Ergo cross-node communication.
…e reset

Add Ergo node to the test harness for direct communication with the
TestController actor. Add methods for fault injection, cloud state
manipulation, and inter-iteration state reset via reconcile destroy.
Two property tests running 100 rapid iterations each:
- SequentialHappyPath: apply/destroy with reconcile mode
- SequentialWithFailures: same plus error/latency injection on all
  plugin operations (Create, Read, Update, Delete)

Both verify invariants (no phantoms, no orphans, property consistency)
against cloud state after each iteration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant