Skip to content

Add optional variables and fix stdout/stderr ordering#8

Open
andreasjansson wants to merge 2 commits intomainfrom
optional-variables
Open

Add optional variables and fix stdout/stderr ordering#8
andreasjansson wants to merge 2 commits intomainfrom
optional-variables

Conversation

@andreasjansson
Copy link
Copy Markdown
Owner

@andreasjansson andreasjansson commented Mar 31, 2026

Two changes:

1. Optional variable modifier for test patterns

Add support for optional variables that mark output lines which may or may not appear.

Syntax

{{ var: optional string }}       # optional with explicit type
{{ var: optional number }}       # optional number
{{ var: optional json object }}  # optional JSON types
{{ var: optional }}              # optional with duck typing

An optional variable must occupy an entire line by itself in the expected output pattern. When the actual output doesn't contain that line, the test still passes and the variable is not captured. When present, the variable captures normally and can be used in constraints.

Example

===
test with optional progress line
===
./my-command
---
{{ progress: optional string }}
result: {{ value: number }}
---
where
* value > 0

This matches both result: 42 (no progress line) and Processing...\nresult: 42 (with progress line). Multiple consecutive optional lines are also supported.

Implementation

  • Parser (cctr-corpus): Extract optional prefix in parse_placeholder, added optional: bool field to VariableDecl
  • Matcher: build_regex wraps optional-only lines in (?:...)? groups with correct newline handling for start/middle/end positions
  • 10 matcher unit tests, 5 parser unit tests, 18 corpus tests

2. Fix stdout/stderr stream ordering

Previously run_command read all of stdout then appended all of stderr:

let combined = format!("{}{}", stdout_str, stderr_str);

This caused stderr to always appear after stdout regardless of when it was actually written. Now uses the same channel-based interleaving approach as run_command_streaming to preserve temporal ordering of output lines from both streams.

6 corpus tests verify ordering with stderr-first, stdout-first, interleaved, and timed delay scenarios.

Add support for optional variables that mark output lines which may or
may not appear. Syntax: {{ var: optional type }} or {{ var: optional }}
for duck-typed.

An optional variable must occupy an entire line by itself. When the
actual output doesn't contain that line, the test still passes and the
variable is not captured. When present, the variable captures normally
and can be used in constraints.

This is useful for commands that conditionally print progress or
diagnostic lines depending on execution time.

Examples:
  {{ progress: optional string }}
  {{ n: optional number }}
  {{ data: optional json object }}
  {{ x: optional }}

Implementation:
- Parser: extract 'optional' prefix in parse_placeholder
- Matcher: build_regex wraps optional-only lines in (?:...)? groups
  with correct newline handling for start/middle/end positions
- 10 matcher unit tests, 5 parser unit tests, 18 corpus tests
@andreasjansson andreasjansson changed the title Add optional variable modifier for test patterns Add optional variables and fix stdout/stderr ordering Mar 31, 2026
Previously run_command read all of stdout then all of stderr,
causing stderr to always appear after stdout regardless of when
it was actually written. Now uses channel-based interleaving
(same approach as run_command_streaming) to preserve temporal
ordering of output lines from both streams.

Adds 6 corpus tests verifying ordering with stderr-first,
stdout-first, interleaved, and timed delay scenarios.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant