Skip to content

Add DAG, Web, CLI modules and barrel export for TypeScript package#10

Open
Copilot wants to merge 11 commits intomainfrom
copilot/create-pixie-qa-package
Open

Add DAG, Web, CLI modules and barrel export for TypeScript package#10
Copilot wants to merge 11 commits intomainfrom
copilot/create-pixie-qa-package

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 4, 2026 •
edited
Loading

  • Explore Python codebase - read all source files
  • Clean up: remove node_modules from git tracking
  • Phase 1: Core types (config, spans, evaluable)
  • Phase 2: Instrumentation (handler, queue, context, observation, processor, instrumentors, handlers, index)
  • Phase 3: Storage (serialization, tree, store, index)
  • Phase 4: Dataset (models, store, index)
  • Phase 5: Evals (evaluation, criteria, scorers, llmEvaluator, rateLimiter, traceCapture, traceHelpers, evalUtils, testResult, scorecard, datasetRunner, index)
  • Phase 6: DAG module (index, traceCheck)
  • Phase 7: Web module (app, server, watcher, index)
  • Phase 8: CLI commands (main, testCommand, initCommand, datasetCommand, traceCommand, analyzeCommand, startCommand, dagCommand)
  • Phase 9: Package exports (index.ts)
  • Phase 10: Build verification (tsc compiles cleanly)
  • Phase 11: Tests (10 test files, 133 tests passing)
  • Phase 12: Documentation (README.md, changelog)

Copilot AI and others added 10 commits April 4, 2026 05:37
@yiouli
Add TypeScript package scaffolding for pixie-qa npm package
6c85bca 
Create ts/ directory with:
- package.json with npm config, dependencies, and scripts
- tsconfig.json with strict TypeScript configuration
- vitest.config.ts for test runner
- Source directories: instrumentation, storage, evals, dataset, cli, dag, web
- tests/ directory for test files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Add core types, models, and instrumentation layer for TS pixie-qa
9adeed2 
Agent-Logs-Url: https://github.com/yiouli/pixie-qa/sessions/211cdef7-77c2-49a1-952d-7e839fbf7130

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Remove ts/node_modules from git tracking, add to .gitignore
0851248 
Agent-Logs-Url: https://github.com/yiouli/pixie-qa/sessions/7eb59954-353e-4621-9bd6-866d349ba094

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Port Python instrumentation, config, and storage modules to TypeScript
3477013 
Create TypeScript equivalents of the following Python modules:
- pixie/config.py → ts/src/config.ts
- pixie/instrumentation/spans.py → ts/src/instrumentation/spans.ts
- pixie/instrumentation/handler.py → ts/src/instrumentation/handler.ts
- pixie/instrumentation/queue.py → ts/src/instrumentation/queue.ts
- pixie/instrumentation/context.py → ts/src/instrumentation/context.ts
- pixie/instrumentation/processor.py → ts/src/instrumentation/processor.ts
- pixie/instrumentation/instrumentors.py → ts/src/instrumentation/instrumentors.ts
- pixie/instrumentation/observation.py → ts/src/instrumentation/observation.ts
- pixie/instrumentation/handlers.py → ts/src/instrumentation/handlers.ts
- pixie/storage/evaluable.py → ts/src/storage/evaluable.ts
- ts/src/instrumentation/index.ts (barrel re-exports)

Remove .gitkeep placeholder files from src subdirectories.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Address review: fix type safety, remove double-serialization, extract…
e5ecbaa 
… constant

- NoOpObservationContext._snapshot() now throws instead of returning null cast
- observe() wrapper passes args/return values directly without JSON round-trip
- Extract NULL_SPAN_ID constant for magic span ID checks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
feat(ts): add storage, dataset, and evals modules
5e3459a 
Port the Python storage, dataset, and evals packages to TypeScript:

Storage module:
- serialization.ts: serializeSpan/deserializeSpan for DB storage
- tree.ts: ObservationNode class with find/findByType/toText + buildTree
- store.ts: ObservationStore using better-sqlite3 (sync API)
- index.ts: barrel export

Dataset module:
- models.ts: Dataset interface
- store.ts: DatasetStore with JSON file-backed CRUD
- index.ts: barrel export

Evals module:
- evaluation.ts: Evaluation interface, Evaluator type, evaluate()
- criteria.ts: ScoreThreshold pass criteria
- scorers.ts: AutoevalsAdapter + 21 factory functions for autoevals
- llmEvaluator.ts: createLlmEvaluator() factory using OpenAI
- rateLimiter.ts: EvalRateLimiter with sliding window rate limiting
- traceCapture.ts: MemoryTraceHandler + captureTraces()
- traceHelpers.ts: lastLlmCall(), root() trace extractors
- evalUtils.ts: assertPass, assertDatasetPass, runAndEvaluate
- testResult.ts: EvaluationResult, EntryResult, RunResult models
- scorecard.ts: DatasetScorecard types + HTML generation
- datasetRunner.ts: BUILTIN_EVALUATOR_NAMES, resolveEvaluatorName, etc.
- index.ts: barrel export

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
refactor(ts): address code review feedback
f849b7a 
- Improve error message in parseDate with expected format hint
- Document token estimation approximation in EvalRateLimiter
- Reduce unnecessary allocation in assertPass default criteria

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Add DAG, Web, CLI modules and barrel export for TypeScript package
06af66c 
Port Python modules to TypeScript:
- dag/index.ts: DAG parsing, validation, Mermaid generation
- dag/traceCheck.ts: trace validation against DAG
- web/app.ts: Express app with SSE, artifact API endpoints
- web/server.ts: HTTP server with lock file management
- web/watcher.ts: chokidar file watcher for live updates
- web/index.ts: barrel export
- cli/main.ts: commander-based CLI entry point with all subcommands
- cli/testCommand.ts: dataset-driven test runner
- cli/initCommand.ts: directory scaffolding
- cli/datasetCommand.ts: dataset CRUD operations
- cli/traceCommand.ts: trace inspection commands
- cli/analyzeCommand.ts: OpenAI-powered test analysis
- cli/startCommand.ts: web UI launcher
- cli/dagCommand.ts: DAG validation and trace checking
- index.ts: main package barrel export

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Address review feedback: named constants, cross-platform stdin
c36ca13 
- Use named constants for Unicode marks in testCommand.ts
- Use fd 0 instead of /dev/stdin for cross-platform stdin reading

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
@yiouli
Add ts/dist/ to .gitignore and remove from tracking
7817ed3 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: yiouli <3508494+yiouli@users.noreply.github.com>
Copilot AI requested a review from yiouli April 4, 2026 15:04
@yiouli yiouli marked this pull request as ready for review April 4, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiouli yiouli Awaiting requested review from yiouli

Assignees

@yiouli yiouli

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yiouli