Skip to content

[Feature] Benchmark suite with CI integration #24

@Siddhant-K-code

Description

@Siddhant-K-code

Summary

Add a Go benchmark suite that measures dedup pipeline performance across different data sizes and configurations. Publish results in CI.

Motivation

Publishable benchmark numbers are useful for:

  • Blog posts and conference talks ("12ms for 50 chunks" backed by reproducible data)
  • Detecting performance regressions on PRs
  • Comparing algorithm variants (e.g., average vs complete linkage)
  • Giving users confidence in production readiness

Benchmarks to add

// pkg/contextlab/
BenchmarkCluster_10Chunks
BenchmarkCluster_50Chunks
BenchmarkCluster_100Chunks
BenchmarkCluster_500Chunks
BenchmarkMMR_10Chunks
BenchmarkMMR_50Chunks
BenchmarkSelector_10Clusters
BenchmarkSelector_50Clusters

// pkg/dedup/
BenchmarkDistanceMatrix_50
BenchmarkDistanceMatrix_200

// pkg/compress/
BenchmarkCompress_ShortText
BenchmarkCompress_LongText

// End-to-end
BenchmarkFullPipeline_50Chunks
BenchmarkFullPipeline_200Chunks

CI integration

  • Add benchmark job to ci.yml that runs go test -bench=. -benchmem ./...
  • Use benchstat to compare against baseline on PRs
  • Store baseline results in testdata/benchmarks/baseline.txt
  • Comment on PR with performance diff if regression > 10%

Deliverables

  • Benchmark functions in *_test.go files
  • Synthetic test data generator (deterministic embeddings)
  • make bench target
  • CI workflow for benchmark comparison
  • BENCHMARKS.md with latest results table

Acceptance Criteria

  • go test -bench=. ./... runs all benchmarks
  • Results are reproducible (deterministic test data)
  • CI detects regressions > 10%
  • README links to benchmark results

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions