Performance significantly worse than reference implementation

[pereorga]

I found this implementation ~35% slower than brotli CLI at compression level 11:

Benchmark

package main

import (
  "fmt"
  "io"
  "net/http"
  "os"
  "os/exec"
  "time"

  "github.com/andybalholm/brotli"
)

func main() {
  // Fetch reproducible test file (~63KB)
  resp, _ := http.Get("https://go.dev")
  content, _ := io.ReadAll(resp.Body)
  resp.Body.Close()
  fmt.Printf("Testing with %d bytes\n\n", len(content))

  // Go library
  start := time.Now()
  f, _ := os.Create("test-go.br")
  w := brotli.NewWriterLevel(f, brotli.BestCompression)
  w.Write(content)
  w.Close()
  f.Close()
  fmt.Printf("Go library: %v\n", time.Since(start))

  // CLI
  tmp, _ := os.CreateTemp("", "test-*.tmp")
  tmp.Write(content)
  tmp.Close()
  start = time.Now()
  exec.Command("brotli", "-q", "11", "-f", "-o", "test-cli.br", tmp.Name()).Run()
  fmt.Printf("CLI: %v\n", time.Since(start))
  os.Remove(tmp.Name())
}

Results

Testing with 63009 bytes

Go library: ~135ms
CLI:        ~99ms

Both produce identical output. CLI is consistently ~35% faster.

Environment

• Ubuntu 24.04 (amd64)
• github.com/andybalholm/brotli v1.2.0
• brotli CLI 1.1.0
• Go 1.24.0

[andybalholm]

It's not surprising at all that C code that was translated to Go is slower than the same C code compiled directly.