Skip to content

Commit 18ca995

Browse files
[Breaking] Use non-delta codec for oneof type (#146)
Resolves #127 This simplifies oneof codec and improves codec speed at the cost of slight increase of uncompressed size. The zstd-compressed size is approximately the same (slightly increased or decresed depending on the dataset). ### Benchmarks Otel schema speed improvement: ``` │ bench_base.txt │ bench_current.txt │ │ sec/point │ sec/point vs base │ SerializeNative/STEF/serialize-10 64.36n ± 1% 63.10n ± 2% -1.96% (p=0.020 n=9) DeserializeNative/STEF/deser-10 24.06n ± 1% 22.70n ± 1% -5.65% (p=0.000 n=9) geomean 39.35n 37.85n -3.82% ``` Otel schema size before the change: ``` File Uncompressed Zstd Bytes hipstershop-otelmetrics.stefz 432716 93702 hostandcollector-otelmetrics.stefz 1258652 83009 astronomy-otelmetrics.stefz 7462303 1652082 ``` After the change: ``` File Uncompressed Zstd Bytes hipstershop-otelmetrics.stefz 441558 95083 hostandcollector-otelmetrics.stefz 1266627 79835 astronomy-otelmetrics.stefz 7557122 1644464 ```
1 parent 55d0a17 commit 18ca995

File tree

17 files changed

+194
-206
lines changed

17 files changed

+194
-206
lines changed

.github/workflows/benchmark.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ jobs:
8585
HEAD=$(git rev-parse HEAD)
8686
git reset --hard ${{ github.event.pull_request.base.sha }}
8787
export REF_NAME=main
88-
cd benchmarks && make bench-run bench-stat
88+
cd benchmarks && make gentestfiles bench-run bench-stat
8989
git reset --hard $HEAD
9090
9191
- name: Benchmark stats

benchmarks/scripts/genoldtestfiles.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ TMPDIR=$(mktemp -d)
1414
cp ./gentestfiles.sh "$TMPDIR/"
1515

1616
# Checkout the base branch to compare to
17-
BASE_BRANCH=21f498a293dbdf6e29c75385ba0043dc0f5e4009
17+
BASE_BRANCH=tigran/oneofcodec # Change this to the main branch commit after tigran/oneofcodec is merged.
1818
git -c advice.detachedHead=false checkout $BASE_BRANCH
1919

2020
# Convert/generate the files.

examples/jsonl/internal/jsonstef/jsonvalue.go

Lines changed: 22 additions & 19 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

examples/profile/internal/profile/labelvalue.go

Lines changed: 22 additions & 19 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

go/otel/oteltef/anyvalue.go

Lines changed: 22 additions & 19 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)