Skip to content

experiment: v2 datafusion no pinning pool#7114

Closed
0ax1 wants to merge 2 commits intodevelopfrom
ad/v2-datafusion-no-pool
Closed

experiment: v2 datafusion no pinning pool#7114
0ax1 wants to merge 2 commits intodevelopfrom
ad/v2-datafusion-no-pool

Conversation

@0ax1
Copy link
Contributor

@0ax1 0ax1 commented Mar 22, 2026

No description provided.

0ax1 added 2 commits March 22, 2026 20:12
@0ax1
feat(io): pinned CPU thread pool for cache-local decode
7fb8999 
Add PinnedCpuPool/PinnedExecutor: a core-pinned thread pool that
overrides spawn_cpu while delegating async I/O to the fallback runtime
(tokio). Avoids work-stealing for decode-heavy ops (bitunpacking, FoR,
dictionary gather), keeping data cache-local.

Expose via RuntimeSessionExt::with_pinned_cpu_pool() and enable it by
default in the benchmark SESSION. Fix executor lifetime bug by storing a
strong Arc in RuntimeSession alongside the Weak-based Handle.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1
bench: use v2 scan API by default, no pinned pool
f652db6 
Enable the v2 VortexTable/VortexDataSource scan path for Vortex formats
in datafusion-bench without requiring VORTEX_USE_SCAN_API=1. Use plain
tokio (.with_tokio()) as the executor — no pinned CPU thread pool.

This serves as a baseline to measure the v2 scan path on its own before
layering in the pinned pool.

Signed-off-by: Alexander Droste <alexander.droste@protonmail.com>
@0ax1 0ax1 added do not merge Pull requests that are not intended to merge action/benchmark-sql Trigger SQL benchmarks to run on this PR labels Mar 22, 2026
@github-actions github-actions bot removed the action/benchmark-sql Trigger SQL benchmarks to run on this PR label Mar 22, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done f652db6 2 Explore Profiling Data
Previous Runs (1)
Status Commit Job Attempt Link
🟢 Done f652db6 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: PolarSignals Profiling

Vortex (geomean): 1.019x ➖

datafusion / vortex-file-compressed (1.019x ➖, 1↑ 3↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
polarsignals_q00/datafusion:vortex-file-compressed 114224399 113797953 1.00
polarsignals_q01/datafusion:vortex-file-compressed 326841788 341194416 0.96
polarsignals_q02/datafusion:vortex-file-compressed 21102956 21658336 0.97
polarsignals_q03/datafusion:vortex-file-compressed 322207682 342750902 0.94
polarsignals_q04/datafusion:vortex-file-compressed 🚀 9745697 11907396 0.82
polarsignals_q05/datafusion:vortex-file-compressed 🚨 18091828 16071343 1.13
polarsignals_q06/datafusion:vortex-file-compressed 🚨 22218658 20061286 1.11
polarsignals_q07/datafusion:vortex-file-compressed 🚨 18562315 12556719 1.48
polarsignals_q08/datafusion:vortex-file-compressed 406122760 426271714 0.95
polarsignals_q09/datafusion:vortex-file-compressed 10586856 11135009 0.95

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: TPC-H SF=1 on NVME

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -2.7%
Vortex (geomean): 0.961x ➖
Parquet (geomean): 0.982x ➖
Shifts: Parquet (control) -1.8% · Median polish -4.3%

datafusion / vortex-file-compressed (0.981x ➖, 12↑ 7↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚀 46447667 54522937 0.85
tpch_q02/datafusion:vortex-file-compressed 🚀 23632181 51742880 0.46
tpch_q03/datafusion:vortex-file-compressed 🚀 25666799 29991782 0.86
tpch_q04/datafusion:vortex-file-compressed 🚨 59692476 19516257 3.06
tpch_q05/datafusion:vortex-file-compressed 🚨 183239934 79906038 2.29
tpch_q06/datafusion:vortex-file-compressed 9375972 10135130 0.93
tpch_q07/datafusion:vortex-file-compressed 🚨 148000071 93968845 1.57
tpch_q08/datafusion:vortex-file-compressed 🚀 33460486 68922855 0.49
tpch_q09/datafusion:vortex-file-compressed 🚀 68167457 89039949 0.77
tpch_q10/datafusion:vortex-file-compressed 76291353 78874523 0.97
tpch_q11/datafusion:vortex-file-compressed 18861078 20171584 0.94
tpch_q12/datafusion:vortex-file-compressed 🚨 44342582 34592697 1.28
tpch_q13/datafusion:vortex-file-compressed 🚨 43323476 27992436 1.55
tpch_q14/datafusion:vortex-file-compressed 🚀 18232141 29664100 0.61
tpch_q15/datafusion:vortex-file-compressed 🚀 20939313 27703859 0.76
tpch_q16/datafusion:vortex-file-compressed 🚀 21034199 35678048 0.59
tpch_q17/datafusion:vortex-file-compressed 🚀 64090861 76027276 0.84
tpch_q18/datafusion:vortex-file-compressed 🚨 209683953 104187137 2.01
tpch_q19/datafusion:vortex-file-compressed 🚀 21729616 35786437 0.61
tpch_q20/datafusion:vortex-file-compressed 🚀 26628789 51876824 0.51
tpch_q21/datafusion:vortex-file-compressed 🚨 219938676 96583915 2.28
tpch_q22/datafusion:vortex-file-compressed 🚀 13553732 15400973 0.88
datafusion / vortex-compact (0.951x ➖, 13↑ 6↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 🚀 51494202 60934626 0.85
tpch_q02/datafusion:vortex-compact 🚀 26636308 54352937 0.49
tpch_q03/datafusion:vortex-compact 🚀 28647531 32729627 0.88
tpch_q04/datafusion:vortex-compact 🚨 57638835 24797494 2.32
tpch_q05/datafusion:vortex-compact 🚨 200986407 85936136 2.34
tpch_q06/datafusion:vortex-compact 🚀 11573818 14082069 0.82
tpch_q07/datafusion:vortex-compact 🚨 156818630 103671805 1.51
tpch_q08/datafusion:vortex-compact 🚀 35973095 76848723 0.47
tpch_q09/datafusion:vortex-compact 🚀 75318165 97871979 0.77
tpch_q10/datafusion:vortex-compact 78897155 81691507 0.97
tpch_q11/datafusion:vortex-compact 21796172 21512219 1.01
tpch_q12/datafusion:vortex-compact 49340466 47088672 1.05
tpch_q13/datafusion:vortex-compact 🚨 47754213 38172860 1.25
tpch_q14/datafusion:vortex-compact 🚀 21071153 32925313 0.64
tpch_q15/datafusion:vortex-compact 🚀 26737800 33762662 0.79
tpch_q16/datafusion:vortex-compact 🚀 25552380 40888574 0.62
tpch_q17/datafusion:vortex-compact 🚀 64894335 81811833 0.79
tpch_q18/datafusion:vortex-compact 🚨 217561352 116393973 1.87
tpch_q19/datafusion:vortex-compact 🚀 31962787 48928514 0.65
tpch_q20/datafusion:vortex-compact 🚀 34467781 59766351 0.58
tpch_q21/datafusion:vortex-compact 🚨 222747579 109226857 2.04
tpch_q22/datafusion:vortex-compact 🚀 14440191 18003244 0.80
datafusion / parquet (0.976x ➖, 1↑ 1↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 🚨 145762679 109530594 1.33
tpch_q02/datafusion:parquet 80048137 84059663 0.95
tpch_q03/datafusion:parquet 🚀 77375042 89749764 0.86
tpch_q04/datafusion:parquet 53019242 53885942 0.98
tpch_q05/datafusion:parquet 163475649 168801090 0.97
tpch_q06/datafusion:parquet 46785603 48500276 0.96
tpch_q07/datafusion:parquet 180917075 181646900 1.00
tpch_q08/datafusion:parquet 171370981 163469532 1.05
tpch_q09/datafusion:parquet 209281429 211391087 0.99
tpch_q10/datafusion:parquet 184882941 194020469 0.95
tpch_q11/datafusion:parquet 59736908 61747124 0.97
tpch_q12/datafusion:parquet 110470689 120475458 0.92
tpch_q13/datafusion:parquet 200964765 217283819 0.92
tpch_q14/datafusion:parquet 80630722 83935596 0.96
tpch_q15/datafusion:parquet 69470687 71725406 0.97
tpch_q16/datafusion:parquet 52407574 55725591 0.94
tpch_q17/datafusion:parquet 174318138 177474744 0.98
tpch_q18/datafusion:parquet 227155153 233543826 0.97
tpch_q19/datafusion:parquet 99576751 104227003 0.96
tpch_q20/datafusion:parquet 116197462 120459973 0.96
tpch_q21/datafusion:parquet 184829028 191961636 0.96
tpch_q22/datafusion:parquet 39726960 40941093 0.97
datafusion / arrow (0.926x ➖, 3↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:arrow 🚀 60779059 73686773 0.82
tpch_q02/datafusion:arrow 23065189 23980283 0.96
tpch_q03/datafusion:arrow 36980997 39270522 0.94
tpch_q04/datafusion:arrow 🚀 29146389 33907145 0.86
tpch_q05/datafusion:arrow 92935293 99499088 0.93
tpch_q06/datafusion:arrow 21291944 22872688 0.93
tpch_q07/datafusion:arrow 178717910 196483500 0.91
tpch_q08/datafusion:arrow 47849013 49134990 0.97
tpch_q09/datafusion:arrow 74710846 80907740 0.92
tpch_q10/datafusion:arrow 56464466 62252806 0.91
tpch_q11/datafusion:arrow 11820321 12780295 0.92
tpch_q12/datafusion:arrow 58266719 62067914 0.94
tpch_q13/datafusion:arrow 50693120 52028650 0.97
tpch_q14/datafusion:arrow 25614647 25912203 0.99
tpch_q15/datafusion:arrow 51347674 56006895 0.92
tpch_q16/datafusion:arrow 23534645 25885821 0.91
tpch_q17/datafusion:arrow 84931638 92360746 0.92
tpch_q18/datafusion:arrow 156971177 168009896 0.93
tpch_q19/datafusion:arrow 41251205 44075049 0.94
tpch_q20/datafusion:arrow 41302272 42532225 0.97
tpch_q21/datafusion:arrow 239596815 260060625 0.92
tpch_q22/datafusion:arrow 🚀 20563584 23206307 0.89
duckdb / vortex-file-compressed (0.959x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 32557668 32998908 0.99
tpch_q02/duckdb:vortex-file-compressed 27826573 27111079 1.03
tpch_q03/duckdb:vortex-file-compressed 28388272 29311010 0.97
tpch_q04/duckdb:vortex-file-compressed 29255940 30996196 0.94
tpch_q05/duckdb:vortex-file-compressed 34457267 33494512 1.03
tpch_q06/duckdb:vortex-file-compressed 11310732 11382029 0.99
tpch_q07/duckdb:vortex-file-compressed 37118436 38531127 0.96
tpch_q08/duckdb:vortex-file-compressed 39012295 40379995 0.97
tpch_q09/duckdb:vortex-file-compressed 76962201 81864298 0.94
tpch_q10/duckdb:vortex-file-compressed 36291283 38975646 0.93
tpch_q11/duckdb:vortex-file-compressed 16081393 16704532 0.96
tpch_q12/duckdb:vortex-file-compressed 23733217 24914354 0.95
tpch_q13/duckdb:vortex-file-compressed 37216402 38013671 0.98
tpch_q14/duckdb:vortex-file-compressed 21301709 23119096 0.92
tpch_q15/duckdb:vortex-file-compressed 15684366 17031257 0.92
tpch_q16/duckdb:vortex-file-compressed 30061125 31152973 0.96
tpch_q17/duckdb:vortex-file-compressed 24955468 25329962 0.99
tpch_q18/duckdb:vortex-file-compressed 49803671 51952498 0.96
tpch_q19/duckdb:vortex-file-compressed 31357468 33962713 0.92
tpch_q20/duckdb:vortex-file-compressed 33199551 36490412 0.91
tpch_q21/duckdb:vortex-file-compressed 102423404 110779587 0.92
tpch_q22/duckdb:vortex-file-compressed 18291374 19189515 0.95
duckdb / vortex-compact (0.955x ➖, 2↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 40277647 40623542 0.99
tpch_q02/duckdb:vortex-compact 36900338 37901597 0.97
tpch_q03/duckdb:vortex-compact 32754727 34725863 0.94
tpch_q04/duckdb:vortex-compact 33928759 36333955 0.93
tpch_q05/duckdb:vortex-compact 38640708 39818545 0.97
tpch_q06/duckdb:vortex-compact 🚀 14164970 15799310 0.90
tpch_q07/duckdb:vortex-compact 42472004 45304616 0.94
tpch_q08/duckdb:vortex-compact 45993306 47106297 0.98
tpch_q09/duckdb:vortex-compact 89788940 93366691 0.96
tpch_q10/duckdb:vortex-compact 40738669 41649259 0.98
tpch_q11/duckdb:vortex-compact 🚀 19595599 21856565 0.90
tpch_q12/duckdb:vortex-compact 37557018 38551506 0.97
tpch_q13/duckdb:vortex-compact 46337899 48609857 0.95
tpch_q14/duckdb:vortex-compact 30974353 32756857 0.95
tpch_q15/duckdb:vortex-compact 19560088 20316996 0.96
tpch_q16/duckdb:vortex-compact 37578970 36651358 1.03
tpch_q17/duckdb:vortex-compact 31662356 32072609 0.99
tpch_q18/duckdb:vortex-compact 50455180 53747135 0.94
tpch_q19/duckdb:vortex-compact 37823326 42018958 0.90
tpch_q20/duckdb:vortex-compact 46483635 46674526 1.00
tpch_q21/duckdb:vortex-compact 110942426 119690773 0.93
tpch_q22/duckdb:vortex-compact 20360387 21553407 0.94
duckdb / parquet (0.989x ➖, 3↑ 4↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 🚨 100725255 80232441 1.26
tpch_q02/duckdb:parquet 39604014 40301331 0.98
tpch_q03/duckdb:parquet 🚀 68990541 83535957 0.83
tpch_q04/duckdb:parquet 47788254 50489348 0.95
tpch_q05/duckdb:parquet 65920784 67398278 0.98
tpch_q06/duckdb:parquet 20664738 21338002 0.97
tpch_q07/duckdb:parquet 🚀 66677787 76205987 0.87
tpch_q08/duckdb:parquet 🚨 91739493 82327440 1.11
tpch_q09/duckdb:parquet 🚀 134351416 156502820 0.86
tpch_q10/duckdb:parquet 🚨 142167690 128811826 1.10
tpch_q11/duckdb:parquet 21918989 22608823 0.97
tpch_q12/duckdb:parquet 45604033 49212801 0.93
tpch_q13/duckdb:parquet 264147031 279407537 0.95
tpch_q14/duckdb:parquet 48414387 50202212 0.96
tpch_q15/duckdb:parquet 🚨 31412658 24815127 1.27
tpch_q16/duckdb:parquet 54491752 57576736 0.95
tpch_q17/duckdb:parquet 59465164 55732272 1.07
tpch_q18/duckdb:parquet 120189694 115142809 1.04
tpch_q19/duckdb:parquet 68743109 70779160 0.97
tpch_q20/duckdb:parquet 73744470 77896768 0.95
tpch_q21/duckdb:parquet 176942311 187973948 0.94
tpch_q22/duckdb:parquet 52103222 53640177 0.97
duckdb / duckdb (0.962x ➖, 3↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:duckdb 16086345 16342277 0.98
tpch_q02/duckdb:duckdb 🚀 11868976 13443938 0.88
tpch_q03/duckdb:duckdb 19661299 20131387 0.98
tpch_q04/duckdb:duckdb 19051942 20028921 0.95
tpch_q05/duckdb:duckdb 19738900 20237072 0.98
tpch_q06/duckdb:duckdb 5255532 5610613 0.94
tpch_q07/duckdb:duckdb 21905934 22792041 0.96
tpch_q08/duckdb:duckdb 21247119 21502692 0.99
tpch_q09/duckdb:duckdb 55070239 56571864 0.97
tpch_q10/duckdb:duckdb 🚀 43837348 48735591 0.90
tpch_q11/duckdb:duckdb 🚀 5481850 6230102 0.88
tpch_q12/duckdb:duckdb 13834555 14102051 0.98
tpch_q13/duckdb:duckdb 36749423 37594061 0.98
tpch_q14/duckdb:duckdb 16309405 17834270 0.91
tpch_q15/duckdb:duckdb 12048814 12247763 0.98
tpch_q16/duckdb:duckdb 22568053 22680334 1.00
tpch_q17/duckdb:duckdb 13548690 13773001 0.98
tpch_q18/duckdb:duckdb 38441528 37967373 1.01
tpch_q19/duckdb:duckdb 27481172 29574750 0.93
tpch_q20/duckdb:duckdb 23442432 22910570 1.02
tpch_q21/duckdb:duckdb 56138685 56156680 1.00
tpch_q22/duckdb:duckdb 24354790 24952367 0.98
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:arrow -17.5% +29.3% -36.2% +23.8% ✅ faster
1 datafusion:vortex-compact -15.5% +29.3% -34.6% +23.8% ✅ faster
1 datafusion:vortex-file-compressed -14.8% +29.3% -34.1% +23.8% ✅ faster
1 duckdb:duckdb -1.6% +29.3% -23.8% +23.8% ✅ faster
1 duckdb:vortex-compact -0.9% +29.3% -23.3% +23.8% ✅ faster
1 duckdb:vortex-file-compressed -1.3% +29.3% -23.7% +23.8% ✅ faster
2 datafusion:arrow -3.8% -3.3% -0.6% +23.8% ➖ noise
2 datafusion:vortex-compact -51.0% -3.3% -49.3% +23.8% ✅ faster
2 datafusion:vortex-file-compressed -54.3% -3.3% -52.8% +23.8% ✅ faster
2 duckdb:duckdb -11.7% -3.3% -8.7% +25.5% ➖ noise
2 duckdb:vortex-compact -2.6% -3.3% +0.6% +23.8% ➖ noise
2 duckdb:vortex-file-compressed +2.6% -3.3% +6.1% +23.8% ➖ noise
3 datafusion:arrow -5.8% -15.6% +11.6% +23.8% ➖ noise
3 datafusion:vortex-compact -12.5% -15.6% +3.7% +23.8% ➖ noise
3 datafusion:vortex-file-compressed -14.4% -15.6% +1.4% +23.8% ➖ noise
3 duckdb:duckdb -2.3% -15.6% +15.7% +23.8% ➖ noise
3 duckdb:vortex-compact -5.7% -15.6% +11.8% +23.8% ➖ noise
3 duckdb:vortex-file-compressed -3.1% -15.6% +14.8% +23.8% ➖ noise
4 datafusion:arrow -14.0% -3.5% -10.9% +23.8% ➖ noise
4 datafusion:vortex-compact +132.4% -3.5% +140.9% +23.8% 🚨 regression
4 datafusion:vortex-file-compressed +205.9% -3.5% +216.9% +23.8% 🚨 regression
4 duckdb:duckdb -4.9% -3.5% -1.4% +23.8% ➖ noise
4 duckdb:vortex-compact -6.6% -3.5% -3.2% +23.8% ➖ noise
4 duckdb:vortex-file-compressed -5.6% -3.5% -2.2% +23.8% ➖ noise
5 datafusion:arrow -6.6% -2.7% -4.0% +23.8% ➖ noise
5 datafusion:vortex-compact +133.9% -2.7% +140.3% +23.8% 🚨 regression
5 datafusion:vortex-file-compressed +129.3% -2.7% +135.6% +23.8% 🚨 regression
5 duckdb:duckdb -2.5% -2.7% +0.2% +23.8% ➖ noise
5 duckdb:vortex-compact -3.0% -2.7% -0.3% +23.8% ➖ noise
5 duckdb:vortex-file-compressed +2.9% -2.7% +5.7% +23.8% ➖ noise
6 datafusion:arrow -6.9% -3.3% -3.7% +23.8% ➖ noise
6 datafusion:vortex-compact -17.8% -3.3% -15.0% +23.8% ➖ noise
6 datafusion:vortex-file-compressed -7.5% -3.3% -4.3% +23.8% ➖ noise
6 duckdb:duckdb -6.3% -3.3% -3.1% +23.8% ➖ noise
6 duckdb:vortex-compact -10.3% -3.3% -7.2% +23.8% ➖ noise
6 duckdb:vortex-file-compressed -0.6% -3.3% +2.8% +23.8% ➖ noise
7 datafusion:arrow -9.0% -6.6% -2.6% +23.8% ➖ noise
7 datafusion:vortex-compact +51.3% -6.6% +62.0% +23.8% 🚨 regression
7 datafusion:vortex-file-compressed +57.5% -6.6% +68.7% +23.8% 🚨 regression
7 duckdb:duckdb -3.9% -6.6% +3.0% +23.8% ➖ noise
7 duckdb:vortex-compact -6.3% -6.6% +0.4% +23.8% ➖ noise
7 duckdb:vortex-file-compressed -3.7% -6.6% +3.2% +23.8% ➖ noise
8 datafusion:arrow -2.6% +8.1% -9.9% +23.8% ➖ noise
8 datafusion:vortex-compact -53.2% +8.1% -56.7% +23.8% ✅ faster
8 datafusion:vortex-file-compressed -51.5% +8.1% -55.1% +23.8% ✅ faster
8 duckdb:duckdb -1.2% +8.1% -8.6% +23.8% ➖ noise
8 duckdb:vortex-compact -2.4% +8.1% -9.7% +23.8% ➖ noise
8 duckdb:vortex-file-compressed -3.4% +8.1% -10.6% +23.8% ➖ noise
9 datafusion:arrow -7.7% -7.8% +0.2% +23.8% ➖ noise
9 datafusion:vortex-compact -23.0% -7.8% -16.5% +23.8% ➖ noise
9 datafusion:vortex-file-compressed -23.4% -7.8% -17.0% +23.8% ➖ noise
9 duckdb:duckdb -2.7% -7.8% +5.6% +23.8% ➖ noise
9 duckdb:vortex-compact -3.8% -7.8% +4.3% +23.8% ➖ noise
9 duckdb:vortex-file-compressed -6.0% -7.8% +2.0% +23.8% ➖ noise
10 datafusion:arrow -9.3% +2.6% -11.6% +23.8% ➖ noise
10 datafusion:vortex-compact -3.4% +2.6% -5.8% +23.8% ➖ noise
10 datafusion:vortex-file-compressed -3.3% +2.6% -5.7% +23.8% ➖ noise
10 duckdb:duckdb -10.1% +2.6% -12.3% +23.8% ➖ noise
10 duckdb:vortex-compact -2.2% +2.6% -4.6% +23.8% ➖ noise
10 duckdb:vortex-file-compressed -6.9% +2.6% -9.2% +23.8% ➖ noise
11 datafusion:arrow -7.5% -3.2% -4.5% +23.8% ➖ noise
11 datafusion:vortex-compact +1.3% -3.2% +4.6% +23.8% ➖ noise
11 datafusion:vortex-file-compressed -6.5% -3.2% -3.5% +23.8% ➖ noise
11 duckdb:duckdb -12.0% -3.2% -9.1% +23.8% ➖ noise
11 duckdb:vortex-compact -10.3% -3.2% -7.4% +23.8% ➖ noise
11 duckdb:vortex-file-compressed -3.7% -3.2% -0.6% +23.8% ➖ noise
12 datafusion:arrow -6.1% -7.8% +1.8% +23.8% ➖ noise
12 datafusion:vortex-compact +4.8% -7.8% +13.7% +23.8% ➖ noise
12 datafusion:vortex-file-compressed +28.2% -7.8% +39.1% +23.8% 🚨 regression
12 duckdb:duckdb -1.9% -7.8% +6.4% +23.8% ➖ noise
12 duckdb:vortex-compact -2.6% -7.8% +5.7% +23.8% ➖ noise
12 duckdb:vortex-file-compressed -4.7% -7.8% +3.3% +23.8% ➖ noise
13 datafusion:arrow -2.6% -6.5% +4.2% +23.8% ➖ noise
13 datafusion:vortex-compact +25.1% -6.5% +33.8% +23.8% 🚨 regression
13 datafusion:vortex-file-compressed +54.8% -6.5% +65.5% +23.8% 🚨 regression
13 duckdb:duckdb -2.2% -6.5% +4.5% +23.8% ➖ noise
13 duckdb:vortex-compact -4.7% -6.5% +1.9% +23.8% ➖ noise
13 duckdb:vortex-file-compressed -2.1% -6.5% +4.7% +23.8% ➖ noise
14 datafusion:arrow -1.1% -3.7% +2.7% +23.8% ➖ noise
14 datafusion:vortex-compact -36.0% -3.7% -33.5% +23.8% ✅ faster
14 datafusion:vortex-file-compressed -38.5% -3.7% -36.1% +23.8% ✅ faster
14 duckdb:duckdb -8.6% -3.7% -5.0% +23.8% ➖ noise
14 duckdb:vortex-compact -5.4% -3.7% -1.8% +23.8% ➖ noise
14 duckdb:vortex-file-compressed -7.9% -3.7% -4.3% +23.8% ➖ noise
15 datafusion:arrow -8.3% +10.7% -17.2% +33.4% ➖ noise
15 datafusion:vortex-compact -20.8% +10.7% -28.5% +23.8% ✅ faster
15 datafusion:vortex-file-compressed -24.4% +10.7% -31.7% +23.8% ✅ faster
15 duckdb:duckdb -1.6% +10.7% -11.2% +23.8% ➖ noise
15 duckdb:vortex-compact -3.7% +10.7% -13.1% +23.8% ➖ noise
15 duckdb:vortex-file-compressed -7.9% +10.7% -16.8% +23.8% ➖ noise
16 datafusion:arrow -9.1% -5.7% -3.6% +23.8% ➖ noise
16 datafusion:vortex-compact -37.5% -5.7% -33.8% +23.8% ✅ faster
16 datafusion:vortex-file-compressed -41.0% -5.7% -37.5% +23.8% ✅ faster
16 duckdb:duckdb -0.5% -5.7% +5.5% +23.8% ➖ noise
16 duckdb:vortex-compact +2.5% -5.7% +8.7% +23.8% ➖ noise
16 duckdb:vortex-file-compressed -3.5% -5.7% +2.3% +23.8% ➖ noise
17 datafusion:arrow -8.0% +2.4% -10.2% +23.8% ➖ noise
17 datafusion:vortex-compact -20.7% +2.4% -22.5% +23.8% ✅ faster
17 datafusion:vortex-file-compressed -15.7% +2.4% -17.7% +23.8% ➖ noise
17 duckdb:duckdb -1.6% +2.4% -3.9% +23.8% ➖ noise
17 duckdb:vortex-compact -1.3% +2.4% -3.6% +23.8% ➖ noise
17 duckdb:vortex-file-compressed -1.5% +2.4% -3.8% +23.8% ➖ noise
18 datafusion:arrow -6.6% +0.8% -7.3% +23.8% ➖ noise
18 datafusion:vortex-compact +86.9% +0.8% +85.5% +23.8% 🚨 regression
18 datafusion:vortex-file-compressed +101.3% +0.8% +99.7% +23.8% 🚨 regression
18 duckdb:duckdb +1.2% +0.8% +0.5% +23.8% ➖ noise
18 duckdb:vortex-compact -6.1% +0.8% -6.8% +23.8% ➖ noise
18 duckdb:vortex-file-compressed -4.1% +0.8% -4.9% +23.8% ➖ noise
19 datafusion:arrow -6.4% -3.7% -2.8% +23.8% ➖ noise
19 datafusion:vortex-compact -34.7% -3.7% -32.2% +23.8% ✅ faster
19 datafusion:vortex-file-compressed -39.3% -3.7% -37.0% +23.8% ✅ faster
19 duckdb:duckdb -7.1% -3.7% -3.5% +23.8% ➖ noise
19 duckdb:vortex-compact -10.0% -3.7% -6.6% +23.8% ➖ noise
19 duckdb:vortex-file-compressed -7.7% -3.7% -4.2% +23.8% ➖ noise
20 datafusion:arrow -2.9% -4.4% +1.6% +23.8% ➖ noise
20 datafusion:vortex-compact -42.3% -4.4% -39.7% +23.8% ✅ faster
20 datafusion:vortex-file-compressed -48.7% -4.4% -46.3% +23.8% ✅ faster
20 duckdb:duckdb +2.3% -4.4% +7.1% +23.8% ➖ noise
20 duckdb:vortex-compact -0.4% -4.4% +4.2% +23.8% ➖ noise
20 duckdb:vortex-file-compressed -9.0% -4.4% -4.8% +23.8% ➖ noise
21 datafusion:arrow -7.9% -4.8% -3.2% +23.8% ➖ noise
21 datafusion:vortex-compact +103.9% -4.8% +114.2% +23.8% 🚨 regression
21 datafusion:vortex-file-compressed +127.7% -4.8% +139.2% +23.8% 🚨 regression
21 duckdb:duckdb -0.0% -4.8% +5.0% +23.8% ➖ noise
21 duckdb:vortex-compact -7.3% -4.8% -2.6% +23.8% ➖ noise
21 duckdb:vortex-file-compressed -7.5% -4.8% -2.9% +23.8% ➖ noise
22 datafusion:arrow -11.4% -2.9% -8.7% +23.8% ➖ noise
22 datafusion:vortex-compact -19.8% -2.9% -17.4% +23.8% ➖ noise
22 datafusion:vortex-file-compressed -12.0% -2.9% -9.4% +23.8% ➖ noise
22 duckdb:duckdb -2.4% -2.9% +0.5% +23.8% ➖ noise
22 duckdb:vortex-compact -5.5% -2.9% -2.7% +23.8% ➖ noise
22 duckdb:vortex-file-compressed -4.7% -2.9% -1.8% +23.8% ➖ noise

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: FineWeb NVMe

Verdict: No clear signal (medium confidence)
Attributed Vortex impact: -8.7%
Vortex (geomean): 0.958x ➖
Parquet (geomean): 1.050x ➖
Shifts: Parquet (control) +5.0% · Median polish +3.5%

datafusion / vortex-file-compressed (0.962x ➖, 4↑ 3↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 🚨 7219535 3893731 1.85
fineweb_q01/datafusion:vortex-file-compressed 🚨 29421227 26425017 1.11
fineweb_q02/datafusion:vortex-file-compressed 🚀 24103456 27497082 0.88
fineweb_q03/datafusion:vortex-file-compressed 🚨 133852954 81046690 1.65
fineweb_q04/datafusion:vortex-file-compressed 226701443 247766663 0.91
fineweb_q05/datafusion:vortex-file-compressed 212756448 233341732 0.91
fineweb_q06/datafusion:vortex-file-compressed 🚀 56135108 65164789 0.86
fineweb_q07/datafusion:vortex-file-compressed 🚀 58877068 66995845 0.88
fineweb_q08/datafusion:vortex-file-compressed 🚀 10159924 27245235 0.37
datafusion / vortex-compact (0.729x ✅, 8↑ 1↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 🚨 8707194 4019759 2.17
fineweb_q01/datafusion:vortex-compact 🚀 102147656 162245384 0.63
fineweb_q02/datafusion:vortex-compact 🚀 130536441 164689013 0.79
fineweb_q03/datafusion:vortex-compact 🚀 882747260 1433728638 0.62
fineweb_q04/datafusion:vortex-compact 🚀 990004519 1554469893 0.64
fineweb_q05/datafusion:vortex-compact 🚀 899312079 1225642173 0.73
fineweb_q06/datafusion:vortex-compact 🚀 506006737 699010497 0.72
fineweb_q07/datafusion:vortex-compact 🚀 519012791 731781192 0.71
fineweb_q08/datafusion:vortex-compact 🚀 7890732 21575588 0.37
datafusion / parquet (1.010x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 6175246 5921069 1.04
fineweb_q01/datafusion:parquet 271647459 277926374 0.98
fineweb_q02/datafusion:parquet 282226194 279148184 1.01
fineweb_q03/datafusion:parquet 282546134 272847097 1.04
fineweb_q04/datafusion:parquet 288415897 281275467 1.03
fineweb_q05/datafusion:parquet 285034175 281367965 1.01
fineweb_q06/datafusion:parquet 278062183 286157301 0.97
fineweb_q07/datafusion:parquet 274662573 270265412 1.02
fineweb_q08/datafusion:parquet 267736845 268672952 1.00
duckdb / vortex-file-compressed (1.120x ❌, 1↑ 5↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 🚨 3674386 3296636 1.11
fineweb_q01/duckdb:vortex-file-compressed 🚨 40225311 26145381 1.54
fineweb_q02/duckdb:vortex-file-compressed 🚨 35088560 28127859 1.25
fineweb_q03/duckdb:vortex-file-compressed 149289579 138591407 1.08
fineweb_q04/duckdb:vortex-file-compressed 246556204 232269629 1.06
fineweb_q05/duckdb:vortex-file-compressed 223570952 218465345 1.02
fineweb_q06/duckdb:vortex-file-compressed 🚀 60281466 68619570 0.88
fineweb_q07/duckdb:vortex-file-compressed 🚨 71637784 64392131 1.11
fineweb_q08/duckdb:vortex-file-compressed 🚨 27258075 23951793 1.14
duckdb / vortex-compact (1.073x ➖, 0↑ 3↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 🚨 4759029 4068248 1.17
fineweb_q01/duckdb:vortex-compact 106050441 107901266 0.98
fineweb_q02/duckdb:vortex-compact 111633338 115127448 0.97
fineweb_q03/duckdb:vortex-compact 894676614 857368156 1.04
fineweb_q04/duckdb:vortex-compact 991809397 908959545 1.09
fineweb_q05/duckdb:vortex-compact 881492870 808411096 1.09
fineweb_q06/duckdb:vortex-compact 🚨 515507520 465328577 1.11
fineweb_q07/duckdb:vortex-compact 521320367 476619288 1.09
fineweb_q08/duckdb:vortex-compact 🚨 23910826 21314654 1.12
duckdb / parquet (1.091x ➖, 0↑ 3↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 🚨 41789390 34771093 1.20
fineweb_q01/duckdb:parquet 99826583 92562283 1.08
fineweb_q02/duckdb:parquet 🚨 109979058 95774569 1.15
fineweb_q03/duckdb:parquet 324140277 307035077 1.06
fineweb_q04/duckdb:parquet 450547516 435385721 1.03
fineweb_q05/duckdb:parquet 419678837 408671009 1.03
fineweb_q06/duckdb:parquet 🚨 218202341 197306460 1.11
fineweb_q07/duckdb:parquet 227373399 208787489 1.09
fineweb_q08/duckdb:parquet 62337249 57237385 1.09
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 datafusion:vortex-compact +116.6% +12.0% +93.5% +70.9% 🚨 regression
0 datafusion:vortex-file-compressed +85.4% +12.0% +65.6% +53.3% 🚨 regression
0 duckdb:vortex-compact +17.0% +12.0% +4.5% +39.5% ➖ noise
0 duckdb:vortex-file-compressed +11.5% +12.0% -0.4% +54.5% ➖ noise
1 datafusion:vortex-compact -37.0% +2.7% -38.7% +21.1% ✅ faster
1 datafusion:vortex-file-compressed +11.3% +2.7% +8.4% +17.5% ➖ noise
1 duckdb:vortex-compact -1.7% +2.7% -4.3% +14.7% ➖ noise
1 duckdb:vortex-file-compressed +53.9% +2.7% +49.9% +28.8% 🚨 regression
2 datafusion:vortex-compact -20.7% +7.7% -26.4% +10.0% ✅ faster
2 datafusion:vortex-file-compressed -12.3% +7.7% -18.6% +21.7% ✅ faster
2 duckdb:vortex-compact -3.0% +7.7% -10.0% +17.5% ➖ noise
2 duckdb:vortex-file-compressed +24.7% +7.7% +15.8% +30.6% ➖ noise
3 datafusion:vortex-compact -38.4% +4.6% -41.1% +10.0% ✅ faster
3 datafusion:vortex-file-compressed +65.2% +4.6% +58.0% +22.6% 🚨 regression
3 duckdb:vortex-compact +4.4% +4.6% -0.2% +10.0% ➖ noise
3 duckdb:vortex-file-compressed +7.7% +4.6% +3.0% +29.4% ➖ noise
4 datafusion:vortex-compact -36.3% +3.0% -38.2% +10.0% ✅ faster
4 datafusion:vortex-file-compressed -8.5% +3.0% -11.2% +10.0% ✅ faster
4 duckdb:vortex-compact +9.1% +3.0% +5.9% +10.0% ➖ noise
4 duckdb:vortex-file-compressed +6.2% +3.0% +3.0% +11.2% ➖ noise
5 datafusion:vortex-compact -26.6% +2.0% -28.1% +10.0% ✅ faster
5 datafusion:vortex-file-compressed -8.8% +2.0% -10.6% +10.0% ✅ faster
5 duckdb:vortex-compact +9.0% +2.0% +6.9% +10.0% ➖ noise
5 duckdb:vortex-file-compressed +2.3% +2.0% +0.3% +10.0% ➖ noise
6 datafusion:vortex-compact -27.6% +3.7% -30.2% +10.0% ✅ faster
6 datafusion:vortex-file-compressed -13.9% +3.7% -16.9% +11.0% ✅ faster
6 duckdb:vortex-compact +10.8% +3.7% +6.9% +10.0% ➖ noise
6 duckdb:vortex-file-compressed -12.2% +3.7% -15.3% +20.8% ➖ noise
7 datafusion:vortex-compact -29.1% +5.2% -32.6% +10.0% ✅ faster
7 datafusion:vortex-file-compressed -12.1% +5.2% -16.5% +10.0% ✅ faster
7 duckdb:vortex-compact +9.4% +5.2% +4.0% +10.0% ➖ noise
7 duckdb:vortex-file-compressed +11.3% +5.2% +5.8% +10.4% ➖ noise
8 datafusion:vortex-compact -63.4% +4.2% -64.9% +24.3% ✅ faster
8 datafusion:vortex-file-compressed -62.7% +4.2% -64.2% +18.3% ✅ faster
8 duckdb:vortex-compact +12.2% +4.2% +7.7% +24.1% ➖ noise
8 duckdb:vortex-file-compressed +13.8% +4.2% +9.2% +13.6% ➖ noise

@0ax1 0ax1 changed the title experiment: 2 datafusion no pool experiment: v2 datafusion no pool Mar 22, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: TPC-H SF=10 on NVME

Verdict: Likely regression (high confidence)
Attributed Vortex impact: +40.1%
Vortex (geomean): 1.541x ❌
Parquet (geomean): 0.991x ➖
Shifts: Parquet (control) -0.9% · Median polish +14.3%

datafusion / vortex-file-compressed (2.063x ❌, 2↑ 18↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚀 404034043 456987618 0.88
tpch_q02/datafusion:vortex-file-compressed 🚨 252254099 169038545 1.49
tpch_q03/datafusion:vortex-file-compressed 🚨 919968739 277217079 3.32
tpch_q04/datafusion:vortex-file-compressed 🚨 488220557 116224371 4.20
tpch_q05/datafusion:vortex-file-compressed 🚨 1609093710 500898495 3.21
tpch_q06/datafusion:vortex-file-compressed 40478881 43597353 0.93
tpch_q07/datafusion:vortex-file-compressed 🚨 808085685 687058295 1.18
tpch_q08/datafusion:vortex-file-compressed 🚨 1929970142 484501547 3.98
tpch_q09/datafusion:vortex-file-compressed 🚨 2684642364 827436805 3.24
tpch_q10/datafusion:vortex-file-compressed 🚨 571862931 285093351 2.01
tpch_q11/datafusion:vortex-file-compressed 🚀 59710646 115377370 0.52
tpch_q12/datafusion:vortex-file-compressed 🚨 362242696 128062721 2.83
tpch_q13/datafusion:vortex-file-compressed 🚨 359881628 223435876 1.61
tpch_q14/datafusion:vortex-file-compressed 🚨 119215003 73373337 1.62
tpch_q15/datafusion:vortex-file-compressed 103650097 110090819 0.94
tpch_q16/datafusion:vortex-file-compressed 🚨 191939724 94177372 2.04
tpch_q17/datafusion:vortex-file-compressed 🚨 1852145728 739128438 2.51
tpch_q18/datafusion:vortex-file-compressed 🚨 2179412891 966655623 2.25
tpch_q19/datafusion:vortex-file-compressed 🚨 199590573 131303915 1.52
tpch_q20/datafusion:vortex-file-compressed 🚨 333824440 208286362 1.60
tpch_q21/datafusion:vortex-file-compressed 🚨 10120026263 752288821 13.45
tpch_q22/datafusion:vortex-file-compressed 🚨 219413235 60120116 3.65
datafusion / vortex-compact (2.023x ❌, 1↑ 18↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 482447670 464731795 1.04
tpch_q02/datafusion:vortex-compact 🚨 253059310 163206474 1.55
tpch_q03/datafusion:vortex-compact 🚨 906321265 257643842 3.52
tpch_q04/datafusion:vortex-compact 🚨 499140790 126444710 3.95
tpch_q05/datafusion:vortex-compact 🚨 1586648482 470344221 3.37
tpch_q06/datafusion:vortex-compact 63425915 62715999 1.01
tpch_q07/datafusion:vortex-compact 🚨 836263096 669828045 1.25
tpch_q08/datafusion:vortex-compact 🚨 1875631596 474097636 3.96
tpch_q09/datafusion:vortex-compact 🚨 2736812794 810136647 3.38
tpch_q10/datafusion:vortex-compact 🚨 594673852 325240644 1.83
tpch_q11/datafusion:vortex-compact 🚀 64108916 127657808 0.50
tpch_q12/datafusion:vortex-compact 🚨 389549680 181980490 2.14
tpch_q13/datafusion:vortex-compact 🚨 403054466 277762259 1.45
tpch_q14/datafusion:vortex-compact 🚨 137927521 90665293 1.52
tpch_q15/datafusion:vortex-compact 165551194 167002857 0.99
tpch_q16/datafusion:vortex-compact 🚨 199660246 97230602 2.05
tpch_q17/datafusion:vortex-compact 🚨 1910498276 749327652 2.55
tpch_q18/datafusion:vortex-compact 🚨 2279842742 987684062 2.31
tpch_q19/datafusion:vortex-compact 🚨 289196868 231132577 1.25
tpch_q20/datafusion:vortex-compact 🚨 355882321 229542122 1.55
tpch_q21/datafusion:vortex-compact 🚨 10475951475 799868456 13.10
tpch_q22/datafusion:vortex-compact 🚨 226934858 70815712 3.20
datafusion / parquet (0.969x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 461723752 495566398 0.93
tpch_q02/datafusion:parquet 263061813 256926502 1.02
tpch_q03/datafusion:parquet 410383956 391136140 1.05
tpch_q04/datafusion:parquet 150855775 147966036 1.02
tpch_q05/datafusion:parquet 578486476 580012159 1.00
tpch_q06/datafusion:parquet 136156324 129919453 1.05
tpch_q07/datafusion:parquet 794793980 796160308 1.00
tpch_q08/datafusion:parquet 645440612 648176896 1.00
tpch_q09/datafusion:parquet 974357668 993834168 0.98
tpch_q10/datafusion:parquet 596436281 611546116 0.98
tpch_q11/datafusion:parquet 196633655 195471373 1.01
tpch_q12/datafusion:parquet 248421399 269358560 0.92
tpch_q13/datafusion:parquet 343575752 376719458 0.91
tpch_q14/datafusion:parquet 196985826 208138724 0.95
tpch_q15/datafusion:parquet 256085750 275163379 0.93
tpch_q16/datafusion:parquet 178233731 182017036 0.98
tpch_q17/datafusion:parquet 834278123 898683617 0.93
tpch_q18/datafusion:parquet 1022027606 1125403483 0.91
tpch_q19/datafusion:parquet 339003872 354102857 0.96
tpch_q20/datafusion:parquet 374007560 394495054 0.95
tpch_q21/datafusion:parquet 891694728 974453568 0.92
tpch_q22/datafusion:parquet 223649208 229690907 0.97
datafusion / arrow (1.073x ➖, 0↑ 11↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:arrow 🚨 846396195 734916740 1.15
tpch_q02/datafusion:arrow 🚨 228042077 205229546 1.11
tpch_q03/datafusion:arrow 🚨 696638515 548162651 1.27
tpch_q04/datafusion:arrow 🚨 502865006 403691440 1.25
tpch_q05/datafusion:arrow 🚨 1271888862 1057259088 1.20
tpch_q06/datafusion:arrow 🚨 393065174 317923596 1.24
tpch_q07/datafusion:arrow 🚨 1682306421 1354676254 1.24
tpch_q08/datafusion:arrow 🚨 1557238746 1319710609 1.18
tpch_q09/datafusion:arrow 🚨 1845249162 1576217608 1.17
tpch_q10/datafusion:arrow 🚨 836609820 700593428 1.19
tpch_q11/datafusion:arrow 🚨 204826235 180275968 1.14
tpch_q12/datafusion:arrow 740482755 813283824 0.91
tpch_q13/datafusion:arrow 542636144 551989573 0.98
tpch_q14/datafusion:arrow 360913074 381193904 0.95
tpch_q15/datafusion:arrow 760883046 811202386 0.94
tpch_q16/datafusion:arrow 122370793 122379178 1.00
tpch_q17/datafusion:arrow 1571206109 1666341620 0.94
tpch_q18/datafusion:arrow 2119494818 2185924250 0.97
tpch_q19/datafusion:arrow 578589444 590048770 0.98
tpch_q20/datafusion:arrow 550175307 574312397 0.96
tpch_q21/datafusion:arrow 4078886579 4130040468 0.99
tpch_q22/datafusion:arrow 180911828 181355780 1.00
duckdb / vortex-file-compressed (1.156x ❌, 0↑ 11↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 177828933 178276247 1.00
tpch_q02/duckdb:vortex-file-compressed 53755541 54187738 0.99
tpch_q03/duckdb:vortex-file-compressed 132563926 130657144 1.01
tpch_q04/duckdb:vortex-file-compressed 153886808 150566598 1.02
tpch_q05/duckdb:vortex-file-compressed 140264087 133150239 1.05
tpch_q06/duckdb:vortex-file-compressed 48936437 48163214 1.02
tpch_q07/duckdb:vortex-file-compressed 149325066 148117343 1.01
tpch_q08/duckdb:vortex-file-compressed 184798161 177784157 1.04
tpch_q09/duckdb:vortex-file-compressed 358829332 336089053 1.07
tpch_q10/duckdb:vortex-file-compressed 🚨 186709316 155472138 1.20
tpch_q11/duckdb:vortex-file-compressed 🚨 46855786 34670563 1.35
tpch_q12/duckdb:vortex-file-compressed 🚨 112880334 96804628 1.17
tpch_q13/duckdb:vortex-file-compressed 🚨 327018579 247856528 1.32
tpch_q14/duckdb:vortex-file-compressed 🚨 103416171 70880320 1.46
tpch_q15/duckdb:vortex-file-compressed 🚨 133343623 97461231 1.37
tpch_q16/duckdb:vortex-file-compressed 🚨 102311590 81181606 1.26
tpch_q17/duckdb:vortex-file-compressed 🚨 135972122 102634661 1.32
tpch_q18/duckdb:vortex-file-compressed 326531235 300076845 1.09
tpch_q19/duckdb:vortex-file-compressed 99359355 90688314 1.10
tpch_q20/duckdb:vortex-file-compressed 🚨 197846766 164359328 1.20
tpch_q21/duckdb:vortex-file-compressed 🚨 815236023 558792159 1.46
tpch_q22/duckdb:vortex-file-compressed 🚨 87978510 77108464 1.14
duckdb / vortex-compact (1.169x ❌, 0↑ 16↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 🚨 256001899 232393643 1.10
tpch_q02/duckdb:vortex-compact 🚨 72079477 57671716 1.25
tpch_q03/duckdb:vortex-compact 🚨 183299706 140939499 1.30
tpch_q04/duckdb:vortex-compact 🚨 215561740 176858853 1.22
tpch_q05/duckdb:vortex-compact 159740038 147220954 1.09
tpch_q06/duckdb:vortex-compact 89261528 84843484 1.05
tpch_q07/duckdb:vortex-compact 🚨 229094588 207861694 1.10
tpch_q08/duckdb:vortex-compact 🚨 245874031 194851224 1.26
tpch_q09/duckdb:vortex-compact 🚨 450130040 361464015 1.25
tpch_q10/duckdb:vortex-compact 🚨 243088215 187815432 1.29
tpch_q11/duckdb:vortex-compact 🚨 53586816 41807787 1.28
tpch_q12/duckdb:vortex-compact 🚨 245820527 208398092 1.18
tpch_q13/duckdb:vortex-compact 329283428 301534855 1.09
tpch_q14/duckdb:vortex-compact 🚨 112752532 102037540 1.11
tpch_q15/duckdb:vortex-compact 🚨 133660181 118212258 1.13
tpch_q16/duckdb:vortex-compact 92620209 85409219 1.08
tpch_q17/duckdb:vortex-compact 127223112 117529028 1.08
tpch_q18/duckdb:vortex-compact 319853459 310783755 1.03
tpch_q19/duckdb:vortex-compact 🚨 138062997 116516580 1.18
tpch_q20/duckdb:vortex-compact 🚨 267539320 206800047 1.29
tpch_q21/duckdb:vortex-compact 🚨 808206738 634653842 1.27
tpch_q22/duckdb:vortex-compact 🚨 102105521 90105001 1.13
duckdb / parquet (1.014x ➖, 0↑ 1↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 263354582 257309814 1.02
tpch_q02/duckdb:parquet 96006124 99173665 0.97
tpch_q03/duckdb:parquet 211140571 210954982 1.00
tpch_q04/duckdb:parquet 139731805 135498171 1.03
tpch_q05/duckdb:parquet 219243207 218716690 1.00
tpch_q06/duckdb:parquet 66851147 68380928 0.98
tpch_q07/duckdb:parquet 184053324 182405366 1.01
tpch_q08/duckdb:parquet 256695171 253813157 1.01
tpch_q09/duckdb:parquet 465705536 468132196 0.99
tpch_q10/duckdb:parquet 611822508 604784676 1.01
tpch_q11/duckdb:parquet 🚨 47638208 40535905 1.18
tpch_q12/duckdb:parquet 125883065 124914768 1.01
tpch_q13/duckdb:parquet 445121306 456022674 0.98
tpch_q14/duckdb:parquet 174797594 173111746 1.01
tpch_q15/duckdb:parquet 99738442 98252758 1.02
tpch_q16/duckdb:parquet 163357269 157441898 1.04
tpch_q17/duckdb:parquet 178768845 181959070 0.98
tpch_q18/duckdb:parquet 352137851 347363499 1.01
tpch_q19/duckdb:parquet 278774832 275860355 1.01
tpch_q20/duckdb:parquet 235793645 227926438 1.03
tpch_q21/duckdb:parquet 576617420 570464899 1.01
tpch_q22/duckdb:parquet 296053900 289869317 1.02
duckdb / duckdb (1.187x ❌, 0↑ 20↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:duckdb 126077583 118282260 1.07
tpch_q02/duckdb:duckdb 🚨 53142859 45695169 1.16
tpch_q03/duckdb:duckdb 🚨 117557131 96494139 1.22
tpch_q04/duckdb:duckdb 🚨 170576012 131383277 1.30
tpch_q05/duckdb:duckdb 🚨 136628729 109382635 1.25
tpch_q06/duckdb:duckdb 🚨 44731884 39774062 1.12
tpch_q07/duckdb:duckdb 🚨 100917433 85370077 1.18
tpch_q08/duckdb:duckdb 🚨 138465477 112162874 1.23
tpch_q09/duckdb:duckdb 🚨 338145759 274965171 1.23
tpch_q10/duckdb:duckdb 🚨 252445566 204982966 1.23
tpch_q11/duckdb:duckdb 🚨 19796562 15514421 1.28
tpch_q12/duckdb:duckdb 🚨 95711493 84514787 1.13
tpch_q13/duckdb:duckdb 🚨 295836069 251944673 1.17
tpch_q14/duckdb:duckdb 🚨 83467719 71326522 1.17
tpch_q15/duckdb:duckdb 🚨 88700357 77874123 1.14
tpch_q16/duckdb:duckdb 🚨 89191214 72408935 1.23
tpch_q17/duckdb:duckdb 🚨 94782159 84491479 1.12
tpch_q18/duckdb:duckdb 🚨 262240345 214199388 1.22
tpch_q19/duckdb:duckdb 🚨 134196869 114969888 1.17
tpch_q20/duckdb:duckdb 🚨 126836708 111619094 1.14
tpch_q21/duckdb:duckdb 🚨 388236692 302366274 1.28
tpch_q22/duckdb:duckdb 77229509 70655251 1.09
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:arrow +15.2% -2.3% +17.9% +11.4% 🚨 regression
1 datafusion:vortex-compact +3.8% -2.3% +6.3% +13.8% ➖ noise
1 datafusion:vortex-file-compressed -11.6% -2.3% -9.5% +15.3% ➖ noise
1 duckdb:duckdb +6.6% -2.3% +9.2% +11.9% ➖ noise
1 duckdb:vortex-compact +10.2% -2.3% +12.8% +11.6% 🚨 regression
1 duckdb:vortex-file-compressed -0.3% -2.3% +2.1% +13.5% ➖ noise
2 datafusion:arrow +11.1% -0.4% +11.6% +10.0% 🚨 regression
2 datafusion:vortex-compact +55.1% -0.4% +55.7% +10.0% 🚨 regression
2 datafusion:vortex-file-compressed +49.2% -0.4% +49.9% +10.0% 🚨 regression
2 duckdb:duckdb +16.3% -0.4% +16.8% +10.0% 🚨 regression
2 duckdb:vortex-compact +25.0% -0.4% +25.5% +12.1% 🚨 regression
2 duckdb:vortex-file-compressed -0.8% -0.4% -0.4% +10.0% ➖ noise
3 datafusion:arrow +27.1% +2.5% +24.0% +10.0% 🚨 regression
3 datafusion:vortex-compact +251.8% +2.5% +243.3% +10.0% 🚨 regression
3 datafusion:vortex-file-compressed +231.9% +2.5% +223.8% +10.0% 🚨 regression
3 duckdb:duckdb +21.8% +2.5% +18.9% +10.0% 🚨 regression
3 duckdb:vortex-compact +30.1% +2.5% +26.9% +10.0% 🚨 regression
3 duckdb:vortex-file-compressed +1.5% +2.5% -1.0% +10.0% ➖ noise
4 datafusion:arrow +24.6% +2.5% +21.5% +10.0% 🚨 regression
4 datafusion:vortex-compact +294.8% +2.5% +285.0% +10.0% 🚨 regression
4 datafusion:vortex-file-compressed +320.1% +2.5% +309.7% +10.0% 🚨 regression
4 duckdb:duckdb +29.8% +2.5% +26.6% +10.0% 🚨 regression
4 duckdb:vortex-compact +21.9% +2.5% +18.9% +10.0% 🚨 regression
4 duckdb:vortex-file-compressed +2.2% +2.5% -0.3% +10.0% ➖ noise
5 datafusion:arrow +20.3% -0.0% +20.3% +10.0% 🚨 regression
5 datafusion:vortex-compact +237.3% -0.0% +237.4% +10.0% 🚨 regression
5 datafusion:vortex-file-compressed +221.2% -0.0% +221.3% +10.0% 🚨 regression
5 duckdb:duckdb +24.9% -0.0% +24.9% +10.0% 🚨 regression
5 duckdb:vortex-compact +8.5% -0.0% +8.5% +10.0% ➖ noise
5 duckdb:vortex-file-compressed +5.3% -0.0% +5.4% +10.0% ➖ noise
6 datafusion:arrow +23.6% +1.2% +22.1% +10.0% 🚨 regression
6 datafusion:vortex-compact +1.1% +1.2% -0.1% +10.0% ➖ noise
6 datafusion:vortex-file-compressed -7.2% +1.2% -8.3% +10.0% ➖ noise
6 duckdb:duckdb +12.5% +1.2% +11.1% +15.3% ➖ noise
6 duckdb:vortex-compact +5.2% +1.2% +3.9% +10.0% ➖ noise
6 duckdb:vortex-file-compressed +1.6% +1.2% +0.4% +10.1% ➖ noise
7 datafusion:arrow +24.2% +0.4% +23.7% +10.0% 🚨 regression
7 datafusion:vortex-compact +24.8% +0.4% +24.4% +10.0% 🚨 regression
7 datafusion:vortex-file-compressed +17.6% +0.4% +17.2% +10.0% 🚨 regression
7 duckdb:duckdb +18.2% +0.4% +17.8% +10.0% 🚨 regression
7 duckdb:vortex-compact +10.2% +0.4% +9.8% +10.0% ➖ noise
7 duckdb:vortex-file-compressed +0.8% +0.4% +0.4% +10.0% ➖ noise
8 datafusion:arrow +18.0% +0.4% +17.6% +10.0% 🚨 regression
8 datafusion:vortex-compact +295.6% +0.4% +294.2% +10.0% 🚨 regression
8 datafusion:vortex-file-compressed +298.3% +0.4% +296.9% +10.0% 🚨 regression
8 duckdb:duckdb +23.5% +0.4% +23.0% +10.0% 🚨 regression
8 duckdb:vortex-compact +26.2% +0.4% +25.7% +10.0% 🚨 regression
8 duckdb:vortex-file-compressed +3.9% +0.4% +3.6% +10.0% ➖ noise
9 datafusion:arrow +17.1% -1.2% +18.5% +10.0% 🚨 regression
9 datafusion:vortex-compact +237.8% -1.2% +242.1% +10.0% 🚨 regression
9 datafusion:vortex-file-compressed +224.5% -1.2% +228.5% +10.0% 🚨 regression
9 duckdb:duckdb +23.0% -1.2% +24.5% +10.0% 🚨 regression
9 duckdb:vortex-compact +24.5% -1.2% +26.1% +10.0% 🚨 regression
9 duckdb:vortex-file-compressed +6.8% -1.2% +8.1% +10.0% ➖ noise
10 datafusion:arrow +19.4% -0.7% +20.2% +10.0% 🚨 regression
10 datafusion:vortex-compact +82.8% -0.7% +84.1% +10.0% 🚨 regression
10 datafusion:vortex-file-compressed +100.6% -0.7% +101.9% +10.0% 🚨 regression
10 duckdb:duckdb +23.2% -0.7% +24.0% +10.0% 🚨 regression
10 duckdb:vortex-compact +29.4% -0.7% +30.3% +10.0% 🚨 regression
10 duckdb:vortex-file-compressed +20.1% -0.7% +20.9% +10.0% 🚨 regression
11 datafusion:arrow +13.6% +8.7% +4.5% +10.1% ➖ noise
11 datafusion:vortex-compact -49.8% +8.7% -53.8% +14.3% ✅ faster
11 datafusion:vortex-file-compressed -48.2% +8.7% -52.4% +10.2% ✅ faster
11 duckdb:duckdb +27.6% +8.7% +17.4% +22.4% ➖ noise
11 duckdb:vortex-compact +28.2% +8.7% +17.9% +13.1% 🚨 regression
11 duckdb:vortex-file-compressed +35.1% +8.7% +24.3% +14.3% 🚨 regression
12 datafusion:arrow -9.0% -3.6% -5.6% +38.1% ➖ noise
12 datafusion:vortex-compact +114.1% -3.6% +122.0% +10.0% 🚨 regression
12 datafusion:vortex-file-compressed +182.9% -3.6% +193.4% +10.0% 🚨 regression
12 duckdb:duckdb +13.2% -3.6% +17.5% +10.0% 🚨 regression
12 duckdb:vortex-compact +18.0% -3.6% +22.4% +10.0% 🚨 regression
12 duckdb:vortex-file-compressed +16.6% -3.6% +21.0% +11.6% 🚨 regression
13 datafusion:arrow -1.7% -5.6% +4.2% +10.0% ➖ noise
13 datafusion:vortex-compact +45.1% -5.6% +53.8% +10.0% 🚨 regression
13 datafusion:vortex-file-compressed +61.1% -5.6% +70.7% +10.0% 🚨 regression
13 duckdb:duckdb +17.4% -5.6% +24.5% +10.0% 🚨 regression
13 duckdb:vortex-compact +9.2% -5.6% +15.7% +10.0% 🚨 regression
13 duckdb:vortex-file-compressed +31.9% -5.6% +39.8% +10.0% 🚨 regression
14 datafusion:arrow -5.3% -2.2% -3.1% +10.0% ➖ noise
14 datafusion:vortex-compact +52.1% -2.2% +55.6% +10.0% 🚨 regression
14 datafusion:vortex-file-compressed +62.5% -2.2% +66.2% +10.0% 🚨 regression
14 duckdb:duckdb +17.0% -2.2% +19.7% +14.2% 🚨 regression
14 duckdb:vortex-compact +10.5% -2.2% +13.0% +10.0% 🚨 regression
14 duckdb:vortex-file-compressed +45.9% -2.2% +49.3% +12.4% 🚨 regression
15 datafusion:arrow -6.2% -2.8% -3.5% +10.0% ➖ noise
15 datafusion:vortex-compact -0.9% -2.8% +2.0% +10.0% ➖ noise
15 datafusion:vortex-file-compressed -5.9% -2.8% -3.1% +10.0% ➖ noise
15 duckdb:duckdb +13.9% -2.8% +17.2% +10.0% 🚨 regression
15 duckdb:vortex-compact +13.1% -2.8% +16.3% +10.0% 🚨 regression
15 duckdb:vortex-file-compressed +36.8% -2.8% +40.8% +10.0% 🚨 regression
16 datafusion:arrow -0.0% +0.8% -0.8% +10.0% ➖ noise
16 datafusion:vortex-compact +105.3% +0.8% +103.7% +10.0% 🚨 regression
16 datafusion:vortex-file-compressed +103.8% +0.8% +102.2% +10.3% 🚨 regression
16 duckdb:duckdb +23.2% +0.8% +22.2% +10.6% 🚨 regression
16 duckdb:vortex-compact +8.4% +0.8% +7.6% +11.1% ➖ noise
16 duckdb:vortex-file-compressed +26.0% +0.8% +25.0% +10.0% 🚨 regression
17 datafusion:arrow -5.7% -4.5% -1.3% +10.0% ➖ noise
17 datafusion:vortex-compact +155.0% -4.5% +167.0% +10.0% 🚨 regression
17 datafusion:vortex-file-compressed +150.6% -4.5% +162.4% +10.0% 🚨 regression
17 duckdb:duckdb +12.2% -4.5% +17.5% +10.0% 🚨 regression
17 duckdb:vortex-compact +8.2% -4.5% +13.3% +10.0% 🚨 regression
17 duckdb:vortex-file-compressed +32.5% -4.5% +38.7% +10.0% 🚨 regression
18 datafusion:arrow -3.0% -4.1% +1.1% +11.7% ➖ noise
18 datafusion:vortex-compact +130.8% -4.1% +140.6% +10.0% 🚨 regression
18 datafusion:vortex-file-compressed +125.5% -4.1% +135.0% +10.0% 🚨 regression
18 duckdb:duckdb +22.4% -4.1% +27.6% +10.0% 🚨 regression
18 duckdb:vortex-compact +2.9% -4.1% +7.3% +10.0% ➖ noise
18 duckdb:vortex-file-compressed +8.8% -4.1% +13.4% +13.3% 🚨 regression
19 datafusion:arrow -1.9% -1.6% -0.3% +10.0% ➖ noise
19 datafusion:vortex-compact +25.1% -1.6% +27.2% +10.0% 🚨 regression
19 datafusion:vortex-file-compressed +52.0% -1.6% +54.5% +10.9% 🚨 regression
19 duckdb:duckdb +16.7% -1.6% +18.7% +10.0% 🚨 regression
19 duckdb:vortex-compact +18.5% -1.6% +20.5% +10.0% 🚨 regression
19 duckdb:vortex-file-compressed +9.6% -1.6% +11.4% +10.0% 🚨 regression
20 datafusion:arrow -4.2% -1.0% -3.3% +10.0% ➖ noise
20 datafusion:vortex-compact +55.0% -1.0% +56.6% +10.0% 🚨 regression
20 datafusion:vortex-file-compressed +60.3% -1.0% +61.8% +10.0% 🚨 regression
20 duckdb:duckdb +13.6% -1.0% +14.7% +10.0% 🚨 regression
20 duckdb:vortex-compact +29.4% -1.0% +30.6% +10.0% 🚨 regression
20 duckdb:vortex-file-compressed +20.4% -1.0% +21.5% +10.0% 🚨 regression
21 datafusion:arrow -1.2% -3.8% +2.7% +12.6% ➖ noise
21 datafusion:vortex-compact +1209.7% -3.8% +1261.8% +10.0% 🚨 regression
21 datafusion:vortex-file-compressed +1245.2% -3.8% +1298.7% +10.0% 🚨 regression
21 duckdb:duckdb +28.4% -3.8% +33.5% +10.0% 🚨 regression
21 duckdb:vortex-compact +27.3% -3.8% +32.4% +10.0% 🚨 regression
21 duckdb:vortex-file-compressed +45.9% -3.8% +51.7% +10.0% 🚨 regression
22 datafusion:arrow -0.2% -0.3% +0.0% +10.0% ➖ noise
22 datafusion:vortex-compact +220.5% -0.3% +221.3% +10.0% 🚨 regression
22 datafusion:vortex-file-compressed +265.0% -0.3% +266.0% +10.0% 🚨 regression
22 duckdb:duckdb +9.3% -0.3% +9.6% +10.0% ➖ noise
22 duckdb:vortex-compact +13.3% -0.3% +13.6% +10.0% 🚨 regression
22 duckdb:vortex-file-compressed +14.1% -0.3% +14.4% +10.0% 🚨 regression

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: Statistical and Population Genetics

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +5.0%
Vortex (geomean): 1.000x ➖
Parquet (geomean): 0.953x ➖
Shifts: Parquet (control) -4.7% · Median polish -2.4%

duckdb / vortex-file-compressed (0.973x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-file-compressed 10774288 11557049 0.93
statpopgen_q01/duckdb:vortex-file-compressed 26615948 28511027 0.93
statpopgen_q02/duckdb:vortex-file-compressed 1291598377 1358551544 0.95
statpopgen_q03/duckdb:vortex-file-compressed 2967522355 3179281065 0.93
statpopgen_q04/duckdb:vortex-file-compressed 3055823110 3137292080 0.97
statpopgen_q05/duckdb:vortex-file-compressed 1382632816 1385359657 1.00
statpopgen_q06/duckdb:vortex-file-compressed 2035422973 2046263239 0.99
statpopgen_q07/duckdb:vortex-file-compressed 194142455 201102164 0.97
statpopgen_q08/duckdb:vortex-file-compressed 210198029 204703767 1.03
statpopgen_q09/duckdb:vortex-file-compressed 2848879712 2825623149 1.01
statpopgen_q10/duckdb:vortex-file-compressed 4576548835 4627718717 0.99
duckdb / vortex-compact (1.028x ➖, 0↑ 1↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
statpopgen_q00/duckdb:vortex-compact 10889471 10673304 1.02
statpopgen_q01/duckdb:vortex-compact 🚨 165464752 123171881 1.34
statpopgen_q02/duckdb:vortex-compact 1809630169 1812119978 1.00
statpopgen_q03/duckdb:vortex-compact 3492017298 3460791061 1.01
statpopgen_q04/duckdb:vortex-compact 3508244495 3553088458 0.99
statpopgen_q05/duckdb:vortex-compact 1827867810 1842171510 0.99
statpopgen_q06/duckdb:vortex-compact 2640640816 2674875505 0.99
statpopgen_q07/duckdb:vortex-compact 945716911 950943224 0.99
statpopgen_q08/duckdb:vortex-compact 984278382 979349027 1.01
statpopgen_q09/duckdb:vortex-compact 3304798564 3269578630 1.01
statpopgen_q10/duckdb:vortex-compact 5382123371 5350596404 1.01
duckdb / parquet (0.953x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
statpopgen_q00/duckdb:parquet 306751183 331561584 0.93
statpopgen_q01/duckdb:parquet 393401694 413514957 0.95
statpopgen_q02/duckdb:parquet 973640366 1044888196 0.93
statpopgen_q03/duckdb:parquet 1465159114 1562549113 0.94
statpopgen_q04/duckdb:parquet 1473428111 1618931706 0.91
statpopgen_q05/duckdb:parquet 998158266 1038581778 0.96
statpopgen_q06/duckdb:parquet 1484843394 1515318697 0.98
statpopgen_q07/duckdb:parquet 1875632677 1921401228 0.98
statpopgen_q08/duckdb:parquet 1862221363 1905775300 0.98
statpopgen_q09/duckdb:parquet 1345044845 1420195884 0.95
statpopgen_q10/duckdb:parquet 2667914567 2707929688 0.99
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 duckdb:vortex-compact +2.0% -7.5% +10.3% +10.0% 🚨 regression
0 duckdb:vortex-file-compressed -6.8% -7.5% +0.8% +10.0% ➖ noise
1 duckdb:vortex-compact +34.3% -4.9% +41.2% +13.6% 🚨 regression
1 duckdb:vortex-file-compressed -6.6% -4.9% -1.9% +286.8% ➖ noise
2 duckdb:vortex-compact -0.1% -6.8% +7.2% +10.0% ➖ noise
2 duckdb:vortex-file-compressed -4.9% -6.8% +2.0% +10.0% ➖ noise
3 duckdb:vortex-compact +0.9% -6.2% +7.6% +10.0% ➖ noise
3 duckdb:vortex-file-compressed -6.7% -6.2% -0.5% +10.0% ➖ noise
4 duckdb:vortex-compact -1.3% -9.0% +8.5% +10.0% ➖ noise
4 duckdb:vortex-file-compressed -2.6% -9.0% +7.0% +10.0% ➖ noise
5 duckdb:vortex-compact -0.8% -3.9% +3.2% +10.0% ➖ noise
5 duckdb:vortex-file-compressed -0.2% -3.9% +3.8% +10.0% ➖ noise
6 duckdb:vortex-compact -1.3% -2.0% +0.7% +10.0% ➖ noise
6 duckdb:vortex-file-compressed -0.5% -2.0% +1.5% +10.0% ➖ noise
7 duckdb:vortex-compact -0.5% -2.4% +1.9% +10.0% ➖ noise
7 duckdb:vortex-file-compressed -3.5% -2.4% -1.1% +10.0% ➖ noise
8 duckdb:vortex-compact +0.5% -2.3% +2.9% +10.0% ➖ noise
8 duckdb:vortex-file-compressed +2.7% -2.3% +5.1% +10.0% ➖ noise
9 duckdb:vortex-compact +1.1% -5.3% +6.7% +10.0% ➖ noise
9 duckdb:vortex-file-compressed +0.8% -5.3% +6.5% +10.0% ➖ noise
10 duckdb:vortex-compact +0.6% -1.5% +2.1% +10.0% ➖ noise
10 duckdb:vortex-file-compressed -1.1% -1.5% +0.4% +10.0% ➖ noise

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: TPC-H SF=1 on S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -4.6%
Vortex (geomean): 0.814x ➖
Parquet (geomean): 0.853x ➖
Shifts: Parquet (control) -14.7% · Median polish -17.1%

datafusion / vortex-file-compressed (0.672x ✅, 11↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 164975332 182193373 0.91
tpch_q02/datafusion:vortex-file-compressed 🚀 217736568 577368920 0.38
tpch_q03/datafusion:vortex-file-compressed 🚀 296663103 443860382 0.67
tpch_q04/datafusion:vortex-file-compressed 🚀 204234004 295861536 0.69
tpch_q05/datafusion:vortex-file-compressed 438162615 504942430 0.87
tpch_q06/datafusion:vortex-file-compressed 229733681 319399044 0.72
tpch_q07/datafusion:vortex-file-compressed 404239549 534458966 0.76
tpch_q08/datafusion:vortex-file-compressed 🚀 384975897 718750686 0.54
tpch_q09/datafusion:vortex-file-compressed 383705613 530010325 0.72
tpch_q10/datafusion:vortex-file-compressed 439181034 598459429 0.73
tpch_q11/datafusion:vortex-file-compressed 🚀 170490155 478360488 0.36
tpch_q12/datafusion:vortex-file-compressed 🚀 350666357 650106159 0.54
tpch_q13/datafusion:vortex-file-compressed 252093370 241658887 1.04
tpch_q14/datafusion:vortex-file-compressed 🚀 185047246 281992293 0.66
tpch_q15/datafusion:vortex-file-compressed 329851014 418088639 0.79
tpch_q16/datafusion:vortex-file-compressed 🚀 126353681 209426748 0.60
tpch_q17/datafusion:vortex-file-compressed 255238102 352055693 0.72
tpch_q18/datafusion:vortex-file-compressed 460784327 412240855 1.12
tpch_q19/datafusion:vortex-file-compressed 🚀 345247674 514200837 0.67
tpch_q20/datafusion:vortex-file-compressed 🚀 286426468 536583345 0.53
tpch_q21/datafusion:vortex-file-compressed 540716875 578670269 0.93
tpch_q22/datafusion:vortex-file-compressed 🚀 86051839 193949477 0.44
datafusion / vortex-compact (0.752x ➖, 8↑ 1↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 🚀 218112087 378135840 0.58
tpch_q02/datafusion:vortex-compact 🚀 238323256 442883565 0.54
tpch_q03/datafusion:vortex-compact 301131252 363063009 0.83
tpch_q04/datafusion:vortex-compact 233217815 271459789 0.86
tpch_q05/datafusion:vortex-compact 379254109 487883354 0.78
tpch_q06/datafusion:vortex-compact 🚀 239666310 363090411 0.66
tpch_q07/datafusion:vortex-compact 388482060 503425690 0.77
tpch_q08/datafusion:vortex-compact 🚀 289553175 490588015 0.59
tpch_q09/datafusion:vortex-compact 371446420 498651471 0.74
tpch_q10/datafusion:vortex-compact 337506035 435514766 0.77
tpch_q11/datafusion:vortex-compact 🚀 142191878 284754965 0.50
tpch_q12/datafusion:vortex-compact 348545965 420806225 0.83
tpch_q13/datafusion:vortex-compact 234466841 211296612 1.11
tpch_q14/datafusion:vortex-compact 🚀 176465730 292828552 0.60
tpch_q15/datafusion:vortex-compact 🚀 323473379 467562471 0.69
tpch_q16/datafusion:vortex-compact 133387924 174036576 0.77
tpch_q17/datafusion:vortex-compact 🚀 245407572 364410577 0.67
tpch_q18/datafusion:vortex-compact 🚨 394952650 295445659 1.34
tpch_q19/datafusion:vortex-compact 338276820 403555228 0.84
tpch_q20/datafusion:vortex-compact 330643780 452349241 0.73
tpch_q21/datafusion:vortex-compact 554349966 690875514 0.80
tpch_q22/datafusion:vortex-compact 128362811 130214448 0.99
datafusion / parquet (0.797x ➖, 6↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 🚀 238437418 379395257 0.63
tpch_q02/datafusion:parquet 🚀 414738964 817936633 0.51
tpch_q03/datafusion:parquet 🚀 298695503 474884390 0.63
tpch_q04/datafusion:parquet 🚀 140497180 252177812 0.56
tpch_q05/datafusion:parquet 515937336 627540112 0.82
tpch_q06/datafusion:parquet 🚀 144300281 211963858 0.68
tpch_q07/datafusion:parquet 539404745 674822609 0.80
tpch_q08/datafusion:parquet 691643941 975892155 0.71
tpch_q09/datafusion:parquet 543366093 726436233 0.75
tpch_q10/datafusion:parquet 566854515 726645724 0.78
tpch_q11/datafusion:parquet 378759047 374767843 1.01
tpch_q12/datafusion:parquet 279340498 312165832 0.89
tpch_q13/datafusion:parquet 438839220 506890784 0.87
tpch_q14/datafusion:parquet 🚀 206728433 310974584 0.66
tpch_q15/datafusion:parquet 304983321 341537383 0.89
tpch_q16/datafusion:parquet 180907698 193039713 0.94
tpch_q17/datafusion:parquet 396484950 455433521 0.87
tpch_q18/datafusion:parquet 492805692 582692842 0.85
tpch_q19/datafusion:parquet 316117427 376405042 0.84
tpch_q20/datafusion:parquet 370576088 363561417 1.02
tpch_q21/datafusion:parquet 586455908 580295781 1.01
tpch_q22/datafusion:parquet 146935326 124484944 1.18
duckdb / vortex-file-compressed (0.919x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 226920291 280120249 0.81
tpch_q02/duckdb:vortex-file-compressed 913247164 938350036 0.97
tpch_q03/duckdb:vortex-file-compressed 530145179 612178397 0.87
tpch_q04/duckdb:vortex-file-compressed 342527144 409670126 0.84
tpch_q05/duckdb:vortex-file-compressed 802845639 885058459 0.91
tpch_q06/duckdb:vortex-file-compressed 358445024 434445022 0.83
tpch_q07/duckdb:vortex-file-compressed 723955034 669459975 1.08
tpch_q08/duckdb:vortex-file-compressed 869455445 975989290 0.89
tpch_q09/duckdb:vortex-file-compressed 905227545 937112259 0.97
tpch_q10/duckdb:vortex-file-compressed 717093322 730959262 0.98
tpch_q11/duckdb:vortex-file-compressed 493083437 483500191 1.02
tpch_q12/duckdb:vortex-file-compressed 499358735 491435401 1.02
tpch_q13/duckdb:vortex-file-compressed 457164910 497245908 0.92
tpch_q14/duckdb:vortex-file-compressed 409599364 549114442 0.75
tpch_q15/duckdb:vortex-file-compressed 290254958 306322932 0.95
tpch_q16/duckdb:vortex-file-compressed 343824890 346415424 0.99
tpch_q17/duckdb:vortex-file-compressed 602761069 695665046 0.87
tpch_q18/duckdb:vortex-file-compressed 529735082 554901540 0.95
tpch_q19/duckdb:vortex-file-compressed 431776911 538844497 0.80
tpch_q20/duckdb:vortex-file-compressed 793527320 829364535 0.96
tpch_q21/duckdb:vortex-file-compressed 1040947221 970343410 1.07
tpch_q22/duckdb:vortex-file-compressed 351384366 395785754 0.89
duckdb / vortex-compact (0.943x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 264494343 276959945 0.95
tpch_q02/duckdb:vortex-compact 900995621 852982860 1.06
tpch_q03/duckdb:vortex-compact 486720814 543351997 0.90
tpch_q04/duckdb:vortex-compact 371019393 334082228 1.11
tpch_q05/duckdb:vortex-compact 854984632 865544138 0.99
tpch_q06/duckdb:vortex-compact 413528208 345821617 1.20
tpch_q07/duckdb:vortex-compact 664151443 752720225 0.88
tpch_q08/duckdb:vortex-compact 821635438 865648525 0.95
tpch_q09/duckdb:vortex-compact 825431686 878461699 0.94
tpch_q10/duckdb:vortex-compact 703797484 655481015 1.07
tpch_q11/duckdb:vortex-compact 473364893 517473793 0.91
tpch_q12/duckdb:vortex-compact 461928796 430606974 1.07
tpch_q13/duckdb:vortex-compact 433723392 442244155 0.98
tpch_q14/duckdb:vortex-compact 459733799 504211483 0.91
tpch_q15/duckdb:vortex-compact 292087303 350125787 0.83
tpch_q16/duckdb:vortex-compact 340068456 354477797 0.96
tpch_q17/duckdb:vortex-compact 577601446 634111765 0.91
tpch_q18/duckdb:vortex-compact 441298435 510585576 0.86
tpch_q19/duckdb:vortex-compact 455902717 578950202 0.79
tpch_q20/duckdb:vortex-compact 743578703 882054221 0.84
tpch_q21/duckdb:vortex-compact 972590472 1206723450 0.81
tpch_q22/duckdb:vortex-compact 332062884 357616123 0.93
duckdb / parquet (0.912x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 503455107 663210253 0.76
tpch_q02/duckdb:parquet 1316113112 1388595870 0.95
tpch_q03/duckdb:parquet 938878836 1082473522 0.87
tpch_q04/duckdb:parquet 580452541 731825860 0.79
tpch_q05/duckdb:parquet 1451833144 1788654040 0.81
tpch_q06/duckdb:parquet 380934713 510126096 0.75
tpch_q07/duckdb:parquet 1266811999 1301546115 0.97
tpch_q08/duckdb:parquet 1626454352 1865052104 0.87
tpch_q09/duckdb:parquet 1298649512 1467428942 0.88
tpch_q10/duckdb:parquet 1320428507 1511709026 0.87
tpch_q11/duckdb:parquet 839884780 828249614 1.01
tpch_q12/duckdb:parquet 568510907 618664387 0.92
tpch_q13/duckdb:parquet 875866917 866302377 1.01
tpch_q14/duckdb:parquet 676558368 670096697 1.01
tpch_q15/duckdb:parquet 487722915 591667413 0.82
tpch_q16/duckdb:parquet 534096431 597859911 0.89
tpch_q17/duckdb:parquet 748323229 792142480 0.94
tpch_q18/duckdb:parquet 846228601 800573471 1.06
tpch_q19/duckdb:parquet 806570501 742077848 1.09
tpch_q20/duckdb:parquet 951073672 1117918879 0.85
tpch_q21/duckdb:parquet 1164012636 1142605127 1.02
tpch_q22/duckdb:parquet 475853689 467663440 1.02
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:vortex-compact -42.3% -30.9% -16.5% +83.2% ➖ noise
1 datafusion:vortex-file-compressed -9.5% -30.9% +31.1% +111.8% ➖ noise
1 duckdb:vortex-compact -4.5% -30.9% +38.3% +42.2% ➖ noise
1 duckdb:vortex-file-compressed -19.0% -30.9% +17.3% +53.9% ➖ noise
2 datafusion:vortex-compact -46.2% -30.7% -22.4% +45.3% ➖ noise
2 datafusion:vortex-file-compressed -62.3% -30.7% -45.6% +42.2% ✅ faster
2 duckdb:vortex-compact +5.6% -30.7% +52.4% +42.2% 🚨 regression
2 duckdb:vortex-file-compressed -2.7% -30.7% +40.4% +42.2% ➖ noise
3 datafusion:vortex-compact -17.1% -26.1% +12.3% +76.5% ➖ noise
3 datafusion:vortex-file-compressed -33.2% -26.1% -9.5% +60.1% ➖ noise
3 duckdb:vortex-compact -10.4% -26.1% +21.3% +42.2% ➖ noise
3 duckdb:vortex-file-compressed -13.4% -26.1% +17.2% +42.2% ➖ noise
4 datafusion:vortex-compact -14.1% -33.5% +29.2% +42.2% ➖ noise
4 datafusion:vortex-file-compressed -31.0% -33.5% +3.8% +42.2% ➖ noise
4 duckdb:vortex-compact +11.1% -33.5% +67.1% +42.2% 🚨 regression
4 duckdb:vortex-file-compressed -16.4% -33.5% +25.8% +42.2% ➖ noise
5 datafusion:vortex-compact -22.3% -18.3% -4.8% +42.2% ➖ noise
5 datafusion:vortex-file-compressed -13.2% -18.3% +6.2% +42.2% ➖ noise
5 duckdb:vortex-compact -1.2% -18.3% +20.9% +42.2% ➖ noise
5 duckdb:vortex-file-compressed -9.3% -18.3% +11.0% +42.2% ➖ noise
6 datafusion:vortex-compact -34.0% -28.7% -7.4% +61.8% ➖ noise
6 datafusion:vortex-file-compressed -28.1% -28.7% +0.9% +42.2% ➖ noise
6 duckdb:vortex-compact +19.6% -28.7% +67.7% +42.2% 🚨 regression
6 duckdb:vortex-file-compressed -17.5% -28.7% +15.7% +66.1% ➖ noise
7 datafusion:vortex-compact -22.8% -11.8% -12.5% +42.2% ➖ noise
7 datafusion:vortex-file-compressed -24.4% -11.8% -14.2% +42.2% ➖ noise
7 duckdb:vortex-compact -11.8% -11.8% +0.0% +42.2% ➖ noise
7 duckdb:vortex-file-compressed +8.1% -11.8% +22.6% +42.2% ➖ noise
8 datafusion:vortex-compact -41.0% -21.4% -24.9% +52.2% ➖ noise
8 datafusion:vortex-file-compressed -46.4% -21.4% -31.9% +42.2% ✅ faster
8 duckdb:vortex-compact -5.1% -21.4% +20.7% +42.2% ➖ noise
8 duckdb:vortex-file-compressed -10.9% -21.4% +13.3% +42.2% ➖ noise
9 datafusion:vortex-compact -25.5% -18.6% -8.4% +42.2% ➖ noise
9 datafusion:vortex-file-compressed -27.6% -18.6% -11.0% +42.2% ➖ noise
9 duckdb:vortex-compact -6.0% -18.6% +15.5% +42.2% ➖ noise
9 duckdb:vortex-file-compressed -3.4% -18.6% +18.7% +42.2% ➖ noise
10 datafusion:vortex-compact -22.5% -17.5% -6.1% +42.2% ➖ noise
10 datafusion:vortex-file-compressed -26.6% -17.5% -11.1% +42.2% ➖ noise
10 duckdb:vortex-compact +7.4% -17.5% +30.1% +42.2% ➖ noise
10 duckdb:vortex-file-compressed -1.9% -17.5% +18.8% +42.2% ➖ noise
11 datafusion:vortex-compact -50.1% +1.2% -50.7% +42.2% ✅ faster
11 datafusion:vortex-file-compressed -64.4% +1.2% -64.8% +42.2% ✅ faster
11 duckdb:vortex-compact -8.5% +1.2% -9.6% +42.2% ➖ noise
11 duckdb:vortex-file-compressed +2.0% +1.2% +0.7% +42.2% ➖ noise
12 datafusion:vortex-compact -17.2% -9.3% -8.7% +42.2% ➖ noise
12 datafusion:vortex-file-compressed -46.1% -9.3% -40.5% +42.2% ✅ faster
12 duckdb:vortex-compact +7.3% -9.3% +18.3% +42.2% ➖ noise
12 duckdb:vortex-file-compressed +1.6% -9.3% +12.1% +42.2% ➖ noise
13 datafusion:vortex-compact +11.0% -6.4% +18.6% +48.0% ➖ noise
13 datafusion:vortex-file-compressed +4.3% -6.4% +11.5% +81.2% ➖ noise
13 duckdb:vortex-compact -1.9% -6.4% +4.8% +42.2% ➖ noise
13 duckdb:vortex-file-compressed -8.1% -6.4% -1.7% +42.2% ➖ noise
14 datafusion:vortex-compact -39.7% -18.1% -26.4% +42.2% ➖ noise
14 datafusion:vortex-file-compressed -34.4% -18.1% -19.9% +48.4% ➖ noise
14 duckdb:vortex-compact -8.8% -18.1% +11.3% +58.9% ➖ noise
14 duckdb:vortex-file-compressed -25.4% -18.1% -9.0% +42.2% ➖ noise
15 datafusion:vortex-compact -30.8% -14.2% -19.4% +42.2% ➖ noise
15 datafusion:vortex-file-compressed -21.1% -14.2% -8.0% +42.2% ➖ noise
15 duckdb:vortex-compact -16.6% -14.2% -2.8% +42.2% ➖ noise
15 duckdb:vortex-file-compressed -5.2% -14.2% +10.4% +44.2% ➖ noise
16 datafusion:vortex-compact -23.4% -8.5% -16.2% +42.2% ➖ noise
16 datafusion:vortex-file-compressed -39.7% -8.5% -34.1% +42.2% ✅ faster
16 duckdb:vortex-compact -4.1% -8.5% +4.8% +42.2% ➖ noise
16 duckdb:vortex-file-compressed -0.7% -8.5% +8.5% +42.2% ➖ noise
17 datafusion:vortex-compact -32.7% -9.3% -25.7% +42.2% ➖ noise
17 datafusion:vortex-file-compressed -27.5% -9.3% -20.1% +42.2% ➖ noise
17 duckdb:vortex-compact -8.9% -9.3% +0.4% +42.2% ➖ noise
17 duckdb:vortex-file-compressed -13.4% -9.3% -4.5% +42.2% ➖ noise
18 datafusion:vortex-compact +33.7% -5.5% +41.4% +42.2% ➖ noise
18 datafusion:vortex-file-compressed +11.8% -5.5% +18.2% +42.2% ➖ noise
18 duckdb:vortex-compact -13.6% -5.5% -8.6% +42.2% ➖ noise
18 duckdb:vortex-file-compressed -4.5% -5.5% +1.0% +42.2% ➖ noise
19 datafusion:vortex-compact -16.2% -4.5% -12.3% +42.2% ➖ noise
19 datafusion:vortex-file-compressed -32.9% -4.5% -29.7% +42.2% ✅ faster
19 duckdb:vortex-compact -21.3% -4.5% -17.6% +42.2% ➖ noise
19 duckdb:vortex-file-compressed -19.9% -4.5% -16.1% +42.2% ➖ noise
20 datafusion:vortex-compact -26.9% -6.9% -21.5% +42.2% ➖ noise
20 datafusion:vortex-file-compressed -46.6% -6.9% -42.7% +42.2% ✅ faster
20 duckdb:vortex-compact -15.7% -6.9% -9.5% +42.2% ➖ noise
20 duckdb:vortex-file-compressed -4.3% -6.9% +2.7% +42.2% ➖ noise
21 datafusion:vortex-compact -19.8% +1.5% -20.9% +42.2% ➖ noise
21 datafusion:vortex-file-compressed -6.6% +1.5% -7.9% +42.2% ➖ noise
21 duckdb:vortex-compact -19.4% +1.5% -20.6% +42.2% ➖ noise
21 duckdb:vortex-file-compressed +7.3% +1.5% +5.7% +42.2% ➖ noise
22 datafusion:vortex-compact -1.4% +9.6% -10.0% +42.2% ➖ noise
22 datafusion:vortex-file-compressed -55.6% +9.6% -59.5% +42.2% ✅ faster
22 duckdb:vortex-compact -7.1% +9.6% -15.3% +42.2% ➖ noise
22 duckdb:vortex-file-compressed -11.2% +9.6% -19.0% +42.2% ➖ noise

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: FineWeb S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: -20.8%
Vortex (geomean): 0.830x ➖
Parquet (geomean): 1.048x ➖
Shifts: Parquet (control) +4.8% · Median polish -5.1%

datafusion / vortex-file-compressed (0.855x ➖, 1↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-file-compressed 32157619 33826178 0.95
fineweb_q01/datafusion:vortex-file-compressed 571702586 590974585 0.97
fineweb_q02/datafusion:vortex-file-compressed 379836180 530580784 0.72
fineweb_q03/datafusion:vortex-file-compressed 1384222657 1380552968 1.00
fineweb_q04/datafusion:vortex-file-compressed 1509965642 1391921572 1.08
fineweb_q05/datafusion:vortex-file-compressed 1425337572 1372546807 1.04
fineweb_q06/datafusion:vortex-file-compressed 1523132166 1481667774 1.03
fineweb_q07/datafusion:vortex-file-compressed 1376838802 1409625638 0.98
fineweb_q08/datafusion:vortex-file-compressed 🚀 188917389 575061921 0.33
datafusion / vortex-compact (0.713x ➖, 3↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/datafusion:vortex-compact 36593784 36041576 1.02
fineweb_q01/datafusion:vortex-compact 614274209 744899585 0.82
fineweb_q02/datafusion:vortex-compact 566518360 704295489 0.80
fineweb_q03/datafusion:vortex-compact 🚀 1366139170 1992550100 0.69
fineweb_q04/datafusion:vortex-compact 🚀 1428767743 2413466637 0.59
fineweb_q05/datafusion:vortex-compact 1261827668 1733960409 0.73
fineweb_q06/datafusion:vortex-compact 1241473556 1515368524 0.82
fineweb_q07/datafusion:vortex-compact 1173595518 1297457660 0.90
fineweb_q08/datafusion:vortex-compact 🚀 134593980 416954465 0.32
datafusion / parquet (1.108x ➖, 0↑ 2↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/datafusion:parquet 🚨 1362631110 985755640 1.38
fineweb_q01/datafusion:parquet 1921105810 1851075277 1.04
fineweb_q02/datafusion:parquet 2270953451 1951846220 1.16
fineweb_q03/datafusion:parquet 1983871791 1983224510 1.00
fineweb_q04/datafusion:parquet 🚨 2726659634 1995535497 1.37
fineweb_q05/datafusion:parquet 2016623316 1850462294 1.09
fineweb_q06/datafusion:parquet 1906526040 1819109974 1.05
fineweb_q07/datafusion:parquet 1940715389 1938650592 1.00
fineweb_q08/datafusion:parquet 2044673434 2127387620 0.96
duckdb / vortex-file-compressed (0.882x ➖, 0↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-file-compressed 108294026 121271912 0.89
fineweb_q01/duckdb:vortex-file-compressed 652128276 663431024 0.98
fineweb_q02/duckdb:vortex-file-compressed 506213315 532872737 0.95
fineweb_q03/duckdb:vortex-file-compressed 1477732461 1986408903 0.74
fineweb_q04/duckdb:vortex-file-compressed 1497958169 1745596671 0.86
fineweb_q05/duckdb:vortex-file-compressed 1465302834 1556038138 0.94
fineweb_q06/duckdb:vortex-file-compressed 1667194507 1746192986 0.95
fineweb_q07/duckdb:vortex-file-compressed 1445377617 1641570609 0.88
fineweb_q08/duckdb:vortex-file-compressed 659270370 863986685 0.76
duckdb / vortex-compact (0.882x ➖, 1↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/duckdb:vortex-compact 🚀 83868122 126282928 0.66
fineweb_q01/duckdb:vortex-compact 564995961 606663349 0.93
fineweb_q02/duckdb:vortex-compact 603627712 587204090 1.03
fineweb_q03/duckdb:vortex-compact 1627901345 1922743993 0.85
fineweb_q04/duckdb:vortex-compact 1702080974 2052117837 0.83
fineweb_q05/duckdb:vortex-compact 1510903032 1816809122 0.83
fineweb_q06/duckdb:vortex-compact 1447798291 1532589467 0.94
fineweb_q07/duckdb:vortex-compact 1343505348 1310065381 1.03
fineweb_q08/duckdb:vortex-compact 474591513 526357962 0.90
duckdb / parquet (0.992x ➖, 0↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
fineweb_q00/duckdb:parquet 1126277449 1056692084 1.07
fineweb_q01/duckdb:parquet 1607440550 1783446421 0.90
fineweb_q02/duckdb:parquet 1677335593 1663208077 1.01
fineweb_q03/duckdb:parquet 4278297626 4260105177 1.00
fineweb_q04/duckdb:parquet 1954846900 1949826332 1.00
fineweb_q05/duckdb:parquet 2836399884 2972673189 0.95
fineweb_q06/duckdb:parquet 3676392365 3425925858 1.07
fineweb_q07/duckdb:parquet 4161770753 4203179360 0.99
fineweb_q08/duckdb:parquet 1126192188 1196739871 0.94
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 datafusion:vortex-compact +1.5% +21.4% -16.4% +93.9% ➖ noise
0 datafusion:vortex-file-compressed -4.9% +21.4% -21.7% +170.9% ➖ noise
0 duckdb:vortex-compact -33.6% +21.4% -45.3% +34.2% ✅ faster
0 duckdb:vortex-file-compressed -10.7% +21.4% -26.4% +41.5% ➖ noise
1 datafusion:vortex-compact -17.5% -3.3% -14.7% +30.0% ➖ noise
1 datafusion:vortex-file-compressed -3.3% -3.3% +0.0% +49.2% ➖ noise
1 duckdb:vortex-compact -6.9% -3.3% -3.7% +30.0% ➖ noise
1 duckdb:vortex-file-compressed -1.7% -3.3% +1.6% +30.0% ➖ noise
2 datafusion:vortex-compact -19.6% +8.3% -25.7% +30.0% ✅ faster
2 datafusion:vortex-file-compressed -28.4% +8.3% -33.9% +30.0% ✅ faster
2 duckdb:vortex-compact +2.8% +8.3% -5.1% +30.0% ➖ noise
2 duckdb:vortex-file-compressed -5.0% +8.3% -12.3% +30.0% ➖ noise
3 datafusion:vortex-compact -31.4% +0.2% -31.6% +30.0% ✅ faster
3 datafusion:vortex-file-compressed +0.3% +0.2% +0.0% +30.0% ➖ noise
3 duckdb:vortex-compact -15.3% +0.2% -15.5% +30.0% ➖ noise
3 duckdb:vortex-file-compressed -25.6% +0.2% -25.8% +30.0% ✅ faster
4 datafusion:vortex-compact -40.8% +17.0% -49.4% +30.0% ✅ faster
4 datafusion:vortex-file-compressed +8.5% +17.0% -7.3% +30.0% ➖ noise
4 duckdb:vortex-compact -17.1% +17.0% -29.1% +30.0% ✅ faster
4 duckdb:vortex-file-compressed -14.2% +17.0% -26.7% +30.0% ✅ faster
5 datafusion:vortex-compact -27.2% +2.0% -28.6% +30.0% ✅ faster
5 datafusion:vortex-file-compressed +3.8% +2.0% +1.8% +30.0% ➖ noise
5 duckdb:vortex-compact -16.8% +2.0% -18.4% +30.0% ➖ noise
5 duckdb:vortex-file-compressed -5.8% +2.0% -7.7% +30.0% ➖ noise
6 datafusion:vortex-compact -18.1% +6.1% -22.7% +30.0% ➖ noise
6 datafusion:vortex-file-compressed +2.8% +6.1% -3.1% +30.0% ➖ noise
6 duckdb:vortex-compact -5.5% +6.1% -10.9% +30.0% ➖ noise
6 duckdb:vortex-file-compressed -4.5% +6.1% -10.0% +30.0% ➖ noise
7 datafusion:vortex-compact -9.5% -0.4% -9.1% +30.0% ➖ noise
7 datafusion:vortex-file-compressed -2.3% -0.4% -1.9% +30.0% ➖ noise
7 duckdb:vortex-compact +2.6% -0.4% +3.0% +30.0% ➖ noise
7 duckdb:vortex-file-compressed -12.0% -0.4% -11.6% +30.0% ➖ noise
8 datafusion:vortex-compact -67.7% -4.9% -66.1% +30.0% ✅ faster
8 datafusion:vortex-file-compressed -67.1% -4.9% -65.5% +30.0% ✅ faster
8 duckdb:vortex-compact -9.8% -4.9% -5.2% +30.0% ➖ noise
8 duckdb:vortex-file-compressed -23.7% -4.9% -19.8% +30.0% ➖ noise

@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: Clickbench on NVME

Verdict: No clear signal (low confidence)
Attributed Vortex impact: +6.0%
Vortex (geomean): 1.089x ➖
Parquet (geomean): 1.005x ➖
Shifts: Parquet (control) +0.5% · Median polish +1.0%

datafusion / vortex-file-compressed (1.252x ❌, 4↑ 18↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
clickbench_q00/datafusion:vortex-file-compressed 🚨 17709492 1841716 9.62
clickbench_q01/datafusion:vortex-file-compressed 15608503 14526209 1.07
clickbench_q02/datafusion:vortex-file-compressed 🚨 52067237 31535086 1.65
clickbench_q03/datafusion:vortex-file-compressed 🚨 64260676 35662107 1.80
clickbench_q04/datafusion:vortex-file-compressed 317803764 312306067 1.02
clickbench_q05/datafusion:vortex-file-compressed 🚨 348077335 309717218 1.12
clickbench_q06/datafusion:vortex-file-compressed 🚨 59194626 1665341 35.55
clickbench_q07/datafusion:vortex-file-compressed 🚨 22939941 16922600 1.36
clickbench_q08/datafusion:vortex-file-compressed 389384784 384322616 1.01
clickbench_q09/datafusion:vortex-file-compressed 540532321 569701795 0.95
clickbench_q10/datafusion:vortex-file-compressed 🚨 79045163 71343250 1.11
clickbench_q11/datafusion:vortex-file-compressed 🚨 89275776 80276450 1.11
clickbench_q12/datafusion:vortex-file-compressed 276473569 272927508 1.01
clickbench_q13/datafusion:vortex-file-compressed 421221414 412634836 1.02
clickbench_q14/datafusion:vortex-file-compressed 263896731 258786603 1.02
clickbench_q15/datafusion:vortex-file-compressed 369838238 375956026 0.98
clickbench_q16/datafusion:vortex-file-compressed 737439675 720986064 1.02
clickbench_q17/datafusion:vortex-file-compressed 717271414 711005269 1.01
clickbench_q18/datafusion:vortex-file-compressed 1569444559 1450276193 1.08
clickbench_q19/datafusion:vortex-file-compressed 25215323 23573250 1.07
clickbench_q20/datafusion:vortex-file-compressed 383459734 349036422 1.10
clickbench_q21/datafusion:vortex-file-compressed 408012593 400909390 1.02
clickbench_q22/datafusion:vortex-file-compressed 🚨 619307066 520505230 1.19
clickbench_q23/datafusion:vortex-file-compressed 🚨 1902644988 597806388 3.18
clickbench_q24/datafusion:vortex-file-compressed 🚨 107704273 47028459 2.29
clickbench_q25/datafusion:vortex-file-compressed 🚨 89295332 69775669 1.28
clickbench_q26/datafusion:vortex-file-compressed 🚨 106072499 45714108 2.32
clickbench_q27/datafusion:vortex-file-compressed 🚨 897573825 807810815 1.11
clickbench_q28/datafusion:vortex-file-compressed 🚀 5475990669 6690251866 0.82
clickbench_q29/datafusion:vortex-file-compressed 218602380 242279209 0.90
clickbench_q30/datafusion:vortex-file-compressed 245245544 234974076 1.04
clickbench_q31/datafusion:vortex-file-compressed 274929066 278759057 0.99
clickbench_q32/datafusion:vortex-file-compressed 1324826318 1325847845 1.00
clickbench_q33/datafusion:vortex-file-compressed 🚨 1721587230 1432127475 1.20
clickbench_q34/datafusion:vortex-file-compressed 🚨 1693069964 1435326742 1.18
clickbench_q35/datafusion:vortex-file-compressed 514922703 507774395 1.01
clickbench_q36/datafusion:vortex-file-compressed 🚀 35306869 79177789 0.45
clickbench_q37/datafusion:vortex-file-compressed 🚀 25239574 37144127 0.68
clickbench_q38/datafusion:vortex-file-compressed 25802152 24040731 1.07
clickbench_q39/datafusion:vortex-file-compressed 🚀 55495741 154026159 0.36
clickbench_q40/datafusion:vortex-file-compressed 🚨 25409975 19768962 1.29
clickbench_q41/datafusion:vortex-file-compressed 🚨 22421710 18688166 1.20
clickbench_q42/datafusion:vortex-file-compressed 21496194 21727989 0.99
datafusion / parquet (1.013x ➖, 1↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
clickbench_q00/datafusion:parquet 1574669 1531670 1.03
clickbench_q01/datafusion:parquet 21923001 21162770 1.04
clickbench_q02/datafusion:parquet 48734836 47658338 1.02
clickbench_q03/datafusion:parquet 43780467 42537418 1.03
clickbench_q04/datafusion:parquet 353352494 341008297 1.04
clickbench_q05/datafusion:parquet 391324895 375249420 1.04
clickbench_q06/datafusion:parquet 1556931 1545308 1.01
clickbench_q07/datafusion:parquet 27533346 26832860 1.03
clickbench_q08/datafusion:parquet 424655905 416039842 1.02
clickbench_q09/datafusion:parquet 657423511 653441194 1.01
clickbench_q10/datafusion:parquet 118345395 117106656 1.01
clickbench_q11/datafusion:parquet 145459560 143216677 1.02
clickbench_q12/datafusion:parquet 371644489 368872299 1.01
clickbench_q13/datafusion:parquet 524624303 513220847 1.02
clickbench_q14/datafusion:parquet 374197501 362655797 1.03
clickbench_q15/datafusion:parquet 393047987 398541378 0.99
clickbench_q16/datafusion:parquet 742401029 728977824 1.02
clickbench_q17/datafusion:parquet 727650112 721644150 1.01
clickbench_q18/datafusion:parquet 1524110396 1569026475 0.97
clickbench_q19/datafusion:parquet 🚀 32044162 37850659 0.85
clickbench_q20/datafusion:parquet 618643875 623027131 0.99
clickbench_q21/datafusion:parquet 683646442 675628508 1.01
clickbench_q22/datafusion:parquet 1009781992 996325782 1.01
clickbench_q23/datafusion:parquet 3873806866 3750820428 1.03
clickbench_q24/datafusion:parquet 96868675 92340727 1.05
clickbench_q25/datafusion:parquet 150122511 151583997 0.99
clickbench_q26/datafusion:parquet 96637017 94909900 1.02
clickbench_q27/datafusion:parquet 1071896419 1061186824 1.01
clickbench_q28/datafusion:parquet 6592355016 6455048847 1.02
clickbench_q29/datafusion:parquet 231741012 241101715 0.96
clickbench_q30/datafusion:parquet 381475351 379010555 1.01
clickbench_q31/datafusion:parquet 412303081 405308616 1.02
clickbench_q32/datafusion:parquet 1384465728 1320034356 1.05
clickbench_q33/datafusion:parquet 1555590627 1552785374 1.00
clickbench_q34/datafusion:parquet 1551951735 1529864853 1.01
clickbench_q35/datafusion:parquet 547844638 554709869 0.99
clickbench_q36/datafusion:parquet 148061896 141114625 1.05
clickbench_q37/datafusion:parquet 58959279 58814450 1.00
clickbench_q38/datafusion:parquet 94622457 89551453 1.06
clickbench_q39/datafusion:parquet 286653928 279196000 1.03
clickbench_q40/datafusion:parquet 34148265 36257134 0.94
clickbench_q41/datafusion:parquet 31883989 30122678 1.06
clickbench_q42/datafusion:parquet 34369864 31522266 1.09
duckdb / vortex-file-compressed (0.948x ➖, 7↑ 2↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
clickbench_q00/duckdb:vortex-file-compressed 5902321 5750590 1.03
clickbench_q01/duckdb:vortex-file-compressed 13565786 13390529 1.01
clickbench_q02/duckdb:vortex-file-compressed 21658872 23645650 0.92
clickbench_q03/duckdb:vortex-file-compressed 31290361 30603506 1.02
clickbench_q04/duckdb:vortex-file-compressed 187248917 174666277 1.07
clickbench_q05/duckdb:vortex-file-compressed 196424491 196310588 1.00
clickbench_q06/duckdb:vortex-file-compressed 19243577 18806573 1.02
clickbench_q07/duckdb:vortex-file-compressed 14887322 16173767 0.92
clickbench_q08/duckdb:vortex-file-compressed 261115126 264091995 0.99
clickbench_q09/duckdb:vortex-file-compressed 369878029 375036418 0.99
clickbench_q10/duckdb:vortex-file-compressed 71046116 72310163 0.98
clickbench_q11/duckdb:vortex-file-compressed 78644341 79675251 0.99
clickbench_q12/duckdb:vortex-file-compressed 211407236 213151579 0.99
clickbench_q13/duckdb:vortex-file-compressed 427920916 443840748 0.96
clickbench_q14/duckdb:vortex-file-compressed 238777954 240388977 0.99
clickbench_q15/duckdb:vortex-file-compressed 221520361 206960458 1.07
clickbench_q16/duckdb:vortex-file-compressed 733077141 706163741 1.04
clickbench_q17/duckdb:vortex-file-compressed 543742786 577329293 0.94
clickbench_q18/duckdb:vortex-file-compressed 1223718105 1213729971 1.01
clickbench_q19/duckdb:vortex-file-compressed 🚨 22887722 20480181 1.12
clickbench_q20/duckdb:vortex-file-compressed 384322630 393700220 0.98
clickbench_q21/duckdb:vortex-file-compressed 399743445 413217620 0.97
clickbench_q22/duckdb:vortex-file-compressed 683509897 663626437 1.03
clickbench_q23/duckdb:vortex-file-compressed 🚀 667028990 1406888115 0.47
clickbench_q24/duckdb:vortex-file-compressed 🚨 47381372 43011364 1.10
clickbench_q25/duckdb:vortex-file-compressed 72419548 72889450 0.99
clickbench_q26/duckdb:vortex-file-compressed 40407442 44823458 0.90
clickbench_q27/duckdb:vortex-file-compressed 516467282 526694405 0.98
clickbench_q28/duckdb:vortex-file-compressed 2994246102 2995328124 1.00
clickbench_q29/duckdb:vortex-file-compressed 25840718 28498123 0.91
clickbench_q30/duckdb:vortex-file-compressed 171074528 172957656 0.99
clickbench_q31/duckdb:vortex-file-compressed 256614225 254999685 1.01
clickbench_q32/duckdb:vortex-file-compressed 1320107796 1320219399 1.00
clickbench_q33/duckdb:vortex-file-compressed 1661194756 1659827138 1.00
clickbench_q34/duckdb:vortex-file-compressed 1711695488 1692468453 1.01
clickbench_q35/duckdb:vortex-file-compressed 359713437 365027986 0.99
clickbench_q36/duckdb:vortex-file-compressed 🚀 27243875 32491349 0.84
clickbench_q37/duckdb:vortex-file-compressed 🚀 18416687 23062921 0.80
clickbench_q38/duckdb:vortex-file-compressed 🚀 21070719 25837501 0.82
clickbench_q39/duckdb:vortex-file-compressed 41622987 45669288 0.91
clickbench_q40/duckdb:vortex-file-compressed 🚀 18647335 26195538 0.71
clickbench_q41/duckdb:vortex-file-compressed 🚀 19325636 22995272 0.84
clickbench_q42/duckdb:vortex-file-compressed 🚀 21493973 26173270 0.82
duckdb / parquet (0.997x ➖, 0↑ 0↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
clickbench_q00/duckdb:parquet 213516072 213634416 1.00
clickbench_q01/duckdb:parquet 51078646 49651483 1.03
clickbench_q02/duckdb:parquet 65932814 65575643 1.01
clickbench_q03/duckdb:parquet 63729344 62951241 1.01
clickbench_q04/duckdb:parquet 205864560 204116995 1.01
clickbench_q05/duckdb:parquet 278803503 280134385 1.00
clickbench_q06/duckdb:parquet 59345849 58732751 1.01
clickbench_q07/duckdb:parquet 53740806 51437532 1.04
clickbench_q08/duckdb:parquet 267462428 269932762 0.99
clickbench_q09/duckdb:parquet 394925488 395143263 1.00
clickbench_q10/duckdb:parquet 98101009 98218468 1.00
clickbench_q11/duckdb:parquet 107590507 109202716 0.99
clickbench_q12/duckdb:parquet 298373476 294543094 1.01
clickbench_q13/duckdb:parquet 512146772 504583254 1.01
clickbench_q14/duckdb:parquet 328028841 326011016 1.01
clickbench_q15/duckdb:parquet 239938425 242746884 0.99
clickbench_q16/duckdb:parquet 617444245 604658158 1.02
clickbench_q17/duckdb:parquet 507124410 511858360 0.99
clickbench_q18/duckdb:parquet 1057506917 1050881863 1.01
clickbench_q19/duckdb:parquet 48895030 49523318 0.99
clickbench_q20/duckdb:parquet 438473227 432932908 1.01
clickbench_q21/duckdb:parquet 553585537 559475658 0.99
clickbench_q22/duckdb:parquet 927590802 935520998 0.99
clickbench_q23/duckdb:parquet 317444611 321666682 0.99
clickbench_q24/duckdb:parquet 77382627 77985500 0.99
clickbench_q25/duckdb:parquet 141730061 141048675 1.00
clickbench_q26/duckdb:parquet 51194515 51475302 0.99
clickbench_q27/duckdb:parquet 639304040 641682891 1.00
clickbench_q28/duckdb:parquet 4828626624 4820550428 1.00
clickbench_q29/duckdb:parquet 59788342 60888910 0.98
clickbench_q30/duckdb:parquet 305148478 305080914 1.00
clickbench_q31/duckdb:parquet 349796153 348566776 1.00
clickbench_q32/duckdb:parquet 1142170259 1225642231 0.93
clickbench_q33/duckdb:parquet 1236940894 1347130008 0.92
clickbench_q34/duckdb:parquet 1296072224 1323415431 0.98
clickbench_q35/duckdb:parquet 351543387 351491689 1.00
clickbench_q36/duckdb:parquet 64014393 63282812 1.01
clickbench_q37/duckdb:parquet 53531510 54473597 0.98
clickbench_q38/duckdb:parquet 58060834 59045824 0.98
clickbench_q39/duckdb:parquet 97810324 97714546 1.00
clickbench_q40/duckdb:parquet 55431446 54440789 1.02
clickbench_q41/duckdb:parquet 58414143 59433774 0.98
clickbench_q42/duckdb:parquet 49375248 49472250 1.00
duckdb / duckdb (1.019x ➖, 0↑ 1↓)
name PR f652db6 (ns) base cbc8ad1 (ns) ratio (PR/base)
clickbench_q00/duckdb:duckdb 19102490 17409233 1.10
clickbench_q01/duckdb:duckdb 35840807 33136749 1.08
clickbench_q02/duckdb:duckdb 48500678 46193669 1.05
clickbench_q03/duckdb:duckdb 49508265 46294121 1.07
clickbench_q04/duckdb:duckdb 224473274 221304647 1.01
clickbench_q05/duckdb:duckdb 316232370 349470598 0.90
clickbench_q06/duckdb:duckdb 37527481 34987833 1.07
clickbench_q07/duckdb:duckdb 37263947 34998173 1.06
clickbench_q08/duckdb:duckdb 314356468 321829067 0.98
clickbench_q09/duckdb:duckdb 486732919 499249644 0.97
clickbench_q10/duckdb:duckdb 99696724 98447131 1.01
clickbench_q11/duckdb:duckdb 110076578 107013784 1.03
clickbench_q12/duckdb:duckdb 298106172 315043562 0.95
clickbench_q13/duckdb:duckdb 548703374 544950430 1.01
clickbench_q14/duckdb:duckdb 346722748 349928495 0.99
clickbench_q15/duckdb:duckdb 264241737 265068757 1.00
clickbench_q16/duckdb:duckdb 681376985 677036091 1.01
clickbench_q17/duckdb:duckdb 581274588 589301161 0.99
clickbench_q18/duckdb:duckdb 1240669343 1279991304 0.97
clickbench_q19/duckdb:duckdb 36862605 34789665 1.06
clickbench_q20/duckdb:duckdb 954289617 1012118299 0.94
clickbench_q21/duckdb:duckdb 1058580384 1110845394 0.95
clickbench_q22/duckdb:duckdb 1085551676 1104009389 0.98
clickbench_q23/duckdb:duckdb 250722874 251951383 1.00
clickbench_q24/duckdb:duckdb 63605219 61212954 1.04
clickbench_q25/duckdb:duckdb 126218442 124118130 1.02
clickbench_q26/duckdb:duckdb 58515305 54974178 1.06
clickbench_q27/duckdb:duckdb 1129110166 1110014371 1.02
clickbench_q28/duckdb:duckdb 4576572040 4656719699 0.98
clickbench_q29/duckdb:duckdb 50424664 48167456 1.05
clickbench_q30/duckdb:duckdb 312130767 310815193 1.00
clickbench_q31/duckdb:duckdb 446861241 441862507 1.01
clickbench_q32/duckdb:duckdb 1437862120 1528960672 0.94
clickbench_q33/duckdb:duckdb 2129617096 1959075640 1.09
clickbench_q34/duckdb:duckdb 🚨 2223062314 2020896314 1.10
clickbench_q35/duckdb:duckdb 264375458 263687898 1.00
clickbench_q36/duckdb:duckdb 39107173 38776130 1.01
clickbench_q37/duckdb:duckdb 33548704 31035923 1.08
clickbench_q38/duckdb:duckdb 33214892 31085548 1.07
clickbench_q39/duckdb:duckdb 61212499 62784669 0.97
clickbench_q40/duckdb:duckdb 33704079 31287504 1.08
clickbench_q41/duckdb:duckdb 32064901 29727255 1.08
clickbench_q42/duckdb:duckdb 33641521 31307083 1.07
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
0 datafusion:vortex-file-compressed +861.6% +1.4% +848.6% +502.3% 🚨 regression
0 duckdb:duckdb +9.7% +1.4% +8.2% +310.3% ➖ noise
0 duckdb:vortex-file-compressed +2.6% +1.4% +1.3% +368.3% ➖ noise
1 datafusion:vortex-file-compressed +7.5% +3.2% +4.1% +27.0% ➖ noise
1 duckdb:duckdb +8.2% +3.2% +4.8% +79.2% ➖ noise
1 duckdb:vortex-file-compressed +1.3% +3.2% -1.9% +15.5% ➖ noise
2 datafusion:vortex-file-compressed +65.1% +1.4% +62.8% +31.1% 🚨 regression
2 duckdb:duckdb +5.0% +1.4% +3.5% +31.0% ➖ noise
2 duckdb:vortex-file-compressed -8.4% +1.4% -9.7% +14.8% ➖ noise
3 datafusion:vortex-file-compressed +80.2% +2.1% +76.5% +139.0% ➖ noise
3 duckdb:duckdb +6.9% +2.1% +4.8% +75.1% ➖ noise
3 duckdb:vortex-file-compressed +2.2% +2.1% +0.2% +29.5% ➖ noise
4 datafusion:vortex-file-compressed +1.8% +2.2% -0.5% +10.0% ➖ noise
4 duckdb:duckdb +1.4% +2.2% -0.8% +10.0% ➖ noise
4 duckdb:vortex-file-compressed +7.2% +2.2% +4.9% +10.0% ➖ noise
5 datafusion:vortex-file-compressed +12.4% +1.9% +10.3% +10.0% 🚨 regression
5 duckdb:duckdb -9.5% +1.9% -11.2% +10.0% ✅ faster
5 duckdb:vortex-file-compressed +0.1% +1.9% -1.8% +10.0% ➖ noise
6 datafusion:vortex-file-compressed +3454.5% +0.9% +3422.9% +33.8% 🚨 regression
6 duckdb:duckdb +7.3% +0.9% +6.3% +31.5% ➖ noise
6 duckdb:vortex-file-compressed +2.3% +0.9% +1.4% +19.2% ➖ noise
7 datafusion:vortex-file-compressed +35.6% +3.5% +30.9% +11.2% 🚨 regression
7 duckdb:duckdb +6.5% +3.5% +2.8% +11.8% ➖ noise
7 duckdb:vortex-file-compressed -8.0% +3.5% -11.1% +10.0% ✅ faster
8 datafusion:vortex-file-compressed +1.3% +0.6% +0.7% +10.0% ➖ noise
8 duckdb:duckdb -2.3% +0.6% -2.9% +10.0% ➖ noise
8 duckdb:vortex-file-compressed -1.1% +0.6% -1.7% +10.0% ➖ noise
9 datafusion:vortex-file-compressed -5.1% +0.3% -5.4% +10.0% ➖ noise
9 duckdb:duckdb -2.5% +0.3% -2.8% +10.0% ➖ noise
9 duckdb:vortex-file-compressed -1.4% +0.3% -1.6% +10.0% ➖ noise
10 datafusion:vortex-file-compressed +10.8% +0.5% +10.3% +11.4% ➖ noise
10 duckdb:duckdb +1.3% +0.5% +0.8% +10.0% ➖ noise
10 duckdb:vortex-file-compressed -1.7% +0.5% -2.2% +10.0% ➖ noise
11 datafusion:vortex-file-compressed +11.2% +0.0% +11.2% +10.0% 🚨 regression
11 duckdb:duckdb +2.9% +0.0% +2.8% +10.0% ➖ noise
11 duckdb:vortex-file-compressed -1.3% +0.0% -1.3% +10.0% ➖ noise
12 datafusion:vortex-file-compressed +1.3% +1.0% +0.3% +10.0% ➖ noise
12 duckdb:duckdb -5.4% +1.0% -6.3% +10.0% ➖ noise
12 duckdb:vortex-file-compressed -0.8% +1.0% -1.8% +10.0% ➖ noise
13 datafusion:vortex-file-compressed +2.1% +1.9% +0.2% +10.0% ➖ noise
13 duckdb:duckdb +0.7% +1.9% -1.1% +10.0% ➖ noise
13 duckdb:vortex-file-compressed -3.6% +1.9% -5.3% +10.0% ➖ noise
14 datafusion:vortex-file-compressed +2.0% +1.9% +0.1% +10.0% ➖ noise
14 duckdb:duckdb -0.9% +1.9% -2.8% +10.0% ➖ noise
14 duckdb:vortex-file-compressed -0.7% +1.9% -2.5% +10.0% ➖ noise
15 datafusion:vortex-file-compressed -1.6% -1.3% -0.4% +10.0% ➖ noise
15 duckdb:duckdb -0.3% -1.3% +1.0% +10.0% ➖ noise
15 duckdb:vortex-file-compressed +7.0% -1.3% +8.4% +10.0% ➖ noise
16 datafusion:vortex-file-compressed +2.3% +2.0% +0.3% +10.0% ➖ noise
16 duckdb:duckdb +0.6% +2.0% -1.3% +10.0% ➖ noise
16 duckdb:vortex-file-compressed +3.8% +2.0% +1.8% +17.0% ➖ noise
17 datafusion:vortex-file-compressed +0.9% -0.1% +0.9% +10.0% ➖ noise
17 duckdb:duckdb -1.4% -0.1% -1.3% +10.0% ➖ noise
17 duckdb:vortex-file-compressed -5.8% -0.1% -5.8% +10.0% ➖ noise
18 datafusion:vortex-file-compressed +8.2% -1.1% +9.5% +10.0% ➖ noise
18 duckdb:duckdb -3.1% -1.1% -2.0% +10.0% ➖ noise
18 duckdb:vortex-file-compressed +0.8% -1.1% +2.0% +10.0% ➖ noise
19 datafusion:vortex-file-compressed +7.0% -8.6% +17.0% +26.1% ➖ noise
19 duckdb:duckdb +6.0% -8.6% +15.9% +22.3% ➖ noise
19 duckdb:vortex-file-compressed +11.8% -8.6% +22.2% +39.8% ➖ noise
20 datafusion:vortex-file-compressed +9.9% +0.3% +9.6% +179.7% ➖ noise
20 duckdb:duckdb -5.7% +0.3% -6.0% +35.2% ➖ noise
20 duckdb:vortex-file-compressed -2.4% +0.3% -2.7% +10.0% ➖ noise
21 datafusion:vortex-file-compressed +1.8% +0.1% +1.7% +10.0% ➖ noise
21 duckdb:duckdb -4.7% +0.1% -4.8% +11.4% ➖ noise
21 duckdb:vortex-file-compressed -3.3% +0.1% -3.3% +10.0% ➖ noise
22 datafusion:vortex-file-compressed +19.0% +0.2% +18.7% +18.7% ➖ noise
22 duckdb:duckdb -1.7% +0.2% -1.9% +10.0% ➖ noise
22 duckdb:vortex-file-compressed +3.0% +0.2% +2.7% +20.0% ➖ noise
23 datafusion:vortex-file-compressed +218.3% +1.0% +215.3% +53.0% 🚨 regression
23 duckdb:duckdb -0.5% +1.0% -1.4% +10.0% ➖ noise
23 duckdb:vortex-file-compressed -52.6% +1.0% -53.0% +26.7% ✅ faster
24 datafusion:vortex-file-compressed +129.0% +2.0% +124.5% +12.9% 🚨 regression
24 duckdb:duckdb +3.9% +2.0% +1.8% +10.0% ➖ noise
24 duckdb:vortex-file-compressed +10.2% +2.0% +8.0% +33.1% ➖ noise
25 datafusion:vortex-file-compressed +28.0% -0.2% +28.3% +10.0% 🚨 regression
25 duckdb:duckdb +1.7% -0.2% +1.9% +10.0% ➖ noise
25 duckdb:vortex-file-compressed -0.6% -0.2% -0.4% +10.0% ➖ noise
26 datafusion:vortex-file-compressed +132.0% +0.6% +130.6% +11.4% 🚨 regression
26 duckdb:duckdb +6.4% +0.6% +5.8% +14.8% ➖ noise
26 duckdb:vortex-file-compressed -9.9% +0.6% -10.4% +21.3% ➖ noise
27 datafusion:vortex-file-compressed +11.1% +0.3% +10.8% +10.0% 🚨 regression
27 duckdb:duckdb +1.7% +0.3% +1.4% +10.0% ➖ noise
27 duckdb:vortex-file-compressed -1.9% +0.3% -2.3% +10.0% ➖ noise
28 datafusion:vortex-file-compressed -18.1% +1.1% -19.1% +10.0% ✅ faster
28 duckdb:duckdb -1.7% +1.1% -2.8% +10.0% ➖ noise
28 duckdb:vortex-file-compressed -0.0% +1.1% -1.2% +10.0% ➖ noise
29 datafusion:vortex-file-compressed -9.8% -2.9% -7.1% +10.0% ➖ noise
29 duckdb:duckdb +4.7% -2.9% +7.8% +10.0% ➖ noise
29 duckdb:vortex-file-compressed -9.3% -2.9% -6.7% +14.7% ➖ noise
30 datafusion:vortex-file-compressed +4.4% +0.3% +4.0% +10.0% ➖ noise
30 duckdb:duckdb +0.4% +0.3% +0.1% +10.0% ➖ noise
30 duckdb:vortex-file-compressed -1.1% +0.3% -1.4% +10.0% ➖ noise
31 datafusion:vortex-file-compressed -1.4% +1.0% -2.4% +10.0% ➖ noise
31 duckdb:duckdb +1.1% +1.0% +0.1% +10.0% ➖ noise
31 duckdb:vortex-file-compressed +0.6% +1.0% -0.4% +10.0% ➖ noise
32 datafusion:vortex-file-compressed -0.1% -1.1% +1.1% +13.4% ➖ noise
32 duckdb:duckdb -6.0% -1.1% -4.9% +10.0% ➖ noise
32 duckdb:vortex-file-compressed -0.0% -1.1% +1.1% +10.0% ➖ noise
33 datafusion:vortex-file-compressed +20.2% -4.1% +25.3% +13.5% 🚨 regression
33 duckdb:duckdb +8.7% -4.1% +13.3% +10.3% 🚨 regression
33 duckdb:vortex-file-compressed +0.1% -4.1% +4.4% +10.1% ➖ noise
34 datafusion:vortex-file-compressed +18.0% -0.3% +18.3% +10.0% 🚨 regression
34 duckdb:duckdb +10.0% -0.3% +10.4% +10.0% 🚨 regression
34 duckdb:vortex-file-compressed +1.1% -0.3% +1.5% +10.0% ➖ noise
35 datafusion:vortex-file-compressed +1.4% -0.6% +2.0% +10.0% ➖ noise
35 duckdb:duckdb +0.3% -0.6% +0.9% +10.0% ➖ noise
35 duckdb:vortex-file-compressed -1.5% -0.6% -0.8% +10.0% ➖ noise
36 datafusion:vortex-file-compressed -55.4% +3.0% -56.7% +10.9% ✅ faster
36 duckdb:duckdb +0.9% +3.0% -2.1% +12.1% ➖ noise
36 duckdb:vortex-file-compressed -16.2% +3.0% -18.6% +10.0% ✅ faster
37 datafusion:vortex-file-compressed -32.0% -0.7% -31.5% +11.7% ✅ faster
37 duckdb:duckdb +8.1% -0.7% +8.9% +10.0% ➖ noise
37 duckdb:vortex-file-compressed -20.1% -0.7% -19.5% +12.0% ✅ faster
38 datafusion:vortex-file-compressed +7.3% +1.9% +5.3% +18.3% ➖ noise
38 duckdb:duckdb +6.8% +1.9% +4.8% +10.0% ➖ noise
38 duckdb:vortex-file-compressed -18.4% +1.9% -20.0% +10.0% ✅ faster
39 datafusion:vortex-file-compressed -64.0% +1.4% -64.5% +10.0% ✅ faster
39 duckdb:duckdb -2.5% +1.4% -3.8% +14.9% ➖ noise
39 duckdb:vortex-file-compressed -8.9% +1.4% -10.1% +10.0% ✅ faster
40 datafusion:vortex-file-compressed +28.5% -2.1% +31.3% +12.5% 🚨 regression
40 duckdb:duckdb +7.7% -2.1% +10.0% +10.0% 🚨 regression
40 duckdb:vortex-file-compressed -28.8% -2.1% -27.3% +11.7% ✅ faster
41 datafusion:vortex-file-compressed +20.0% +2.0% +17.6% +10.0% 🚨 regression
41 duckdb:duckdb +7.9% +2.0% +5.8% +10.0% ➖ noise
41 duckdb:vortex-file-compressed -16.0% +2.0% -17.6% +10.0% ✅ faster
42 datafusion:vortex-file-compressed -1.1% +4.3% -5.2% +10.0% ➖ noise
42 duckdb:duckdb +7.5% +4.3% +3.0% +10.0% ➖ noise
42 duckdb:vortex-file-compressed -17.9% +4.3% -21.3% +13.1% ✅ faster

@0ax1 0ax1 changed the title experiment: v2 datafusion no pool experiment: v2 datafusion no pinning pool Mar 22, 2026
@0ax1 0ax1 closed this Mar 22, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Mar 22, 2026 •
edited
Loading

Benchmarks: TPC-H SF=10 on S3

Verdict: No clear signal (environment too noisy confidence)
Attributed Vortex impact: +11.3%
Vortex (geomean): 1.017x ➖
Parquet (geomean): 0.914x ➖
Shifts: Parquet (control) -8.6% · Median polish -3.8%

datafusion / vortex-file-compressed (1.105x ➖, 3↑ 6↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-file-compressed 🚀 606202162 868452366 0.70
tpch_q02/datafusion:vortex-file-compressed 🚀 533125966 1185957979 0.45
tpch_q03/datafusion:vortex-file-compressed 1280472111 1168646202 1.10
tpch_q04/datafusion:vortex-file-compressed 731699929 760032315 0.96
tpch_q05/datafusion:vortex-file-compressed 🚨 2040921329 1300459859 1.57
tpch_q06/datafusion:vortex-file-compressed 🚀 585028432 857044416 0.68
tpch_q07/datafusion:vortex-file-compressed 1245091368 1391847320 0.89
tpch_q08/datafusion:vortex-file-compressed 🚨 2484742683 1509030908 1.65
tpch_q09/datafusion:vortex-file-compressed 🚨 3380931826 1642887541 2.06
tpch_q10/datafusion:vortex-file-compressed 1057933662 1075255380 0.98
tpch_q11/datafusion:vortex-file-compressed 360466873 511454516 0.70
tpch_q12/datafusion:vortex-file-compressed 976930466 1005382194 0.97
tpch_q13/datafusion:vortex-file-compressed 519187563 453536758 1.14
tpch_q14/datafusion:vortex-file-compressed 542255722 604178852 0.90
tpch_q15/datafusion:vortex-file-compressed 1054298722 1098118108 0.96
tpch_q16/datafusion:vortex-file-compressed 375677564 359567365 1.04
tpch_q17/datafusion:vortex-file-compressed 🚨 2203778819 1219371356 1.81
tpch_q18/datafusion:vortex-file-compressed 🚨 2474147631 1465521974 1.69
tpch_q19/datafusion:vortex-file-compressed 611460832 765956786 0.80
tpch_q20/datafusion:vortex-file-compressed 948623005 1237146151 0.77
tpch_q21/datafusion:vortex-file-compressed 🚨 10902018030 1747780365 6.24
tpch_q22/datafusion:vortex-file-compressed 423610421 409946692 1.03
datafusion / vortex-compact (1.022x ➖, 5↑ 6↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:vortex-compact 637210728 901437380 0.71
tpch_q02/datafusion:vortex-compact 🚀 489479648 1645917008 0.30
tpch_q03/datafusion:vortex-compact 1221918464 1558411098 0.78
tpch_q04/datafusion:vortex-compact 688649034 710203975 0.97
tpch_q05/datafusion:vortex-compact 🚨 1999349823 1253538615 1.59
tpch_q06/datafusion:vortex-compact 504825363 671971227 0.75
tpch_q07/datafusion:vortex-compact 1155502163 1190212801 0.97
tpch_q08/datafusion:vortex-compact 🚨 2326806801 1207722141 1.93
tpch_q09/datafusion:vortex-compact 🚨 3131444586 1366132876 2.29
tpch_q10/datafusion:vortex-compact 1036374118 1031400055 1.00
tpch_q11/datafusion:vortex-compact 🚀 291639760 470664018 0.62
tpch_q12/datafusion:vortex-compact 918139729 1265069322 0.73
tpch_q13/datafusion:vortex-compact 641366602 572892894 1.12
tpch_q14/datafusion:vortex-compact 🚀 529144368 923398471 0.57
tpch_q15/datafusion:vortex-compact 🚀 905389000 1428919120 0.63
tpch_q16/datafusion:vortex-compact 🚀 328389407 543922341 0.60
tpch_q17/datafusion:vortex-compact 🚨 2220895296 1391447763 1.60
tpch_q18/datafusion:vortex-compact 🚨 2365888539 1339017879 1.77
tpch_q19/datafusion:vortex-compact 604817969 677117681 0.89
tpch_q20/datafusion:vortex-compact 901414433 925399127 0.97
tpch_q21/datafusion:vortex-compact 🚨 10647702217 1462606126 7.28
tpch_q22/datafusion:vortex-compact 464891137 464982353 1.00
datafusion / parquet (0.911x ➖, 1↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/datafusion:parquet 900953388 894668375 1.01
tpch_q02/datafusion:parquet 760375279 830281050 0.92
tpch_q03/datafusion:parquet 958682837 1108443770 0.86
tpch_q04/datafusion:parquet 461612477 510764423 0.90
tpch_q05/datafusion:parquet 1192911290 1406634525 0.85
tpch_q06/datafusion:parquet 528470508 563875203 0.94
tpch_q07/datafusion:parquet 1390907073 1522445326 0.91
tpch_q08/datafusion:parquet 1657510847 1622918996 1.02
tpch_q09/datafusion:parquet 1806496790 1835015951 0.98
tpch_q10/datafusion:parquet 2026230271 2164629500 0.94
tpch_q11/datafusion:parquet 559719382 565349639 0.99
tpch_q12/datafusion:parquet 684443621 673054558 1.02
tpch_q13/datafusion:parquet 688012582 735647963 0.94
tpch_q14/datafusion:parquet 804338294 828216580 0.97
tpch_q15/datafusion:parquet 1257021884 1223076274 1.03
tpch_q16/datafusion:parquet 469499367 461315951 1.02
tpch_q17/datafusion:parquet 1388201606 1502119993 0.92
tpch_q18/datafusion:parquet 1585892986 1796387322 0.88
tpch_q19/datafusion:parquet 982427817 1116213556 0.88
tpch_q20/datafusion:parquet 🚀 1129939670 1807076720 0.63
tpch_q21/datafusion:parquet 1689711075 2036747290 0.83
tpch_q22/datafusion:parquet 661789357 897307372 0.74
duckdb / vortex-file-compressed (0.937x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-file-compressed 623444004 640088961 0.97
tpch_q02/duckdb:vortex-file-compressed 1047652580 1134234582 0.92
tpch_q03/duckdb:vortex-file-compressed 904109513 985731616 0.92
tpch_q04/duckdb:vortex-file-compressed 630601884 687131814 0.92
tpch_q05/duckdb:vortex-file-compressed 1059625216 1089615734 0.97
tpch_q06/duckdb:vortex-file-compressed 796818843 821392994 0.97
tpch_q07/duckdb:vortex-file-compressed 1042617374 1186944783 0.88
tpch_q08/duckdb:vortex-file-compressed 1331283941 1514949727 0.88
tpch_q09/duckdb:vortex-file-compressed 1401281642 1486659688 0.94
tpch_q10/duckdb:vortex-file-compressed 1038327465 1129550224 0.92
tpch_q11/duckdb:vortex-file-compressed 615995077 688991140 0.89
tpch_q12/duckdb:vortex-file-compressed 714225168 803633958 0.89
tpch_q13/duckdb:vortex-file-compressed 899648485 1126114004 0.80
tpch_q14/duckdb:vortex-file-compressed 853699659 921983570 0.93
tpch_q15/duckdb:vortex-file-compressed 571956122 557192395 1.03
tpch_q16/duckdb:vortex-file-compressed 492382618 463446541 1.06
tpch_q17/duckdb:vortex-file-compressed 969932538 1016279883 0.95
tpch_q18/duckdb:vortex-file-compressed 901943438 1020700786 0.88
tpch_q19/duckdb:vortex-file-compressed 772671560 806000314 0.96
tpch_q20/duckdb:vortex-file-compressed 1280662143 1188563325 1.08
tpch_q21/duckdb:vortex-file-compressed 2002030125 2032770743 0.98
tpch_q22/duckdb:vortex-file-compressed 589789498 649991174 0.91
duckdb / vortex-compact (1.012x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:vortex-compact 624502029 621712622 1.00
tpch_q02/duckdb:vortex-compact 970282453 959609879 1.01
tpch_q03/duckdb:vortex-compact 829053200 942282464 0.88
tpch_q04/duckdb:vortex-compact 539058459 483562743 1.11
tpch_q05/duckdb:vortex-compact 1020087456 995279404 1.02
tpch_q06/duckdb:vortex-compact 785060979 757596030 1.04
tpch_q07/duckdb:vortex-compact 999322628 1140756131 0.88
tpch_q08/duckdb:vortex-compact 1398475497 1396653280 1.00
tpch_q09/duckdb:vortex-compact 1344441029 1356268917 0.99
tpch_q10/duckdb:vortex-compact 1066574283 1088900675 0.98
tpch_q11/duckdb:vortex-compact 654523978 590554044 1.11
tpch_q12/duckdb:vortex-compact 642894372 647370298 0.99
tpch_q13/duckdb:vortex-compact 857938075 901700365 0.95
tpch_q14/duckdb:vortex-compact 862895773 847122791 1.02
tpch_q15/duckdb:vortex-compact 621882517 578140291 1.08
tpch_q16/duckdb:vortex-compact 434677711 381019272 1.14
tpch_q17/duckdb:vortex-compact 911598692 855636338 1.07
tpch_q18/duckdb:vortex-compact 841467253 698550915 1.20
tpch_q19/duckdb:vortex-compact 740157008 712214792 1.04
tpch_q20/duckdb:vortex-compact 1132499058 1187215876 0.95
tpch_q21/duckdb:vortex-compact 1697611417 1726206032 0.98
tpch_q22/duckdb:vortex-compact 601001216 680198195 0.88
duckdb / parquet (0.916x ➖, 0↑ 0↓)
name PR f652db6 (ns) base 2325594 (ns) ratio (PR/base)
tpch_q01/duckdb:parquet 852082481 1033007614 0.82
tpch_q02/duckdb:parquet 1510649255 1624079475 0.93
tpch_q03/duckdb:parquet 1525042029 1594131873 0.96
tpch_q04/duckdb:parquet 1006050861 1044951711 0.96
tpch_q05/duckdb:parquet 2013208430 2290129949 0.88
tpch_q06/duckdb:parquet 705319972 734467240 0.96
tpch_q07/duckdb:parquet 1685003800 1988187234 0.85
tpch_q08/duckdb:parquet 2393213485 2593791390 0.92
tpch_q09/duckdb:parquet 2208158740 2351756583 0.94
tpch_q10/duckdb:parquet 2936289457 3103263598 0.95
tpch_q11/duckdb:parquet 877572719 904696452 0.97
tpch_q12/duckdb:parquet 923934041 976457298 0.95
tpch_q13/duckdb:parquet 1164002634 1164856787 1.00
tpch_q14/duckdb:parquet 1147531754 1366579276 0.84
tpch_q15/duckdb:parquet 813542686 869251710 0.94
tpch_q16/duckdb:parquet 751120542 830891812 0.90
tpch_q17/duckdb:parquet 1270842197 1439550791 0.88
tpch_q18/duckdb:parquet 1262071402 1415025507 0.89
tpch_q19/duckdb:parquet 1374367702 1532453623 0.90
tpch_q20/duckdb:parquet 1663771151 1783413867 0.93
tpch_q21/duckdb:parquet 1750242467 2030784522 0.86
tpch_q22/duckdb:parquet 969510436 1021657007 0.95
Full attributed analysis
Query Config Raw Δ Control Δ Attributed α Noise floor Significant?
1 datafusion:vortex-compact -29.3% -8.9% -22.4% +46.3% ➖ noise
1 datafusion:vortex-file-compressed -30.2% -8.9% -23.4% +55.1% ➖ noise
1 duckdb:vortex-compact +0.4% -8.9% +10.2% +30.0% ➖ noise
1 duckdb:vortex-file-compressed -2.6% -8.9% +6.9% +30.0% ➖ noise
2 datafusion:vortex-compact -70.3% -7.7% -67.8% +30.0% ✅ faster
2 datafusion:vortex-file-compressed -55.0% -7.7% -51.3% +50.6% ✅ faster
2 duckdb:vortex-compact +1.1% -7.7% +9.6% +35.6% ➖ noise
2 duckdb:vortex-file-compressed -7.6% -7.7% +0.1% +30.0% ➖ noise
3 datafusion:vortex-compact -21.6% -9.0% -13.8% +54.0% ➖ noise
3 datafusion:vortex-file-compressed +9.6% -9.0% +20.5% +55.4% ➖ noise
3 duckdb:vortex-compact -12.0% -9.0% -3.3% +43.5% ➖ noise
3 duckdb:vortex-file-compressed -8.3% -9.0% +0.8% +38.5% ➖ noise
4 datafusion:vortex-compact -3.0% -6.7% +4.0% +36.8% ➖ noise
4 datafusion:vortex-file-compressed -3.7% -6.7% +3.2% +30.0% ➖ noise
4 duckdb:vortex-compact +11.5% -6.7% +19.5% +30.0% ➖ noise
4 duckdb:vortex-file-compressed -8.2% -6.7% -1.6% +30.0% ➖ noise
5 datafusion:vortex-compact +59.5% -13.7% +84.7% +30.0% 🚨 regression
5 datafusion:vortex-file-compressed +56.9% -13.7% +81.8% +30.0% 🚨 regression
5 duckdb:vortex-compact +2.5% -13.7% +18.7% +30.0% ➖ noise
5 duckdb:vortex-file-compressed -2.8% -13.7% +12.6% +30.0% ➖ noise
6 datafusion:vortex-compact -24.9% -5.1% -20.8% +30.5% ➖ noise
6 datafusion:vortex-file-compressed -31.7% -5.1% -28.0% +35.5% ✅ faster
6 duckdb:vortex-compact +3.6% -5.1% +9.2% +30.0% ➖ noise
6 duckdb:vortex-file-compressed -3.0% -5.1% +2.3% +30.6% ➖ noise
7 datafusion:vortex-compact -2.9% -12.0% +10.3% +30.0% ➖ noise
7 datafusion:vortex-file-compressed -10.5% -12.0% +1.7% +30.0% ➖ noise
7 duckdb:vortex-compact -12.4% -12.0% -0.4% +30.0% ➖ noise
7 duckdb:vortex-file-compressed -12.2% -12.0% -0.2% +30.0% ➖ noise
8 datafusion:vortex-compact +92.7% -2.9% +98.5% +30.0% 🚨 regression
8 datafusion:vortex-file-compressed +64.7% -2.9% +69.6% +30.0% 🚨 regression
8 duckdb:vortex-compact +0.1% -2.9% +3.1% +30.0% ➖ noise
8 duckdb:vortex-file-compressed -12.1% -2.9% -9.5% +30.0% ➖ noise
9 datafusion:vortex-compact +129.2% -3.9% +138.4% +30.0% 🚨 regression
9 datafusion:vortex-file-compressed +105.8% -3.9% +114.0% +30.0% 🚨 regression
9 duckdb:vortex-compact -0.9% -3.9% +3.1% +30.0% ➖ noise
9 duckdb:vortex-file-compressed -5.7% -3.9% -2.0% +30.0% ➖ noise
10 datafusion:vortex-compact +0.5% -5.9% +6.8% +30.0% ➖ noise
10 datafusion:vortex-file-compressed -1.6% -5.9% +4.5% +30.0% ➖ noise
10 duckdb:vortex-compact -2.1% -5.9% +4.1% +30.0% ➖ noise
10 duckdb:vortex-file-compressed -8.1% -5.9% -2.3% +30.0% ➖ noise
11 datafusion:vortex-compact -38.0% -2.0% -36.8% +30.0% ✅ faster
11 datafusion:vortex-file-compressed -29.5% -2.0% -28.1% +30.0% ✅ faster
11 duckdb:vortex-compact +10.8% -2.0% +13.1% +30.0% ➖ noise
11 duckdb:vortex-file-compressed -10.6% -2.0% -8.8% +30.0% ➖ noise
12 datafusion:vortex-compact -27.4% -1.9% -26.0% +30.0% ✅ faster
12 datafusion:vortex-file-compressed -2.8% -1.9% -0.9% +30.0% ➖ noise
12 duckdb:vortex-compact -0.7% -1.9% +1.2% +30.0% ➖ noise
12 duckdb:vortex-file-compressed -11.1% -1.9% -9.4% +30.0% ➖ noise
13 datafusion:vortex-compact +12.0% -3.3% +15.8% +39.7% ➖ noise
13 datafusion:vortex-file-compressed +14.5% -3.3% +18.4% +30.0% ➖ noise
13 duckdb:vortex-compact -4.9% -3.3% -1.6% +34.4% ➖ noise
13 duckdb:vortex-file-compressed -20.1% -3.3% -17.4% +30.0% ➖ noise
14 datafusion:vortex-compact -42.7% -9.7% -36.5% +30.0% ✅ faster
14 datafusion:vortex-file-compressed -10.2% -9.7% -0.6% +30.0% ➖ noise
14 duckdb:vortex-compact +1.9% -9.7% +12.8% +30.0% ➖ noise
14 duckdb:vortex-file-compressed -7.4% -9.7% +2.5% +30.0% ➖ noise
15 datafusion:vortex-compact -36.6% -1.9% -35.4% +30.0% ✅ faster
15 datafusion:vortex-file-compressed -4.0% -1.9% -2.1% +30.0% ➖ noise
15 duckdb:vortex-compact +7.6% -1.9% +9.7% +30.0% ➖ noise
15 duckdb:vortex-file-compressed +2.6% -1.9% +4.7% +30.0% ➖ noise
16 datafusion:vortex-compact -39.6% -4.1% -37.1% +50.1% ✅ faster
16 datafusion:vortex-file-compressed +4.5% -4.1% +8.9% +33.5% ➖ noise
16 duckdb:vortex-compact +14.1% -4.1% +18.9% +30.0% ➖ noise
16 duckdb:vortex-file-compressed +6.2% -4.1% +10.8% +30.0% ➖ noise
17 datafusion:vortex-compact +59.6% -9.7% +76.7% +30.0% 🚨 regression
17 datafusion:vortex-file-compressed +80.7% -9.7% +100.1% +30.0% 🚨 regression
17 duckdb:vortex-compact +6.5% -9.7% +18.0% +30.0% ➖ noise
17 duckdb:vortex-file-compressed -4.6% -9.7% +5.7% +30.0% ➖ noise
18 datafusion:vortex-compact +76.7% -11.3% +99.1% +30.0% 🚨 regression
18 datafusion:vortex-file-compressed +68.8% -11.3% +90.3% +30.0% 🚨 regression
18 duckdb:vortex-compact +20.5% -11.3% +35.8% +30.0% 🚨 regression
18 duckdb:vortex-file-compressed -11.6% -11.3% -0.4% +30.0% ➖ noise
19 datafusion:vortex-compact -10.7% -11.2% +0.5% +30.0% ➖ noise
19 datafusion:vortex-file-compressed -20.2% -11.2% -10.1% +30.0% ➖ noise
19 duckdb:vortex-compact +3.9% -11.2% +17.0% +30.0% ➖ noise
19 duckdb:vortex-file-compressed -4.1% -11.2% +7.9% +30.0% ➖ noise
20 datafusion:vortex-compact -2.6% -23.6% +27.5% +30.0% ➖ noise
20 datafusion:vortex-file-compressed -23.3% -23.6% +0.4% +34.2% ➖ noise
20 duckdb:vortex-compact -4.6% -23.6% +24.9% +30.0% ➖ noise
20 duckdb:vortex-file-compressed +7.7% -23.6% +41.1% +30.0% 🚨 regression
21 datafusion:vortex-compact +628.0% -15.4% +760.9% +30.0% 🚨 regression
21 datafusion:vortex-file-compressed +523.8% -15.4% +637.7% +30.0% 🚨 regression
21 duckdb:vortex-compact -1.7% -15.4% +16.3% +30.0% ➖ noise
21 duckdb:vortex-file-compressed -1.5% -15.4% +16.5% +30.0% ➖ noise
22 datafusion:vortex-compact -0.0% -16.3% +19.5% +33.9% ➖ noise
22 datafusion:vortex-file-compressed +3.3% -16.3% +23.5% +30.0% ➖ noise
22 duckdb:vortex-compact -11.6% -16.3% +5.6% +32.8% ➖ noise
22 duckdb:vortex-file-compressed -9.3% -16.3% +8.5% +30.0% ➖ noise

@0ax1 0ax1 deleted the ad/v2-datafusion-no-pool branch March 22, 2026 21:21
@0ax1 0ax1 restored the ad/v2-datafusion-no-pool branch March 23, 2026 11:33
@0ax1 0ax1 reopened this Mar 23, 2026
@0ax1 0ax1 added the action/benchmark-sql Trigger SQL benchmarks to run on this PR label Mar 23, 2026
@github-actions github-actions bot removed the action/benchmark-sql Trigger SQL benchmarks to run on this PR label Mar 23, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 23, 2026

Merging this PR will degrade performance by 10.2%

❌ 1 regressed benchmark
✅ 1015 untouched benchmarks
⏩ 1522 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Simulation map_each[BufferMut<i32>, 128] 770.6 ns 858.1 ns -10.2%

Comparing ad/v2-datafusion-no-pool (f652db6) with develop (cbc8ad1)

Open in CodSpeed

Footnotes

  1. 1522 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@0ax1 0ax1 closed this Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

do not merge Pull requests that are not intended to merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@0ax1