Skip to content

Implement native interleave for ListView#9558

Open
vegarsti wants to merge 5 commits intoapache:mainfrom
vegarsti:list-view-interleave-native
Open

Implement native interleave for ListView#9558
vegarsti wants to merge 5 commits intoapache:mainfrom
vegarsti:list-view-interleave-native

Conversation

@vegarsti
Copy link
Contributor

@vegarsti vegarsti commented Mar 15, 2026

Closes #9342.

This PR adds a native implementation of interleave for the ListView type. Also adds a benchmark.

Performance improves by more than 30% in all cases in the benchmark:

Benchmark Time (branch) Change vs baseline Speedup
list_view<i64>(0.1,0.1,20) 100 2.40 µs −41.2% 1.70×
list_view<i64>(0.1,0.1,20) 400 6.50 µs −36.6% 1.58×
list_view<i64>(0.1,0.1,20) 1024 15.67 µs −38.1% 1.62×
list_view<i64>(0.1,0.1,20) 1024 4-arr 15.76 µs −35.7% 1.55×
list_view<i64>(0.0,0.0,20) 100 1.49 µs −41.0% 1.70×
list_view<i64>(0.0,0.0,20) 400 3.99 µs −30.3% 1.43×
list_view<i64>(0.0,0.0,20) 1024 9.58 µs −40.7% 1.69×
list_view<i64>(0.0,0.0,20) 1024 4-arr 9.45 µs −36.2% 1.57×

@github-actions github-actions bot added the arrow Changes to the arrow crate label Mar 15, 2026
@vegarsti vegarsti force-pushed the list-view-interleave-native branch from a1131f2 to 16f2287 Compare March 15, 2026 15:42
@brancz
Copy link
Contributor

brancz commented Mar 16, 2026

Do you mind comparing this to the fallthrough performance of #9562 ?

@vegarsti
Copy link
Contributor Author

Do you mind comparing this to the fallthrough performance of #9562 ?

Oh for sure, thanks for reminding me!

@vegarsti
Copy link
Contributor Author

vegarsti commented Mar 16, 2026

Updated the description with results now. It's not looking like a win..!

@brancz
Copy link
Contributor

brancz commented Mar 16, 2026

I would say let's merge the fallthrough and iterate on this version. I'm sure there are several possibilities for optimizations.

@asubiotto
Copy link
Contributor

FWIW I pushed up the branch I've had marinating locally for a month or two in case it's helpful: main...polarsignals:arrow-rs:asubiotto/lvinterleave. I believe the benchmarks showed a slight regression for interleaves of small lists, but overall the perf was an improvement. I'm not able to take a closer look right now, but sharing in case it's helpful.

@vegarsti
Copy link
Contributor Author

FWIW I pushed up the branch I've had marinating locally for a month or two in case it's helpful: main...polarsignals:arrow-rs:asubiotto/lvinterleave. I believe the benchmarks showed a slight regression for interleaves of small lists, but overall the perf was an improvement. I'm not able to take a closer look right now, but sharing in case it's helpful.

Thank you!

@vegarsti vegarsti force-pushed the list-view-interleave-native branch from 6e8412e to b18c3a6 Compare March 19, 2026 12:48
@vegarsti
Copy link
Contributor Author

Updated implementation and results now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add native ListView support for interleave kernel

3 participants