Skip to content
This repository was archived by the owner on Jul 19, 2023. It is now read-only.

Conversation

@simonswine
Copy link
Collaborator

@simonswine simonswine commented Jul 11, 2023

This retrieves rowGroups in parallel, I don't think this is the way to go.

We rather should look at switchting the querying order to align completely with the block order (seriesId, then timeNanos)

❯ benchstat before.txt after.txt
name                                                                                                             old time/op    new time/op    delta
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:contentions:count:contentions:count-12       284ms ± 6%     269ms ± 1%   -5.25%  (p=0.016 n=5+4)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:delay:nanoseconds:contentions:count-12       289ms ± 8%     283ms ± 7%     ~     (p=0.151 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-goroutine:goroutine:count:goroutine:count-12       145ms ±15%     129ms ± 2%  -10.77%  (p=0.016 n=5+4)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_objects:count:space:bytes-12          123ms ± 6%     107ms ± 1%  -13.13%  (p=0.016 n=5+4)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_space:bytes:space:bytes-12            116ms ± 0%     106ms ± 5%   -8.92%  (p=0.016 n=4+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_objects:count:space:bytes-12          106ms ± 4%      95ms ± 6%  -10.19%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_space:bytes:space:bytes-12            106ms ± 5%      99ms ± 7%     ~     (p=0.056 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:contentions:count:contentions:count-12       128ms ± 2%     121ms ± 3%   -5.52%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:delay:nanoseconds:contentions:count-12       836ms ± 4%     810ms ± 0%   -3.05%  (p=0.016 n=5+4)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:cpu:nanoseconds:cpu:nanoseconds-12     1.53s ± 1%     1.53s ± 1%     ~     (p=0.690 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:samples:count:cpu:nanoseconds-12       1.47s ± 1%     1.48s ± 3%     ~     (p=0.841 n=5+5)

name                                                                                                             old alloc/op   new alloc/op   delta
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:contentions:count:contentions:count-12       492MB ± 0%     522MB ± 1%   +5.98%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:delay:nanoseconds:contentions:count-12       466MB ± 1%     495MB ± 1%   +6.14%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-goroutine:goroutine:count:goroutine:count-12       157MB ± 0%     186MB ± 1%  +18.44%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_objects:count:space:bytes-12          113MB ± 2%     142MB ± 0%  +25.31%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_space:bytes:space:bytes-12            122MB ± 1%     148MB ± 5%  +21.11%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_objects:count:space:bytes-12          101MB ± 2%     129MB ± 2%  +27.60%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_space:bytes:space:bytes-12            108MB ± 1%     138MB ± 3%  +26.92%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:contentions:count:contentions:count-12       158MB ± 1%     189MB ± 0%  +19.35%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:delay:nanoseconds:contentions:count-12       785MB ± 0%     813MB ± 1%   +3.56%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:cpu:nanoseconds:cpu:nanoseconds-12    1.56GB ± 0%    1.59GB ± 0%   +1.87%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:samples:count:cpu:nanoseconds-12      1.58GB ± 0%    1.61GB ± 0%   +1.65%  (p=0.008 n=5+5)

name                                                                                                             old allocs/op  new allocs/op  delta
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:contentions:count:contentions:count-12       66.6k ± 0%     67.3k ± 0%   +1.07%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-block:delay:nanoseconds:contentions:count-12       80.7k ± 0%     81.3k ± 0%   +0.80%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-goroutine:goroutine:count:goroutine:count-12       90.4k ± 0%     91.2k ± 0%   +0.86%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_objects:count:space:bytes-12           124k ± 0%      125k ± 0%   +0.56%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:alloc_space:bytes:space:bytes-12             113k ± 0%      114k ± 0%   +0.66%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_objects:count:space:bytes-12           111k ± 0%      112k ± 0%   +0.70%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-memory:inuse_space:bytes:space:bytes-12             108k ± 0%      109k ± 0%   +0.81%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:contentions:count:contentions:count-12       69.5k ± 0%     70.4k ± 0%   +1.33%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-mutex:delay:nanoseconds:contentions:count-12       1.30M ± 0%     1.30M ± 0%   +0.06%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:cpu:nanoseconds:cpu:nanoseconds-12     3.12M ± 0%     3.12M ± 0%   +0.03%  (p=0.008 n=5+5)
SelectMatchingProfilesRealBlock/block-01H406NQE5JEQPV6QBCC39BCHK-process_cpu:samples:count:cpu:nanoseconds-12       3.09M ± 0%     3.09M ± 0%   +0.03%  (p=0.008 n=5+5)

@simonswine simonswine changed the title 20230711 parallelize row groups Parallelize row groups Jul 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant