Skip to content

Store accumulator-stage lookups directly#10645

Open
charlesmyu wants to merge 1 commit intocharles.yu/djm-0000/fix-spark-plan-metricsfrom
charles.yu/djm-0000/stop-storing-stage-accumulators
Open

Store accumulator-stage lookups directly#10645
charlesmyu wants to merge 1 commit intocharles.yu/djm-0000/fix-spark-plan-metricsfrom
charles.yu/djm-0000/stop-storing-stage-accumulators

Conversation

@charlesmyu
Copy link
Contributor

What Does This Do

Stores the map of accumulator ID to stage ID directly in a Long - Integer map, rather than storing the stage-level accumulators themselves.

Motivation

The stage-level accumulators naively roll up metrics from the task level by summing all values. This means they are not accurate in all cases, particular for metrics that are better visualized as a distribution of values across all tasks instead of a single sum.

In #10553 we rollup task-level metrics ourselves and encode them into the Spark SQL metrics as distributions in order to improve the granularity of information collected. This left one remaining use of the stage-level accumulators - mapping operations in Spark SQL plans to their respective stages.

Since we do not need the entire accumulator to accomplish this, we should simplify that to a ID-ID map instead to save space. We can also remove the 50k limit since we are not storing the entire accumulator, allowing us to avoid creating orphaned operations (previously occurred when the EldestHashMap overflowed).

Additional Notes

I didn't feel the explicit need for a FF here, since the change is fairly straightforwards and shouldn't negatively impact perf. However, if we feel strongly otherwise I'm happy to make that happen.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@charlesmyu charlesmyu added the inst: apache spark Apache Spark instrumentation label Feb 19, 2026
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from e52fbc5 to e413d1d Compare February 19, 2026 22:02
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/stop-storing-stage-accumulators branch from e1c532c to 9594586 Compare February 19, 2026 22:02
@pr-commenter
Copy link

pr-commenter bot commented Feb 19, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/stop-storing-stage-accumulators
git_commit_date 1772631867 1772638451
git_commit_sha 70410da a43ff79
release_version 1.61.0-SNAPSHOT~70410da0e2 1.61.0-SNAPSHOT~a43ff794a8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1772640389 1772640389
ci_job_id 1475494340 1475494340
ci_pipeline_id 100352311 100352311
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-middy767 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-middy767 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 66 metrics, 5 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.07 s) : 0, 1070092
Total [baseline] (11.116 s) : 0, 11116255
Agent [candidate] (1.062 s) : 0, 1061823
Total [candidate] (11.137 s) : 0, 11137160
section appsec
Agent [baseline] (1.244 s) : 0, 1243631
Total [baseline] (11.083 s) : 0, 11082833
Agent [candidate] (1.251 s) : 0, 1251480
Total [candidate] (11.311 s) : 0, 11310560
section iast
Agent [baseline] (1.224 s) : 0, 1224323
Total [baseline] (11.337 s) : 0, 11336532
Agent [candidate] (1.226 s) : 0, 1226177
Total [candidate] (11.406 s) : 0, 11406111
section profiling
Agent [baseline] (1.179 s) : 0, 1178973
Total [baseline] (11.122 s) : 0, 11122103
Agent [candidate] (1.187 s) : 0, 1187389
Total [candidate] (11.059 s) : 0, 11059356
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent appsec 1.244 s 173.539 ms (16.2%)
Agent iast 1.224 s 154.231 ms (14.4%)
Agent profiling 1.179 s 108.882 ms (10.2%)
Total tracing 11.116 s -
Total appsec 11.083 s -33.422 ms (-0.3%)
Total iast 11.337 s 220.278 ms (2.0%)
Total profiling 11.122 s 5.848 ms (0.1%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.251 s 189.658 ms (17.9%)
Agent iast 1.226 s 164.354 ms (15.5%)
Agent profiling 1.187 s 125.566 ms (11.8%)
Total tracing 11.137 s -
Total appsec 11.311 s 173.4 ms (1.6%)
Total iast 11.406 s 268.951 ms (2.4%)
Total profiling 11.059 s -77.804 ms (-0.7%)
gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (632.193 ms) : 0, 632193
BytebuddyAgent [candidate] (629.988 ms) : 0, 629988
AgentMeter [baseline] (29.482 ms) : 0, 29482
AgentMeter [candidate] (29.073 ms) : 0, 29073
GlobalTracer [baseline] (259.401 ms) : 0, 259401
GlobalTracer [candidate] (257.434 ms) : 0, 257434
AppSec [baseline] (31.848 ms) : 0, 31848
AppSec [candidate] (31.447 ms) : 0, 31447
Debugger [baseline] (59.991 ms) : 0, 59991
Debugger [candidate] (59.363 ms) : 0, 59363
Remote Config [baseline] (604.726 µs) : 0, 605
Remote Config [candidate] (584.436 µs) : 0, 584
Telemetry [baseline] (8.765 ms) : 0, 8765
Telemetry [candidate] (8.629 ms) : 0, 8629
Flare Poller [baseline] (10.415 ms) : 0, 10415
Flare Poller [candidate] (8.106 ms) : 0, 8106
section appsec
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.205 ms) : 0, 1205
BytebuddyAgent [baseline] (656.343 ms) : 0, 656343
BytebuddyAgent [candidate] (661.935 ms) : 0, 661935
AgentMeter [baseline] (12.013 ms) : 0, 12013
AgentMeter [candidate] (12.063 ms) : 0, 12063
GlobalTracer [baseline] (257.741 ms) : 0, 257741
GlobalTracer [candidate] (259.006 ms) : 0, 259006
IAST [baseline] (23.919 ms) : 0, 23919
IAST [candidate] (24.069 ms) : 0, 24069
AppSec [baseline] (177.316 ms) : 0, 177316
AppSec [candidate] (177.899 ms) : 0, 177899
Debugger [baseline] (65.635 ms) : 0, 65635
Debugger [candidate] (65.646 ms) : 0, 65646
Remote Config [baseline] (567.23 µs) : 0, 567
Remote Config [candidate] (579.597 µs) : 0, 580
Telemetry [baseline] (8.938 ms) : 0, 8938
Telemetry [candidate] (9.082 ms) : 0, 9082
Flare Poller [baseline] (3.554 ms) : 0, 3554
Flare Poller [candidate] (3.648 ms) : 0, 3648
section iast
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (794.87 ms) : 0, 794870
BytebuddyAgent [candidate] (795.485 ms) : 0, 795485
AgentMeter [baseline] (11.304 ms) : 0, 11304
AgentMeter [candidate] (11.294 ms) : 0, 11294
GlobalTracer [baseline] (246.86 ms) : 0, 246860
GlobalTracer [candidate] (247.06 ms) : 0, 247060
IAST [baseline] (25.019 ms) : 0, 25019
IAST [candidate] (25.07 ms) : 0, 25070
AppSec [baseline] (26.153 ms) : 0, 26153
AppSec [candidate] (26.208 ms) : 0, 26208
Debugger [baseline] (62.813 ms) : 0, 62813
Debugger [candidate] (63.768 ms) : 0, 63768
Remote Config [baseline] (520.239 µs) : 0, 520
Remote Config [candidate] (522.959 µs) : 0, 523
Telemetry [baseline] (14.665 ms) : 0, 14665
Telemetry [candidate] (14.7 ms) : 0, 14700
Flare Poller [baseline] (4.853 ms) : 0, 4853
Flare Poller [candidate] (4.893 ms) : 0, 4893
section profiling
crashtracking [baseline] (1.171 ms) : 0, 1171
crashtracking [candidate] (1.166 ms) : 0, 1166
BytebuddyAgent [baseline] (680.515 ms) : 0, 680515
BytebuddyAgent [candidate] (685.958 ms) : 0, 685958
AgentMeter [baseline] (8.6 ms) : 0, 8600
AgentMeter [candidate] (8.617 ms) : 0, 8617
GlobalTracer [baseline] (215.01 ms) : 0, 215010
GlobalTracer [candidate] (216.261 ms) : 0, 216261
AppSec [baseline] (31.74 ms) : 0, 31740
AppSec [candidate] (32.079 ms) : 0, 32079
Debugger [baseline] (62.77 ms) : 0, 62770
Debugger [candidate] (63.797 ms) : 0, 63797
Remote Config [baseline] (570.959 µs) : 0, 571
Remote Config [candidate] (586.029 µs) : 0, 586
Telemetry [baseline] (10.554 ms) : 0, 10554
Telemetry [candidate] (9.797 ms) : 0, 9797
Flare Poller [baseline] (3.473 ms) : 0, 3473
Flare Poller [candidate] (3.51 ms) : 0, 3510
ProfilingAgent [baseline] (93.95 ms) : 0, 93950
ProfilingAgent [candidate] (94.765 ms) : 0, 94765
Profiling [baseline] (94.515 ms) : 0, 94515
Profiling [candidate] (95.335 ms) : 0, 95335
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059391
Total [baseline] (8.827 s) : 0, 8826834
Agent [candidate] (1.061 s) : 0, 1060789
Total [candidate] (8.875 s) : 0, 8875262
section iast
Agent [baseline] (1.23 s) : 0, 1229657
Total [baseline] (9.569 s) : 0, 9568743
Agent [candidate] (1.226 s) : 0, 1226441
Total [candidate] (9.58 s) : 0, 9579910
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent iast 1.23 s 170.266 ms (16.1%)
Total tracing 8.827 s -
Total iast 9.569 s 741.91 ms (8.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent iast 1.226 s 165.652 ms (15.6%)
Total tracing 8.875 s -
Total iast 9.58 s 704.648 ms (7.9%)
gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.222 ms) : 0, 1222
crashtracking [candidate] (1.201 ms) : 0, 1201
BytebuddyAgent [baseline] (628.89 ms) : 0, 628890
BytebuddyAgent [candidate] (629.752 ms) : 0, 629752
AgentMeter [baseline] (29.133 ms) : 0, 29133
AgentMeter [candidate] (29.09 ms) : 0, 29090
GlobalTracer [baseline] (256.925 ms) : 0, 256925
GlobalTracer [candidate] (257.641 ms) : 0, 257641
AppSec [baseline] (31.38 ms) : 0, 31380
AppSec [candidate] (31.613 ms) : 0, 31613
Debugger [baseline] (58.569 ms) : 0, 58569
Debugger [candidate] (58.768 ms) : 0, 58768
Remote Config [baseline] (591.097 µs) : 0, 591
Remote Config [candidate] (605.631 µs) : 0, 606
Telemetry [baseline] (8.639 ms) : 0, 8639
Telemetry [candidate] (8.713 ms) : 0, 8713
Flare Poller [baseline] (7.905 ms) : 0, 7905
Flare Poller [candidate] (7.283 ms) : 0, 7283
section iast
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.195 ms) : 0, 1195
BytebuddyAgent [baseline] (799.076 ms) : 0, 799076
BytebuddyAgent [candidate] (796.116 ms) : 0, 796116
AgentMeter [baseline] (11.305 ms) : 0, 11305
AgentMeter [candidate] (11.339 ms) : 0, 11339
GlobalTracer [baseline] (247.427 ms) : 0, 247427
GlobalTracer [candidate] (247.327 ms) : 0, 247327
AppSec [baseline] (27.058 ms) : 0, 27058
AppSec [candidate] (27.234 ms) : 0, 27234
Debugger [baseline] (62.031 ms) : 0, 62031
Debugger [candidate] (61.705 ms) : 0, 61705
Remote Config [baseline] (526.946 µs) : 0, 527
Remote Config [candidate] (524.963 µs) : 0, 525
Telemetry [baseline] (14.697 ms) : 0, 14697
Telemetry [candidate] (14.827 ms) : 0, 14827
Flare Poller [baseline] (4.891 ms) : 0, 4891
Flare Poller [candidate] (4.88 ms) : 0, 4880
IAST [baseline] (25.177 ms) : 0, 25177
IAST [candidate] (25.21 ms) : 0, 25210
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/stop-storing-stage-accumulators
git_commit_date 1772631867 1772638451
git_commit_sha 70410da a43ff79
release_version 1.61.0-SNAPSHOT~70410da0e2 1.61.0-SNAPSHOT~a43ff794a8
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1772640874 1772640874
ci_job_id 1475494343 1475494343
ci_pipeline_id 100352311 100352311
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-90869w4h 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-90869w4h 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 3 performance regressions! Performance is the same for 14 metrics, 16 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:iast_FULL:high_load better
[-413.455µs; -203.414µs] or [-7.611%; -3.744%]
better
[-965.779µs; -360.198µs] or [-7.466%; -2.785%]
unstable
[-37.163op/s; +119.475op/s] or [-4.899%; +15.751%]
5.124ms 12.273ms 799.688op/s 5.432ms 12.936ms 758.531op/s
scenario:load:petclinic:no_agent:high_load worse
[+0.354ms; +1.740ms] or [+2.143%; +10.519%]
unstable
[-0.234ms; +2.942ms] or [-0.832%; +10.468%]
unstable
[-47.945op/s; +16.820op/s] or [-17.643%; +6.189%]
17.590ms 29.462ms 256.188op/s 16.543ms 28.108ms 271.750op/s
scenario:load:petclinic:tracing:high_load worse
[+0.886ms; +1.660ms] or [+5.154%; +9.661%]
unsure
[+0.366ms; +1.826ms] or [+1.298%; +6.477%]
unstable
[-47.741op/s; +12.679op/s] or [-17.942%; +4.765%]
18.460ms 29.292ms 248.562op/s 17.187ms 28.196ms 266.094op/s
scenario:load:petclinic:profiling:high_load worse
[+378.617µs; +952.695µs] or [+2.103%; +5.292%]
unsure
[+0.496ms; +1.826ms] or [+1.685%; +6.203%]
unstable
[-37.176op/s; +20.989op/s] or [-14.685%; +8.291%]
18.668ms 30.596ms 245.062op/s 18.002ms 29.435ms 253.156op/s
scenario:load:petclinic:code_origins:high_load better
[-1029.530µs; -675.343µs] or [-5.740%; -3.765%]
unsure
[-1515.733µs; -201.575µs] or [-5.173%; -0.688%]
unstable
[-18.522op/s; +42.209op/s] or [-7.288%; +16.610%]
17.084ms 28.440ms 265.969op/s 17.936ms 29.298ms 254.125op/s
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.172 ms) : 1160, 1184
.   : milestone, 1172,
iast (3.164 ms) : 3122, 3206
.   : milestone, 3164,
iast_FULL (6.099 ms) : 6036, 6162
.   : milestone, 6099,
iast_GLOBAL (3.523 ms) : 3464, 3583
.   : milestone, 3523,
profiling (2.027 ms) : 2010, 2044
.   : milestone, 2027,
tracing (1.798 ms) : 1782, 1814
.   : milestone, 1798,
section candidate
no_agent (1.175 ms) : 1163, 1186
.   : milestone, 1175,
iast (3.224 ms) : 3180, 3267
.   : milestone, 3224,
iast_FULL (5.782 ms) : 5723, 5840
.   : milestone, 5782,
iast_GLOBAL (3.474 ms) : 3424, 3524
.   : milestone, 3474,
profiling (2.122 ms) : 2101, 2143
.   : milestone, 2122,
tracing (1.772 ms) : 1758, 1786
.   : milestone, 1772,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.172 ms [1.16 ms, 1.184 ms] -
iast 3.164 ms [3.122 ms, 3.206 ms] 1.992 ms (170.0%)
iast_FULL 6.099 ms [6.036 ms, 6.162 ms] 4.927 ms (420.4%)
iast_GLOBAL 3.523 ms [3.464 ms, 3.583 ms] 2.351 ms (200.6%)
profiling 2.027 ms [2.01 ms, 2.044 ms] 855.58 µs (73.0%)
tracing 1.798 ms [1.782 ms, 1.814 ms] 626.195 µs (53.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.175 ms [1.163 ms, 1.186 ms] -
iast 3.224 ms [3.18 ms, 3.267 ms] 2.049 ms (174.4%)
iast_FULL 5.782 ms [5.723 ms, 5.84 ms] 4.607 ms (392.2%)
iast_GLOBAL 3.474 ms [3.424 ms, 3.524 ms] 2.299 ms (195.8%)
profiling 2.122 ms [2.101 ms, 2.143 ms] 947.435 µs (80.7%)
tracing 1.772 ms [1.758 ms, 1.786 ms] 597.334 µs (50.9%)
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.168 ms) : 16995, 17340
.   : milestone, 17168,
appsec (18.67 ms) : 18484, 18856
.   : milestone, 18670,
code_origins (18.364 ms) : 18183, 18545
.   : milestone, 18364,
iast (17.841 ms) : 17661, 18020
.   : milestone, 17841,
profiling (18.431 ms) : 18248, 18614
.   : milestone, 18431,
tracing (17.533 ms) : 17360, 17706
.   : milestone, 17533,
section candidate
no_agent (18.214 ms) : 18025, 18403
.   : milestone, 18214,
appsec (19.075 ms) : 18875, 19276
.   : milestone, 19075,
code_origins (17.541 ms) : 17368, 17715
.   : milestone, 17541,
iast (17.946 ms) : 17765, 18128
.   : milestone, 17946,
profiling (19.045 ms) : 18852, 19238
.   : milestone, 19045,
tracing (18.777 ms) : 18590, 18964
.   : milestone, 18777,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.168 ms [16.995 ms, 17.34 ms] -
appsec 18.67 ms [18.484 ms, 18.856 ms] 1.502 ms (8.8%)
code_origins 18.364 ms [18.183 ms, 18.545 ms] 1.196 ms (7.0%)
iast 17.841 ms [17.661 ms, 18.02 ms] 673.174 µs (3.9%)
profiling 18.431 ms [18.248 ms, 18.614 ms] 1.263 ms (7.4%)
tracing 17.533 ms [17.36 ms, 17.706 ms] 365.781 µs (2.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.214 ms [18.025 ms, 18.403 ms] -
appsec 19.075 ms [18.875 ms, 19.276 ms] 861.131 µs (4.7%)
code_origins 17.541 ms [17.368 ms, 17.715 ms] -672.717 µs (-3.7%)
iast 17.946 ms [17.765 ms, 18.128 ms] -267.941 µs (-1.5%)
profiling 19.045 ms [18.852 ms, 19.238 ms] 830.994 µs (4.6%)
tracing 18.777 ms [18.59 ms, 18.964 ms] 562.784 µs (3.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master charles.yu/djm-0000/stop-storing-stage-accumulators
git_commit_date 1772631867 1772638451
git_commit_sha 70410da a43ff79
release_version 1.61.0-SNAPSHOT~70410da0e2 1.61.0-SNAPSHOT~a43ff794a8
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1772640569 1772640569
ci_job_id 1475494346 1475494346
ci_pipeline_id 100352311 100352311
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-3-3n820lpp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-3-3n820lpp 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
.   : milestone, 1472,
appsec (3.801 ms) : 3579, 4024
.   : milestone, 3801,
iast (2.251 ms) : 2182, 2321
.   : milestone, 2251,
iast_GLOBAL (2.294 ms) : 2225, 2364
.   : milestone, 2294,
profiling (2.074 ms) : 2020, 2129
.   : milestone, 2074,
tracing (2.068 ms) : 2014, 2123
.   : milestone, 2068,
section candidate
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (3.805 ms) : 3583, 4028
.   : milestone, 3805,
iast (2.249 ms) : 2180, 2318
.   : milestone, 2249,
iast_GLOBAL (2.295 ms) : 2225, 2364
.   : milestone, 2295,
profiling (2.089 ms) : 2034, 2145
.   : milestone, 2089,
tracing (2.061 ms) : 2007, 2114
.   : milestone, 2061,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.46 ms, 1.483 ms] -
appsec 3.801 ms [3.579 ms, 4.024 ms] 2.33 ms (158.3%)
iast 2.251 ms [2.182 ms, 2.321 ms] 779.775 µs (53.0%)
iast_GLOBAL 2.294 ms [2.225 ms, 2.364 ms] 822.789 µs (55.9%)
profiling 2.074 ms [2.02 ms, 2.129 ms] 602.914 µs (41.0%)
tracing 2.068 ms [2.014 ms, 2.123 ms] 596.702 µs (40.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 3.805 ms [3.583 ms, 4.028 ms] 2.333 ms (158.5%)
iast 2.249 ms [2.18 ms, 2.318 ms] 776.571 µs (52.8%)
iast_GLOBAL 2.295 ms [2.225 ms, 2.364 ms] 822.627 µs (55.9%)
profiling 2.089 ms [2.034 ms, 2.145 ms] 617.143 µs (41.9%)
tracing 2.061 ms [2.007 ms, 2.114 ms] 588.356 µs (40.0%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~a43ff794a8, baseline=1.61.0-SNAPSHOT~70410da0e2
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.623 s) : 14623000, 14623000
.   : milestone, 14623000,
appsec (15.183 s) : 15183000, 15183000
.   : milestone, 15183000,
iast (17.684 s) : 17684000, 17684000
.   : milestone, 17684000,
iast_GLOBAL (17.615 s) : 17615000, 17615000
.   : milestone, 17615000,
profiling (14.929 s) : 14929000, 14929000
.   : milestone, 14929000,
tracing (15.265 s) : 15265000, 15265000
.   : milestone, 15265000,
section candidate
no_agent (15.015 s) : 15015000, 15015000
.   : milestone, 15015000,
appsec (15.04 s) : 15040000, 15040000
.   : milestone, 15040000,
iast (17.95 s) : 17950000, 17950000
.   : milestone, 17950000,
iast_GLOBAL (17.616 s) : 17616000, 17616000
.   : milestone, 17616000,
profiling (15.669 s) : 15669000, 15669000
.   : milestone, 15669000,
tracing (15.174 s) : 15174000, 15174000
.   : milestone, 15174000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.623 s [14.623 s, 14.623 s] -
appsec 15.183 s [15.183 s, 15.183 s] 560.0 ms (3.8%)
iast 17.684 s [17.684 s, 17.684 s] 3.061 s (20.9%)
iast_GLOBAL 17.615 s [17.615 s, 17.615 s] 2.992 s (20.5%)
profiling 14.929 s [14.929 s, 14.929 s] 306.0 ms (2.1%)
tracing 15.265 s [15.265 s, 15.265 s] 642.0 ms (4.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.015 s [15.015 s, 15.015 s] -
appsec 15.04 s [15.04 s, 15.04 s] 25.0 ms (0.2%)
iast 17.95 s [17.95 s, 17.95 s] 2.935 s (19.5%)
iast_GLOBAL 17.616 s [17.616 s, 17.616 s] 2.601 s (17.3%)
profiling 15.669 s [15.669 s, 15.669 s] 654.0 ms (4.4%)
tracing 15.174 s [15.174 s, 15.174 s] 159.0 ms (1.1%)

@charlesmyu charlesmyu added the type: enhancement Enhancements and improvements label Feb 20, 2026
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from e413d1d to 89df516 Compare February 23, 2026 21:52
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/stop-storing-stage-accumulators branch from 9594586 to cb6648d Compare February 23, 2026 21:52
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from 89df516 to 8651527 Compare February 24, 2026 20:06
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/stop-storing-stage-accumulators branch from cb6648d to 780cd6f Compare February 24, 2026 20:06
Copy link
Contributor Author

charlesmyu commented Feb 24, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/stop-storing-stage-accumulators branch from 780cd6f to a43ff79 Compare March 4, 2026 15:37
@charlesmyu charlesmyu force-pushed the charles.yu/djm-0000/fix-spark-plan-metrics branch from 8651527 to 7e4b7de Compare March 4, 2026 15:37
@charlesmyu charlesmyu marked this pull request as ready for review March 4, 2026 15:49
@charlesmyu charlesmyu requested a review from a team as a code owner March 4, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: apache spark Apache Spark instrumentation type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant