Removing Wrong Spark Spans for Inactive Databricks Clusters#10651
Open
larakulkarni1 wants to merge 10 commits intomasterfrom
Open
Removing Wrong Spark Spans for Inactive Databricks Clusters#10651larakulkarni1 wants to merge 10 commits intomasterfrom
larakulkarni1 wants to merge 10 commits intomasterfrom
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 63 metrics, 8 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.068 s) : 0, 1068268
Total [baseline] (8.742 s) : 0, 8742119
Agent [candidate] (1.064 s) : 0, 1063760
Total [candidate] (8.725 s) : 0, 8725119
section iast
Agent [baseline] (1.241 s) : 0, 1240905
Total [baseline] (9.389 s) : 0, 9388949
Agent [candidate] (1.229 s) : 0, 1229366
Total [candidate] (9.343 s) : 0, 9343460
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.205 ms) : 0, 1205
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (629.63 ms) : 0, 629630
BytebuddyAgent [candidate] (627.189 ms) : 0, 627189
AgentMeter [baseline] (29.161 ms) : 0, 29161
AgentMeter [candidate] (29.017 ms) : 0, 29017
GlobalTracer [baseline] (258.432 ms) : 0, 258432
GlobalTracer [candidate] (257.397 ms) : 0, 257397
AppSec [baseline] (33.136 ms) : 0, 33136
AppSec [candidate] (32.867 ms) : 0, 32867
Debugger [baseline] (63.082 ms) : 0, 63082
Debugger [candidate] (65.013 ms) : 0, 65013
Remote Config [baseline] (622.576 µs) : 0, 623
Remote Config [candidate] (609.907 µs) : 0, 610
Telemetry [baseline] (12.308 ms) : 0, 12308
Telemetry [candidate] (9.804 ms) : 0, 9804
Flare Poller [baseline] (4.553 ms) : 0, 4553
Flare Poller [candidate] (4.554 ms) : 0, 4554
section iast
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (802.869 ms) : 0, 802869
BytebuddyAgent [candidate] (795.335 ms) : 0, 795335
AgentMeter [baseline] (11.577 ms) : 0, 11577
AgentMeter [candidate] (11.277 ms) : 0, 11277
GlobalTracer [baseline] (248.859 ms) : 0, 248859
GlobalTracer [candidate] (246.871 ms) : 0, 246871
AppSec [baseline] (35.384 ms) : 0, 35384
AppSec [candidate] (33.903 ms) : 0, 33903
Debugger [baseline] (64.946 ms) : 0, 64946
Debugger [candidate] (65.417 ms) : 0, 65417
Remote Config [baseline] (540.767 µs) : 0, 541
Remote Config [candidate] (536.418 µs) : 0, 536
Telemetry [baseline] (8.634 ms) : 0, 8634
Telemetry [candidate] (8.612 ms) : 0, 8612
Flare Poller [baseline] (3.484 ms) : 0, 3484
Flare Poller [candidate] (3.485 ms) : 0, 3485
IAST [baseline] (27.326 ms) : 0, 27326
IAST [candidate] (26.937 ms) : 0, 26937
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067205
Total [baseline] (10.865 s) : 0, 10865314
Agent [candidate] (1.069 s) : 0, 1068654
Total [candidate] (10.792 s) : 0, 10792031
section appsec
Agent [baseline] (1.243 s) : 0, 1243124
Total [baseline] (11.085 s) : 0, 11084929
Agent [candidate] (1.236 s) : 0, 1236018
Total [candidate] (10.963 s) : 0, 10962738
section iast
Agent [baseline] (1.228 s) : 0, 1228059
Total [baseline] (11.197 s) : 0, 11197416
Agent [candidate] (1.23 s) : 0, 1229523
Total [candidate] (11.224 s) : 0, 11223814
section profiling
Agent [baseline] (1.198 s) : 0, 1197521
Total [baseline] (10.889 s) : 0, 10888522
Agent [candidate] (1.196 s) : 0, 1196336
Total [candidate] (10.946 s) : 0, 10945821
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.192 ms) : 0, 1192
crashtracking [candidate] (1.19 ms) : 0, 1190
BytebuddyAgent [baseline] (628.955 ms) : 0, 628955
BytebuddyAgent [candidate] (631.307 ms) : 0, 631307
AgentMeter [baseline] (29.276 ms) : 0, 29276
AgentMeter [candidate] (29.195 ms) : 0, 29195
GlobalTracer [baseline] (257.7 ms) : 0, 257700
GlobalTracer [candidate] (258.49 ms) : 0, 258490
AppSec [baseline] (33.137 ms) : 0, 33137
AppSec [candidate] (33.24 ms) : 0, 33240
Debugger [baseline] (65.076 ms) : 0, 65076
Debugger [candidate] (63.279 ms) : 0, 63279
Remote Config [baseline] (610.336 µs) : 0, 610
Remote Config [candidate] (620.14 µs) : 0, 620
Telemetry [baseline] (10.645 ms) : 0, 10645
Telemetry [candidate] (9.91 ms) : 0, 9910
Flare Poller [baseline] (4.576 ms) : 0, 4576
Flare Poller [candidate] (5.36 ms) : 0, 5360
section appsec
crashtracking [baseline] (1.199 ms) : 0, 1199
crashtracking [candidate] (1.204 ms) : 0, 1204
BytebuddyAgent [baseline] (659.816 ms) : 0, 659816
BytebuddyAgent [candidate] (655.428 ms) : 0, 655428
AgentMeter [baseline] (12.03 ms) : 0, 12030
AgentMeter [candidate] (11.976 ms) : 0, 11976
GlobalTracer [baseline] (258.852 ms) : 0, 258852
GlobalTracer [candidate] (257.603 ms) : 0, 257603
IAST [baseline] (25.617 ms) : 0, 25617
IAST [candidate] (25.241 ms) : 0, 25241
AppSec [baseline] (168.147 ms) : 0, 168147
AppSec [candidate] (167.462 ms) : 0, 167462
Debugger [baseline] (67.264 ms) : 0, 67264
Debugger [candidate] (67.259 ms) : 0, 67259
Remote Config [baseline] (708.128 µs) : 0, 708
Remote Config [candidate] (722.389 µs) : 0, 722
Telemetry [baseline] (9.61 ms) : 0, 9610
Telemetry [candidate] (9.487 ms) : 0, 9487
Flare Poller [baseline] (3.729 ms) : 0, 3729
Flare Poller [candidate] (3.716 ms) : 0, 3716
section iast
crashtracking [baseline] (1.18 ms) : 0, 1180
crashtracking [candidate] (1.192 ms) : 0, 1192
BytebuddyAgent [baseline] (793.41 ms) : 0, 793410
BytebuddyAgent [candidate] (793.704 ms) : 0, 793704
AgentMeter [baseline] (11.276 ms) : 0, 11276
AgentMeter [candidate] (11.326 ms) : 0, 11326
GlobalTracer [baseline] (246.613 ms) : 0, 246613
GlobalTracer [candidate] (246.728 ms) : 0, 246728
IAST [baseline] (26.984 ms) : 0, 26984
IAST [candidate] (27.18 ms) : 0, 27180
AppSec [baseline] (34.766 ms) : 0, 34766
AppSec [candidate] (33.751 ms) : 0, 33751
Debugger [baseline] (65.245 ms) : 0, 65245
Debugger [candidate] (66.856 ms) : 0, 66856
Remote Config [baseline] (527.064 µs) : 0, 527
Remote Config [candidate] (534.11 µs) : 0, 534
Telemetry [baseline] (8.649 ms) : 0, 8649
Telemetry [candidate] (8.72 ms) : 0, 8720
Flare Poller [baseline] (3.512 ms) : 0, 3512
Flare Poller [candidate] (3.535 ms) : 0, 3535
section profiling
crashtracking [baseline] (1.169 ms) : 0, 1169
crashtracking [candidate] (1.175 ms) : 0, 1175
BytebuddyAgent [baseline] (686.395 ms) : 0, 686395
BytebuddyAgent [candidate] (686.16 ms) : 0, 686160
AgentMeter [baseline] (8.641 ms) : 0, 8641
AgentMeter [candidate] (8.64 ms) : 0, 8640
GlobalTracer [baseline] (216.895 ms) : 0, 216895
GlobalTracer [candidate] (217.156 ms) : 0, 217156
AppSec [baseline] (32.785 ms) : 0, 32785
AppSec [candidate] (32.891 ms) : 0, 32891
Debugger [baseline] (67.605 ms) : 0, 67605
Debugger [candidate] (67.124 ms) : 0, 67124
Remote Config [baseline] (631.226 µs) : 0, 631
Remote Config [candidate] (633.648 µs) : 0, 634
Telemetry [baseline] (8.914 ms) : 0, 8914
Telemetry [candidate] (8.869 ms) : 0, 8869
Flare Poller [baseline] (3.81 ms) : 0, 3810
Flare Poller [candidate] (3.725 ms) : 0, 3725
ProfilingAgent [baseline] (99.576 ms) : 0, 99576
ProfilingAgent [candidate] (98.951 ms) : 0, 98951
Profiling [baseline] (100.158 ms) : 0, 100158
Profiling [candidate] (99.517 ms) : 0, 99517
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section baseline
no_agent (17.33 ms) : 17157, 17504
. : milestone, 17330,
appsec (18.676 ms) : 18490, 18862
. : milestone, 18676,
code_origins (18.504 ms) : 18313, 18694
. : milestone, 18504,
iast (17.686 ms) : 17513, 17860
. : milestone, 17686,
profiling (18.774 ms) : 18586, 18962
. : milestone, 18774,
tracing (17.996 ms) : 17818, 18174
. : milestone, 17996,
section candidate
no_agent (18.212 ms) : 18025, 18399
. : milestone, 18212,
appsec (18.857 ms) : 18667, 19047
. : milestone, 18857,
code_origins (17.563 ms) : 17391, 17736
. : milestone, 17563,
iast (17.644 ms) : 17468, 17821
. : milestone, 17644,
profiling (18.749 ms) : 18563, 18936
. : milestone, 18749,
tracing (17.905 ms) : 17725, 18086
. : milestone, 17905,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.191 ms) : 1179, 1202
. : milestone, 1191,
iast (3.157 ms) : 3116, 3199
. : milestone, 3157,
iast_FULL (5.896 ms) : 5837, 5956
. : milestone, 5896,
iast_GLOBAL (3.495 ms) : 3444, 3547
. : milestone, 3495,
profiling (2.323 ms) : 2301, 2345
. : milestone, 2323,
tracing (1.898 ms) : 1881, 1914
. : milestone, 1898,
section candidate
no_agent (1.267 ms) : 1254, 1281
. : milestone, 1267,
iast (3.151 ms) : 3110, 3192
. : milestone, 3151,
iast_FULL (5.89 ms) : 5831, 5949
. : milestone, 5890,
iast_GLOBAL (3.411 ms) : 3361, 3460
. : milestone, 3411,
profiling (2.139 ms) : 2121, 2157
. : milestone, 2139,
tracing (1.817 ms) : 1800, 1833
. : milestone, 1817,
DacapoParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 0 unstable metrics.
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section baseline
no_agent (1.467 ms) : 1455, 1478
. : milestone, 1467,
appsec (3.691 ms) : 3475, 3908
. : milestone, 3691,
iast (2.25 ms) : 2181, 2319
. : milestone, 2250,
iast_GLOBAL (2.288 ms) : 2219, 2357
. : milestone, 2288,
profiling (2.094 ms) : 2038, 2151
. : milestone, 2094,
tracing (2.062 ms) : 2008, 2115
. : milestone, 2062,
section candidate
no_agent (1.472 ms) : 1460, 1483
. : milestone, 1472,
appsec (2.564 ms) : 2506, 2622
. : milestone, 2564,
iast (2.243 ms) : 2174, 2312
. : milestone, 2243,
iast_GLOBAL (2.288 ms) : 2219, 2357
. : milestone, 2288,
profiling (2.071 ms) : 2017, 2125
. : milestone, 2071,
tracing (2.054 ms) : 2001, 2108
. : milestone, 2054,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~2efadac8aa, baseline=1.60.0-SNAPSHOT~1c44c6f9a7
dateFormat X
axisFormat %s
section baseline
no_agent (15.466 s) : 15466000, 15466000
. : milestone, 15466000,
appsec (14.726 s) : 14726000, 14726000
. : milestone, 14726000,
iast (18.25 s) : 18250000, 18250000
. : milestone, 18250000,
iast_GLOBAL (17.751 s) : 17751000, 17751000
. : milestone, 17751000,
profiling (14.888 s) : 14888000, 14888000
. : milestone, 14888000,
tracing (14.875 s) : 14875000, 14875000
. : milestone, 14875000,
section candidate
no_agent (14.98 s) : 14980000, 14980000
. : milestone, 14980000,
appsec (15.275 s) : 15275000, 15275000
. : milestone, 15275000,
iast (17.931 s) : 17931000, 17931000
. : milestone, 17931000,
iast_GLOBAL (17.779 s) : 17779000, 17779000
. : milestone, 17779000,
profiling (14.606 s) : 14606000, 14606000
. : milestone, 14606000,
tracing (14.854 s) : 14854000, 14854000
. : milestone, 14854000,
|
8bf15a0 to
fb1450e
Compare
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
This PR focuses on removing phantom spans for Databricks Clusters. Currently, DBX Clusters can show up as Spark Jobs on the Batch Jobs Table due to the SparkListener incorrectly emitting a spark.application span for the cluster. These should not be appearing on the table. More information available on this doc
Motivation
Incorrect information on the Batch Jobs Table (customers affected).
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: DJM-1120
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.