[Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes #7833

dudikeleti · 2025-11-17T15:52:29Z

Summary of changes

Added support to load log/metric/span/span‑decoration probes from a JSON file (DebuggerSettings.ProbeFile)

Reason for change

Dynamic Instrumentation only worked with RCM‑delivered probes. There was no support for loading probes from a local file, and the internal configuration state wasn’t designed for combining file and RCM probes or handling file‑only

Implementation details

Added support to load log/metric/span/span‑decoration probes from a JSON file (DebuggerSettings.ProbeFile) during DynamicInstrumentation initialization.
Ensured Dynamic Instrumentation can operate in three modes: RCM‑only, file‑only, and combined file+RCM.
Implemented source‑level merge (MergeProbes) where RCM wins on ID conflicts with file probes, and configuration‑level union (MergeConfigurations).
Updated initialization to start background processing and status/snapshot upload when either RCM is available or file probes exist.

Test coverage

Updated DynamicInstrumentationTests to cover file probe loading (multiple probe types, invalid/missing/empty files, no file configured, partially valid file entries, and duplicate IDs within a file) and merge semantics between file and RCM probes.

dd-trace-dotnet-ci-bot · 2025-11-17T17:06:06Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7833) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	75.31 ± (75.34 - 76.11) ms	75.31 ± (75.19 - 75.87) ms	+0.0%	✅⬆️
.NET Framework 4.8 - Bailout
duration	81.00 ± (81.11 - 82.07) ms	80.29 ± (80.12 - 80.72) ms	-0.9%	✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1116.47 ± (1118.26 - 1127.95) ms	1116.83 ± (1116.29 - 1124.84) ms	+0.0%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	23.24 ± (23.14 - 23.33) ms	23.04 ± (22.96 - 23.13) ms	-0.8%	✅
process.time_to_main_ms	88.83 ± (88.44 - 89.22) ms	87.79 ± (87.47 - 88.11) ms	-1.2%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.91 ± (10.90 - 10.91) MB	10.91 ± (10.91 - 10.91) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.99 ± (22.91 - 23.07) ms	22.83 ± (22.76 - 22.90) ms	-0.7%	✅
process.time_to_main_ms	89.88 ± (89.45 - 90.31) ms	89.23 ± (88.79 - 89.67) ms	-0.7%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.96 ± (10.96 - 10.96) MB	10.96 ± (10.95 - 10.96) MB	-0.0%	✅
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	219.73 ± (218.39 - 221.06) ms	221.91 ± (220.25 - 223.57) ms	+1.0%	✅⬆️
process.time_to_main_ms	551.35 ± (550.12 - 552.57) ms	547.16 ± (545.97 - 548.35) ms	-0.8%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	52.55 ± (52.53 - 52.57) MB	52.56 ± (52.54 - 52.58) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	-0.1%	✅
.NET 6 - Baseline
process.internal_duration_ms	21.86 ± (21.78 - 21.94) ms	21.96 ± (21.88 - 22.04) ms	+0.4%	✅⬆️
process.time_to_main_ms	76.93 ± (76.56 - 77.29) ms	77.05 ± (76.67 - 77.44) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.58 ± (10.58 - 10.59) MB	10.62 ± (10.62 - 10.63) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	21.82 ± (21.75 - 21.90) ms	21.78 ± (21.71 - 21.85) ms	-0.2%	✅
process.time_to_main_ms	77.81 ± (77.36 - 78.25) ms	77.57 ± (77.24 - 77.90) ms	-0.3%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.66 ± (10.65 - 10.66) MB	10.67 ± (10.66 - 10.67) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	208.76 ± (206.81 - 210.70) ms	207.44 ± (206.18 - 208.71) ms	-0.6%	✅
process.time_to_main_ms	519.84 ± (518.66 - 521.02) ms	509.26 ± (508.14 - 510.39) ms	-2.0%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	51.73 ± (51.70 - 51.75) MB	51.63 ± (51.59 - 51.66) MB	-0.2%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	-0.2%	✅
.NET 8 - Baseline
process.internal_duration_ms	20.05 ± (19.97 - 20.13) ms	19.84 ± (19.77 - 19.91) ms	-1.0%	✅
process.time_to_main_ms	76.09 ± (75.70 - 76.47) ms	75.25 ± (74.93 - 75.57) ms	-1.1%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.65 ± (7.64 - 7.65) MB	7.63 ± (7.63 - 7.64) MB	-0.1%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	20.06 ± (20.00 - 20.13) ms	19.85 ± (19.79 - 19.91) ms	-1.1%	✅
process.time_to_main_ms	77.97 ± (77.60 - 78.33) ms	76.70 ± (76.39 - 77.01) ms	-1.6%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.70 ± (7.69 - 7.71) MB	7.73 ± (7.72 - 7.74) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	191.61 ± (190.71 - 192.52) ms	191.47 ± (190.49 - 192.44) ms	-0.1%	✅
process.time_to_main_ms	494.34 ± (493.33 - 495.35) ms	485.92 ± (484.76 - 487.07) ms	-1.7%	✅
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	39.04 ± (39.00 - 39.08) MB	38.95 ± (38.90 - 38.99) MB	-0.3%	✅
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	-0.0%	✅

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	195.24 ± (194.98 - 195.76) ms	192.81 ± (192.77 - 193.62) ms	-1.2%	✅
.NET Framework 4.8 - Bailout
duration	198.91 ± (198.71 - 199.81) ms	196.07 ± (195.93 - 196.70) ms	-1.4%	✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1169.86 ± (1173.47 - 1182.82) ms	1167.69 ± (1173.94 - 1183.55) ms	-0.2%	✅
.NET Core 3.1 - Baseline
process.internal_duration_ms	194.03 ± (193.43 - 194.64) ms	188.64 ± (188.31 - 188.98) ms	-2.8%	✅
process.time_to_main_ms	82.89 ± (82.58 - 83.20) ms	80.97 ± (80.74 - 81.21) ms	-2.3%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	15.99 ± (15.97 - 16.00) MB	16.15 ± (16.12 - 16.17) MB	+1.0%	✅⬆️
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (19 - 20)	-0.1%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	189.67 ± (189.28 - 190.06) ms	187.38 ± (186.99 - 187.76) ms	-1.2%	✅
process.time_to_main_ms	82.26 ± (82.10 - 82.41) ms	81.86 ± (81.70 - 82.02) ms	-0.5%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.07 ± (16.04 - 16.11) MB	16.17 ± (16.14 - 16.20) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	21 ± (21 - 21)	21 ± (21 - 21)	+0.0%	✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	396.14 ± (393.79 - 398.49) ms	397.10 ± (394.42 - 399.79) ms	+0.2%	✅⬆️
process.time_to_main_ms	521.87 ± (520.98 - 522.75) ms	517.34 ± (516.76 - 517.92) ms	-0.9%	✅
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	63.06 ± (62.92 - 63.21) MB	62.79 ± (62.63 - 62.95) MB	-0.4%	✅
runtime.dotnet.threads.count	29 ± (29 - 30)	29 ± (29 - 29)	-0.2%	✅
.NET 6 - Baseline
process.internal_duration_ms	193.24 ± (192.88 - 193.60) ms	192.04 ± (191.71 - 192.37) ms	-0.6%	✅
process.time_to_main_ms	70.31 ± (70.10 - 70.51) ms	70.04 ± (69.84 - 70.24) ms	-0.4%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.23 ± (16.12 - 16.34) MB	16.33 ± (16.24 - 16.41) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (18 - 19)	-0.5%	✅
.NET 6 - Bailout
process.internal_duration_ms	195.25 ± (194.67 - 195.83) ms	191.78 ± (191.45 - 192.11) ms	-1.8%	✅
process.time_to_main_ms	72.09 ± (71.88 - 72.31) ms	71.11 ± (70.94 - 71.29) ms	-1.4%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.38 ± (16.36 - 16.40) MB	16.27 ± (16.16 - 16.39) MB	-0.7%	✅
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (20 - 20)	-2.4%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	414.55 ± (411.90 - 417.21) ms	411.46 ± (408.20 - 414.71) ms	-0.7%	✅
process.time_to_main_ms	493.81 ± (492.80 - 494.82) ms	484.25 ± (483.59 - 484.90) ms	-1.9%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	62.41 ± (62.27 - 62.54) MB	62.03 ± (61.88 - 62.18) MB	-0.6%	✅
runtime.dotnet.threads.count	30 ± (29 - 30)	30 ± (29 - 30)	-0.0%	✅
.NET 8 - Baseline
process.internal_duration_ms	191.72 ± (191.35 - 192.08) ms	191.18 ± (190.78 - 191.58) ms	-0.3%	✅
process.time_to_main_ms	70.28 ± (70.01 - 70.55) ms	69.80 ± (69.60 - 70.01) ms	-0.7%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.74 ± (11.71 - 11.76) MB	11.70 ± (11.67 - 11.73) MB	-0.3%	✅
runtime.dotnet.threads.count	18 ± (18 - 18)	18 ± (18 - 18)	-0.2%	✅
.NET 8 - Bailout
process.internal_duration_ms	193.10 ± (192.66 - 193.53) ms	189.89 ± (189.62 - 190.16) ms	-1.7%	✅
process.time_to_main_ms	71.20 ± (71.04 - 71.36) ms	70.47 ± (70.35 - 70.59) ms	-1.0%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.80 ± (11.77 - 11.82) MB	11.81 ± (11.78 - 11.83) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	-0.8%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	363.93 ± (362.47 - 365.39) ms	365.59 ± (364.13 - 367.04) ms	+0.5%	✅⬆️
process.time_to_main_ms	467.47 ± (466.73 - 468.22) ms	463.00 ± (462.23 - 463.76) ms	-1.0%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	50.44 ± (50.41 - 50.48) MB	50.35 ± (50.32 - 50.38) MB	-0.2%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.1%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (76ms)  : 71, 81
    master - mean (76ms)  : 70, 81

    section Bailout
    This PR (7833) - mean (80ms)  : 76, 85
    master - mean (82ms)  : 76, 87

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (1,121ms)  : 1058, 1183
    master - mean (1,123ms)  : 1054, 1192

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (118ms)  : 111, 125
    master - mean (120ms)  : 112, 128

    section Bailout
    This PR (7833) - mean (120ms)  : 112, 128
    master - mean (120ms)  : 112, 128

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (803ms)  : 760, 847
    master - mean (805ms)  : 765, 846

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (106ms)  : 99, 113
    master - mean (106ms)  : 99, 112

    section Bailout
    This PR (7833) - mean (106ms)  : 99, 113
    master - mean (107ms)  : 98, 116

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (749ms)  : 714, 784
    master - mean (764ms)  : 718, 811

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (103ms)  : 96, 111
    master - mean (105ms)  : 97, 113

    section Bailout
    This PR (7833) - mean (105ms)  : 99, 111
    master - mean (107ms)  : 99, 115

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (709ms)  : 681, 736
    master - mean (718ms)  : 687, 749

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (193ms)  : 189, 197
    master - mean (195ms)  : 191, 200

    section Bailout
    This PR (7833) - mean (196ms)  : 192, 200
    master - mean (199ms)  : 193, 205

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (1,179ms)  : 1108, 1249
    master - mean (1,178ms)  : 1109, 1248

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (278ms)  : 273, 283
    master - mean (285ms)  : 272, 299

    section Bailout
    This PR (7833) - mean (277ms)  : 273, 282
    master - mean (280ms)  : 275, 286

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (946ms)  : 905, 987
    master - mean (956ms)  : 896, 1015

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (271ms)  : 264, 277
    master - mean (272ms)  : 266, 277

    section Bailout
    This PR (7833) - mean (271ms)  : 266, 276
    master - mean (277ms)  : 262, 292

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (928ms)  : 877, 978
    master - mean (943ms)  : 886, 999

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7833) - mean (271ms)  : 266, 276
    master - mean (272ms)  : 266, 278

    section Bailout
    This PR (7833) - mean (270ms)  : 267, 273
    master - mean (274ms)  : 266, 283

    section CallTarget+Inlining+NGEN
    This PR (7833) - mean (859ms)  : 833, 884
    master - mean (863ms)  : 837, 889

Support file-based debugger probes and merge with RCM

1004977

dudikeleti requested a review from a team as a code owner November 17, 2025 15:52

dudikeleti changed the title ~~Support file-based debugger probes and merge with RCM~~ [Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes Nov 17, 2025

This comment has been minimized.

Sign in to view

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes #7833

[Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes #7833

Uh oh!

dudikeleti commented Nov 17, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Nov 17, 2025

FakeDbCommand

HttpMessageHandler

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

[Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes #7833

Are you sure you want to change the base?

[Dynamic Instrumentation] DEBUG-4089 Support file-based debugger probes #7833

Uh oh!

Conversation

dudikeleti commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Nov 17, 2025

Execution-Time Benchmarks Report ⏱️

FakeDbCommand

HttpMessageHandler

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dudikeleti commented Nov 17, 2025 •

edited

Loading