Debug bench regressions #3

MahdiBM · 2025-10-21T17:17:49Z

No description provided.

Run count: '500' Target: '' Filter: '' Skip: '' Committed to auto-thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18426511609

Updated performance benchmarks and formatting in README.

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18441387668

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18468203008

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18470584262

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18471323111

github-actions · 2025-10-21T17:19:52Z

Benchmark Report for aa33444

❌ Benchmark results are outside the specified thresholds 📊

Pushed commit c69cc9a to branch thr-update/mmbm-debug-benchs containing an update of current thresholds with the deviated baseline.

Click to expand comparison result

Benchmark check running at 2025-10-25 11:27:03 UTC

===========================================================================================================
Deviations worse than threshold for DomainNameBenchs:app-analytics-services_dot_com_String_Parsing_CPU_200K
===========================================================================================================

Time (user CPU) (ms, ↔)	p90 threshold	Current-PR	Difference Δ
p90	20 ... 30	30	10

The baseline 'Current-PR' is WORSE than the defined thresholds.

Click to expand benchmark result

Baseline 'Current-PR'

Host '1c2e5c9fdc0d' with 2 'x86_64' processors with 7 GB memory, running:
#71-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 22 16:52:38 UTC 2025

DomainNameBenchs

Equality_Check_CPU_20M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	260	260	260	260	270	270	270	20

Equality_Check_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

app-analytics-services_dot_com_String_Parsing_CPU_200K

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	20	20	20	30	30	30	40	205

app-analytics-services_dot_com_String_Parsing_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	2	2	2	2	2	2	2	10

google_dot_com_String_Parsing_CPU_200K

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	10	20	20	20	20	20	30	254

google_dot_com_String_Parsing_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	2	2	2	2	2	2	2	10

IPAddressBenchs

111_Machine_Warmup_Benchmark

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (μs) *	0	0	0	0	0	10000	10000	31240

IPv4_CIDR_Create_Then_Check_Is_Loopback_100M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	110	110	110	110	120	120	120	45

IPv4_CIDR_Create_Then_Check_Is_Multicast_100M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	110	110	110	110	120	120	120	45

IPv4_CIDR_Create_Then_Check_Is_Multicast_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv4_String_Decoding_Local_Broadcast_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	210	210	220	220	220	220	220	24

IPv4_String_Decoding_Local_Broadcast_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv4_String_Decoding_Local_Broadcast_inet_pton_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	220	230	230	230	240	240	240	22

IPv4_String_Decoding_Local_Broadcast_inet_pton_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv4_String_Decoding_Localhost_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	160	160	160	160	170	170	32

IPv4_String_Decoding_Zero_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	140	140	140	150	150	170	170	35

IPv4_String_Encoding_Local_Broadcast_15M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	180	190	190	190	190	200	200	27

IPv4_String_Encoding_Localhost_15M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	170	170	170	180	180	180	180	29

IPv4_String_Encoding_Mixed_15M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	170	170	180	180	180	180	180	29

IPv4_String_Encoding_Mixed_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv4_String_Encoding_Mixed_inet_ntop_15M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	1510	1511	1520	1530	1530	1530	1530	4

IPv4_String_Encoding_Mixed_inet_ntop_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv4_String_Encoding_Zero_15M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	170	170	170	180	180	180	180	29

IPv6_CIDR_Create_Then_Check_Is_Loopback_100M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	110	110	110	110	110	120	120	45

IPv6_CIDR_Create_Then_Check_Is_Multicast_100M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	110	110	110	110	120	120	120	45

IPv6_CIDR_Create_Then_Check_Is_Multicast_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv6_String_Decoding_2_Groups_Compressed_At_The_Begining_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	140	140	150	150	150	160	160	34

IPv6_String_Decoding_2_Groups_Compressed_At_The_End_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	140	140	150	150	150	160	160	34

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	140	140	140	150	150	150	150	35

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_inet_pton_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	90	90	100	100	100	100	100	52

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_inet_pton_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	0	0	0	0	0	0	0	10

IPv6_String_Decoding_Localhost_Compressed_5M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	160	160	160	160	180	180	32

IPv6_String_Decoding_Uncompressed_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	150	150	160	160	160	160	33

IPv6_String_Decoding_Zero_Compressed_5M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	150	150	160	160	160	160	33

IPv6_String_Decoding_Zero_Uncompressed_3M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	150	150	160	160	160	160	33

IPv6_String_Encoding_Localhost_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	150	150	150	160	160	160	160	33

IPv6_String_Encoding_Max_4M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	200	200	210	210	210	220	220	25

IPv6_String_Encoding_Mixed_4M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	180	180	190	190	190	190	190	27

IPv6_String_Encoding_Mixed_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	1	1	1	1	1	1	1	10

IPv6_String_Encoding_Mixed_inet_ntop_4M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	810	810	810	821	830	830	830	7

IPv6_String_Encoding_Mixed_inet_ntop_Malloc

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Malloc (total) *	1	1	1	1	1	1	1	10

IPv6_String_Encoding_Zero_10M

Metric	p0	p25	p50	p75	p90	p99	p100	Samples
Time (user CPU) (ms) *	190	190	190	200	200	200	200	26

MahdiBM and others added 18 commits October 6, 2025 18:27

Update benchmark thresholds + minor refinements (#1)

09fa7ab

MISC minor refinements

e4bfc8c

README refinements and sync with code changes

bb5b83d

More minor refinements

e737c87

Some updates and refinements

7601b05

Lower down all availability reqs to macOS 15 + many more refinements

357486f

Update of benchmark thresholds with the new baselines

8000444

Run count: '500' Target: '' Filter: '' Skip: '' Committed to auto-thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18426511609

Merge remote-tracking branch 'origin/auto-thr-update/main'

9279dca

Improve README

b41098e

Updated performance benchmarks and formatting in README.

Safer memory bindings

25c7a1d

Update of benchmark thresholds with the deviated baseline

b984644

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18441387668

More accurate domain name implementation

169e4ef

General refinements

86f0fda

Update of benchmark thresholds with the deviated baseline

7869c83

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18468203008

Fix IPv4 -> DomainName conversion + other minor refinements

109240e

Update of benchmark thresholds with the deviated baseline

c5f0269

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18470584262

Update of benchmark thresholds with the deviated baseline

91e5fa9

Committed to thr-update/main by https://github.com/swift-dns/swift-endpoint/actions/runs/18471323111

Merge branch 'mmbm-benchmark-new-string-parsing' into mmbm-debug-benchs

aa33444

Debug bench regressions #3

Are you sure you want to change the base?

Debug bench regressions #3

Uh oh!

Conversation

MahdiBM commented Oct 21, 2025

Uh oh!

github-actions bot commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmark Report for aa33444

Benchmark check running at 2025-10-25 11:27:03 UTC

Baseline 'Current-PR'

DomainNameBenchs

Equality_Check_CPU_20M

Equality_Check_Malloc

app-analytics-services_dot_com_String_Parsing_CPU_200K

app-analytics-services_dot_com_String_Parsing_Malloc

google_dot_com_String_Parsing_CPU_200K

google_dot_com_String_Parsing_Malloc

IPAddressBenchs

111_Machine_Warmup_Benchmark

IPv4_CIDR_Create_Then_Check_Is_Loopback_100M

IPv4_CIDR_Create_Then_Check_Is_Multicast_100M

IPv4_CIDR_Create_Then_Check_Is_Multicast_Malloc

IPv4_String_Decoding_Local_Broadcast_10M

IPv4_String_Decoding_Local_Broadcast_Malloc

IPv4_String_Decoding_Local_Broadcast_inet_pton_10M

IPv4_String_Decoding_Local_Broadcast_inet_pton_Malloc

IPv4_String_Decoding_Localhost_10M

IPv4_String_Decoding_Zero_10M

IPv4_String_Encoding_Local_Broadcast_15M

IPv4_String_Encoding_Localhost_15M

IPv4_String_Encoding_Mixed_15M

IPv4_String_Encoding_Mixed_Malloc

IPv4_String_Encoding_Mixed_inet_ntop_15M

IPv4_String_Encoding_Mixed_inet_ntop_Malloc

IPv4_String_Encoding_Zero_15M

IPv6_CIDR_Create_Then_Check_Is_Loopback_100M

IPv6_CIDR_Create_Then_Check_Is_Multicast_100M

IPv6_CIDR_Create_Then_Check_Is_Multicast_Malloc

IPv6_String_Decoding_2_Groups_Compressed_At_The_Begining_3M

IPv6_String_Decoding_2_Groups_Compressed_At_The_End_3M

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_3M

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_Malloc

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_inet_pton_3M

IPv6_String_Decoding_2_Groups_Compressed_In_The_Middle_No_Brackets_inet_pton_Malloc

IPv6_String_Decoding_Localhost_Compressed_5M

IPv6_String_Decoding_Uncompressed_3M

IPv6_String_Decoding_Zero_Compressed_5M

IPv6_String_Decoding_Zero_Uncompressed_3M

IPv6_String_Encoding_Localhost_10M

IPv6_String_Encoding_Max_4M

IPv6_String_Encoding_Mixed_4M

IPv6_String_Encoding_Mixed_Malloc

IPv6_String_Encoding_Mixed_inet_ntop_4M

IPv6_String_Encoding_Mixed_inet_ntop_Malloc

IPv6_String_Encoding_Zero_10M

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Oct 21, 2025 •

edited

Loading