Nth attempt to resolve port collisions once-and-for-all

[kaleb-himes]

Description

I've been seeing port collisions in Jenkins again in spite of bwrap having worked for us for many years now. This is an attempt to further avoid the probability of port collisions in the make check scripts that use random port generation.

This solution introduces the concept of "remembering ports assigned" in addition to checking "already assigned ports on the system".

Testing:

Many many cycles running openssl/ocsp scripts in tight loops. Probability of collisions in the worst case script (openssl) is estimated to have been:

# 16023 == port range
# 11 == 5x openssl servers & 6x wolfSSL servers in 

  P ≈ 1 − e^(−(11×10) / (2 × 16023))
    = 1 − e^(−110 / 32046)
    = 1 − e^(−0.003433)
    ≈ 0.343% per run

  Expected runs before seeing one collision: 1 / 0.00343 ≈ 292 runs

1 collision per 292 runs of the openssl script (we test more runs than that in each PR due to the many config options)

With this proposed change the probability is estimated to drop from 0.343% to 0% inside a single run of a script and ~0.006% intra-script (multiple copies of the script executing on the same machine). The new logic checks for already-handed-out ports on the machine but there is a small probability remaining that port is free when checked and no longer free by the time it gets used if another script grabs the same one.

I left the test scripts running all weekend without the fix and with the fix.

In the shell running the old code we saw 52 collisions over the course of 48 hours.
In the shell running with these changes in place we saw 0 collisions over the course of 48 hours.

I also created two "simulation" scripts to run many more tests in 10 minutes intervals then can be achieved with the actual live TLS connections in 48 hours and these are those results:

OLD SOLUTION w/ bwrap in place, using /dev/random for port values with no memory of assigned ports:

========================================================================
  wolfSSL port-collision demo  —  OLD generate_port() (no dedup)
========================================================================
  Servers per simulated run : 11
  Port range                : 49512 – 65534  (N=16023)
  Theoretical collision prob: 0.343% per run
  Timeout                   : 48h  (until 2026-03-04 15:17:39)
========================================================================

  elapsed             runs          collisions    obs-rate      theory-rate

  +10m00s              297522        987           0.331%        0.343%

NEW SOLUTION w/ bwrap in place, using /dev/random for port values and having memory of assigned ports:

========================================================================
  wolfSSL port-collision demo  —  NEW generate_port() (with dedup)
========================================================================
  Servers per simulated run : 11
  Port range                : 49512 – 65534  (N=16023)
  Old method theory rate    : 0.343% per run  (for comparison)
  Expected new method rate  : 0.000%  (guaranteed by used_ports[])
  System port check (ss)    : skipped (simulation — no ports bound; set SIMULATE_ONLY=0 to enable)
  Timeout                   : 48h  (until 2026-03-04 15:17:39)
========================================================================

  elapsed             runs          collisions    status

  +10m00s              249485        0             OK

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation