Skip to content

proposal: run-multitests.py --target-wiring #20

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 34 commits into
base: tests-improvements-for-octoprobe
Choose a base branch
from
Open

proposal: run-multitests.py --target-wiring #20

wants to merge 34 commits into from
+463 −221

Conversation

@hmaerki
Copy link

@hmaerki hmaerki commented Oct 20, 2025

Summary

See my comments in octoprobe/testbed_micropython#57

@dpgeorge
tests/extmod/machine_spi_rate.py: Support samd with default SPI.
0fbdd57 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod: Support ESP32-C3 targets.
f8efe44 
Some boards define ESP32C3, some define ESP32-C3.

Probably it should just use default pins, eg SPI(1).

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod/machine_spi_rate.py: Use default pins on ESP32.
2e3e758 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-perfbench.py: Skip bm_hexiom on small targets.
99a9155 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/net_inet: Skip tests on axTLS when necessary.
2c1fe8e 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/perf_bench: Skip tests when vfs doesn't exist.
a2476c1 
Eg for ESP8266-512K.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/float/string_format_modulo2_intbig.py: Get working on esp8266.
f78b65c 
In --via-mpy --emit native mode.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod/vfs_blockdev_invalid.py: Skip if no RAM to mkfs.
c5d0dd2 
On ESP8266 this test can fail with --via-mpy.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/net_hosted/ssl_verify_callback.py: Skip if necessary.
2c7f8a5 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
py/persistentcode: Only select hard-fp arch if compiler uses hard ABI.
8158596 
Because RP2350 uses float-abi=softfp!  Maybe we should change that somehow?

Could also use `defined(__SOFTFP__)` to detect soft (all software fp) or
softfp (hardware for calculations but soft/integer for passing arguments).

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-perfbench.py: Skip tests that have a MemoryError.
14329f4 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-tests.py: Skip various tests on small SAMD targets.
2f4f18f 
Tested on ADAFRUIT_ITSYBITSY_M0_EXPRESS.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/multi_net: Skip tests on axTLS when needed.
697c68b 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/multi_net/udp_data_multi.py: Reduce number of UDP groups.
b2e3761 
On bare-metal lwIP targets, there's only enough buffer space for 4 groups.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-perfbench.py: Skip misc_mandel if target doesn't have complex.
9d7ebcc 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
esp32/modules: Convert AttributeError to ImportError.
430ae9f 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/ports/stm32/adcall.py: Print values if test ADC fails.
a545264 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod/machine_spi_rate.py: Support mimxrt and nrf ports.
9e4e073 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-tests.py: Abort test if serial port fails at start.
dbfeb32 
The test runner will abort with error code 2 if:
- the target could not be detected (enter_raw_repl failed at the
  detection stage)
- unittest is not installed but is needed

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-tests.py: Abort test run if enter_raw_repl fails many times.
863dfd5 
Now, if a test fails and crashes a board to the point it no longer responds
(but the serial connection is still alive and able to read/write), it only
takes a maximum of 30 + 5 * 4 = 50 seconds to detect that state and exit
the test run (previously it would be 30 + 10 seconds * number of remaining
tests = potentially large).

Can test this by adding the following to a test:

    import time, micropython
    micropython.kbd_intr(-1)
    while 1:
        time.sleep(1)

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
esp8266/boards: Include unittest in 512k board variant.
ff9f6f9 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-natmodtests.py: Abort test run if target fails.
81ef984 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests: Move esp32-specific test to general test dir.
15de2a3 
Because it needs internet connection it must be in a test directory that is
expected to have internet connection.

Also, this test can be made to run on more targets in the future.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/net_inet/resolve_on_connect.py: Convert to use unittest.
e8e39d7 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/ports/esp32/check_err_str.py: Open file in root directory.
8644b7b 
Otherwise the test will fail with --via-mpy because the mounted __FS object
will allow any file to be opened.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/misc/rge_sm.py: Use list.append instead of list += [...].
d31ac07 
To reduce heap memory use.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
py/objlist: Make list append allocation more gentle.
8d04fae 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-tests.py: Auto detect -march-flags for mpy-cross.
38c8e0b 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/run-tests.py: Allow port:<dev> specification in run-multitests.py.
60adac5 
This moves the `port:<dev>` check for the test instance to the
`convert_device_shortcut_to_real_device()` helper function, which means
that `run-multitests.py` can use this specification.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod/time_res.py: Properly skip functions not in time module.
64c8c24 
If `time.time` doesn't exist, it tries to use `globals()['time']` which is
the time module itself, and that causes the test to fail.  Instead it
should just skip `time.time`.

Signed-off-by: Damien George <damien@micropython.org>
dpgeorge and others added 4 commits September 26, 2025 22:37
@dpgeorge
tests/net_inet: Update micropython.org certificate for SSL tests.
347e8e1 
Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/extmod/machine_soft_timer.py: Skip on nrf boards.
98fddbc 
They don't have the `freq` argument.

Signed-off-by: Damien George <damien@micropython.org>
@dpgeorge
tests/micropython/ringio_big.py: Print results at end of test.
62f5dcf 
So it can be properly skipped.

Signed-off-by: Damien George <damien@micropython.org>
@hmaerki
proposal: run-multitests.py --target-wiring
3e00073
@hmaerki hmaerki mentioned this pull request Oct 20, 2025
Open
@dpgeorge dpgeorge force-pushed the tests-improvements-for-octoprobe branch 2 times, most recently from 39ba8fe to e143b45 Compare November 19, 2025 02:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hmaerki @dpgeorge