Releases: AdaCore/e3-testsuite
Releases · AdaCore/e3-testsuite
v26.0
DiffTestDriver: handle diff context size customization.- Add a XUnit results importer for report indexes.
AdaCoreLegacyDriver: avoid CRLF line endings in test scripts.AdaCoreLegacyDriver: fix handling of non-ASCII test scripts.- Detailed logs in case of subprocess output decoding error.
e3.testsuite.report.display.generate_report: fix spurious write to
sys.stdout.- Introduce a new testsuite option:
--skip-result. - Fix XUnit reports when logs include control characters.
- Introduce
e3.testsuite.report.rewriting. - GAIA reports: add missing
.resultfile for each test. - Make it easier to generate a GAIA-compatible report without a testsuite run.
- Disable the use of ANSI sequences in logging when
--nocoloris passed. - Make
DiffTestDriverperform a full match for regexp baselines. - Fix memory leaks for test driver data.
- Include total testsuite and testcase durations in XUnit reports.
- Add support for
--xunit-outputine3-testsuite-report. - Add support for
--failure-exit-codeine3-testsuite-report. - Ignore VCS directories like .git when looking for tests.
v25.0
- Testsuite reports on the filesystem are relocatable now.
- XUnit reports: count
XFAILtest results asskippedinstead offailures. - Introduce the
--no-random-temp-subdiroption. --gaia-output: create thediscsfile in more cases.TestsuiteCore.testsuite_main: force UTF-8 for text report
generation.- Introduce the
--cleanup-modecommand line option. - Make
--gaia-output(report formatted for GAIA) faster. - Fix the
--max-consecutive-failurescommand line option. - Introduce multi-processing for test fragment execution.
- Write command line options to the comment file by default.
AdaCoreLegacyDriver: add theTIMEOUTfailure reason on rlimit abort.- GAIA reports: rename
NOT_APPLICABLEtoNOT-APPLICABLE. - Fix test filtering when they don't have dedicated directories.
- Testsuite: enhance the API to ease adding inter-test dependencies.
- Add the
--generate-text-report/--no-generate-text-reportcommand-line
options. - Revamp the
e3-testsuite-reportscript. - Create a "status" file to allow users to check testsuite execution.
- GAIA reports: include discriminants list when available.
- Protect test probing against duplicate test names.
- Emit
ERRORtest results for trouble in test detection/parsing. ClassicTestDriver: use XFAIL message for XPASS result.AdaCoreLegacyTestDriver: stop altering shell scripts.DiffTestDriver: make it possible to refine baselines.- Allow multiple testcases per test directory.
AdaCoreLegacyTestControlCreator: also check for shell scripts (test.sh).- Always enable "cross" support for testsuites.
- Make the default testsuite failure exit code customizable (1 is the "default
default").
v24.0
AdaCoreLegacyTestDriver: fix working directory substitution.LineByLine: new output refiner combinator, refines each line separately.ClassicTestDriver: maketear_downdelete the working directory by
default.DiffTestDriver: fix handling of encodings to read/write baselines.e3-find-skipped-tests: new script to look for always skipped tests.e3-testsuite-report: new script to display results on the terminal.- Enable the display of error outputs (
-E/--show-error-output) by default on
TTYs. AdaCoreLegacyTestDriverenhance support for generated test scripts.- Add a lightweight report index.
DiffTestDriver: do not run output refiners on baselines.- Add the
--show-time-infocommand-line option. - Add support for the
--max-consecutive-failurescommand-line option. - Follow symbolic links to find testcases.
- Force the use of UTF-8 for GAIA reports.
- Fix
ReplacePathto work on Windows. AdaCoreLegacyTestDriver: when rewriting empty baselines with
default file (test.out), just remove the baseline file.driver.diff.PathSubstitute: userealpathrather thanabspath`.- Fix CRLF handling when truncating too long logs.
AdaCoreLegacyTestDriver: use Bash as soon as we have a Bourne shell script.- Accept invalid regexps as testcase filters.
- Make
DiffTestDriveradd aDIFFfailure reason (useful for advanced
viewers). - Switch to more common statuses in the GAIA output (
--gaia-output: OK
instead ofPASSED,XFAILinstead ofXFAILED, ...). - Do not rewrite baselines for tests with expected failures.
v23.0
- Small revamp of the current API (rename properties).
- Rework the list of test status.
- Rework and specify the structure of test results.
- Add mechanisms to control testcase discovery and execution.
- Add new general purpose drivers: classic and diff.
- Add compatibility layers for legacy AdaCore testsuites.
- Add a developer-friendly testsuite mode (--dev-temp/-d).
- Add a --failure-exit-code option, useful in continuous integration scripts.
- Add a detailed documentation.