Add support for pre/post unit test run scripts

Author: hlovdal

CHANGELOG.md

@@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - Added support for `ARDUINO_CI_UNIT_TEST_EXTRA_COMPILER_FLAGS` environment variable.
 - Added support for overriding the shell used to execute `CUSTOM_INIT_SCRIPT`
   by setting `CUSTOM_INIT_SCRIPT_SHELL` (defaults to `/bin/sh`).
+- Added support running scripts before and/or after each run of unit tests with
+  `ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT` and `ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT`.
 
 ### Changed
 

REFERENCE.md

@@ -57,6 +57,31 @@ action to install custom library versions (i.e. a version of a library that
 is different than what the library manager would automatically install by name)
 prior to CI test runs.
 
+### `ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT` and `ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT` environment variables
+
+If set, the corresponding script will be run before/after each run of unit tests
+for each configured platform and for each compiler, e.g. if `.arduino-ci.yaml`
+contains
+
+```yaml
+unittest:
+  compilers:
+    - g++-10
+    - g++-11
+  libraries: ~
+  platforms:
+    - leonardo
+    - uno
+```
+
+the scripts will be invoked four times, with the current platform name being
+tested as the first parameter to the script and the current compiler used as
+the second parameter. It is not necessary to define both PRE and POST script;
+if you only want to run something before or after unit tests that's fine.
+By default the scripts are executed by `/bin/sh`, you can override by setting
+`ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT_SHELL` and
+`ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT_SHELL` respectively.
+
 ### `USE_SUBDIR` environment variable
 
 If set, testing will be conducted in this subdirectory (relative to the working directory).  This is for monorepos or other layouts where the library directory and project root directory are different.

exe/arduino_ci.rb

@@ -15,6 +15,8 @@
 VAR_USE_SUBDIR         = "USE_SUBDIR".freeze
 VAR_EXPECT_EXAMPLES    = "EXPECT_EXAMPLES".freeze
 VAR_EXPECT_UNITTESTS   = "EXPECT_UNITTESTS".freeze
+VAR_ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT  = "ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT".freeze
+VAR_ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT = "ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT".freeze
 
 @failure_count = 0
 @passfail = proc { |result| result ? "✓" : "✗" }
@@ -68,6 +70,8 @@ def self.parse(options)
         puts "       prior to any automated library installation or testing (e.g. to install unofficial libraries)"
         puts " - #{VAR_CUSTOM_INIT_SCRIPT_SHELL} - if set, this will override the"
         puts "       default shell (/bin/sh) used to execute #{VAR_CUSTOM_INIT_SCRIPT} with."
+        puts " - #{VAR_ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT} and/or #{VAR_ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT}"
+        puts "       if set, run the script before/after each unit test run"
         puts " - #{VAR_USE_SUBDIR} - if set, the script will install the library from this subdirectory of the cwd"
         puts " - #{VAR_EXPECT_EXAMPLES} - if set, testing will fail if no example sketches are present"
         puts " - #{VAR_EXPECT_UNITTESTS} - if set, testing will fail if no unit tests are present"
@@ -438,6 +442,7 @@ def perform_unit_tests(cpp_library, file_config)
   platforms.each do |p|
     puts
     compilers.each do |gcc_binary|
+      run_custom_script(VAR_ARDUINO_CI_PRE_UNIT_TEST_RUN_SCRIPT, p, gcc_binary)
       # before compiling the tests, build a shared library of everything except the test code
       next unless build_shared_library(gcc_binary, p, config, cpp_library)
 
@@ -457,6 +462,7 @@ def perform_unit_tests(cpp_library, file_config)
           cpp_library.run_test_file(exe)
         end
       end
+      run_custom_script(VAR_ARDUINO_CI_POST_UNIT_TEST_RUN_SCRIPT, p, gcc_binary)
     end
   end
 end