Add NPU provider (New) by vandah · Pull Request #2198 · canonical/checkbox

vandah · 2025-11-12T14:11:31Z

Description

This PR adds a provider for testing devices with an NPU.
It is currently focused on Intel NPUs which use the accel kernel interface.
The driver for these NPUs is distributed through the intel-npu-driver snap which also includes a gtest-based testing utility npu-umd-test. The tests in this provider check the appropriate firmware version is loaded, the user has the appropriate permissions and the rest runs individual tests from the npu-umd-test utility.

The latest version of the intel-npu-driver snap now includes also the configuration file and model files necessary for running the npu-umd-test as well as a known-failures command which I've used to simplify the test filtering. The configuration file and list of known failures will be maintained as part of the intel-npu-driver snap as they can change from one version to another.

Known issues

Some of the test names coming from the npu-umd-test test suite are longer than 80 characters which triggers a warning in checkbox.

Tests

Tests have been run on Meteor Lake and Arrow Lake devices.

antlassagne · 2025-11-26T11:01:47Z

providers/npu/README.md

@@ -0,0 +1,5 @@
+# Checkbox Provider - NPU


Is there anything that should be done when creating a new provider? CI-wise, packaging-wise?

Yes, indeed. It will require to add at least the provider into the tox-checkbox.yaml and create a tox file for this provider.
Nevertheless, I discussed this morning with Max and, since these tests are fairly simple, and they are all under a new manifest entry, we could add them directly to the base provider.
I will finish the review to make sure that's the case, but that way we won't need to create a new provider.

I have been told by @farshidtz that to run the coverage and different python version tests, the provider needs to be added in the tox-checkbox workflow.
However, I am not sure if there is more to do for a new provider.

antlassagne · 2025-11-26T11:06:28Z

providers/npu/units/jobs.pxu

+estimated_duration: 2s
+command: check_accel_permissions.py
+imports: from com.canonical.plainbox import manifest
+requires: manifest.has_npu == 'True'


I think that new manifest entries must now be submitted to the certification team to be added to C3's feature set

C3 gets them from https://github.com/canonical/blueprints so you could also make a PR there, I think.

vandah · 2025-12-12T12:11:22Z

@fernando79513 and @tomli380576, can you please review this PR?

pseudocc

Just a passersby reviewer, see inline comments.

pseudocc · 2025-12-19T08:02:55Z

providers/npu/README.md

@@ -0,0 +1,5 @@
+# Checkbox Provider - NPU
+
+This provider includes tests for devices with an NPU. As of right now, it is intended only for Intel NPUs.  The tests only run as long as the manifest entry `has_npu` is set to `true`.


Should check this instead of defining has_npu

modinfo -Falias intel_vpu devenv-shell-env pci:v00008086d0000FD3Esv*sd*bc*sc*i* pci:v00008086d0000B03Esv*sd*bc*sc*i* pci:v00008086d0000643Esv*sd*bc*sc*i* pci:v00008086d0000AD1Dsv*sd*bc*sc*i* pci:v00008086d00007D1Dsv*sd*bc*sc*i*

Do you mean there should be no manifest entry at all ?
I have now added a job to check modinfo output, is that what you had in mind?

I was thinking about reading the modinfo output in a resource job, but now I think it may not be the ideal way.

Say we have:

6.17 kernel and a Panther Lake CPU (modaliases for Panther Lake were added in 6.18).

Then all NPU tests would be skipped, which shouldn't be what we expect. Let's just keep the manifest implementation; it's good.

Yes, I think in these cases it's better to go to the manifest. If we know if a device should test for NPU(via the manifest), then we can easily detect an error in case the NPU is not detected correctly.

providers/npu/bin/check_accel_permissions.py

providers/npu/bin/intel_npu_gtest_resource.py

tomli380576

Test scripts lgtm! Could you provide a bit of documentation on what the expectations are for NPU_UMD_TEST_CONFIG like where it's supposed to be placed at, permissions, what exactly are the expected contents (for example the tree output of a correct setup), etc.

One small question: is the driver snap expected to come preinstalled? If not, I think we should print an error somewhere or mention it in the manifest.

tomli380576 · 2025-12-22T08:54:34Z

providers/npu/bin/intel_npu_gtest_resource.py

+
+
+def main():
+    config_path = os.environ.get("NPU_UMD_TEST_CONFIG")


I think NPU_UMD_TEST_CONFIG also needs to be an absolute path since it's passed to dirname in one of the jobs

assert Path(config_path).is_absolute()

This file (along with the model file) might have to be placed inside the driver snap's current directory or the umd tests will say "the file is bad" and trigger an apparmor deny message. For example if I put the config file in $HOME, this happens:

[Tue Dec 23 10:47:03 2025] audit: type=1400 audit(1766458023.464:331): apparmor="DENIED" operation="open" class="file" profile="snap.intel-npu-driver.npu-umd-test" name="/home/ubuntu/basic.yaml" pid=31451 comm="npu-umd-test" requested_mask="r" denied_mask="r" fsuid=1000 ouid=1000

Currently this file is placed in the current directory by the script which installs the intel-npu-driver snap. The file is pretty much static but we're planning to have the file located directly inside the intel-npu-driver snap (since the format could change between versions of npu-umd-test which is already distributed as a binary in the intel-npu-driver snap).

Does this mean the model files would be bundled with the snap in the future? If so, I think you can use the path of the bundled files as the default inside the test case and only use the environment variable as an override. This would also make it easier to run the test since we won't have to specify NPU_UMD_TEST_CONFIG every time.

The config file as well as the model files are now bundled with the intel-npu-driver snap on the latest/edge channel. The default config and model files are used if you specify --config=basic.yaml.

I have updated the code to make the NPU_UMD_TEST_CONFIG optional.

providers/npu/units/jobs.pxu

vandah · 2026-01-23T07:53:59Z

One small question: is the driver snap expected to come preinstalled? If not, I think we should print an error somewhere or mention it in the manifest.

The snap is not pre-installed but ideally the devices that do have the has_npu manifest entry should install the snap before running checkbox... Is there any way in checkbox to define this dependency and maybe even have the snap auto-installed by checkbox?

pseudocc

Looks good to me now, thanks!

tomli380576 · 2026-01-23T09:13:23Z

To make this case depend on whether the driver snap exists, add snap.name == "intel-npu-driver" in the requires: section of the test job; but note that this would make checkbox "silently" skip the job and put it in the "job with failed dependencies" pile if the snap is not installed even if the manifest is set to true.

codecov · 2026-01-23T14:46:18Z

Codecov Report

❌ Patch coverage is 96.78068% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.83%. Comparing base (278c069) to head (eaea343).
⚠️ Report is 83 commits behind head on main.

Files with missing lines	Patch %	Lines
...roviders/npu/tests/test_check_accel_permissions.py	94.73%	7 Missing ⚠️
providers/npu/bin/check_accel_permissions.py	86.95%	3 Missing ⚠️
providers/npu/bin/check_firmware_version.py	97.05%	1 Missing ⚠️
providers/npu/bin/intel_npu_gtest_resource.py	97.36%	1 Missing ⚠️
providers/npu/bin/min_kernel_version.py	95.65%	1 Missing ⚠️
providers/npu/tests/test_check_firmware_version.py	99.27%	1 Missing ⚠️
...oviders/npu/tests/test_intel_npu_gtest_resource.py	97.43%	1 Missing ⚠️
providers/npu/tests/test_min_kernel_version.py	98.50%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2198      +/-   ##
==========================================
+ Coverage   53.34%   55.83%   +2.49%     
==========================================
  Files         399      422      +23     
  Lines       42907    45417    +2510     
  Branches     7945     8194     +249     
==========================================
+ Hits        22887    25358    +2471     
- Misses      19214    19287      +73     
+ Partials      806      772      -34

Flag	Coverage Δ
checkbox-ng	`71.69% <ø> (+0.28%)`	⬆️
checkbox-support	`70.89% <ø> (+5.59%)`	⬆️
provider-base	`32.17% <ø> (+2.34%)`	⬆️
provider-certification-client	`57.14% <ø> (ø)`
provider-certification-server	`57.14% <ø> (ø)`
provider-genio	`96.90% <ø> (ø)`
provider-gpgpu	`93.14% <ø> (ø)`
provider-iiotg	`100.00% <ø> (ø)`
provider-npu	`53.21% <96.78%> (?)`
provider-resource	`39.57% <ø> (+0.26%)`	⬆️
provider-sru	`97.97% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

vandah · 2026-01-29T14:38:59Z

To make this case depend on whether the driver snap exists, add snap.name == "intel-npu-driver" in the requires: section of the test job; but note that this would make checkbox "silently" skip the job and put it in the "job with failed dependencies" pile if the snap is not installed even if the manifest is set to true.

I see, I have actually avoided using the dependency exactly so that it would be an explicit failure if the snap is not present but has_npu. In our testing pipeline, we install the snap with testflinger before running checkbox, but I was wondering how to make this more obvious if it gets run by someone else (and how does it work with dependencies when a device gets certified).

As a side note, I think it would be useful to have something like Ansible's failed_when which would allow the tests to fail based on the value of checkbox variables since as I understand, skipped tests have a bit different meaning than failed tests (not applicable vs. something is wrong).

fernando79513

Sorry for taking so long to review it. Thanks a lot for all the effort.
I will continue the review on Monday, but I'm leaving some comments in the meantime.

providers/npu/bin/check_accel_permissions.py

fernando79513 · 2025-12-16T08:03:30Z

providers/npu/bin/check_firmware_version.py

+    raise SystemExit(
+        "The loaded firmware does not match any version in the snap files."
+    )


Same comment as before. If the versions mismatch, it will be interesting to log the loaded version and the version for the snap to investigate the issue.

I have now updated it to add the loaded version and the versions found in the snap in the error message.

providers/npu/bin/intel_npu_gtest_resource.py

fernando79513 · 2025-12-16T12:24:40Z

providers/npu/README.md

@@ -0,0 +1,5 @@
+# Checkbox Provider - NPU


Yes, indeed. It will require to add at least the provider into the tox-checkbox.yaml and create a tox file for this provider.
Nevertheless, I discussed this morning with Max and, since these tests are fairly simple, and they are all under a new manifest entry, we could add them directly to the base provider.
I will finish the review to make sure that's the case, but that way we won't need to create a new provider.

fernando79513 · 2025-12-16T12:31:11Z

providers/npu/bin/check_firmware_version.py

+        # This can happen with corrupted files or if 'strings' isn't installed
+        return None
+    return None


No need to retunr None in python.
I would log here the error though, to know that it could not read any file

Suggested change

# This can happen with corrupted files or if 'strings' isn't installed

return None

return None

print("Version pattern not found in file")

I have made this raise SystemExit now.

fernando79513 · 2026-01-30T13:29:26Z

providers/npu/bin/check_accel_permissions.py

+        except (
+            IOError,
+            FileNotFoundError,
+        ):


Nitpick, I just feel it's more readable this way.

Suggested change

except (

IOError,

FileNotFoundError,

):

except (IOError, FileNotFoundError):

This is now updated.

fernando79513 · 2026-01-30T13:33:16Z

providers/npu/bin/min_kernel_version.py

+    try:
+        # Find all sequences of one or more digits in the string
+        numbers = re.findall(
+            r"\d+",
+            version_str,
+        )
+        if not numbers:
+            raise ValueError("No version numbers found in the string.")
+
+        # Convert the list of number strings to a tuple of integers
+        return tuple(
+            map(
+                int,
+                numbers,
+            )
+        )
+    except (
+        ValueError,
+        TypeError,
+    ):
+        return None


Nitpick: This may be an autoformatting config thing, but this 20 lines could be written in just 9, with a more readable structure (and more consistent with the rest of the checkbox codebase)
Maybe it's an issue with the trailing commas after each list/touple item?
I saw you have already fixed it in some parts of the code.

Suggested change

try:

# Find all sequences of one or more digits in the string

numbers = re.findall(

r"\d+",

version_str,

)

if not numbers:

raise ValueError("No version numbers found in the string.")

# Convert the list of number strings to a tuple of integers

return tuple(

map(

int,

numbers,

)

)

except (

ValueError,

TypeError,

):

return None

try:

# Find all sequences of one or more digits in the string

numbers = re.findall(r"\d+", version_str)

if not numbers:

raise ValueError("No version numbers found in the string.")

# Convert the list of number strings to a tuple of integers

return tuple(map(int, numbers))

except (ValueError, TypeError):

return None

I'm pretty sure this formatting comes from running black on it, I don't use any other autoformatter. It could be wrong comma placement though, I'll check if I can make it format it in less lines.

Considering how many of these formattings are there I wonder if I haven't given black wrong line length at some point...

I think it's putting trailing commas by default. I also use black, but when I remove the trailing comma on list/touples, I get a more reasonable result.

Yeah I somehow think the commas were also added by black, but honestly can't be sure anymore. I think it stemmed somehow from the fact that black considers 80-82 characters still fine for -l 79 but then the flake8 test fails and so I've tried to put smaller numbers to black to make the lines actually max 79 characters long.

Anyways, I've reformatted as much as I saw that could be fit one less lines now 🙂

providers/npu/bin/check_firmware_version.py

fernando79513 · 2026-01-30T14:08:28Z

providers/npu/bin/check_firmware_version.py

+    result = subprocess.run(
+        [
+            "journalctl",
+            "--dmesg",
+        ],
+        capture_output=True,
+        text=True,
+        check=True,
+        encoding="utf-8",
+    )


It should be okay just using subprocess.check_output(). Also I don't think "encoding=" is available for python 3.5, but I don't think we are ever going to run these tests in core16.

Suggested change

result = subprocess.run(

[

"journalctl",

"--dmesg",

],

capture_output=True,

text=True,

check=True,

encoding="utf-8",

)

result = subprocess.check_output(

["journalctl", "--dmesg"], universal_newlines=True, encoding="utf-8"

)

I have updated all of the calls to use check_output now. And I don't think the encoding parameter is necessary there anyways.

fernando79513 · 2026-01-30T14:09:10Z

providers/npu/bin/check_firmware_version.py

+        result = subprocess.run(
+            [
+                "strings",
+                filepath,
+            ],
+            capture_output=True,
+            text=True,
+            check=True,
+            encoding="utf-8",
+        )


Suggested change

result = subprocess.run(

[

"strings",

filepath,

],

capture_output=True,

text=True,

check=True,

encoding="utf-8",

)

result = subprocess.check_output(

["strings", filepath], universal_newlines=True, encoding="utf-8"

)

This is now updated. 👍

Remove unreachable return Co-authored-by: Fernando Bravo <39527354+fernando79513@users.noreply.github.com>

fernando79513

I left a few more comments on the PR.
My main concern is with the "non-blocker" jobs. I am not sure under which conditions you expect them to fail, but we should not add them to the test plan if they are not expected to pass.

Apart from that, could you add some submissions of devices in the lab running the NPU tests with the new provider?

fernando79513 · 2026-02-02T15:17:27Z

providers/npu/bin/min_kernel_version.py

+    current_version = parse_version(current_version_str)
+    required_version = parse_version(args.required_version)


We have packaging.version available. If you import from packaging import version you can use it to parse kernel versions, in a similar way to this script.

Suggested change

current_version = parse_version(current_version_str)

required_version = parse_version(args.required_version)

current_version = version.Version(current_version_str.rsplit('-', 1)[0]))

required_version = version.Version(current_version_str.rsplit('-', 1)[0]))

It would be great to move this functionality to checkbo_ support/helpers, but I don't think we are including the packaging module there.
We could have a simple kernel version parser there thoug:

def parse_kernel_version(version_str): version, flavor = version_str.rsplit("-", 1) return version, flavor

fernando79513 · 2026-02-02T15:27:36Z

providers/npu/bin/intel_npu_gtest_resource.py

+    known_failures = (
+        subprocess.run(
+            ["intel-npu-driver.known-failures"],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            universal_newlines=True,
+        )
+        .stdout.strip()
+        .splitlines()
+    )


This is a bit simpler:

Suggested change

known_failures = (

subprocess.run(

["intel-npu-driver.known-failures"],

stdout=subprocess.PIPE,

stderr=subprocess.PIPE,

universal_newlines=True,

)

.stdout.strip()

.splitlines()

)

known_failures = (

subprocess.check_output(

["intel-npu-driver.known-failures"],

universal_newlines=True,

)

.strip()

.splitlines()

)

Thanks, I changed all of the subprocess.run to subprocess.check_output 👍

fernando79513 · 2026-02-02T16:08:00Z

providers/npu/bin/intel_npu_gtest_resource.py

+        help=" 'blocker' prints only tests that aren't known failures,"
+        "'non-blocker' prints only known failures.",


I think the naming is confusing here. We don't expect any of the tests in checkbox to fail in normal circumstances, even if they are "non-blocker" tests.

If they only pass some of the time, just call them fleaky. If they only pass on certain architectures, use another resource and filter them. The "blocker"/"non-blocker" category should be reserved only for the certification process.

They are known failures, the list is already filtered by cpu generation and running kernel version. However the failures might get fixed by Intel in future driver versions and that's why it would be useful to be able to have the information about if they passed/failed on a certain version.

If you just want to keep this information from run to run, you could just have an informational job.
You could have a job that runs all these tests that are expected to fail and logs their results, without the actual job failing. You should make it clear in the tests this purpose.
something like:

id: tracked_known_issues category_id: intel_npu flags: simple _description: Keeps track of tests with known failures that may be fixed in the future. estimated_duration: 2s command: failed_tests_info

fernando79513 · 2026-02-02T16:19:20Z

providers/npu/bin/check_firmware_version.py

+)
+
+FIRMWARE_SEARCH_DIR = Path("/var/snap/intel-npu-driver/current/intel/vpu")
+VERSION_PATTERN = re.compile(r"^(\d{8}\*|[A-Z][a-z]{2}\s+\d{1,2}\s+\d{4}\*).*")


This regex pattern is quite hard to read. If we can't avoid using it (in favor of builtin python functions), at least try to explain what it's trying to achieve and maybe add an example.

fernando79513 · 2026-02-02T16:27:39Z

providers/npu/units/resource.pxu

+plugin: resource
+id: intel_npu_gtest_resource_non_blocker
+category_id: intel_npu
+_summary: Produces a resource per test of Intel NPU Umd test
+_description: Creates a resource for accessing tests for NPU drivers.
+imports: from com.canonical.plainbox import manifest
+requires: manifest.has_npu == 'True'
+command: intel_npu_gtest_resource.py -m non-blocker


As I said before. If they are known failures, either you make sure you run them only on the environments where they always pass, or you add the results as an informational job. Jobs that are included in the test plan are expected to pass if the machine has no issues.

or you add the results as an informational job

What do you mean with this? Exclude it from the test plan and just keep it as an optional job that could be included by another test plan?

Yes, that was my approach. But after your previous comment, I think we should not have them in any test plan. If we decide to include any of these tests for a specific platform, we can add a "platform" option in intel_npu_gtest_resource.py to filter them.

fernando79513 · 2026-02-02T16:43:53Z

providers/npu/units/resource.pxu

+plugin: resource
+_description: Creates resource for firwmare version logging support in dmesg
+estimated_duration: 2s
+command: min_kernel_version.py 6.8


If you are not getting any information from the resource job, apart from whether it passes or not, you could get rid of the resource altogether and just have this check as a dependency of the rest of the NPU tests:

id: kmd/char_device_permissions ... depends: info/kernel_config

id: check_min_kernel_version category_id: intel_npu flags: simple _description: Check if the firmware version is supported estimated_duration: 2s command: min_kernel_version.py 6.8

If you are not getting any information from the resource job, apart from whether it passes or not

Can you please clarify what you mean here? The resource job always passes but sets the resource to either supported or unsupported, not in the sense if the driver is supported or not but if getting the firmware version out of dmesg is supported. The job kmd/firmware_version is requiring that the state is supported because it inspects dmesg logs to check the loaded firmware version and that it matches the one in the tested snap. If getting the fw version from dmesg is not supported, the job kmd/firmware_version would fail but the driver is still expected to work also with kernels older than 6.8.

fernando79513 · 2026-02-04T10:16:07Z

Also, I have a couple of questions.

How are you going to install the intel-npu-driver snap on the devices that require it for testing? For that, we could use the setup-include feature. But since it's not fully implemented (It's lacking documentation and a few key features), I think we should include it in the corresponding testflinger job.
Can you share the submission tar file of the checkbox run in the PR description? Or if you want, submit the checkbox job to C3, that's also OK.

vandah · 2026-02-12T15:23:12Z

Also, I have a couple of questions.

How are you going to install the intel-npu-driver snap on the devices that require it for testing? For that, we could use the setup-include feature. But since it's not fully implemented (It's lacking documentation and a few key features), I think we should include it in the corresponding testflinger job.

@fernando79513 Thank you for your review! The intel-npu-driver snap is a functional dependency that's needed for the NPU to be used so it is expected to be present on the tested device so installing in a testflinger job sounds perfect 👍. We do that for the intel-npu-driver snap CI already but are there any extra steps needed for certification ? I.e. do we need to provide you with the testflinger jobs and then it gets run by the cert team directly or do we just run the tests and upload to C3 ourselves?

Can you share the submission tar file of the checkbox run in the PR description? Or if you want, submit the checkbox job to C3, that's also OK.

Sure, I'll upload those.

vandah changed the title ~~New: Add NPU provider~~ Add NPU provider (New) Nov 12, 2025

vandah force-pushed the npu-provider branch 5 times, most recently from 40b763c to 3c0ef4f Compare November 13, 2025 09:29

vandah marked this pull request as draft November 13, 2025 15:24

Add NPU provider (New)

37a1b04

vandah force-pushed the npu-provider branch from 3c0ef4f to 37a1b04 Compare November 14, 2025 12:53

vandah marked this pull request as ready for review November 14, 2025 12:54

antlassagne reviewed Nov 26, 2025

View reviewed changes

pseudocc reviewed Dec 20, 2025

View reviewed changes

tomli380576 requested changes Jan 5, 2026

View reviewed changes

vandah added 3 commits January 22, 2026 16:28

Check RW permissions together, not separately

0b3b357

Replace find with in for string search

333a665

Add a job to check modinfo

24378fa

vandah requested a review from pseudocc January 23, 2026 07:55

pseudocc previously approved these changes Jan 23, 2026

View reviewed changes

Add the npu provider to CI

8611ac7

vandah dismissed pseudocc’s stale review via 8611ac7 January 23, 2026 14:45

vandah added 3 commits January 23, 2026 15:51

Add the npu provider to CI (paths)

2fd4a1f

Add tox.ini

1ba3488

remove the pytest import from tests

5c0b940

vandah force-pushed the npu-provider branch from e5835e8 to 5c0b940 Compare January 29, 2026 14:22

Remove type hints for python<3.10 compatibility

dfeade4

vandah force-pushed the npu-provider branch from 3075be1 to dfeade4 Compare January 29, 2026 14:43

Remove assert_called_once for python<3.6 compatibility

c724555

vandah force-pushed the npu-provider branch from e2355f3 to 6130420 Compare January 29, 2026 15:18

Use OrderedDict for python<3.6 compatibility

3ca070a

vandah force-pushed the npu-provider branch from 6130420 to 3ca070a Compare January 29, 2026 15:23

vandah added 4 commits January 29, 2026 16:35

Make it optional to specify the config file

1a6595b

Update according to the latest version of intel-npu-driver

eaaeb68

Fix certification-status

ffe045b

Fix CI

2b617ca

vandah requested review from fernando79513 and tomli380576 January 29, 2026 22:51

fernando79513 requested changes Jan 30, 2026

View reviewed changes

Update providers/npu/bin/check_firmware_version.py

eaea343

Remove unreachable return Co-authored-by: Fernando Bravo <39527354+fernando79513@users.noreply.github.com>

fernando79513 requested changes Feb 2, 2026

View reviewed changes

vandah added 4 commits February 2, 2026 22:18

Reformatting for less lines

ce390a0

Replace subprocess.run with subprocess.check_output

f4e2a3b

Print more information when fw not found

20236aa

Add error message when fw version not found in file.

34b18fd

		@@ -0,0 +1,5 @@
		# Checkbox Provider - NPU

		This provider includes tests for devices with an NPU. As of right now, it is intended only for Intel NPUs. The tests only run as long as the manifest entry `has_npu` is set to `true`.



		def main():
		config_path = os.environ.get("NPU_UMD_TEST_CONFIG")

		current_version = parse_version(current_version_str)
		required_version = parse_version(args.required_version)

		help=" 'blocker' prints only tests that aren't known failures,"
		"'non-blocker' prints only known failures.",

Conversation

vandah commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Known issues

Tests

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vandah commented Dec 12, 2025

Uh oh!

pseudocc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tomli380576 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tomli380576 Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vandah Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vandah commented Jan 23, 2026

Uh oh!

pseudocc left a comment

Choose a reason for hiding this comment

Uh oh!

tomli380576 commented Jan 23, 2026

Uh oh!

codecov bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

vandah commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fernando79513 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vandah commented Nov 12, 2025 •

edited

Loading

tomli380576 left a comment •

edited

Loading

tomli380576 Jan 23, 2026 •

edited

Loading

vandah Jan 29, 2026 •

edited

Loading

codecov bot commented Jan 23, 2026 •

edited

Loading

vandah commented Jan 29, 2026 •

edited

Loading