Skip to content

Conversation

@cvanelteren
Copy link
Collaborator

@cvanelteren cvanelteren commented Dec 12, 2025

Closes #424
As we expand our tests, the runners are running longer and longer. I propose that we move from visual tests to hashes on the runners. The idea would be that we can see what tests fails such that they can be testsed locally to see how they differ visually.

We should move also to nox in case the tests are visually off in a specific combination of python and matplotlib.

@cvanelteren
Copy link
Collaborator Author

Would love your input here @beckermr!

Copy link
Collaborator

@beckermr beckermr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slow part here is making the plots. This still makes the plots for both the main branch and the PR.

To get a 2x speedup, we should cache the results from main using the github cache.

@cvanelteren
Copy link
Collaborator Author

Caching is a good idea (see latest commit). I was under the impression that the hash comparison was also faster. The generation on the baseline would be slow, but the hash check is fast.

@cvanelteren
Copy link
Collaborator Author

I will do some speed tests hang on

@beckermr
Copy link
Collaborator

The slowest part of this is making the plots. Comparing them via hashes or directly through differencing the images will have a time difference but the change will be small relative to everything else.

@cvanelteren
Copy link
Collaborator Author

Ok yeah great. Another would be to use pytest-xdist which recently got updated to allow to work with pytest-mpl. The latter will give a minor boost as I think there are only 2 cores per runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FIX/Discussion] Slow unittests

3 participants