Fuzzing Numpy

[devdanzin]

Fuzzing Numpy with the default Python fuzzer failed to exercise most of the API and didn't uncover free-threaded issues (likely due to the absence of shared objects between threads). Probably creating and using tricky arrays and other objects would increase the hits. Tricky data would also be interesting and suitable to finding security issues.

Numpy maintainers prefer that hits from analytic tools be condensed in single issues.

Issues found so far:

• BUG: ASAN detects heap-buffer-overflow from numpy.strings.find numpy/numpy#28791
• BUG: segfault from calling __class__ on functions like atleast_1d numpy/numpy#28788
• BUG: segfault by rounding number by large negative value numpy/numpy#28787
• BUG: Slot * of type X succeeded with an exception set numpy/numpy#28786
• BUG: segfault from calling __func__ of Cython methods numpy/numpy#28785
• BUG: segfault from random_raw from instance initialized from invalid value numpy/numpy#28784
• BUG: Failed !PyErr_Occurred() assertion numpy/numpy#28783
• BUG: Segfault/stack-overflow from initializing numpy.random._mt19937.MT19937 with recursive list numpy/numpy#28822
• BUG: crashes found by fuzzing with fusil numpy/numpy#28829

[ngoldbaum]

Probably creating and using tricky arrays and other objects would increase the hits.

Just a head's up that mutating shared arrays is explicitly not supported. It would be much more valuable to get reports where crashes or other breakage happens without any mutation on shared objects.

[devdanzin]

Thank you very much for the guidance!

I'll avoid mutation of shared arrays while fuzzing Numpy.

[devdanzin]

Fuzzing has stopped finding new hits and there are many crashes happening repeatedly (which makes checking for new hits harder), so considering this one done for now.

All in all, only shallow, low value crashes have been found. No adaptation of the fuzzer has been done except for adding a couple of arrays with dubious values, which hasn't found any hits. Some SystemError hits haven't been reported due to expectations that they'd be of even lower value.

Leaving open because #41 might make Numpy an interesting target again.