Running detect_spikes(multi_process = True) fails and hangs because Macs hate forking

[danielsvedberg]

Running detect_spikes(multi_process = True) on mac hangs and produces numerous copies of the following message:

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

Doing some reading online has led me to find that Forking multiprocessing processes (as is "proper") on mac is "unsafe" and will cause problems. Unfortunately we're stuck with Joblib's multiprocessing backend here.

Anyways if there's a fix to be made, it starts here:

blechpy/blechpy/datastructures/dataset.py

Line 790 in 567b6eb

backend='multiprocessing')(delayed(run_joblib_process)

And for those of you reading at home with the same problem, run with multi_process = False for now

[danielsvedberg]

Quick update on this issue--for whatever reason, I've started to encounter this behavior on Ubuntu as well even though I'm working with my "magic" yml file. I'm working on changing my branch back to loky backend for joblib in dataset.detect_spikes() and dataset.blech_clust_run(). Seems that the issue with loky backend isn't anything intrinsic to the library itself, but rather, the sub-functions inside these are trying to execute parallel processes within the parallel processes, causing memory/stability problems. Keeping the version as-is for now until I validate the fixes.