ciftify_recon_all fails occasionally with returncode 1

[CompuNeuroPsychiatryLabEinKerem]

Hi,

I'm running ciftify_bids_app using singularity. Usually it works great, but sometimes it fails with message:

cmd: ciftify_recon_all --n_cpus 1 --ciftify-work-dir /data/ciftify --fs-subjects-dir /data/freesurfer --surf-reg MSMSulc sub-SBJ1
 Failed with returncode 1

setNativeLocks failed: Resource temporarily unavailable
. . .

For the subjects I get this error for, I always get this error - also if I run them again. However, if I try to run them via docker on my desktop with the same inputs, it runs properly.

The shell I'm using is:

singularity run --cleanenv \
    -B $DATA_DIR:/data \
    -B $FS_LICENSE:/fs-license \
    $LAB_PATH/Singularity/tigrlab_fmriprep_ciftify_latest-2019-08-16-6ea864911724.simg \
    /data/BIDS /data participant  \
    --participant_label=${SBJ} \
    --fs-license="/fs-license/license.txt"  \
    --read-from-derivatives /data --debug --anat_only

And I'm running it via sbatch

Any Ideas?

Thanks you so much!

[DESm1th]

This is a very late response, and probably not needed anymore buuuut... are any of those directories you're using on NFS? That error crops up sometimes with singularity and it tends to happen much more frequently with NFS directories. Unfortunately it's not really something we can fix on our end

[edickie]

Also sorry to late respond to this. We've seen the "setNativeLocks failed:" message in other runs of ciftify - but it ciftify still finishes correctly - so it's more of an anoying warning message than a try error.