`sciunit repeat` Error: FileNotFoundError in `xarray` to `zarr` Conversion

[Maricaya]

I encountered an issue when running sciunit repeat. The process fails with a FileNotFoundError during the conversion from xarray to zarr. Here are the details of the error message:

File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/xarray/core/dataset.py", line 2549, in to_zarr
    return to_zarr(  # type: ignore[call-overload,misc]
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/xarray/backends/api.py", line 1661, in to_zarr
    zstore = backends.ZarrStore.open_group(
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/xarray/backends/zarr.py", line 483, in open_group
    zarr_group, consolidate_on_close, close_store_on_close = _get_open_params(
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/xarray/backends/zarr.py", line 1332, in _get_open_params
    zarr_group = zarr.open_group(store, **open_kwargs)
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/zarr/hierarchy.py", line 1581, in open_group
    init_group(store, overwrite=True, path=path, chunk_store=chunk_store)
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/zarr/storage.py", line 682, in init_group
    _init_group_metadata(store=store, overwrite=overwrite, path=path, chunk_store=chunk_store)
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/zarr/storage.py", line 704, in _init_group_metadata
    rmdir(store, path)
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/zarr/storage.py", line 212, in rmdir
    store.rmdir(path)
  File "/opt/conda/envs/ecopro/lib/python3.10/site-packages/zarr/storage.py", line 1266, in rmdir
    shutil.rmtree(dir_path)
  File "/opt/conda/envs/ecopro/lib/python3.10/shutil.py", line 725, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/opt/conda/envs/ecopro/lib/python3.10/shutil.py", line 654, in _rmtree_safe_fd
    onerror(os.open, fullname, sys.exc_info())
  File "/opt/conda/envs/ecopro/lib/python3.10/shutil.py", line 651, in _rmtree_safe_fd
    dirfd = os.open(entry.name, os.O_RDONLY, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'cwd'

Here are some additional details that might be helpful:
I found the same issue when I didn't use Docker.
Command Used: sciunit repeat
Docker Setup: Readme
Operating System: Ubuntu 20.04
Python Version: Python 3.10
Conda Environment: ecopro

[Maricaya]

I reproduced the problem on this python file

import xarray as xr
import numpy as np
import pandas as pd

def create_sample_dataset():
    # Create some example data
    data = np.random.rand(4, 3)
    time = pd.date_range('2024-01-01', periods=4)
    lat = [10, 20, 30]
    
    # Create an xarray Dataset
    dataset = xr.Dataset(
        {
            "temperature": (["time", "lat"], data)
        },
        coords={
            "time": time,
            "lat": lat
        }
    )
    return dataset

def write_to_zarr(dataset, outputfile):
    # Write the dataset to a Zarr file
    write_job = dataset.to_zarr(
        outputfile, mode='w', compute=False, consolidated=True
    )
    return write_job

if __name__ == "__main__":
    # Define the output file
    outputfile = 'sample_dataset.zarr'
    
    # Create a sample dataset
    dataset = create_sample_dataset()
    
    # Write the dataset to a Zarr file
    write_to_zarr(dataset, outputfile)

    print(f"Dataset successfully written to {outputfile}")

[Maricaya]

https://github.com/depaul-dice/provenance-to-use It's the PTE that can't delete folders

[Maricaya]

I reproduced the issue on this python file

import shutil
import os

file_path = './test'

try:
    shutil.rmtree(file_path)
    print(f"The directory {file_path} and all its contents have been removed successfully.")
except Exception as e:
    print(f"An error occurred while removing the directory: {e}")