Unable to Read N5 Files

[william-silversmith]

• pyn5 version: 1.1.1
• Python version: 3.9
• Operating System: MacOS 12.3

Description

Trying to read an existing N5 image chunk, but unable to figure out how to read in bytes. I was also unable to read in an existing file.

What I Did

import pyn5
from io import BytesIO

f = open("synapses_dt_reblocked_s5_14_8_5 14-10-08-526.n5", "rb")
binary = f.read()
b = BytesIO(b)
n5file = pyn5.File(b, mode=pyn5.Mode.READ_ONLY)

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/wms/.virtualenvs/cv/lib/python3.9/site-packages/pyn5/file_group.py", line 180, in __init__
    super().__init__(name, mode)
  File "/Users/wms/.virtualenvs/cv/lib/python3.9/site-packages/h5py_like/file.py", line 12, in __init__
    self.filename: Path = Path(name).resolve()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1082, in __new__
    self = cls._from_parts(args, init=False)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 707, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 691, in _parse_args
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not BytesIO

I also tried to read from a file:

import pyn5

n5file = pyn5.File("synapses_dt_reblocked_s5_14_8_5 14-10-08-526.n5", mode=pyn5.Mode.READ_ONLY)

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/USER/.virtualenvs/cv/lib/python3.9/site-packages/pyn5/file_group.py", line 181, in __init__
    self._require_dir(self.filename)
  File "/Users/USER/.virtualenvs/cv/lib/python3.9/site-packages/pyn5/file_group.py", line 206, in _require_dir
    raise FileExistsError("File found at desired location of directory")
FileExistsError: File found at desired location of directory

[william-silversmith]

Here is a sample of the data I was trying to read. The block size should be 256x256x208

synapses_dt_reblocked_s5_14_8_5 14-10-08-526.n5.zip

[aschampion]

I looked quickly through pyn5 source since I haven't touched it in ages, and while this use case of reading a block from a buffer (while presumably providing the dataset attributes manually) is possible in rust-n5, I don't think the necessary interfaces are exposed in pyn5.

[william-silversmith]

I ended up managing to do it by reading the n5 spec and using numpy for my super basic use case. Thanks for taking a look Andrew!

…

On Tue, Jul 19, 2022, 7:07 AM Andrew Champion ***@***.***> wrote: I looked quickly through pyn5 source since I haven't touched it in ages, and while this use case of reading a block from a buffer (while presumably providing the dataset attributes manually) is possible in rust-n5, I don't think the necessary interfaces are exposed in pyn5. — Reply to this email directly, view it on GitHub <#175 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AATGQSLJYIIK6BXGNWMZIQ3VU2EAXANCNFSM53TONORA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[sharmishtaa]

@william-silversmith Could you please share how you did this?

[william-silversmith]

Hi @sharmishtaa, you can find my implementation in CloudVolume here:

https://github.com/seung-lab/cloud-volume/tree/master/cloudvolume/datasource/n5