Skip to content

KeyError when DirCache is disabled #153

@sutao

Description

@sutao

When I tried to disable DirCache by passing use_listings_cache=False or listings_expiry_time=0, any listing operation would raise a KeyError, this is due to fsspec's DirCache raising KeyError if no cached item is found.

The cause of the error is from ossfs/core.py line 183.

        try:
            self.dircache[norm_path] = self._get_object_info_list(
                bucket_name, prefix, delimiter, connect_timeout
            )
            return self.dircache[norm_path]  # <-- Raises KeyError if DirCache is disabled
        except oss2.exceptions.AccessDenied:
            return []

It should be changed to:

        try:
            obj_info_list = self._get_object_info_list(
                bucket_name, prefix, delimiter, connect_timeout
            )
            self.dircache[norm_path] = obj_info_list
            return obj_info_list
        except oss2.exceptions.AccessDenied:
            return []

There might be other places in the SDK that need fixing.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions