Rebuild opendalfs via opendal python binding

[Xuanwo]

I made a mistake.

---

When I started the opendalfs project, my goal was to build a native integration for fsspec using Rust. My motivation was to ensure an optimized fsspec implementation without the additional overhead from opendal’s Python bindings. However, things didn’t go as planned.

On one hand, as a new project, opendalfs doesn’t have many contributors yet. (A big thank you to @PsiACE, @wey-gu, and @BeautyyuYanli.) Unfortunately, we missed our planned release schedule: #6

On the other hand, opendal’s python bindings have continued to improve. They are much better now than they were a year ago when I first started this project.

Recently, there has been a lot of discussion within the opendal community about implementing fsspec support using the opendal Python binding: apache/opendal#5972 and apache/opendal#6249. These conversations made me realize that my decision a year ago may not have been the right one.

What the community truly wants is not for the opendal team to take over this integration, but simply to have a usable and reliable product. I still believe that opendalfs is better to be under fsspec and with guidance from @martindurant.

The good news is that it’s never too late to correct my mistakes. Therefore, I propose to rebuild opendalfs using the opendal Python binding and to prepare for our release within the next one or two weeks.

[wey-gu]

Learnt a lot from the previous design and also loved the new approach, having more of opendalfs push to the python-native space with py-binding could indeed bring more maintainability in python community && accelerate the evolution powered by its downstream user community.

Great, brave and wise call/action @Xuanwo.

PS. the previous design made a lot of sense and I wouldn't call it a mistake as it follows the philosophy of rust and OpenDAL to make every inch rust/fast, it's totally worth it.

[umangbudhwar]

Hi @Xuanwo,

Thanks for the update — it’s great to see this direction being clarified and aligned with recent discussion in the OpenDAL community.

I haven’t started any implementation yet, but I’d be interested in contributing to this effort if it’s helpful. Would you be open to me trying my hand at developing the fsspec.AsyncFileSystem integration using the OpenDAL Python bindings?

Happy to collaborate wherever needed — just let me know how you’d prefer to proceed or if there are any specific areas you’d like help with.

[martindurant]

I am available to help where I can, but I'm glad to see community collaboration here.