Skip to content

feat(zb-write): Support write mode in Zonal File #726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ankitaluthra1 merged 23 commits into from
Dec 26, 2025
Merged

feat(zb-write): Support write mode in Zonal File #726

ankitaluthra1 merged 23 commits into from
Dec 26, 2025

Conversation

@suni72
Copy link
Contributor

@suni72 suni72 commented Dec 17, 2025 •
edited by ankitaluthra1
Loading

Key Changes

  • ZonalFile._ init_: Updated to initialize AsyncAppendableObjectWriter (AAOW) when the file mode is write-enabled ('w' or 'a' mode).
  • Support append mode in Zonal buckets
  • zb_hns_utils._init_aaow: Added a helper method to initialize the AAOW.
  • ZonalFile.write: Overriden to write data to gcs using AAOW.
  • ZonalFile.flush: Overridden to flush data from AAOW.
  • ZonalFile.close: Overridden to ensure the AAOW is closed properly.
    • If finalize_on_close=True, the object is finalized on close.
    • If finalize_on_close=False (default), the object remains appendable.
  • ZonalFile.discard: Reimplemented to log a warning ("Discard is unavailable for Zonal Buckets..."), as Bidi streams cannot be discarded/cancelled once written.
  • core: Added logic to route upload methods to zonal implementation for zonal buckets
  • Unit Test: Added unit tests for new methods in zb_hns_utils and zonal_file
  • e2e Test: Added e2e tests for zonal write in real gcs

@suni72 suni72 changed the title Zb write latest feat(zb-write): Support write mode in Zonal File Dec 17, 2025
@suni72
Added tests for Zonal writes
714c087 
Close GCSFile before aaow/mrd
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 19, 2025

/gcbrun

@suni72 suni72 marked this pull request as ready for review December 19, 2025 08:15
ankitaluthra1
ankitaluthra1 requested changes Dec 19, 2025
View reviewed changes
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 22, 2025

/gcbrun

@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 23, 2025

/gcbrun

1 similar comment
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 24, 2025

/gcbrun

ankitaluthra1
ankitaluthra1 requested changes Dec 24, 2025
View reviewed changes
ankitaluthra1
ankitaluthra1 reviewed Dec 24, 2025
View reviewed changes
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 24, 2025

/gcbrun

Mahalaxmibejugam
Mahalaxmibejugam reviewed Dec 24, 2025
View reviewed changes
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 24, 2025

/gcbrun

@Mahalaxmibejugam
Update assertions in mv tests
ec9aa91 
touch method's side effect shouldn't be counted in the assertion of the rename test
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 24, 2025

/gcbrun

1 similar comment
@ankitaluthra1
Copy link
Collaborator

ankitaluthra1 commented Dec 26, 2025

/gcbrun

@ankitaluthra1 ankitaluthra1 merged commit c5ac0e3 into fsspec:main Dec 26, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ankitaluthra1 ankitaluthra1 ankitaluthra1 requested changes

+1 more reviewer

@Mahalaxmibejugam Mahalaxmibejugam Mahalaxmibejugam left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@suni72 @ankitaluthra1 @Mahalaxmibejugam