Skip to content

perf(files_external/S3): slightly optimize memory in opendir #58681

Draft
joshtrichards wants to merge 3 commits intomasterfrom
jtr/refactor-external-s3-opendir
Draft

perf(files_external/S3): slightly optimize memory in opendir #58681
joshtrichards wants to merge 3 commits intomasterfrom
jtr/refactor-external-s3-opendir

Conversation

@joshtrichards
Copy link
Member

@joshtrichards joshtrichards commented Mar 3, 2026

  • Resolves: #

Summary

  • A small optimization from only carrying forward the key we care about into the final array.
  • Also: slightly better variable naming and exception logging (rather than silently swallowing every S3Exception)

TODO

  • A much bigger win (in terms of memory/CPU usage for larger directories) would be a rewind-safe iterator.

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@joshtrichards joshtrichards marked this pull request as ready for review March 3, 2026 03:04
@joshtrichards joshtrichards requested a review from a team as a code owner March 3, 2026 03:04
@joshtrichards joshtrichards requested review from Altahrim, icewind1991, leftybournes and salmart-dev and removed request for a team March 3, 2026 03:04
@joshtrichards joshtrichards added 2. developing Work in progress and removed 3. to review Waiting for reviews labels Mar 3, 2026
@joshtrichards joshtrichards marked this pull request as draft March 3, 2026 10:18
Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards joshtrichards force-pushed the jtr/refactor-external-s3-opendir branch from 9b1cd65 to bd6cb66 Compare March 3, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant