[GLACIER | NSFS | NC] Replace GLACIER_DA with DEEP_ARCHIVE #9278
Conversation
WalkthroughRemoved the pre-redirect storage-class override call and deleted the override implementation; renamed STORAGE_CLASS_GLACIER_DA → STORAGE_CLASS_DEEP_ARCHIVE and introduced GLACIER_STORAGE_CLASSES; updated parsing, runtime checks, and TypeScript storage-class type to include DEEP_ARCHIVE and treat multiple glacier classes as glacier. Changes
Sequence Diagram(s)mermaid rect rect mermaid note over Component: Old: storage_class === STORAGE_CLASS_GLACIER note right of Component: This broadens which classes are treated as "glacier" for restore/expiry logic Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/endpoint/s3/s3_utils.js (1)
24-24: Consider renaming the constant to match its new value.The constant
STORAGE_CLASS_GLACIER_DAnow holds the value'DEEP_ARCHIVE', creating a semantic mismatch. While this maintains export compatibility, it may confuse developers.Consider this refactor to improve clarity:
-const STORAGE_CLASS_GLACIER_DA = 'DEEP_ARCHIVE'; // "S3 Deep Archive Storage Class" +const STORAGE_CLASS_DEEP_ARCHIVE = 'DEEP_ARCHIVE'; // "S3 Deep Archive Storage Class"And update the export on line 832:
-exports.STORAGE_CLASS_GLACIER_DA = STORAGE_CLASS_GLACIER_DA; +exports.STORAGE_CLASS_DEEP_ARCHIVE = STORAGE_CLASS_DEEP_ARCHIVE;Alternatively, if maintaining the old export name is critical for API compatibility, you could keep both:
+const STORAGE_CLASS_DEEP_ARCHIVE = 'DEEP_ARCHIVE'; // "S3 Deep Archive Storage Class" -const STORAGE_CLASS_GLACIER_DA = 'DEEP_ARCHIVE'; // "S3 Deep Archive Storage Class" +const STORAGE_CLASS_GLACIER_DA = STORAGE_CLASS_DEEP_ARCHIVE; // Deprecated, use STORAGE_CLASS_DEEP_ARCHIVE
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
src/endpoint/s3/s3_rest.js(0 hunks)src/endpoint/s3/s3_utils.js(2 hunks)src/sdk/nb.d.ts(1 hunks)
💤 Files with no reviewable changes (1)
- src/endpoint/s3/s3_rest.js
🧰 Additional context used
🧬 Code graph analysis (1)
src/endpoint/s3/s3_utils.js (3)
src/endpoint/s3/ops/s3_put_object.js (1)
storage_class(23-23)src/endpoint/s3/ops/s3_post_object_uploads.js (1)
storage_class(16-16)config.js (1)
config(7-7)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build Noobaa Image
- GitHub Check: run-package-lock-validation
- GitHub Check: run-jest-unit-tests
🔇 Additional comments (3)
src/sdk/nb.d.ts (1)
20-20: Type definition updated to reflect DEEP_ARCHIVE storage class.The change aligns with the runtime constant value update in
s3_utils.js(line 24). TypeScript will now correctly validate 'DEEP_ARCHIVE' as a valid StorageClass value.src/endpoint/s3/s3_utils.js (2)
384-387: Inline override logic correctly implements the new mechanism.The override logic has been successfully moved inline from the removed
override_storage_classfunction. WhenNSFS_GLACIER_FORCE_STORAGE_CLASSis enabled and storage class is missing or STANDARD, the function now returns GLACIER, achieving the PR objective of not overwriting the storage class HTTP header (by handling it during parsing instead).
383-393: ****The review comment incorrectly assumes a breaking change that does not exist. The storage_class_enum schema only includes ['STANDARD', 'GLACIER', 'GLACIER_IR', 'Glacier', 'DEEP_ARCHIVE', 'INTELLIGENT_TIERING', 'ONEZONE_IA', 'STANDARD_IA']—'GLACIER_DA' was never a valid stored value. STORAGE_CLASS_GLACIER_DA has always been defined as 'DEEP_ARCHIVE', and no 'GLACIER_DA' string literals exist in the codebase. The parse_storage_class function correctly validates inputs against the defined constants and rejects unknown values. No data migration or backward compatibility handling is required.
Likely an incorrect or invalid review comment.
Signed-off-by: Utkarsh Srivastava <srivastavautkarsh8097@gmail.com> drop support for GLACIER_IR and fix migration and restore path Signed-off-by: Utkarsh Srivastava <srivastavautkarsh8097@gmail.com>
tangledbytes
force-pushed
the
utkarsh/fix/glacier_da-storage-class
branch
from
d58d998 to
65c6c42
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
src/endpoint/s3/ops/s3_get_object.js (1)
44-50: Broadened Glacier gate via GLACIER_STORAGE_CLASSES.Good change. Minor hardening: normalize via parse_storage_class to respect force flags uniformly.
- if (s3_utils.GLACIER_STORAGE_CLASSES.includes(object_md.storage_class)) { + if (s3_utils.GLACIER_STORAGE_CLASSES.includes( + s3_utils.parse_storage_class(object_md.storage_class) + )) {src/sdk/glacier.js (1)
333-335: Gate now covers all Glacier-like classes; fix stale comment.Logic LGTM. Please update the note to “Returns undefined if storage_class is not in GLACIER_STORAGE_CLASSES.”
- * NOTE: Returns undefined if `user.storage_class` attribute is not - * `GLACIER` + * NOTE: Returns undefined if storage_class is not one of GLACIER_STORAGE_CLASSESAlso applies to: 342-345
src/sdk/namespace_fs.js (1)
1366-1369: Auto-enqueue migrate WAL when Glacier-like class is uploaded.Looks right. Consider guarding with NSFS_GLACIER_LOGS_ENABLED check here (optional; currently handled inside append_to_migrate_wal).
src/endpoint/s3/s3_utils.js (2)
24-25: Introduce DEEP_ARCHIVE and GLACIER_STORAGE_CLASSES.Good centralization. Consider Object.freeze(GLACIER_STORAGE_CLASSES) to prevent accidental mutation.
-const GLACIER_STORAGE_CLASSES = [ +const GLACIER_STORAGE_CLASSES = Object.freeze([ STORAGE_CLASS_GLACIER, STORAGE_CLASS_DEEP_ARCHIVE, -]; +]);Also applies to: 47-52
389-397: parse_storage_class: normalize input for robustness.Optional: trim/uppercase header before comparisons to avoid case/space pitfalls.
-function parse_storage_class(storage_class) { +function parse_storage_class(storage_class) { + if (typeof storage_class === 'string') storage_class = storage_class.trim(); + // AWS sends uppercase; normalizing defensively + if (typeof storage_class === 'string') storage_class = storage_class.toUpperCase();
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
src/endpoint/s3/ops/s3_get_object.js(1 hunks)src/endpoint/s3/s3_rest.js(0 hunks)src/endpoint/s3/s3_utils.js(5 hunks)src/sdk/glacier.js(1 hunks)src/sdk/namespace_fs.js(7 hunks)src/sdk/nb.d.ts(1 hunks)
💤 Files with no reviewable changes (1)
- src/endpoint/s3/s3_rest.js
🧰 Additional context used
🧬 Code graph analysis (2)
src/endpoint/s3/s3_utils.js (2)
src/endpoint/s3/ops/s3_put_object.js (1)
storage_class(23-23)src/endpoint/s3/ops/s3_post_object_uploads.js (1)
storage_class(16-16)
src/sdk/namespace_fs.js (2)
src/test/unit_tests/nsfs/test_nsfs_glacier_backend.js (4)
s3_utils(13-13)params(223-230)params(539-546)params(570-578)src/endpoint/s3/s3_utils.js (1)
storage_class(322-322)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Build Noobaa Image
- GitHub Check: run-jest-unit-tests
- GitHub Check: run-package-lock-validation
🔇 Additional comments (7)
src/sdk/namespace_fs.js (5)
1080-1086: Read gate aligned with GLACIER_STORAGE_CLASSES.Correct and consistent with restore semantics.
1310-1321: Copy-from Glacier objects requires restore and forces fallback.Good parity with S3 behavior. No issues.
2300-2301: Restore reply now echoes actual storage_class from xattr.Good improvement for accuracy.
3616-3617: Force-expire-on-get covers Glacier-like classes.Matches new model. LGTM.
3544-3547: GLACIER_IR exclusion from NSFS is intentional by design.The
GLACIER_STORAGE_CLASSESconstant is specifically defined for NSFS glacier support and contains onlySTORAGE_CLASS_GLACIERandSTORAGE_CLASS_DEEP_ARCHIVE—STORAGE_CLASS_GLACIER_IRis intentionally excluded. This is consistent across the codebase: inbucketspace_fs.js, onlySTORAGE_CLASS_GLACIER(notGLACIER_IRorDEEP_ARCHIVE) is advertised as a supported storage class whenNSFS_GLACIER_ENABLEDis configured. The exclusion is not an oversight but a deliberate design choice limiting NSFS glacier support to specific storage classes.src/endpoint/s3/s3_utils.js (1)
838-838: Verify documentation updates for new public API exports.The exports of
STORAGE_CLASS_DEEP_ARCHIVEandGLACIER_STORAGE_CLASSESat lines 838 and 882 expose new public API surface. These constants are actively used internally by SDK modules (glacier.js, namespace_fs.js, s3_get_object.js). Please confirm that:
- Any relevant developer or SDK documentation has been updated to document these exports
- If external SDK consumers depend on these constants, they have been notified
src/sdk/nb.d.ts (1)
20-20: StorageClass refactoring verified — no issues found.The type definition and s3_utils constants are correctly aligned:
- StorageClass union includes all four classes: STANDARD, GLACIER, GLACIER_IR, DEEP_ARCHIVE
- s3_utils exports STORAGE_CLASS_DEEP_ARCHIVE with proper JSDoc type annotation
- GLACIER_STORAGE_CLASSES array correctly includes both GLACIER and DEEP_ARCHIVE
- Zero GLACIER_DA references remain in the codebase
- Type consistency maintained throughout
| const OBJECT_ATTRIBUTES_UNSUPPORTED = Object.freeze(['Checksum', 'ObjectParts']); | ||
|
|
||
| /** @type {nb.StorageClass[]} */ | ||
| const GLACIER_STORAGE_CLASSES = [ |
There was a problem hiding this comment.
Just a small comment worth added here I think - that this list is the classes that require restore-object, which is why GLACIER_IR is not included.
| if (!storage_class) return STORAGE_CLASS_STANDARD; | ||
| if (storage_class === STORAGE_CLASS_STANDARD) return STORAGE_CLASS_STANDARD; | ||
| if (!storage_class || storage_class === STORAGE_CLASS_STANDARD) { | ||
| if (config.NSFS_GLACIER_FORCE_STORAGE_CLASS) return STORAGE_CLASS_GLACIER; |
There was a problem hiding this comment.
I think we can get more flexibility if the config specifies the forced storage class name (string), and then this function can be written to validate it too
function parse_storage_class(storage_class) {
if (config.NSFS_GLACIER_FORCE_STORAGE_CLASS) {
storage_class = config.NSFS_GLACIER_FORCE_STORAGE_CLASS;
}
if (!storage_class) return STORAGE_CLASS_STANDARD;
if (storage_class === STORAGE_CLASS_STANDARD) return STORAGE_CLASS_STANDARD;
if (storage_class === STORAGE_CLASS_GLACIER) return STORAGE_CLASS_GLACIER;
if (storage_class === STORAGE_CLASS_DEEP_ARCHIVE) return STORAGE_CLASS_DEEP_ARCHIVE;
if (storage_class === STORAGE_CLASS_GLACIER_IR) return STORAGE_CLASS_GLACIER_IR;
throw new Error(`No such s3 storage class ${storage_class}`);
}2 participants
Describe the Problem
This PR replaces GLACIER_DA with DEEP_ARCHIVE and changes the way we override the storage class without overwriting the storage class HTTP header.
Issues: Fixed #xxx / Gap #xxx
Testing Instructions:
Summary by CodeRabbit
Bug Fixes
Refactor
SDK