-
Notifications
You must be signed in to change notification settings - Fork 431
refactor: use versioned index for index file #7309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: discord9 <discord9@163.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR refactors the index file system from using separate index_file_id fields to using versioned indexes with an index_version field. The main change replaces index_file_id: Option<String> with index_version: u64 in ManifestSstEntry, and introduces a new RegionIndexId type that combines file ID with version number.
Key Changes
- Replaced
index_file_id: Option<FileId>withindex_version: u64inFileMetaandManifestSstEntry - Added
RegionIndexIdstruct combiningRegionFileIdand version - Updated index file path generation to support versioned filenames:
{file_id}.{version}.puffin - Modified
FileType::Puffinto include version field
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/store-api/src/sst_entry.rs | Changed ManifestSstEntry to use index_version instead of index_file_id |
| src/mito2/src/sst/file.rs | Added RegionIndexId struct, updated FileMeta and FileHandle methods |
| src/mito2/src/sst/location.rs | Added versioned index file path functions and parsing logic |
| src/mito2/src/cache/file_cache.rs | Updated FileType::Puffin to include version field |
| src/mito2/src/sst/index.rs | Added index_version field to Indexer, updated IndexOutput |
| src/mito2/src/region/opener.rs | Updated cache loading to use versioned index keys |
| src/mito2/src/gc.rs | Updated GC worker (with known limitation for version support) |
| src/mito2/src/access_layer.rs | Added build_index_file_path_with_version to FilePathProvider trait |
| Multiple test files | Updated test utilities and expected values for new field |
| Multiple index files | Updated index appliers and creators to use RegionIndexId |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
…ete file Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
as title, use versioned index for index file, so rebuild index file can be easier to track
PR Checklist
Please convert it to a draft if some of the following conditions are not met.