Skip to content

Drop Storage abstraction, revise fs cleanup logic for round reset#361

Merged
apruden2008 merged 8 commits intomasterfrom
simplify_storage
Sep 9, 2021
Merged

Drop Storage abstraction, revise fs cleanup logic for round reset#361
apruden2008 merged 8 commits intomasterfrom
simplify_storage

Conversation

@jules
Copy link
Copy Markdown

@jules jules commented Aug 24, 2021

Fixes #360

@ibaryshnikov
Copy link
Copy Markdown

Since it also touches a lot of files, I would prefer to not merge the storage refactoring before we merge #363, otherwise it may eliminate the work I've put into fixing tests.
On the other hand, it would be nice to have the round reset as a standalone tool, with an option to check the round integrity. With such a tool we would be able to stop the coordinator, do the reset, and start the coordinator again.

self.clear_dir_files(round_dir.into(), false)
}

fn clear_dir_files(&mut self, path: PathBuf, delete_initial_contribution: bool) -> Result<()> {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this approach. Resetting the round looks like a well defined problem after checking the folder structure.

}

impl Storage for Disk {
impl Disk {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this be in the same impl block as the one below?


impl Storage for Disk {
impl Disk {
pub fn clear_round_files(&mut self, current_round_height: u64) -> Result<()> {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't necessarily have to be the current round? could perhaps just be round_height?

/// the [crate::storage::Storage] to reflect the changes to the
/// round state. `remove_participants` is a list of participants
/// to remove from the round.
pub(crate) fn reset(&mut self, remove_participants: &[Participant]) -> Vec<StorageAction> {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice to see lots of code getting deleted :) despite having worked so hard on this!

self.storage.process(round.reset(&reset_action.remove_participants))?;

// Clear all files
self.storage.clear_round_files(current_round_height)?;
Copy link
Copy Markdown

@kellpossible kellpossible Aug 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to represent clear_round_files as a storage action, returned by round.reset() instead of calling it directly here. I think it better fits with the existing pattern and improves testability.

/// Returns the size of the object stored at the given locator.
fn size(&self, locator: &Locator) -> Result<u64, CoordinatorError>;

/// Process a [StorageAction] which mutates the storage.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that this description is missing from the process() implementation in Disk, if any other of these doc comments are missing, would be good to copy them across

@kellpossible
Copy link
Copy Markdown

@julesdesmit I guess you're good to merge this one?

@apruden2008 apruden2008 merged commit 30d459d into master Sep 9, 2021
@apruden2008 apruden2008 deleted the simplify_storage branch September 9, 2021 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drop Storage abstraction, and work directly with directories on the filesystem

4 participants