Skip to content

Perform WAL checkpointing ten times per second #89

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
OpsBotPrime merged 2 commits into from
Apr 1, 2026
Merged

Perform WAL checkpointing ten times per second #89

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3e99a4c
Perform WAL checkpointing ten times per second
Qqwy Apr 1, 2026
3104526
Merge #89: Perform WAL checkpointing ten times per second
OpsBotPrime Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ members = [
]

[workspace.package]
version = "0.34.3"
version = "0.34.4"


[workspace.lints.clippy]
Expand Down
2 changes: 1 addition & 1 deletion opsqueue/src/db/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ impl DBPools {
}

pub async fn periodically_checkpoint_wal(&self) {
const EXPLICIT_WAL_CHECK_INTERVAL: Duration = Duration::from_secs(1);
const EXPLICIT_WAL_CHECK_INTERVAL: Duration = Duration::from_millis(100);
let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL);
Copy link
Copy Markdown

@bertptrs bertptrs Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want to set the missed tick strategy, otherwise this will repeatedly issue checkpoints whenever the checkpoints takes too long. This will be exacerbated by the interval being shortened.

Suggested change
let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL);
let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL);
interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);

The default behaviour is "burst", i.e. fire all missed ticks right now. This is not what we want here. We don't actually rely on the number of checkpoints, we just want to fire them regularly.

Copy link
Copy Markdown
Contributor Author

@Qqwy Qqwy Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing a checkpoint when there is (almost) nothing to checkpoint is (almost) free, though it will of course add to the congestion on the one writer DB connection.

Changing to Skip is a nice next step. I'll also add explicit checkpoints to the two write-heavy points of the app, in a follow-up PR.

loop {
let _ = self.perform_explicit_wal_checkpoint().await;
Expand Down
Loading