Perform WAL checkpointing ten times per second#89
Conversation
ReinierMaas
left a comment
ReinierMaas
left a comment
There was a problem hiding this comment.
🚀 Let's get the WAL size under control.
|
@OpsBotPrime merge and tag |
Approved-by: Qqwy Priority: Normal Auto-deploy: false
|
Rebased as 3104526, waiting for CI … |
|
CI job 🟡 started. |
| pub async fn periodically_checkpoint_wal(&self) { | ||
| const EXPLICIT_WAL_CHECK_INTERVAL: Duration = Duration::from_secs(1); | ||
| const EXPLICIT_WAL_CHECK_INTERVAL: Duration = Duration::from_millis(100); | ||
| let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL); |
There was a problem hiding this comment.
I think you want to set the missed tick strategy, otherwise this will repeatedly issue checkpoints whenever the checkpoints takes too long. This will be exacerbated by the interval being shortened.
| let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL); | |
| let mut interval = tokio::time::interval(EXPLICIT_WAL_CHECK_INTERVAL); | |
| interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip); |
The default behaviour is "burst", i.e. fire all missed ticks right now. This is not what we want here. We don't actually rely on the number of checkpoints, we just want to fire them regularly.
There was a problem hiding this comment.
Doing a checkpoint when there is (almost) nothing to checkpoint is (almost) free, though it will of course add to the congestion on the one writer DB connection.
Changing to Skip is a nice next step. I'll also add explicit checkpoints to the two write-heavy points of the app, in a follow-up PR.
4 participants
Before: once per second
With this PR: ten times per second.