Skip to content

downloader: add torrents from disk after snapshot sync stage#19712

Merged
AskAlexSharov merged 8 commits intomainfrom
anacrolix/add-torrents-to-disk-after-sync
Mar 16, 2026
Merged

downloader: add torrents from disk after snapshot sync stage#19712
AskAlexSharov merged 8 commits intomainfrom
anacrolix/add-torrents-to-disk-after-sync

Conversation

@anacrolix
Copy link
Copy Markdown
Contributor

@anacrolix anacrolix commented Mar 7, 2026

Summary

  • Moves AddTorrentsFromDisk call from initDownloader (at startup, before any sync) to an afterSnapshotDownload callback invoked once the snapshot sync stage completes on the first sync cycle.
  • This eliminates false-positive incomplete torrent reports that occurred because we were registering on-disk snapshots before the sync stage had a chance to complete them.
  • The afterDownload callback is threaded through StageSnapshotsCfg, NewDefaultStages, and NewPipelineStages with a nil default so callers that don't need it are unaffected.
  • Incomplete snapshot warnings after this callback are now flagged as unexpected (sync just finished), with a comment noting the preverified-set edge case that remains unresolved.

Background

Issue #19435 first identified this problem: AddTorrentsFromDisk was being called before initial sync status was checked, causing old or stale .torrent files on disk to interfere with the sync stage's ability to fetch the correct (preverified) versions of snapshots.

PR #19316 by @Giulio2002 addressed a related symptom — converting the fatal "snapshot already loaded with different infohash" error in addPreverifiedSnapshotForDownload into a warning — but this was treating a symptom rather than the root cause.

Other related changes to this area:

@mh0lt — please take a look, particularly at the scenario where extra snapshots land on disk that weren't part of the sync set (e.g. after an upgrade or downgrade that changes the preverified set).

Previously AddTorrentsFromDisk was called during initDownloader, before
any sync had occurred. This caused false-positive incomplete torrent
reports for snapshots that the sync stage hadn't yet completed.

Move the call into an afterSnapshotDownload callback that fires once the
snapshot sync stage completes on the first cycle. Thread the callback
through StageSnapshotsCfg, NewDefaultStages, and NewPipelineStages with
a nil default so existing callers are unaffected.

Incomplete torrent warnings emitted by the callback are now annotated as
unexpected (sync just finished), pointing to the unresolved preverified-
set edge case as the likely cause.
@anacrolix
Copy link
Copy Markdown
Contributor Author

Manually dispatched QA - Snapshot Download on this branch to exercise the scenario where snapshots are already present on disk (e.g. after an upgrade/downgrade), which is the primary case the new afterSnapshotDownload callback is meant to handle.

@yperbasis yperbasis requested a review from sudeepdino008 March 11, 2026 09:36
@AskAlexSharov AskAlexSharov enabled auto-merge (squash) March 16, 2026 03:12
@AskAlexSharov AskAlexSharov merged commit 3ca13e0 into main Mar 16, 2026
37 checks passed
@AskAlexSharov AskAlexSharov deleted the anacrolix/add-torrents-to-disk-after-sync branch March 16, 2026 03:45
lupin012 pushed a commit that referenced this pull request Mar 17, 2026
## Summary

- Moves `AddTorrentsFromDisk` call from `initDownloader` (at startup,
before any sync) to an `afterSnapshotDownload` callback invoked once the
snapshot sync stage completes on the first sync cycle.
- This eliminates false-positive incomplete torrent reports that
occurred because we were registering on-disk snapshots before the sync
stage had a chance to complete them.
- The `afterDownload` callback is threaded through `StageSnapshotsCfg`,
`NewDefaultStages`, and `NewPipelineStages` with a `nil` default so
callers that don't need it are unaffected.
- Incomplete snapshot warnings after this callback are now flagged as
unexpected (sync just finished), with a comment noting the
preverified-set edge case that remains unresolved.

## Background

Issue #19435 first identified this problem: `AddTorrentsFromDisk` was
being called before initial sync status was checked, causing old or
stale `.torrent` files on disk to interfere with the sync stage's
ability to fetch the correct (preverified) versions of snapshots.

PR #19316 by @Giulio2002 addressed a related symptom — converting the
fatal "snapshot already loaded with different infohash" error in
`addPreverifiedSnapshotForDownload` into a warning — but this was
treating a symptom rather than the root cause.

Other related changes to this area:
- PR #18056 — "Final downloader cleanup": Changed `AddTorrentsFromDisk`
to only add completed snapshots (anything incomplete is assumed a
mistake), and removed it as a default at startup.
- PR #18270 — "Quick fix for --downloader.verify assertion": Fixed a
panic in `addCompleteTorrentFromMetainfo` when called from
`AddTorrentsFromDisk` on already-completed files (#18165).
- PR #15043 — "Fix a collection of downloader, snapshot sync and torrent
related issues": Earlier broad fix addressing how disk-loaded torrents
interact with preverified snapshots.
- PR #16655 — "Require torrents added from disk to complete on
`downloader.verify`": Enforced completion check for disk-loaded torrents
during verify.

@mh0lt — please take a look, particularly at the scenario where extra
snapshots land on disk that weren't part of the sync set (e.g. after an
upgrade or downgrade that changes the preverified set).

---------

Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants