downloader: add torrents from disk after snapshot sync stage#19712
Merged
AskAlexSharov merged 8 commits intomainfrom Mar 16, 2026
Merged
downloader: add torrents from disk after snapshot sync stage#19712AskAlexSharov merged 8 commits intomainfrom
AskAlexSharov merged 8 commits intomainfrom
Conversation
Previously AddTorrentsFromDisk was called during initDownloader, before any sync had occurred. This caused false-positive incomplete torrent reports for snapshots that the sync stage hadn't yet completed. Move the call into an afterSnapshotDownload callback that fires once the snapshot sync stage completes on the first cycle. Thread the callback through StageSnapshotsCfg, NewDefaultStages, and NewPipelineStages with a nil default so existing callers are unaffected. Incomplete torrent warnings emitted by the callback are now annotated as unexpected (sync just finished), pointing to the unresolved preverified- set edge case as the likely cause.
Contributor
Author
|
Manually dispatched QA - Snapshot Download on this branch to exercise the scenario where snapshots are already present on disk (e.g. after an upgrade/downgrade), which is the primary case the new |
AskAlexSharov
approved these changes
Mar 16, 2026
lupin012
pushed a commit
that referenced
this pull request
Mar 17, 2026
## Summary - Moves `AddTorrentsFromDisk` call from `initDownloader` (at startup, before any sync) to an `afterSnapshotDownload` callback invoked once the snapshot sync stage completes on the first sync cycle. - This eliminates false-positive incomplete torrent reports that occurred because we were registering on-disk snapshots before the sync stage had a chance to complete them. - The `afterDownload` callback is threaded through `StageSnapshotsCfg`, `NewDefaultStages`, and `NewPipelineStages` with a `nil` default so callers that don't need it are unaffected. - Incomplete snapshot warnings after this callback are now flagged as unexpected (sync just finished), with a comment noting the preverified-set edge case that remains unresolved. ## Background Issue #19435 first identified this problem: `AddTorrentsFromDisk` was being called before initial sync status was checked, causing old or stale `.torrent` files on disk to interfere with the sync stage's ability to fetch the correct (preverified) versions of snapshots. PR #19316 by @Giulio2002 addressed a related symptom — converting the fatal "snapshot already loaded with different infohash" error in `addPreverifiedSnapshotForDownload` into a warning — but this was treating a symptom rather than the root cause. Other related changes to this area: - PR #18056 — "Final downloader cleanup": Changed `AddTorrentsFromDisk` to only add completed snapshots (anything incomplete is assumed a mistake), and removed it as a default at startup. - PR #18270 — "Quick fix for --downloader.verify assertion": Fixed a panic in `addCompleteTorrentFromMetainfo` when called from `AddTorrentsFromDisk` on already-completed files (#18165). - PR #15043 — "Fix a collection of downloader, snapshot sync and torrent related issues": Earlier broad fix addressing how disk-loaded torrents interact with preverified snapshots. - PR #16655 — "Require torrents added from disk to complete on `downloader.verify`": Enforced completion check for disk-loaded torrents during verify. @mh0lt — please take a look, particularly at the scenario where extra snapshots land on disk that weren't part of the sync set (e.g. after an upgrade or downgrade that changes the preverified set). --------- Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AddTorrentsFromDiskcall frominitDownloader(at startup, before any sync) to anafterSnapshotDownloadcallback invoked once the snapshot sync stage completes on the first sync cycle.afterDownloadcallback is threaded throughStageSnapshotsCfg,NewDefaultStages, andNewPipelineStageswith anildefault so callers that don't need it are unaffected.Background
Issue #19435 first identified this problem:
AddTorrentsFromDiskwas being called before initial sync status was checked, causing old or stale.torrentfiles on disk to interfere with the sync stage's ability to fetch the correct (preverified) versions of snapshots.PR #19316 by @Giulio2002 addressed a related symptom — converting the fatal "snapshot already loaded with different infohash" error in
addPreverifiedSnapshotForDownloadinto a warning — but this was treating a symptom rather than the root cause.Other related changes to this area:
AddTorrentsFromDiskto only add completed snapshots (anything incomplete is assumed a mistake), and removed it as a default at startup.addCompleteTorrentFromMetainfowhen called fromAddTorrentsFromDiskon already-completed files (--downloader.verify asserts for already completed files #18165).downloader.verify#16655 — "Require torrents added from disk to complete ondownloader.verify": Enforced completion check for disk-loaded torrents during verify.@mh0lt — please take a look, particularly at the scenario where extra snapshots land on disk that weren't part of the sync set (e.g. after an upgrade or downgrade that changes the preverified set).