-
-
Notifications
You must be signed in to change notification settings - Fork 71
Add nohardlinks category directory ignore option #865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
<!--pre-commit.ci start--> updates: - [github.com/astral-sh/ruff-pre-commit: v0.11.13 → v0.12.2](astral-sh/ruff-pre-commit@v0.11.13...v0.12.2) <!--pre-commit.ci end--> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds an option to ignore hardlinks within the same category directory, providing more granular control over hardlink detection. This is ideal for preventing configuring share limits for cross-seeds to be removed if and only if their source files have been removed. The option defaults to True for existing categories and can be configured in the config.yml file.
for more information, see https://pre-commit.ci
Currently |
|
Closing as |
@bobokun I'm afraid that's not correct. Take my setup, described below, as an example.
With the above setup, qbit_manage is able to avoid marking This option is required for my use case because But what if I want to keep it seeding (from This is where For additional context, I have included my config below. Of note in particular is the # This is an example configuration file that documents all the options.
# It will need to be modified for your specific use case.
# Please refer to the link below for more details on how to set up the configuration file
# https://github.com/StuffAnThings/qbit_manage/wiki/Config-Setup
commands:
# The commands defined below will IGNORE any commands used in command line and docker env variables.
dry_run: false
recheck: false
cat_update: false
tag_update: true
rem_unregistered: false
tag_tracker_error: false
rem_orphaned: false
tag_nohardlinks: true
share_limits: true
skip_qb_version_check: true
skip_cleanup: false
qbt:
# qBittorrent parameters
host: [REDACTED]
user: [REDACTED]
pass: [REDACTED]
settings:
force_auto_tmm: false # Will force qBittorrent to enable Automatic Torrent Management for each torrent.
force_auto_tmm_ignore_tags: #Torrents with these tags will be ignored when force_auto_tmm is enabled.
- Upload
tracker_error_tag: issue # Will set the tag of any torrents that do not have a working tracker.
nohardlinks_tag: noHL # Will set the tag of any torrents with no hardlinks.
share_limits_tag: ~share_limit # Will add this tag when applying share limits to provide an easy way to filter torrents by share limit group/priority for each torrent
share_limits_min_seeding_time_tag: MinSeedTimeNotReached # Tag to be added to torrents that have not yet reached the minimum seeding time
share_limits_min_num_seeds_tag: MinSeedsNotMet # Tag to be added to torrents that have not yet reached the minimum number of seeds
share_limits_last_active_tag: LastActiveLimitNotReached # Tag to be added to torrents that have not yet reached the last active limit
cat_filter_completed: true # Filters for completed torrents only when running cat_update command
share_limits_filter_completed: true # Filters for completed torrents only when running share_limits command
tag_nohardlinks_filter_completed: true # Filters for completed torrents only when running tag_nohardlinks command
cat_update_all: true # Checks and updates all torrent categories if set to True when running cat_update command, otherwise only update torrents that are uncategorized
disable_qbt_default_share_limits: true # Allows QBM to handle share limits by disabling qBittorrents default Share limits. Only active when the share_limits command is set to True
rem_unregistered_filter_completed: false
tag_stalled_torrents: true
rem_unregistered_ignore_list: []
stalled_tag: stalledDL
directory:
# Do not remove these
# root_dir var: </your/path/here/> # Root downloads directory used to check for orphaned files, noHL, and RecycleBin.
# <OPTIONAL> remote_dir var: </your/path/here/> # Path of docker host mapping of root_dir.
# remote_dir must be set if you're running qbit_manage locally and qBittorrent/cross_seed is in a docker
# remote_dir should not be set if qbit_manage is running in a container
# <OPTIONAL> recycle_bin var: </your/path/here/> # Path of the RecycleBin folder. Default location is set to remote_dir/.RecycleBin
# <OPTIONAL> torrents_dir var: </your/path/here/> # Path of the your qbittorrent torrents directory. Required for `save_torrents` attribute in recyclebin
# <OPTIONAL> orphaned_dir var: </your/path/here/> # Path of the the Orphaned Data folder. This is similar to RecycleBin, but only for orphaned data.
root_dir: D:\Downloads
remote_dir:
recycle_bin:
torrents_dir:
orphaned_dir:
cat:
# Category & Path Parameters
# All save paths in qbittorent must be populated below.
# If you want to leave a save_path as uncategorized you can use the key 'Uncategorized' as the name of the category.
# <Category Name> : <save_path> # Path of your save directory.
Anime: D:\Downloads\anime
Apps: D:\Downloads\apps
Movies: D:\Downloads\movies
TV: D:\Downloads\tv
XSeed: D:\Downloads\xseed
Prowlarr: D:\Downloads\prowlarr
Radarr: D:\Downloads\radarr
Sonarr: D:\Downloads\sonarr
Web: D:\Downloads\web
cat_change:
# This moves all the torrents from one category to another category. This executes on --cat-update
# WARNING: if the paths are different and Default Torrent Management Mode is set to automatic the files could be moved !!!
# <Old Category Name> : <New Category>
tracker:
# Mandatory
# Tag Parameters
# <Tracker URL Keyword>: # <MANDATORY> This is the keyword in the tracker url. You can define multiple tracker urls by splitting with `|` delimiter
# <MANDATORY> Set tag name. Can be a list of tags or a single tag
# tag: <Tag Name>
# <OPTIONAL> Set the category based on tracker URL. This category option takes priority over the category defined by save directory
# cat: <Category Name>
# <OPTIONAL> Set this to the notifiarr react name. This is used to add indexer reactions to the notifications sent by Notifiarr
# notifiarr: <notifiarr indexer>
animebytes.tv:
tag: AnimeBytes
animetorrents:
tag: AnimeTorrents
avistaz:
tag: Avistaz
notifiarr: avistaz
beyond-hd:
tag: Beyond-HD
cat: Movies
notifiarr: beyondhd
bgp|empirehost|stackoverflow:
tag: IPTorrents
blutopia:
tag: Blutopia
notifiarr: blutopia
cartoonchaos:
tag: CartoonChaos
cinemaz:
tag: CinemaZ
digitalcore:
tag: DigitalCore
notifiarr: digitalcore
gazellegames:
tag: GGn
hawke:
tag: Hawke-Uno
hdts:
tag: HD-Torrents
landof.tv:
tag: BroadcasTheNet
notifiarr: broadcasthenet
lst:
tag: LST
milkie:
tag: Milkie
myanonamouse:
tag: MaM
passthepopcorn:
tag: PassThePopcorn
notifiarr: passthepopcorn
privatehd:
tag: PrivateHD
notifiarr:
seedpool:
tag: SeedPool
torrentdb:
tag: TorrentDB
notifiarr: torrentdb
torrentleech|tleechreload:
tag: TorrentLeech
notifiarr: torrentleech
tv-vault:
tag: TV-Vault
upload.cx:
tag: ULCX
yu-scene:
tag: YUSCENE
# The "other" key is a special keyword and if defined will tag any other trackers that don't match the above trackers into this tag
other:
tag: other
nohardlinks:
# Tag Movies/Series that are not hard linked outside the root directory
# Mandatory to fill out directory parameter above to use this function (root_dir/remote_dir)
# This variable should be set to your category name of your completed movies/completed series in qbit. Acceptable variable can be any category you would like to tag if there are no hardlinks found
Sonarr:
# <OPTIONAL> exclude_tags var: Will exclude torrents with any of the following tags when searching through the category.
exclude_tags:
- retain
# <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True).
ignore_root_dir: true
Radarr:
# <OPTIONAL> exclude_tags var: Will exclude torrents with any of the following tags when searching through the category.
exclude_tags:
- retain
# <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True).
ignore_root_dir: true
Prowlarr:
# <OPTIONAL> exclude_tags var: Will exclude torrents with any of the following tags when searching through the category.
exclude_tags:
- retain
# <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True).
ignore_root_dir: true
XSeed:
# <OPTIONAL> exclude_tags var: Will exclude torrents with any of the following tags when searching through the category.
exclude_tags:
- retain
# <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True).
ignore_root_dir: false
# <OPTIONAL> ignore_category_dir var: Will ignore any hardlinks detected in the same category save path (Default True).
ignore_category_dir: true
Movies:
exclude_tags:
- retain
ignore_root_dir: true
TV:
exclude_tags:
- retain
ignore_root_dir: true
Anime:
exclude_tags:
- retain
ignore_root_dir: true
share_limits:
# Control how torrent share limits are set depending on the priority of your grouping
# Each torrent will be matched with the share limit group with the highest priority that meets the group filter criteria.
# Each torrent can only be matched with one share limit group
# This variable is mandatory and is a text defining the name of your grouping. This can be any string you want
default:
# <MANDATORY> priority: <int/float> # This is the priority of your grouping. The lower the number the higher the priority
priority: 999
# <OPTIONAL> include_all_tags: <list> # Filter the group based on one or more tags. Multiple include_all_tags are checked with an AND condition
# All tags defined here must be present in the torrent for it to be included in this group
include_all_tags:
# <OPTIONAL> include_any_tags: <list> # Filter the group based on one or more tags. Multiple include_any_tags are checked with an OR condition
# Any tags defined here must be present in the torrent for it to be included in this group
include_any_tags:
# <OPTIONAL> exclude_all_tags: <list> # Filter by excluding one or more tags. Multiple exclude_all_tags are checked with an AND condition
# This is useful to combine with the category filter to exclude one or more tags from an entire category
# All tags defined here must be present in the torrent for it to be excluded in this group
exclude_all_tags:
# <OPTIONAL> exclude_any_tags: <list> # Filter by excluding one or more tags. Multiple exclude_any_tags are checked with an OR condition
# This is useful to combine with the category filter to exclude one or more tags from an entire category
# Any tags defined here must be present in the torrent for it to be excluded in this group
exclude_any_tags:
# <OPTIONAL> categories: <list> # Filter by including one or more categories. Multiple categories are checked with an OR condition
# Since one torrent can only be associated with a single category, multiple categories are checked with an OR condition
categories:
# <OPTIONAL> max_ratio <float>: Will set the torrent Maximum share ratio until torrent is stopped from seeding/uploading and may be cleaned up / removed if the minimums have been met.
# Will default to -1 (no limit) if not specified for the group.
max_ratio:
# <OPTIONAL> max_seeding_time <str>: Will set the torrent Maximum seeding time until torrent is stopped from seeding/uploading and may be cleaned up / removed if the minimums have been met.
# See Some examples of valid time expressions (https://github.com/onegreyonewhite/pytimeparse2)
# 32m, 2h32m, 3d2h32m, 1w3d2h32m
# Will default to -1 (no limit) if not specified for the group. (Max value of 1 year (525600 minutes))
max_seeding_time:
# <OPTIONAL> min_seeding_time <str>: Will prevent torrent deletion by cleanup variable if torrent has not yet minimum seeding time (minutes).
# This should only be set if you are using this in conjunction with max_seeding_time and max_ratio. If you are not setting a max_ratio, then use max_seeding_time instead.
# If the torrent has not yet reached this minimum seeding time, it will change the share limits back to no limits and resume the torrent to continue seeding.
# See Some examples of valid time expressions (https://github.com/onegreyonewhite/pytimeparse2)
# 32m, 2h32m, 3d2h32m, 1w3d2h32m
# Will default to 0 if not specified for the group.
min_seeding_time:
# <OPTIONAL> Limit Upload Speed <int>: Will limit the upload speed KiB/s (KiloBytes/second) (`-1` : No Limit)
limit_upload_speed:
# <OPTIONAL> Enable Group Upload Speed <bool>: Upload speed limits are applied at the group level. This will take limit_upload_speed defined and divide it equally among the number of torrents in the group.
enable_group_upload_speed:
# <OPTIONAL> cleanup <bool>: WARNING!! Setting this as true Will remove and delete contents of any torrents that satisfies the share limits (max time OR max ratio)
cleanup: false
# <OPTIONAL> resume_torrent_after_change <bool>: This variable will resume your torrent after changing share limits. Default is true
resume_torrent_after_change:
# <OPTIONAL> add_group_to_tag <bool>: This adds your grouping as a tag with a prefix defined in settings . Default is true
# Example: A grouping defined as noHL will have a tag set to ~share_limit.noHL (if using the default prefix)
add_group_to_tag:
# <OPTIONAL> min_num_seeds <int>: Will prevent torrent deletion by cleanup variable if the number of seeds is less than the value set here.
# If the torrent has less number of seeds than the min_num_seeds, the share limits will be changed back to no limits and resume the torrent to continue seeding.
# Will default to 0 if not specified for the group.
min_num_seeds:
# <OPTIONAL> custom_tag <str>: Apply a custom tag name for this particular group. **WARNING (This tag MUST be unique as it will be used to determine share limits. Please ensure it does not overlap with any other tags in qbt)**
custom_tag: seed
min_last_active:
noSeed:
priority: 1
include_all_tags:
- other
- noHL
exclude_all_tags:
- retain
max_ratio: 0
max_seeding_time: 0
custom_tag: noSeed
cleanup: true
xseed:
priority: 2
include_all_tags:
- noHL
exclude_all_tags:
- retain
categories:
- XSeed
max_ratio: 0
max_seeding_time: 0
custom_tag: xseed
cleanup: true
DigitalCore:
priority: 500
include_all_tags:
- DigitalCore
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 5d
cleanup: true
TorrentLeech:
priority: 501
include_all_tags:
- TorrentLeech
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 7d
cleanup: true
SeedPool:
priority: 502
include_all_tags:
- SeedPool
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_ratio: 0
max_seeding_time: 0
cleanup: true
Hawke-Uno:
priority: 503
include_all_tags:
- Hawke-Uno
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 5d
cleanup: true
IPTorrents:
priority: 504
include_all_tags:
- IPTorrents
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 14d
cleanup: true
LST:
priority: 505
include_all_tags:
- LST
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 3d
cleanup: true
YUSCENE:
priority: 506
include_all_tags:
- YUSCENE
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 5d
cleanup: true
ULCX:
priority: 507
include_all_tags:
- ULCX
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 2d
cleanup: true
CinemaZ:
priority: 508
include_all_tags:
- CinemaZ
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_seeding_time: 20d
cleanup: true
AnimeTorrents:
priority: 509
include_all_tags:
- AnimeTorrents
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_ratio: 1
cleanup: true
HD-Torrents:
priority: 510
include_all_tags:
- HD-Torrents
- noHL
exclude_all_tags:
- retain
- noHnR
categories:
- Movies
- TV
- Radarr
- Sonarr
- Prowlarr
max_ratio: 1
cleanup: true
recyclebin:
# Recycle Bin method of deletion will move files into the recycle bin (Located in /root_dir/.RecycleBin) instead of directly deleting them in qbit
# By default the Recycle Bin will be emptied on every run of the qbit_manage script if empty_after_x_days is defined.
enabled: false
# <OPTIONAL> empty_after_x_days var:
# Will automatically remove all files and folders in recycle bin after x days. (Checks every script run)
# If this variable is not defined it, the RecycleBin will never be emptied.
# WARNING: Setting this variable to 0 will delete all files immediately upon script run!
empty_after_x_days:
# <OPTIONAL> save_torrents var:
# If this option is set to true you MUST fill out the torrents_dir in the directory attribute.
# This will save a copy of your .torrent and .fastresume file in the recycle bin before deleting it from qbittorrent
save_torrents: false
# <OPTIONAL> split_by_category var:
# This will split the recycle bin folder by the save path defined in the `cat` attribute
# and add the base folder name of the recycle bin that was defined in the `recycle_bin` sub-attribute under directory.
split_by_category: false
orphaned:
# Orphaned files are those in the root_dir download directory that are not referenced by any active torrents.
# Will automatically remove all files and folders in orphaned data after x days. (Checks every script run)
# If this variable is not defined it, the orphaned data will never be emptied.
# WARNING: Setting this variable to 0 will delete all files immediately upon script run!
empty_after_x_days:
# File patterns that will not be considered orphaned files. Handy for generated files that aren't part of the torrent but belong with the torrent's files
exclude_patterns:
- "**/.DS_Store"
- "**/Thumbs.db"
- "**/@eaDir"
- "**/*.!qB"
- "**/*_unpackerred"
# Set your desired threshold for the maximum number of orphaned files qbm will delete in a single run. (-1 to disable safeguards)
# This will help reduce the number of accidental large amount orphaned deletions in a single run
# WARNING: Setting this variable to -1 will not safeguard against any deletions
max_orphaned_files_to_delete: 50
apprise:
# Apprise integration with webhooks
# Leave Empty/Blank to disable
# Mandatory to fill out the url of your apprise API endpoint
api_url:
# Mandatory to fill out the notification url/urls based on the notification services provided by apprise. https://github.com/caronc/apprise/wiki
notify_url:
notifiarr:
# Notifiarr integration with webhooks
# Leave Empty/Blank to disable
# Mandatory to fill out API Key
apikey: ####################################
# <OPTIONAL> Set to a unique value (could be your username on notifiarr for example)
instance:
webhooks:
# Webhook notifications:
# Possible values:
# Set value to notifiarr if using notifiarr integration
# Set value to apprise if using apprise integration
# Set value to a valid webhook URL
# Set value to nothing (leave Empty/Blank) to disable
error:
run_start:
run_end:
function:
cross_seed:
recheck:
cat_update:
tag_update:
rem_unregistered:
tag_tracker_error:
rem_orphaned:
tag_nohardlinks:
share_limits:
cleanup_dirs: |
|
This is exactly what I need to solve my issues.
|
I need this in my life |
|
This is exactly what I want.... I want it to mark all the cross-seeded torrents as nohardlink only if the original torrent source files are gone and mark those as nohardlink only if the plex files are gone, kind of like two layers to an onion. Plex (del) > original category movie as nohardlink, and then ONLY IF the original category file is gone because I cleaned it up in qbit, mark all the cross-seeded torrents as nohardlink too. |
|
@bobokun Please could you consider reopening this PR? After sharing it in IRC over at SeedPool, it appears I'm not the only one who wants this feature. E.g., I downloaded a big movie torrent from HDT, it got upgraded by Radarr, and now I can choose between letting it be cleared up automatically when it reaches HDT's share limit (I have it set to a ratio of 1), or I can decide to clear it manually when I think it's uploaded enough to maintain a good ratio. Some users will prefer the first option, and others prefer the flexibility and double-checking of the second option. The "second layer" which @drtaru spoke of simplifies the second option by automating the removal of cross-seeds after manual removal from your Radarr library. It also enables the prevention of cross-seeds from being removed due to hitting share limits before the original torrent, which is my primary use case: the way I achieve this is by assigning cross-seeds their own tag and share limit, |
…or handling - Use GitHub Actions bot credentials for git operations - Add proper branch existence checks and creation logic - Implement comprehensive error handling for push operations - Add detailed logging for debugging workflow issues - Use PAT token fallback for branch protection bypass - Include [skip ci] flag to prevent recursive workflow triggers
for more information, see https://pre-commit.ci
bobokun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First pass seems okay, just had a couple questions
| cross-seed: | ||
| # <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True). | ||
| ignore_root_dir: false | ||
| # <OPTIONAL> ignore_category_dir var: Will ignore any hardlinks detected in the same category save path (Default True). | ||
| ignore_category_dir: true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| cross-seed: | |
| # <OPTIONAL> ignore_root_dir var: Will ignore any hardlinks detected in the same root_dir (Default True). | |
| ignore_root_dir: false | |
| # <OPTIONAL> ignore_category_dir var: Will ignore any hardlinks detected in the same category save path (Default True). | |
| ignore_category_dir: true | |
| # <OPTIONAL> ignore_category_dir var: Will ignore any hardlinks detected in the same category save path (Default True). | |
| ignore_category_dir: true |
I'm not sure why there is a cross-seed key and duplicate ignore_root_dir
| return os.stat(file).st_nlink - self.inode_count.get(os.stat(file).st_ino, 1) > 0 | ||
| return os.stat(file).st_nlink - self.root_inode_count.get(os.stat(file).st_ino, 1) > 0 | ||
| elif ignore_category_dir: | ||
| return os.stat(file).st_nlink - self.categories_inode_count.get(category, {}).get(os.stat(file).st_ino, 1) > 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Has this logic been thoroughly tested? Could you provide some trace log output for specific examples to show it's working as intended?
Co-authored-by: bobokun <12660469+bobokun@users.noreply.github.com>
|
@jacobcxdev I think there are still a few unaddressed comments from the previous review |
Just wanted to note that I should have some free time this weekend to respond properly. My apologies for leaving it a few days. |
|
@jacobcxdev Are you going to review this? |
a7c6eb9 to
849dc1e
Compare
(Also removed a couple
->instances because it was causing issues with syntax highlighting. Not strictly necessary for this PR though, I suppose.)