qbit_manage
qbit_manage copied to clipboard
StuffAnThings
[FR]: Remove all Cross-Seed torrents if the original is deleted
Is your feature request related to a problem? Please elaborate.
When I delete a torrent and it's files if there are cross-seeds for that torrent too they get left behind until I notice them
Describe the solution you'd like
If torrent and it's files are deleted, delete all torrents that are also cross-seeding those files
Does your solution involve any of the following?
- [X] New config option
- [ ] New command option
Describe alternatives you've considered
Custom script but that seems like work that I no wanna do
Who will this benefit?
All Cross-Seed users
Additional Information
No response
Currently this functionality is sorta implemented via hardlinks, if Radarr upgrades a movie, the previous torrent would no longer be hardlinked and therefore all cross-seeds would be removed eventually via noHL.
Regarding your FR, how would qbm know which file is the "original" torrent? If you do remove an original torrent manually, there isn't really any way for qbm to know that you deleted an original torrent or cross seed.
Regarding your FR, how would qbm know which file is the "original" torrent? If you do remove an original torrent manually, there isn't really any way for qbm to know that you deleted an original torrent or cross seed.
This very kind person filed the FR for me, so I'll weigh in.
I'd love to use the product, the current issue is if you remove a cross-seeded torrent this orphans the data for the rest. So the solution is incredibly simple (that I wrote my own solution for) - if the name matches, group them all, and treat the weakest one (lowest ratio/time/etc) as the one to be scored.
Another addition is to check if the data is even there, if not then the original has been deleted so it's safe to delete the cross-seeds
Regarding your FR, how would qbm know which file is the "original" torrent? If you do remove an original torrent manually, there isn't really any way for qbm to know that you deleted an original torrent or cross seed.
The one without the cross-seed tag. If only one has no cross-seed tag, it's the original, would that work?
The one without the cross-seed tag. If only one has no cross-seed tag, it's the original, would that work?
We can't generalize that everyone tags cross-seeded torrents with a cross-seed tag, not everyone who uses qbm will have a setup like this.
I'd love to use the product, the current issue is if you remove a cross-seeded torrent this orphans the data for the rest. So the solution is incredibly simple (that I wrote my own solution for) - if the name matches, group them all, and treat the weakest one (lowest ratio/time/etc) as the one to be scored.
If you manually delete a torrent with contents, all the cross-seeded torrents should end up with errors since the files will be missing. I'm trying to understand better what the use case of removing the cross-seeded torrents are. Is it to remove any broken that have files missing?
Do you mind sharing the solution that you created so that I can better understand the use case?
s := fmt.Sprintf("%s%s%s%04d%02d%02d%02d%03d", rls.MustNormalize(r.Artist), rls.MustNormalize(r.Title), rls.MustNormalize(r.Subtitle), r.Year, r.Month, r.Day, r.Series, r.Episode)
Then from there scan the torrent list, if they all match (including Cut / Edition / etc), then treat them as the same.
Wouldn't it be best to leave all torrents, regardless of if they're a cross seed or not, to be left to their respective tracker settings in qbitmanage? If tracker A requires 2 week seed time and tracker B has a cross seedable target 13 days after you've been seeding, and they also have a minimum seed time, don't you want the cross seeded version to still seed even if the original has been fulfilled and since removed?
Assuming none of these torrents have hardlinks anymore.
Shouldn't this be left up to the orphaned torrents feature to pick up orphaned data once both versions are no longer seeding?
There won't be any orphaned data as when the torrents are removed the corresponding data would be.
but yes seeding to the tracker's mins makes sense as well
Doesn't this raise the question that torrents shouldn't have their corresponding data removed then? Only the torrent should be removed and not data, and when it's orphaned then the orphaned data function will fix it.
Probably would make more sense to adjust the logic and remove data if and only if it's the last torrent (if it doesn't already work that way)
For what it's worth, I wrote a script with my preferred logic to handle this. Perhaps it's of some use.
orphan_cross-seeds.py
#!/usr/bin/env python3
import qbittorrentapi # pip install qbittorrent-api
import os.path
client = qbittorrentapi.Client(
host="localhost",
port=8080,
username="username",
password="password"
)
xseed_tag = "cross-seed"
xseed_cat = None
paths = []
for x in client.torrents_info(): # for torrents that:
if (x.downloaded > 0 or # 1. downloaded data, or
x.upspeed > 0 or # 2. are currently uploading, or
not ( # 3. are not marked as cross-seed...
xseed_tag in x.tags.split(", ") or
xseed_cat == x.category
)):
for f in x.files: # consider their files “needed”...
p = x.save_path + "/" + f.name
if os.path.isfile(p): # if they exists.
paths.append(p)
for x in client.torrents_info( # then delete torrents if:
tags=xseed_tag, category=xseed_cat): # 1. marked as cross-seed, and
if (x.downloaded == 0 and # 2. no data was downloaded, but
x.seeding_time > 0 and # 3. have been seeded, and
((x.seeding_time/60) > x.max_seeding_time or # 4. meet seeding requirements, and...
x.ratio > x.max_ratio)):
needed = False
for f in x.files: # 5. have no “needed” files.
if x.save_path + "/" + f.name in paths:
needed = True
break
if not needed:
x.delete(delete_files=False) # orphan the data.
If, like me, you use a category instead of a tag to mark cross-seeds, something like this will work:
xseed_tag = None
xseed_cat = "Cross-Seed"
Even if you don't mark them at all the script still works because it deletes only torrents that never downloaded data and have been in the Seeding state, like cross-seeds, but that could also be a torrent you uploaded, so beware.
xseed_tag = None
xseed_cat = None
Or you could use both.
xseed_tag = "cross-seed"
xseed_cat = "Shows"
Probably would make more sense to adjust the logic and remove data if and only if it's the last torrent (if it doesn't already work that way)
The current logic is set to do this 👍
Probably would make more sense to adjust the logic and remove data if and only if it's the last torrent (if it doesn't already work that way)
The current logic is set to do this 👍
it doesn't though, because the other units of work are deleted and there's no grouping of them. Carlitos artifact does it more scientifically than mine, where the actual contents are compared to generate the dependency graph.
I currently use qbit_manage with cross seeds and it works fine without deleting any of the data until there's only one torrent left without any orphan files.
This is the part of the code logic that does the check.
https://github.com/StuffAnThings/qbit_manage/blob/6b6001ec0ee576ecb905546c737a7b9222717075/modules/qbittorrent.py#L746-L760
I currently use qbit_manage with cross seeds and it works fine without deleting any of the data until there's only one torrent left without any orphan files.
This is the part of the code logic that does the check.
https://github.com/StuffAnThings/qbit_manage/blob/6b6001ec0ee576ecb905546c737a7b9222717075/modules/qbittorrent.py#L746-L760
Yes, that's understood. However, I'm sure you can understand that it's a bit (for others, very) silly to do that as the load isn't on having the torrents in client, but on the disk itself. Having another entry to maintain tracker stats costs a couple kb of RAM and the torrent file on disk. The data itself is where the "expense" is here. Keeping around the other units of work is, comparatively speaking, free, which is the desired goal of the feature request.
I read the entire discussion and I still can't seem to figure out if there's already some sort of solution or something way simpler (as an option) would suffice.
For me, personally, I handle torrent durations by their original grab. This makes sense to me, because different trackers have different values, rules, priorities, etc.
However, when an "original" torrent's requirements are met, it will then get cleaned up (torrent and files deleted). This will lead to any possible cross-seed that's left behind starting a new download. If there was an optional step that deleted all torrents that do NOT point to an existing file/folder, right after the cleanup, it would solve my issue and now delete the orphaned torrent files.
I can't really think of a nice way to name things - but since we're calling files without a torrent orphans - what's a torrent without a file? Childless?
when an "original" torrent's requirements are met, it will then get cleaned up (torrent and files deleted). This will lead to any possible cross-seed that's left behind starting a new download.
QBm does not delete data that is still in use (I.e. same path) as existing torrents
I read the entire discussion and I still can't seem to figure out if there's already some sort of solution or something way simpler (as an option) would suffice.
For me, personally, I handle torrent durations by their original grab. This makes sense to me, because different trackers have different values, rules, priorities, etc.
However, when an "original" torrent's requirements are met, it will then get cleaned up (torrent and files deleted). This will lead to any possible cross-seed that's left behind starting a new download. If there was an optional step that deleted all torrents that do NOT point to an existing file/folder, right after the cleanup, it would solve my issue and now delete the orphaned torrent files.
I can't really think of a nice way to name things - but since we're calling files without a torrent orphans - what's a torrent without a file? Childless?
It took about 20 minutes of effort from start to finish: upgraderr doesn't have this bug.
It took about 20 minutes of effort from start to finish: upgraderr doesn't have this bug.
Sorry for asking here, but is there any documentation? I have qbit trigger cross-seed atm and it would be somewhat easy to just trigger upgraderr - but not for complete searches if I add a new tracker to the "rotation2.
Lastly, how does this solve the issue of cleanup? qbm would still delete file and torrent when cleaning up and not catch the cross-seeds. I could imagine getting around qbm's cross-seed detection via symlinks for your cross-seeded torrents, but I can't see how they would get cleaned up in the end.
Thanks for taking the time.
https://github.com/StuffAnThings/qbit_manage/wiki/Config-Setup#share_limits
Lastly, how does this solve the issue of cleanup? qbm would still delete file and torrent when cleaning up
it doesn't?
also upgraderr like many of Kyle's creations likely has no documentation nor bug reporting as it is for "adults" not sure why Kyle is deciding to derail this issue
I guess I'll have to write my own cleanup code or adjust Carlito's code then. At this point, cross-seeds will never get cleaned up because they seed infinitely.
If I don't cross-seed infinitely, torrents then get cleaned up on per-tracker share limits and cross-seed will likely add them back if one tracker is set to seed 2-3 months and another is set to 2 weeks.
Anyway, thanks for the explanation. I can probably think of a way to make this work.
cross-seed will likely add them back if one tracker is set to seed 2-3 months and another is set to 2 weeks.
Cross-seed doesn't re-add, only of you wipe its data.
It took about 20 minutes of effort from start to finish: upgraderr doesn't have this bug.
Sorry for asking here, but is there any documentation? I have qbit trigger cross-seed atm and it would be somewhat easy to just trigger upgraderr - but not for complete searches if I add a new tracker to the "rotation2.
Lastly, how does this solve the issue of cleanup? qbm would still delete file and torrent when cleaning up and not catch the cross-seeds. I could imagine getting around qbm's cross-seed detection via symlinks for your cross-seeded torrents, but I can't see how they would get cleaned up in the end.
Thanks for taking the time.
I gave up on waiting and literally spent 20 minutes on the solution 😆. The author of this issue was kind enough to open it for me when I explained the original problem, which has still not been resolved.
use qbittools with orphaned to cleanup the data on disk. upgraderr won't remove dupes from the client until they all meet the expression. This is very trivial to fix. If you don't want to use qbittools I can open up another tool to do it.
Also feel free to ignore bakerboy, he literally just follows from issue to issue and has zero development experience with worthless feedback. It's very disappointing.
Hey all, I was going to create a new thread, but I think this one is essentially stating my issue as well. The problem:
- After sonarr upgrades from episodes to a season pack or upgrades episodes themselves, the original downloaded content gets deleted by qbit as it should be, however if in the meantime cross-seed has found matches on other platforms and those torrents are active, the content will actually never be deleted unless I do a manual sweep (currently in the lengthy process of doing this).
I'm not sure if this is something that can be fixed with qbitmanage in its current state or this is something that is being worked on (or not).
Thanks for your time.
Excluding Kyle or other off topic commentary. Can any users provide specific use cases with specific non-working share_limits that do not resolve this at this point in time?
Excluding Kyle or other off topic commentary. Can any users provide specific use cases with specific non-working share_limits that do not resolve this at this point in time?
Tracker A has a movie that is grabbed by radarr. Once the download finishes, cross-seeds are found in tracker B and C. In 2 days, this movie is removed from tracker A because a better version has been found. The better version is grabbed and is now used in radarr as the default one. The first version from tracker A is now tagged as noHL and the torrent will get removed. However, the cross-seed torrents are still active and seeding in tracker B and C and I believe those will remain in the torrent client.
I believe this is the process, however ideally once the original torrent is deleted so should its contents as well as the cross seeding torrents. This avoids having many versions of the same media occupying space unecessarily, since most of us really only care about having the best possible version on our servers.
I could be missing something, so please let me know if there is a way to avoid this using share_limits.
with specific non-working share_limits that do not resolve this at this point in time?
provide the share limits being used and not working. NoHL and cross-seed tags in a share limit group should easily accomplish that
@ahatdude , this can definitely be achieved with share limits.
The first version from tracker A is now tagged as noHL and the torrent will get removed.
When tracker A is tagged as noHL, those cross seeds from tracker B and C will also be tagged as noHL, so your cross-seeds will also be deleted alongside tracker A at the same time.
If you want the cross-seeds to have a different share limit than tracker A, you can always create another share limit with a much smaller max_seeding_time to capture those torrents using the filter: include_all_tags: [noHL, cross-seed]
Thanks to both for responding and I apologise if I'm missing something.
The cross seed situation doesn't seem to be happening, so it may have been my error in the past. However, I just checked my qbit for movies and currently there a few noHL torrents that have been upgraded for a few weeks despite my cleanup being at 7 days (10080 mins).
Any help is appreciated!
commands:
# The commands defined below will IGNORE any commands used in command line and docker env variables.
dry_run: false
cross_seed: true
recheck: true
cat_update: true
tag_update: true
rem_unregistered: true
tag_tracker_error: true
rem_orphaned: true
tag_nohardlinks: true
skip_recycle: false
skip_cleanup: false
skip_qb_version_check: true
share_limits: false
qbt:
# qBittorrent parameters
host: 192.168.1.80:8080
user: REDACTED
pass: REDACTED
settings:
force_auto_tmm: false
tracker_error_tag: issue
nohardlinks_tag: noHL
share_limits_tag: ~share_limit
ignoreTags_OnUpdate: # When running tag-update function, it will update torrent tags for a given torrent even if the torrent has one or more of the tags defined here.
- noHL
- issue
- cross-seed
directory:
# Do not remove these
# Cross-seed var: </your/path/here/> # Output directory of cross-seed
# root_dir var: </your/path/here/> # Root downloads directory used to check for orphaned files, noHL, and RecycleBin.
# <OPTIONAL> remote_dir var: </your/path/here/> # Path of docker host mapping of root_dir.
# Must be set if you're running qbit_manage locally and qBittorrent/cross_seed is in a docker
# <OPTIONAL> recycle_bin var: </your/path/here/> # Path of the RecycleBin folder. Default location is set to remote_dir/.RecycleBin
# <OPTIONAL> torrents_dir var: </your/path/here/> # Path of the your qbittorrent torrents directory. Required for `save_torrents` attribute in recyclebin
root_dir: /data/torrents/qbit-movies
remote_dir: /data/torrents/qbit-movies
cross_seed: /data/torrents/qbit-movies/input
recycle_bin: /data/torrents/qbit-movies/.RecycleBin
torrents_dir: /qbit_appdata/qbittorrent-vpn-movies/data/BT_backup/
orphaned_dir: /data/torrents/qbit-movies/orphaned_data
cat:
# Category & Path Parameters
# <Category Name> : <save_path> # Path of your save directory.
movies: /data/torrents/qbit-movies/movies
movies-completed: /data/torrents/qbit-movies/movies-completed
# cat_change:
# # This moves all the torrents from one category to another category. This executes on --cat-update
# # WARNING: if the paths are different and Default Torrent Management Mode is set to automatic the files could be moved !!!
# # <Old Category Name> : <New Category>
# Radarr-HD.cross-seed: movies-hd
# Radarr-UHD.cross-seed: movies-uhd
# movies-hd.cross-seed: movies-hd
# movies-uhd.cross-seed: movies-uhd
tracker:
# Tag Parameters
# <Tracker URL Keyword>: # <MANDATORY> This is the keyword in the tracker url
# <MANDATORY> Set tag name. Can be a list of tags or a single tag
# tag: <Tag Name>
# <OPTIONAL> Will set the torrent Maximum share ratio until torrent is stopped from seeding/uploading. -2 means the global limit should be used, -1 means no limit.
# max_ratio: 5.0
# <OPTIONAL> Will set the torrent Maximum seeding time (min) until torrent is stopped from seeding. -2 means the global limit should be used, -1 means no limit.
# max_seeding_time: 129600
# <OPTIONAL> Will limit the upload speed KiB/s (KiloBytes/second) (-1 sets the limit to infinity)
# limit_upload_speed: 150
# <OPTIONAL> Set this to the notifiarr react name. This is used to add indexer reactions to the notifications sent by Notifiarr
# notifiarr: <notifiarr indexer>
alpharatio:
tag: AlphaRatio
limit_upload_speed: 1000
notifiarr: alpharatio
animetosho:
tag: AnimeTosho
max_ratio: 1
max_seeding_time: 720
limit_upload_speed: 250
notifiarr: animetosho
animeTorrents:
tag: AnimeTorrents
notifiarr: animetorrents
audionews:
tag: AudioNews
notifiarr: audionews
beyond-hd:
tag: Beyond-HD
notifiarr: beyondhd
avistaz:
tag: Avistaz
notifiarr: avistaz
animebytes.tv:
tag: AnimeBytes
notifiarr: animebytestv
blutopia:
tag: Blutopia
notifiarr: blutopia
limit_upload_speed: 1000
cartoonchaos:
tag: CartoonChaos
notifiarr: cartoonchaos
cgpeers:
tag: CGPeers
notifiarr: cgpeers
cinemaz:
tag: Cinemaz
notifiarr: cinemaz
filelist:
tag: Filelist
limit_upload_speed: 1000
notifiarr: filelist
gazellegames:
tag: GGn
notifiarr: gazellegames
hdts:
tag: HD-Torrents
notifiarr: hdtorrents
landof.tv:
tag: BroadcasTheNet
notifiarr: broadcasthenet
passthepopcorn:
tag: PassThePopcorn
notifiarr: passthepopcorn
stackoverflow:
tag: IPTorrents
limit_upload_speed: 1000
notifiarr: iptorrents
morethantv:
tag: MoreThanTV
notifiarr: morethantv
myanonamouse:
tag: MaM
notifiarr: myanonamouse
opsfet:
tag: Orpheus
notifiarr: orpheus
portugas:
tag: Portugas
notifiarr: portugas
privatehd:
tag: PrivateHD
notifiarr: privatehd
sportscult:
tag: SportsCult
notifiarr: sportscult
tleechreload:
tag: TorrentLeech
limit_upload_speed: 1000
notifiarr: torrentleech
tv-vault:
tag: TV-Vault
notifiarr: tvvault
pretome:
tag: PreToMe
notifiarr: pretome
revolutiontt:
tag: RevolutionTT
notifiarr: revolutiontt
nyaa:
tag: Nyaa.si
max_ratio: 1
max_seeding_time: 720
limit_upload_speed: 250
notifiarr: nyaasi
tracker.torrentleech.org:
tag: TorrentLeech
notifiarr: torrentleech
reactor.flro.org:
tag: Filelist
notifiarr: filelist
routing.bgp.technology:
tag: IPTorrents
notifiarr: iptorrents
ssl.empirehost.me:
tag: IPTorrents
notifiarr: iptorrents
science.docspedia.world:
tag: DocsPedia
notifiarr: docspedia
animetorrents.me:
tag: animetorrents.me
animetracker.cc:
tag: animetracker.cc
banana.80085455.xyz:
tag: banana.80085455.xyz
nohardlinks:
# Tag Movies/Series that are not hard linked
# Mandatory to fill out directory parameter above to use this function (root_dir/remote_dir)
# This variable should be set to your category name of your completed movies/completed series in qbit. Acceptable variable can be any category you would like to tag if there are no hardlinks found
movies-completed:
# <OPTIONAL> exclude_tags var: Will exclude the following tags when searching through the category.
# exclude_tags:
# - Beyond-HD
# - AnimeBytes
# - MaM
# <OPTIONAL> cleanup var: WARNING!! Setting this as true Will remove and delete contents of any torrents that are in paused state and has the NoHL tag
cleanup: true
# <OPTIONAL> max_ratio var: Will set the torrent Maximum share ratio until torrent is stopped from seeding/uploading
max_ratio: 1.25
# <OPTIONAL> max seeding time var: Will set the torrent Maximum seeding time (min) until torrent is stopped from seeding
max_seeding_time: 21600
# <OPTIONAL> Limit Upload Speed var: Will limit the upload speed KiB/s (KiloBytes/second) (`-1` : No Limit)
limit_upload_speed:
# <OPTIONAL> min seeding time var: Will ensure that torrent is not deleted by cleanup variable if torrent does not meet minimum seeding time (min).
min_seeding_time: 0
exclude_tags:
resume_torrent_after_untagging_noHL: true
recyclebin:
# Recycle Bin method of deletion will move files into the recycle bin (Located in /root_dir/.RecycleBin) instead of directly deleting them in qbit
# By default the Recycle Bin will be emptied on every run of the qbit_manage script if empty_after_x_days is defined.
enabled: true
# <OPTIONAL> empty_after_x_days var:
# Will automatically remove all files and folders in recycle bin after x days. (Checks every script run)
# If this variable is not defined it, the RecycleBin will never be emptied.
# WARNING: Setting this variable to 0 will delete all files immediately upon script run!
empty_after_x_days: 5
# <OPTIONAL> save_torrents var:
# If this option is set to true you MUST fill out the torrents_dir in the directory attribute.
# This will save a copy of your .torrent and .fastresume file in the recycle bin before deleting it from qbittorrent
save_torrents: true
# <OPTIONAL> split_by_category var:
# This will split the recycle bin folder by the save path defined in the `cat` attribute
# and add the base folder name of the recycle bin that was defined in the `recycle_bin` sub-attribute under directory.
split_by_category: false
orphaned:
# Orphaned files are those in the root_dir download directory that are not referenced by any active torrents.
# File patterns that will not be considered orphaned files. Handy for generated files that aren't part of the torrent but belong with the torrent's files
exclude_patterns:
- '**/.DS_Store'
- '**/Thumbs.db'
- '**/@eaDir'
- /data/torrents/temp/**
- '**/*.!qB'
That config file is invalid for v4+ and has been invalid for almost a year. You're failing to use sharelimits thus nothing will ever be removed and no seed targets will ever be set. Redo your file.
why are you ignoring what you were told to do? Use share_limits and are doing absolutely nothing at all with them?