prune download directory for orphaned files

[brainchild0]

Background

As downloading occurs, the application creates files under a user-identified download directory, usually a destination shared for all or most torrents currently downloaded or previously downloaded. Over time, files and sub-directories accumulate under this directory. Files can be removed if the user deletes a torrent with the option selected to delete content, or if the user removes the torrent and then manually deletes or moves the content. Over time, however, human error can cause an accumulation of "orphan" files, that is, files corresponding to torrents that were previously removed but whose contents were subsequently not managed effectively by the user. For example, a user may remove a torrent, then move some of the content to a different location, neglecting to move or to delete the remaining content. If the download directory contains both orphaned files and active content, then finding and identifying orphaned files can be extremely cumbersome. This situation suggests a need for a "prune" feature, to help the user to manage the accumulation of orphaned files mixed throughout the download directory.

Suggested Improvement

Add a feature that scans the tree under the download directory for orphaned files, then displays a list to the user, providing options to delete all such files or a selection of them.

[Kolcha]

I wrote simple Python script for this case. You can use it too. But beware of it, it doesn't ask anything, it just removes ANY files in download folder which is not belong to any torrent. You can easily change it to view what be removed just replacing os.remove(f) with print(f) at the last line. P.S.> scripts relies on WebUI without authentication for localhost

[brainchild0]

@Kolcha Thank you. The script is a very helpful workaround.

Of course, I would still want the request to get reviewed as a candidate for a supported feature.

[taboo180]

I wrote simple Python script for this case.

Hi @Kolcha, seems the dropbox link is dead now. I'd really love this script as I'm suffering from the same issue as OP. Absolutely any change that you'd still have it a year later? @

[Kolcha]

@taboo180 link updated

[HeikoBornholdt]

@Kolcha link is dead again.

[Kolcha]

@heikobornholdt link is updated, again

[Cyno01]

I tried the script, changing remove to print, but instead of listing the orphaned files (i have a miss-match of 97 between client and temp folder), it output a list of files that were COMPLETED and removed my my torrent client nearly a year ago, which dont have anything to do with anything?

Is the script no longer compatible with v4+? Im stuck on 4.1.9.1 specifically.

[Kolcha]

I'm still using this script with qBittorrent 4.2.5 with no changes, at least on Linux. not tested on any other OS.

if it tries to remove something unexpected, it is very likely that path reported by qBittorrent in 'save_path' field is different rather than actual file path. I saw such things very long time ago, so I just patched affected .fastresume files to set correct download path. also it can be affected by symlinks, I even don't know are they resolved or not...

[Cyno01]

Ah, that was probably it, there was a bunch of stuff active in qBt that i had pointed to move elsewhere when completed so i could seed without symlinks. I eventually got something figured out.

https://www.reddit.com/r/torrents/comments/hnvkhx/cleaning_temp_folder_of_abandoned_files/

[emhl]

in case the download link brakes again. here's the script for future travellers:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import os
import json
import urllib.request

base_url = "http://localhost:8080/api/v2"

torrents = json.load(urllib.request.urlopen(base_url + "/torrents/info"), encoding="utf-8")

known_files = {}

for t in torrents:
    save_path = t["save_path"]
    if save_path not in known_files:
        known_files[save_path] = set()

    t_url = "{}/torrents/files?hash={}".format(base_url, t["hash"])
    files = json.load(urllib.request.urlopen(t_url), encoding="utf-8")
    for f in files:
        known_files[save_path].add(f["name"])

for dst_dir in known_files:
    req_files = {os.path.join(dst_dir, f) for f in known_files[dst_dir]}
    all_files = {os.path.join(dp, f) for dp, dn, filenames in os.walk(dst_dir) for f in filenames}
    for f in all_files - req_files:
        print("removing:", f)
        os.remove(f)

[trim21]

You guys should consider using gist...

https://gist.github.com/

[Syer10]

The script above gave me a bunch of issues after changing the os to WebDav. Here is a fixed version for remote use with WebDav

#!/usr/bin/env python3

import json
import requests
from http.cookiejar import CookieJar
from webdav3.client import Client
from webdav3.exceptions import RemoteResourceNotFound


# WebDAV client configuration
options = {
    'webdav_hostname': "https://webdav.example.com",
    'webdav_login': "username",
    'webdav_password': "password"
}
client = Client(options)
client.webdav.disable_check = True

base_url = "https://qbittorrent.example.com/api/v2"

sid_cookie = "<qbittorrent_sid>"  # Replace with your actual SID value

# Set up the session with the SID cookie
session = requests.Session()
session.cookies.set("SID", sid_cookie, domain="qbittorrent.example.com")

response = session.get(base_url + "/torrents/info")
torrents = response.json()

known_files = {}

def get_string_after_first_slash(s):
    # Find the index of the first '/'
    index = s.find('/')
    # Return the substring after the first '/'
    if index != -1:
        return s[index + 1:]
    return s


for t in torrents:
    content_path = t["content_path"].removeprefix("/")
    if content_path not in known_files:
        known_files[content_path] = set()

    t_url = "{}/torrents/files?hash={}".format(base_url, t["hash"])
    response = session.get(t_url)
    files = response.json()
    for f in files:
        known_files[content_path].add(f["name"])

for dst_dir in known_files:
    try:
        info = client.info(dst_dir)
        if (info["size"] is None): # Check if folder
            formatted_files = {get_string_after_first_slash(f) for f in known_files[dst_dir]}
            req_files = {f"{dst_dir}/{f}" for f in formatted_files}
            all_files = {f"{dst_dir}/{f}" for f in set(client.list(dst_dir))}

            for f in all_files - req_files:
                info = client.info(f)
                if (info["size"] is not None): # Check if not folder
                    print("removing:", f)
                    client.clean(f)
    except RemoteResourceNotFound:
        continue