Add ability to easily import/export qBittorrent data

[olfek]

By "qBittorrent data" I mean data/information about the torrents and not their actual files.

Intro

Lots of people who download torrents, say to a separate partition or drive and move to a different OS or to a device with a different OS, lose their ability to seed as I have many times due to qBittorrent data not being easily portable and requiring some technical effort and time to do which I imagine for most people just isn't worthwhile.

qBittorrent is very popular, I assure you that we are losing seeders in great numbers because of this, this needs to be addressed.

Current process

These days, when I do a move, I have to do the following:

1. Save ~/.local/share/data/qBittorrent folder.
2. Restore that folder.
3. Launch the application. Lots of "missing files" errors at this point if the location of downloaded torrents has changed. This would happen for example when moving from Windows to Linux, partition/drive path will be different as it will previously have been addressed by a letter such as D: which Linux does not do. When I did this, I saw all the torrents show a "missing files" error except one which started downloading the torrent to the old Windows path on a Linux system before also showing a "missing files" error.
4. Set the download location of all torrents to the partition or drive on which they exist.
5. Wait for all torrents to complete file checks and resume.
6. Done.

Proposed process

1. Click export - a JSON file is produced containing all qBittorrent data.
2. On new OS/device, import the exported JSON file.
3. qBittorrent asks if the same root path should be used and provides an input field to enter a new root path (by root path I mean the location where all torrent files reside including those under nested directories).
4. Done.

Notes

This is a more formal proposal of #13248

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[glassez]

3. by root path I mean the location where all torrent files reside including those under nested directories

How about the torrents that store their files under different root paths (e.g. on different disks)?

[j1warren]

I'd say just an option for xml/json export/import, user can mass edit/replace paths in their favorite text editor.

[glassez]

Well, maybe... I wonder what the data should be there exactly? All the same as in the internal form, or a reduced subset?

[olfek]

@glassez The minimal amount I think - only that which is neccessary to facilitate an export/import in which the destination qBittorrent is identical to the source qBittorrent.

[olfek]

@j1warren Yep, that sounds good 👍

[olfek]

similar to #9912

[olfek]

Sorry about that :point_up:, trying out the mobile app. Anybody working on this? I have some time, I'll try and have a go at adding it, any tips/advice?

[glassez]

I have some time, I'll try and have a go at adding it, any tips/advice?

First, it would be nice to know the details of what exactly you intend to implement, so that you avoid unnecessary work.

[glassez]

You also need to wait for @Chocobo1 comment. AFAIK, he has some experience in converting Bencoded data to JSON. Although this task does not seem difficult to me. I'm more concerned with the overall concept of this feature.

[Chocobo1]

First, it would be nice to know the details of what exactly you intend to implement

There seems to be 2 tasks here, one is exporting torrents data to some human-friendly format, the other is introducing another way of adding torrents from the new format.

AFAIK, he has some experience in converting Bencoded data to JSON.

For general cases, it isn't pretty. But for specific tasks, it shouldn't pose a problem. I'm talking about representing binary data in JSON format.

[glassez]

the other is introducing another way of adding torrents from the new format.

Can't we just convert it back and load it in the regular way?

[olfek]

What if the current storage solution was just replaced with JSON? Then, the export/import process would be as simple as copying the JSON file out or replacing it.

[asorel1942]

@olfek I just attempted your proposed manual process on a clean install of Manjaro; I made a copy of .local/share/data/qBittorrent, and copied that folder into that same location on the new install. However, when I open up qBittorrent, my torrent list is empty. Do you have any suggestions as to how I may troubleshoot this?

[olfek]

@asorel1942 The default location of app data has changed from .local/share/data/qBittorrent to .local/share/qBittorrent.

[ctminime]

I wrote a script to convert from QB on Windows to QB on Linux that does some of what you are looking for. With the new version I posted today, I also made it into a standalone executable. https://github.com/ctminime/QB_Migrate_to_Linux/releases/tag/2.0

[gsm2441]

I also appreciate this idea, I will test the executable right now.

[DumboJetEngine]

I need this feature too, for a slightly different purpose: I too have Windows and Linux computers that I need to migrate data to/from, but I also have multiple Windows PCs that have their own qBitTorrent data. So, what I want to do is to merge data from one Windows OS to another one. Not replace their existing data, but merge their data with data from another qBitTorrent installation. This way, one can even sync all his qBitTorrent installations to have the same data (putting aside the problems with differences in drive letters or drive mounts).

I also guess that an export feature would be a nice way for backups too. After some point, you might be seeding a lot of torrents and starting up qBitTorrent might be a pain (due to delays, CPU usage, etc). Creating a backup file and starting from scratch (by deleting some or all torrents) would be nice. And you can always restore the old torrents by merging the backup.

Looking at the qBitTorrent data folder, I can't figure out what is going on. The BT_backup folder holds the original torrent files, but if you rename a file inside a torrent during download, the torrent file is not updated. So, you cannot easily use this data to 100% restore your previous state, since qBitTorrent won't find the renamed files (as far as I can tell). I am not sure where this missing information is stored. Is there another file that keeps the updated file/folder names?

I was using uTorrent in the past, and that was updating the actual .torrent files on the disk when you renamed files/folders. This doesn't help with drive letters and mount names that might change, but at least it helps figuring out what the actual names are.

It's been a while since I last used C++ and my Qt experience is a bit limited, but perhaps I could try adding some initial code in a PR for exporting data in JSON format, if this is not an ongoing work already...

[glassez]

The BT_backup folder holds the original torrent files, but if you rename a file inside a torrent during download, the torrent file is not updated. I am not sure where this missing information is stored. Is there another file that keeps the updated file/folder names?

qBittorrent stores original (immutable) torrents metadata in such .torrent files. All other data are stored in corresponding .fastresume files in the same folder (It's strange that you didn't notice them there).

[DumboJetEngine]

@glassez Aha! I did notice those files, but I tried renaming a file in a torrent in the qBitTorrent GUI, then checked the corresponding .fastresume file, I saw the file change-date was updated, but the file contents were the same as before. Also, there were no file names inside of it. So, that's why I assumed this information was missing. Have I missed anything?

[glassez]

Also, there were no file names inside of it. So, that's why I assumed this information was missing. Have I missed anything?

There is mapped_files in it. But it doesn't exist if there is no renamed files.

[GitHubinatrix]

Please make it clear for me - a fed-up former user of uTorrent

[Long story short] Because of often crashes and of fear of loosing data and time / energy, while also not wanting to unnecessary use up my SSDs with moving files around, until a few days ago my modus operandi for torrenting was this:

1] uTorrent installed on drive [i.e. not merely a volume] C 2] automatic copy of up-to-date settings of uTorrent [thus info about progress of downloads] stored on drive Z 3] all [auto created] .TORRENT files [acquired by clicking on magnet links] stored on drive T [in folder named links] 4] copies of all .TORRENT files stored on drive Z 5] all downloads in progress and finished tasks stored on drive T [in folder named data]

In other words - in my setup: ● drive C was for the software and its settings with info about progress in it ● drive T was for hosting all current download jobs and ready-to-be-used finished files ● drive Z was a backup for storing both settings / progress info and [as additional fail-safe] actual .TORRENT files ● there was no automatic moving of finished files an only rarely they were moved / copied manually ● when no longer needed a content from data and links folders was deleted from drive T

But how do I replicate this MO when using now qBitorrent?

I know I can set

Tools > Options > Downloads > Saving Management > Default Save Path

to

T:\data

in order to use drive T for the main body of work; while also I can set

Tools > Options > Downloads > Saving Management > Copy .torrent files to

to

T:\links

for the purpose of maintaining a crude backup of tasks

But if I copy the content of

C:\Users\YOUR-USER-NAME\AppData\Local\qBittorrent\BT_backup

folder, e.g. install new system with qBitorrent and copy to its newly created BT_backup folder my old back-copied .TORRENT and .FASTRESUME files [that had those gibberish names applied to them on old system by qBitorrent] - will then work on them wtihin qBitorrent be resumed without any rechecking? Or will I have to also import old settings to

C:\Users\YOUR-USER-NAME\AppData\Roaming\qBittorrent

folder? Or will even that not be sufficient and I need to secure something more?

[And yes- I did rename all !UT files to !Q i between using of uTorrent and qBitorrent]

[thalieht]

.FASTRESUME files [that had those gibberish names applied to them on old system by qBitorrent]

It's the torrent's hash.

will then work on them wtihin qBitorrent be resumed without any rechecking?

It should work (no rechecking) except maybe if you use "Automatic Torrent Management" for your torrents with categories. Although it may still work, i don't actually know, i would backup and restore "categories.json" and the settings file "qbittorrent.ini" because they contain info about the categories and where they (categories) define where torrents should be saved. Edit: Better restore "qbittorrent.ini" no matter what, to be sure.

[And yes- I did rename all !UT files to !Q i between using of uTorrent and qBitorrent]

qBt doesn't use "!Q". It uses "!qB" and only if the corresponding option in settings is enabled.

[GitHubinatrix]

Yes, !QB; that was a typo

For now, i.e. on the within the same system without, coping of just

C:\Users\YOUR-USER-NAME\AppData\Local\qBittorrent\BT_backup

seems to work - but if I were to restore my OS from an image or install a new one I would surely copy in

C:\Users\YOUR-USER-NAME\AppData\Roaming\qBittorrent

also

As for adjusting BT_backup folder- it is currently impossible to change its path / name?

[thalieht]

As for adjusting BT_backup folder- it is currently impossible to change its path / name?

This wiki page is more help than i can offer https://github.com/qbittorrent/qBittorrent/wiki/How-to-use-portable-mode

[GitHubinatrix]

Yep: the transition to using portable version was smooth [with the small exception of having to once again choose

Tools > Options > Behavior > Interface > Use custom UI Theme

location, as I had stored .QBTHEME file along my config files of the installed version [which I have removed in its entirety] - but it did not resolve the issue of BT_backup folder having a fixed path; hence me creating a proper request: https://github.com/qbittorrent/qBittorrent/issues/18963#issuecomment-1544407942

But thank you for pointing me to the right info

[glassez]

but it did not resolve the issue of BT_backup folder having a fixed path;

It has a fixed path relative to profile/config directory as well as paths of all another internal files/folders. You can set the path to profile/config directory as you want. Something beyond that is overkill.

[GitHubinatrix]

I disagree- with the logic behind such approach

Because I want all my installed and portable programs to be on drive C - but all other important information / files on some other / non-system drive. So that when my operating system or the drive its holding goes kaput, I will have one less thing to wory about: i.e. how many tasks I have added or removed from qBittorent since my last backup of the state of affairs. And I do not want to mix settings with torrents, because this would make me force to remember about this exception. [And I already have to many tweaks and adjustments running on my machine I need to be more or less aware of, thus sucking out life energy out of me]

For the same purpose I e.g. download my e-mail from servers to the same non-system drive, keeping on C only the Thunderbird and its profile. This way when something bad happens and I restore my system from an image I loose zero messages stored offline and in no me I can be back up and running

[stalkerok]

@GitHubinatrix, place the portable qbittorrent on the C drive and back up the entire client folder periodically.