I/O error on torrent

[gyurmogyuri]

Hello, I want torrenting 70 GB file in one file. I got I/O error on tray. I want to unmount drive, and I got error message:

Unable to get volume information, The specified path cannot be found.
cannot unmount G: The data space passed to system call is too small.

Unable to get volume information, Not enough system resources to complete the requested service.
cannot unmount G: wait on umount returned an error unknown windows error 0

Do you have any idea?

[bailey27]

Hi.

Which torrent client program were you using?

[gyurmogyuri]

Hello, qbittorrent

[bailey27]

I got qbittorrent and downloaded a 7GB CentOS Linux iso, and I didn't have any errors.

When the problem has to do with file size, it usually happens as you cross the 2GB boundary (because a signed 32-bit integer overflows) or when you cross the 4GB boundary (because an unsigned 32-bit integer overflows).

It shouldn't make any difference if the file is 4.1GB or 70GB.

cppcryptfs is supposed to be doing all file offset arithmetic using 64-bit integers. Even the 32-bit version of cppcryptfs should be doing this.

Could you try downloading this torrent and see if it also works for you?

http://mirror.mojohost.com/centos/8.1.1911/isos/x86_64/CentOS-8.1.1911-x86_64-dvd1.torrent

Also, are you sure the filesystem you have the encrypted files on is on something that supports files larger than 4GB, e.g. NTFS or exFAT?

Also, are you using a recent version of cppcryptfs?

[gyurmogyuri]

Hello, I use NTFS, Have access for full. Dirve. I'm in group. The group have full access to mounted directory. Directory owner is administrator. The permissions are hereditary. I use cppcryptfs, Version 1.4.1.4 64-bit; AES-NI detected Copyright (C) 2016-2020 Bailey Brown. All rights reserved. linked with OpenSSL 1.1.1d; using Dokany 1.3.1. 1000 I have 111 GB free space I use as administrator

Still have IO error and not enough system resources to complete the requested services on Qbitorrent. Thanks for your help.

[bailey27]

I tried really hard to find a freely available torrent with a file anywhere near 70GB, but I was unable to.

It does sound like something (cppcryptfs, dokany or qbittorrent) is causing Windows to run out of some resource.

Maybe you could try limiting the number of connections used by qbittorrent from the default 100 to something like 10?

Also, when you start downloading a file in qbittorent, if you right-click on the file, you can specify to download it in sequential order. Maybe that would help as well?

I was looking on some torrent forums, and some people complained that some creators of torrents for huge files were creating them with more than 65536 chunks when they should be creating them with a maximum 65536 chunks and with a larger chunk size. Maybe this torrrent you're downloading has more than 65536 chunks, and that's causing the problem?

[bailey27]

Maybe a different torrent client would work better?

[Nodens-]

There's nothing better than qbittorrent on windows. Probably the only client that does everything according to specification. I'm willing to bet this is most likely a dokany issue. @bailey27 You can download a huge rainbow table from here for testing: https://freerainbowtables.com/

[bailey27]

I was able to reproduce the (or at least a) problem using one of the rainbow table torrents.

It looks like it's a race condition in cppcryptfs when being driven by qbittorrent.

If I go to the "Settings" tab and change "Per-filesystem threads" from the default (5) to 1, then I can't make it happen.

Could you please change Per-filesystem threads to 1 and mount that way and see if you still have the problem?

[gyurmogyuri]

Hello I cant find that settings. Could you show this in https://github.com/qbittorrent/qBittorrent/wiki/Explanation-of-Options-in-qBittorrent ?

[Nodens-]

I can not find that option either. Perhaps he's referring to Asynchronous I/O Threads in libtorrent section under advanced. It would make sense if this is a race condition related to OoO read/writes.

[null-python]

@Nodens- @gyurmogyuri I'm pretty sure he's talking about the option in cppcryptfs itself, not qbittorrent. If you go to the settings tabs you'll see it.

[bailey27]

Yes, I meant in cppcryptfs

[null-python]

Can you estimate the performance loss from going from 5 to 1 threads, if there is any?

[bailey27]

The performance loss will depend on the workload.

I can use qbittorrent to download the CentOS linux image mentioned above at 7MB/sec (56mbits/sec) with the filesystem mounted with cppcryptfs using only one thread. My download speed is supposed to be capped at 50mbits/sec but my provider seems to let it float up to 60mbits/sec. I am sure the limiting factor is my connection.

I'm working on a solution to the race condition.

[bailey27]

I'm sorry for taking forever to release something to help with this.

There are two issues here. One is the the race condition which causes qBitttorent to display a CRC (aka checksum) error occurred. This was happening due to a race condition in cppcryptfs which could be worked around by setting the number of per-fileystem threads in the cppcryptfs settings tab to 1.

The race condition (CRC error) issue has been fixed in the latest release https://github.com/bailey27/cppcryptfs/releases/tag/1.4.2.0

You should be fine now using however many per-fileysystem threads that you want in cppcryptfs.

The other issue is the "not enough resources..." error. That is a Dokany issue. I can reproduce it using the mirror.exe sample app from the Dokany distribution and having qBittorent download to the drive letter from mirror.

It appears to me that what is happening with the "not enough resources..." issue it that something qBittorent is doing is overwhelming Dokany driver. I looked at the Dokany source, and I found that the NT_STATUS that translates to this error (STATUS_INSUFFICIENT_RESOURCES) is used in a lot of places for a lot of different errors.

When this happens, as far as I can tell cppcryptfs never even knows about the i/o operation that fails.

I think what's is probably happening is qBittorrent is creating too many in-flight i/o operations at the same time, and Dokany can't handle it.

I would think that changing the max number of connections in qBittorrent from the default of 300 to 14 and change the number of per-filesystem threads used by cppcrypts from the default of 5 to 14 (the max Dokany supports) would help, but it doesn't seem to help that much.

Another weird, thing is that when the error happens, it usually takes a long time to unmount the drive in cppcryptfs. And if it takes a really long time, then usually that drive letter is unusable for re-mounting the filesystem. But sometimes the drive letter is ok to use.

But just getting the drive (or a drive) mounted and qBittorrent pointed at the right drive seems resolve it, at least for a while.

Anyway, I think this problem should be reported to the Dokany team.