Too many hash_failed alerts

[i96751414]

Please provide the following information

libtorrent version (or branch): v2.0.8

platform/architecture: Android ARM64

Hi @arvidn.

I am seeing a strange behavior with libtorrent for which I would like some hints to really understand what's causing it. Essentially, I am seeing too many hash_failed alerts when downloading a torrent. The odd thing is that on another device (same platform/architecture) I don't see any of these.

What could be causing these hash failed alerts?

[i96751414]

I just realized this only happens when I use Android emulated storage (storage/emulated/0).

[i96751414]

Update: This behavior is seen on a different platform/arch (linux armv7) as well. In this case, when using an external USB (ext4) as a download folder there are too many hash_failed alerts and the download gets "stuck".

[arvidn]

Are you confident your drive is OK? If you bought a USB drive from China for well below market prize, for instance, chances are that the true capacity is a lot smaller than what the driver reports.

[i96751414]

Yes, I am. On the first scenario for instance I don't use an external storage at all. The simple fact I use a storage/emulated/0/... path makes this errors appear

[i96751414]

Hi @arvidn. I tried to replicate the same on libtorrent 1.2.18 and I only had 1 hash_failed, it worked great. I then tested again on libtorrent 2.0.8 just to be completely sure I wasn't missing anything and I had a ton of hash_failed alerts. It's like the download gets stuck.

[ValeZAA]

So you are 100% sure that is a regression? Can you modify it to print the hash it gets? Also can you tell if it is all the same segments that are at issue or always different?

and I only had 1 hash_failed

That is still bad.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[i96751414]

Hi @arvidn

Is there any way I can debug this? This is still an unsolved mystery to me, and I would like to understand what could be causing this.

[arvidn]

you could try to run the disk_io_stress_test, under tools in the libtorrent repo. it stress tests disk I/O subsystem in libtorrent, but it also ensures that bytes written to disk are read-back correctly. I don't believe it has coverage of the hashing functions specifically, but reading back bytes might be a good enough proxy.

The expected source of hash failures are:

1. peers sending corrupt data (which can happen for a number of reasons, media players updating id3 tags or other metadata used to be an issue)
2. switches, firewalls or ISPs changing the content of network packets for whatever reason. Some routers look for any 4 bytes that look like your internal IP address and replaces it with your external one. Some firewalls may have heuristics that identifies well known worms, viruses or shell code, and clear it. Any false positive in that logic would cause corruption.

It could be related to the mmap disk I/O. You could test that by switching to the posix disk I/O back-end instead.