Fetch icon indicating copy to clipboard operation
Fetch copied to clipboard

[CRITICAL BUG] Wrong download total size after resuming

Open XiXiongMaoXiong opened this issue 5 years ago • 14 comments

At first shot the total size gets shown correctly but after a cancel and resume, or a pause and resume the total size is alreadyDownloadedSize + totalSize!

This bug is only in version 3.1.5 of Android X... Version 3.1.4 doesn't have this bug @tonyofrancis

XiXiongMaoXiong avatar Oct 04 '20 12:10 XiXiongMaoXiong

@tonyofrancis !!!

XiXiongMaoXiong avatar Oct 05 '20 15:10 XiXiongMaoXiong

This bug is only in version 3.1.5 of Android X... Version 3.1.4 doesn't have this bug I just tested.

XiXiongMaoXiong avatar Oct 05 '20 17:10 XiXiongMaoXiong

I am having the same issue with the non-Android X version and posted my issues in an old, closed issue (#486.) I'll post it here again in case it helps:

"Hi, this issue is still present for me in 3.0.11. I noticed a few things:

  • When you kill the app while it's downloading and then restart the app, it adds the amount of bytes that have been downloaded up to that point to the file. So for example with Big Bunny, the initial file size is 151 MB. If it has downloaded 50 MB and you kill and restart the app, the file size will be set to 201 MB.
  • This breaks the values returned in OnProgress in FetchListener, because it now calculates the progress based on the 201 MB file size, but still correctly downloads only 151 MB.
  • This in turn causes OnComplete to never be called. If you kill and restart the app again at that point, the downloader attempts to download the file again, but this results in a "Request is Unsuccessful" error, probably because it did actually download the amount of bytes it should have downloaded.
  • In the case of the Big Bunny video, playback of the video after 151 MB has been downloaded goes fine, despite the wrong file size. This might be specific to video files.
  • I think a similar issue occurs when pausing & resuming, but I have not tested that enough to confirm it yet. EDIT: just confirmed this happens with pausing & resuming as well. To see it clearly: let the download progress till ~50%, then pause it and resume it. The progress will fall to somewhere in the 30s and the download finishes at 60/70s. Even no need to kill the app.

I can send you my project and an APK for testing if you need it, but I implemented it in a Unity project so I'm not sure if it will be of any particular help. Let me know if you need it and I'll send it ASAP.

EDIT: Also I forgot to mention, if you set preAllocateFileOnCreation to false, the final file size will be correct, but the Download object will still have the incorrect total size."

Dezley-VROwl avatar Oct 06 '20 08:10 Dezley-VROwl

I am having the same issue with the non-Android X version and posted my issues in an old, closed issue (#486.) I'll post it here again in case it helps:

"Hi, this issue is still present for me in 3.0.11. I noticed a few things:

  • When you kill the app while it's downloading and then restart the app, it adds the amount of bytes that have been downloaded up to that point to the file. So for example with Big Bunny, the initial file size is 151 MB. If it has downloaded 50 MB and you kill and restart the app, the file size will be set to 201 MB.
  • This breaks the values returned in OnProgress in FetchListener, because it now calculates the progress based on the 201 MB file size, but still correctly downloads only 151 MB.
  • This in turn causes OnComplete to never be called. If you kill and restart the app again at that point, the downloader attempts to download the file again, but this results in a "Request is Unsuccessful" error, probably because it did actually download the amount of bytes it should have downloaded.
  • In the case of the Big Bunny video, playback of the video after 151 MB has been downloaded goes fine, despite the wrong file size. This might be specific to video files.
  • I think a similar issue occurs when pausing & resuming, but I have not tested that enough to confirm it yet. EDIT: just confirmed this happens with pausing & resuming as well. To see it clearly: let the download progress till ~50%, then pause it and resume it. The progress will fall to somewhere in the 30s and the download finishes at 60/70s. Even no need to kill the app.

I can send you my project and an APK for testing if you need it, but I implemented it in a Unity project so I'm not sure if it will be of any particular help. Let me know if you need it and I'll send it ASAP.

EDIT: Also I forgot to mention, if you set preAllocateFileOnCreation to false, the final file size will be correct, but the Download object will still have the incorrect total size."

Yep same problem in my case. For right now I have fallen back to 3.1.4 version but it feels more like getting out from the shower and stand under the rain, because 3.1.4 has a ton of bugs and issues that were all fixed in 3.1.5 as the dev himself said that he doesn't bother his lazy bum to work in this library. So now I'm in pitfall of going with a buggy version or using a new version but implementing my own way of calculating the file size until the dev bother himself to fix the issue. So my idea is to turn off the database in fetch configuration and save download information my own way using a SQL or NoSQL method (maybe PaperDB), and implement my own code in onProgress for calculating the file size. Simply like: val x = File(targetFile) val size = x.length() progress = size * xyz / 100... yada yada...

The other workaround is to completely override the DatabaseManager in Fetch config and override each and every function which takes a lot of effort and time so I prefer the first way.

But this is just a quick hack for the time being.

Also not to forget mentioning that enabling or disabling preAllocate doesn't make a change in the value, either ways you get the wrong file size. And also it is not just when you kill and restart, it also happens even when you don't kill and just pause or cancel.

By the way this is the example project in case if you need to take a look... It is simple and doesn't have a lot of stuff just a download function and a custom resolver class.

https://drive.google.com/file/d/1OzLUJwdPgcn7JDsYtRoP_fxZX6f08d3t/view?usp=sharing

RAR password: 123

Oh, and let me thank you very much for your reply. That's so kind of you ^_^

XiXiongMaoXiong avatar Oct 06 '20 11:10 XiXiongMaoXiong

@NotNotMarshall, @Dezley-VROwl does this happen with you in Parallel File Download or Sequential File Download

badrazizi avatar Nov 17 '20 16:11 badrazizi

@badrazizi On both settings

Dezley-VROwl avatar Nov 17 '20 16:11 Dezley-VROwl

When you kill the app while it's downloading and then restart the app, it adds the amount of bytes that have been downloaded up to that point to the file. So for example with Big Bunny, the initial file size is 151 MB. If it has downloaded 50 MB and you kill and restart the app, the file size will be set to 201 MB.

i was able to reproduce this only with Sequential File Download, Parallel File Download give me correct total size

EDIT: just confirmed this happens with pausing & resuming as well. To see it clearly: let the download progress till ~50%, then pause it and resume it. The progress will fall to somewhere in the 30s and the download finishes at 60/70s. Even no need to kill the app.

@Dezley-VROwl i could not reproduce it, could share a link to test with.

badrazizi avatar Nov 17 '20 18:11 badrazizi

@NotNotMarshall, @Dezley-VROwl does this happen with you in Parallel File Download or Sequential File Download

Both

XiXiongMaoXiong avatar Nov 20 '20 11:11 XiXiongMaoXiong

When you kill the app while it's downloading and then restart the app, it adds the amount of bytes that have been downloaded up to that point to the file. So for example with Big Bunny, the initial file size is 151 MB. If it has downloaded 50 MB and you kill and restart the app, the file size will be set to 201 MB.

i was able to reproduce this only with Sequential File Download, Parallel File Download give me correct total size

EDIT: just confirmed this happens with pausing & resuming as well. To see it clearly: let the download progress till ~50%, then pause it and resume it. The progress will fall to somewhere in the 30s and the download finishes at 60/70s. Even no need to kill the app.

@Dezley-VROwl i could not reproduce it, could share a link to test with.

This will reproduce the error: https://drive.google.com/file/d/1OzLUJwdPgcn7JDsYtRoP_fxZX6f08d3t/view?usp=sharing RAR password: 123

XiXiongMaoXiong avatar Nov 20 '20 11:11 XiXiongMaoXiong

@NotNotMarshall I have some time this week to look into this issue. Will test and report back.

tonyofrancis avatar Dec 22 '20 00:12 tonyofrancis

@NotNotMarshall I have some time this week to look into this issue. Will test and report back.

Thank you very much, I appreciate your hard work! ♥

XiXiongMaoXiong avatar Dec 23 '20 11:12 XiXiongMaoXiong

@NotNotMarshall Thanks. I am working on releasing the new version today. Will report back

tonyofrancis avatar Dec 23 '20 15:12 tonyofrancis

@NotNotMarshall A new version of Fetch 3.0.12 & AndroidX version 3.1.6 has been released

tonyofrancis avatar Dec 23 '20 20:12 tonyofrancis

@NotNotMarshall A new version of Fetch 3.0.12 & AndroidX version 3.1.6 has been released

Awesome!

XiXiongMaoXiong avatar Dec 26 '20 10:12 XiXiongMaoXiong