downloader-cli icon indicating copy to clipboard operation
downloader-cli copied to clipboard

doesn't skip fully downloaded files

Open ghost opened this issue 3 years ago • 5 comments

Seems to download unnecessary data onto already completed files instead of skipping it. The file size just ends up larger than the original size. So maybe add a condition for cur_size >= int(original_size) or something https://github.com/deepjyoti30/downloader-cli/blob/dd0b8c6d8f6794e7d0e05e6f7764841de8ace302/downloader_cli/download.py#L118-L123

ghost avatar Apr 23 '21 09:04 ghost

@dhuang1 Can you give me steps to perhaps reproduce the error?

I am not able to understand what you said. By default, downloader-cli will throw an error if a file already exists. If you try to resume download, it will do that and if you try to overwrite, it will do that. So when exactly are you getting this error?

deepjyoti30 avatar Apr 24 '21 05:04 deepjyoti30

If you set continue_download=True and download the exact same file multiple times, it appends data onto the existing file

from downloader_cli.download import Download

dest = "5MB.zip"
url = "http://212.183.159.230/5MB.zip"

# In this case, 5MB.zip will have a file size of 15MB (3*5MB)
n = 3
for _ in range(n):
    Download(url, des=dest, continue_download=True).download()

ghost avatar Apr 24 '21 14:04 ghost

@dhuang1 I copied exactly your code and ran it. I have a file with just 5MB size.

I went ahead and downloaded the same file with curl and then checked the md5 of both the files:

❯ md5sum 5MB.zip
b3215c06647bc550406a9c8ccc378756  5MB.zip

~
❯ md5sum 5MB.curl.zip 
b3215c06647bc550406a9c8ccc378756  5MB.curl.zip

Those are exactly same which means the file was not corrupted.

deepjyoti30 avatar Apr 25 '21 09:04 deepjyoti30

@deepjyoti30 Ah, I think it's because I use windows 10. I tried it out in linux and works properly

ghost avatar Apr 25 '21 13:04 ghost

@dhuang1 I will try to reproduce on Windows next time I have access to a machine!

Windows always throws weird errors!

deepjyoti30 avatar Apr 26 '21 06:04 deepjyoti30