downloader-cli
downloader-cli copied to clipboard
doesn't skip fully downloaded files
Seems to download unnecessary data onto already completed files instead of skipping it. The file size just ends up larger than the original size. So maybe add a condition for cur_size >= int(original_size)
or something
https://github.com/deepjyoti30/downloader-cli/blob/dd0b8c6d8f6794e7d0e05e6f7764841de8ace302/downloader_cli/download.py#L118-L123
@dhuang1 Can you give me steps to perhaps reproduce the error?
I am not able to understand what you said. By default, downloader-cli
will throw an error if a file already exists. If you try to resume download, it will do that and if you try to overwrite, it will do that. So when exactly are you getting this error?
If you set continue_download=True
and download the exact same file multiple times, it appends data onto the existing file
from downloader_cli.download import Download
dest = "5MB.zip"
url = "http://212.183.159.230/5MB.zip"
# In this case, 5MB.zip will have a file size of 15MB (3*5MB)
n = 3
for _ in range(n):
Download(url, des=dest, continue_download=True).download()
@dhuang1 I copied exactly your code and ran it. I have a file with just 5MB size.
I went ahead and downloaded the same file with curl and then checked the md5 of both the files:
❯ md5sum 5MB.zip
b3215c06647bc550406a9c8ccc378756 5MB.zip
~
❯ md5sum 5MB.curl.zip
b3215c06647bc550406a9c8ccc378756 5MB.curl.zip
Those are exactly same which means the file was not corrupted.
@deepjyoti30 Ah, I think it's because I use windows 10. I tried it out in linux and works properly
@dhuang1 I will try to reproduce on Windows next time I have access to a machine!
Windows always throws weird errors!