unzip-http
unzip-http copied to clipboard
offset out of range for 65536-byte buffer
While attempting to download files from an ultra-large zip archive (355Gb) I got the following:
Traceback (most recent call last): File "/Users/sergey.vilov/tmp/test/test.py", line 5, in
binfp = rzf.open('train_images/10005/18667/100.dcm') File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 192, in open f = list(self.matching_files(fn)) File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 186, in matching_files for f in self.files.values(): File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 109, in files self._files = {r.filename:r for r in self.infoiter()} File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 109, in self._files = {r.filename:r for r in self.infoiter()} File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 151, in infoiter struct.unpack_from(self.fmt_cdirentry, resp.data, offset=filehdr_index) struct.error: offset -138557274 out of range for 65536-byte buff
The archive link can be obtained by downloading a Kaggle dataset from here. Unfortunately, I can't provide a direct link without exposing my kaggle credentials