unzip-http icon indicating copy to clipboard operation
unzip-http copied to clipboard

offset out of range for 65536-byte buffer

Open sergeyvilov opened this issue 1 year ago • 6 comments

While attempting to download files from an ultra-large zip archive (355Gb) I got the following:

Traceback (most recent call last): File "/Users/sergey.vilov/tmp/test/test.py", line 5, in binfp = rzf.open('train_images/10005/18667/100.dcm') File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 192, in open f = list(self.matching_files(fn)) File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 186, in matching_files for f in self.files.values(): File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 109, in files self._files = {r.filename:r for r in self.infoiter()} File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 109, in self._files = {r.filename:r for r in self.infoiter()} File "/Users/sergey.vilov/miniconda/envs/kaggle/lib/python3.9/site-packages/unzip_http.py", line 151, in infoiter struct.unpack_from(self.fmt_cdirentry, resp.data, offset=filehdr_index) struct.error: offset -138557274 out of range for 65536-byte buff

The archive link can be obtained by downloading a Kaggle dataset from here. Unfortunately, I can't provide a direct link without exposing my kaggle credentials

sergeyvilov avatar Aug 31 '23 16:08 sergeyvilov