zip icon indicating copy to clipboard operation
zip copied to clipboard

allow passing flags to zip_open via open

Open brentp opened this issue 4 years ago • 5 comments

AFAICT, files larger than 4.2 GB are not actually supported because we need to pass: ZIP_CM_DEFLATE64 to zip_open, but that's not possible via zip.open

I can open a PR changing:

proc open*(z: var ZipArchive, filename: string, mode: FileMode = fmRead): bool =

to

proc open*(z: var ZipArchive, filename: string, mode: FileMode = fmRead, flags:int32=0): bool =

here is the code I am using to check the problem. should fail for any file > 4.2GB.

import zip/zipfiles
import zip/libzip

var z:ZipArchive

# 5 GB zip file
var path  = "gnomad.v3.genome.zip"

if not z.open(path):
  # does not open and no way ? to get err value
  echo "couldn't open"

import zip/libzip

var err:int32
var flags:int32

flags = ZIP_CM_DEFLATE64

var w = zip_open(path, 0, addr(err))
# gives not a zip archive
echo err


err = 0
w = zip_open(path, flags, addr(err))
# gives 0
echo err

brentp avatar Oct 22 '19 15:10 brentp

just adding that flag causes zip_open not to error, but it doesn't resolve the underlying issue. not sure how to fix.

brentp avatar Oct 22 '19 16:10 brentp

can't we always add this flag in the implementation? Why expose this silly non-sense to zip's users.

Araq avatar Oct 23 '19 13:10 Araq

that seems fine, but I can't get it to work even with that flag. the file will open, but I can't extract. I'll look into it more.

brentp avatar Oct 23 '19 15:10 brentp

http://home.chpc.utah.edu/~u6000771/gnomad.v3.genome.zip

here is a test archive. the first file in the archive has a compressed size > 4.2GB. which i think is creating additional problems.

brentp avatar Oct 23 '19 22:10 brentp

https://github.com/nih-at/libzip/blob/178c445ef82beae9670ca68668865082eb0e1e91/lib/zip.h#L157 ZIP_CM_DEFLATE64 commented as Reserved for Tokenizing compression algorithm so may not the right use here

bung87 avatar May 14 '20 20:05 bung87