camlzip
camlzip copied to clipboard
xavierleroy
ZIP64 support
Camlzip does not support ZIP64 extensions. We are currently running into limit of 64k files. Has anyone worked on adding ZIP64 support?
I agree it would be nice to overcome the limitations of the original ZIP format. I remember reading the ZIP64 specifications a long time ago and recoiling in horror :-( I don't think this is something I can implement all by myself, but I'd welcome contributions that go in the direction of supporting ZIP64.
Camlzip does not support ZIP64 extensions. We are currently running into limit of 64k files. Has anyone worked on adding ZIP64 support?
Out of curiosity, do you need ZIP64 support for reading, writing or both?
I agree the ZIP64 specification seems unnecessarily complicated, which was why I was wondering if anyone else had started on this. Or perhaps had implemented an Ocaml wrapper for libarchive.
We need to both write and read large zip files.
One thing we found is that the old ZIP format seems to be able to handle more than 64k files as long as an exception isn't raised, since the termination when reading zip file is based on file offsets and not file count. However, our zip files are between 3GB and 4GB and there are some other errors in camlzip in these situations.... converting from int32 to int64 sign-extends the high order bit in file offset, treating the 32-bit value as signed instead of unsigned.
Depending on what we do, we'll consider submitting contributions to camlzip.
Just to mention that I would also be interested in zip64 support: I'm running into this assert failure when reading some zip file and my guess is that this is due to the file being larger than 4GB (it's actually a 13GB file containing weights for a large model) - also happy to provide a repo if it helps.
