icsharpcode
Deflate64
SD-183, originally created on 7/21/2004 21:04:18 by John Reilly
Goes hand in hand with Zip64 pretty much, gives better compression that standard deflate
Comment from David Pierson on 9/2/2010 02:00:49: "Deflate64 is a proprietary and undocumented extension of the protocol"
Do we really want to do this in an open source project?
unzip on Ubuntu is able to extract an archive that uses Deflate64. Is open source really a barrier?
Hypothetically, would there be any interest in implementing this using the private Deflate64 implementation from CoreFX? (It's MIT licenced, and sharpcompress added decompression support by taking a cut of that code. I had a quick test and it does seem possible as a simple approach).
Saying that, I only tried with ZipFile, not sure of the ramifications on the zip input/output streams.
@Numpsy if you want to, sure go ahead! But no, I don't think this has been requested much and I have yet to find a Deflate64 file out in the wild.
I am closing this to keep the Issue count down, but I can open it again if there is further interest or activity.
But no, I don't think this has been requested much and I have yet to find a Deflate64 file out in the wild.
When Windows OS create archive with size more than 2GB it create it with Deflate64 method. And there are too many archives in world created with Deflate64. I'm a developer who is trying to find a library which can extract files from archives like this. I have checked 3 libraries and them can't extract files from Deflate64 archives.
- System.IO.Compression too slow, required .NET 4.6 to extract Deflate64 (but I have to use 4.5)
- DotNetZip can't extract Deflate64 (but has merge request with needed code!)
- SharpZipLib can't extract. (piksel closed this topic)
The reason for System.IO.Compression being 'slow' might be because (in older runtimes at least, I can't say offhand if .NET 5 is different), plain deflate used an optimized build of ZLib to do the decompression, where deflate64 used a managed implementation.
My thought (and from memory, the thought from adding support to DotNetZip) was to integrate the internal MS implementation, in which case the perf would be the same (if you're using .NET Core 3+ then that can be optimized with intrinsics, but that's another piece of work)
I believe (I haven't looked since I made that last comment from last year) that SharpCompress has previously integrated that same MS code, so maybe have a look at that?
Hey, checking whether there has been any movement regarding support for Deflate64 on the library. I am most interested in it supporting it on the InputStream path.
The reason I ask is the same @maximkotelnikov mentioned, Windows uses Deflate64 as default for large files, and it'd be nice to have this support.
No, not that I am aware of. But if windows creates files using deflate64 I guess this warrants keeping the issue open. That being said, that doesn't mean that it will be included anytime soon unless someone actually starts working on it.
I'd still hoped to have a go with the MS code at some point, but I haven't had any time to look at anything complicated recently :-(
Noticed that DotNetZip have implemented Deflate64 support when unzipping https://github.com/haf/DotNetZip.Semverd/pull/182 . Hopefully it won't be very difficult to port this to SharpZipLib. 🤔
Just want to add another voice to it: SharpZipLib works very well for my project. But because others can input any zip file they have, I sadly also came across the "Deflate64 not supported" issue. It would still be great if that would be added. If possible, it would also be nice if the exception "Compression method not supported" would have specifically stated "Deflate64 compression method not supported" to save time on understanding what goes wrong. Thanks!
