ezgzip icon indicating copy to clipboard operation
ezgzip copied to clipboard

Experimenting with decompress

Open hcarty opened this issue 4 years ago • 2 comments

Early experiment using decompress instead of camlzip. Still very incomplete so this is draft PR.

Initial impressions:

  • Compression seems to be slower (not unexpected) and less space efficient (maybe unexpected?) than camlzip/zlib
  • Decompress's gzip support is easy to work with
  • It should be fairly straightforward to add bigstring/bigarray support to ezgzip

hcarty avatar Apr 26 '20 19:04 hcarty

Compression seems to be slower

Note that the underlying LZ77 implementation used to compress is not the best as I can - but I don't have the time to focus on this part.

less space efficient (maybe unexpected?) than camlzip/zlib

I'm interesting about this feedback. By essence, it's sure that decompress is less space efficient than zlib where it needs to allocates OCaml values (which is, by definition, bigger than C data-structures used by zlib). However, I would like to know if memory consumption is stable for a large GZip flow or if decompress continuously allocates and do a pressure on the GC.

dinosaure avatar Sep 11 '20 09:09 dinosaure

Just to revive a bit the PR, decompress.1.3.0 was out and it should be better about compression:

  • a bug was found about a systematic internal allocation of a static array (it was deleted)
  • levels are implemented and they propose not exactly the same ratio as zlib (see https://github.com/mirage/decompress/pull/108)

dinosaure avatar Mar 14 '21 09:03 dinosaure