php-ext-zstd icon indicating copy to clipboard operation
php-ext-zstd copied to clipboard

Unspecified buffer size for decompression.

Open TheBlackPlague opened this issue 6 years ago • 3 comments

This extension doesn't save a decompressed size field value making it impossible to decompress strings compressed by this using any other binding such as ZstdNet.

TheBlackPlague avatar Apr 26 '20 21:04 TheBlackPlague

This extension support the various Zstd compression modes

  • simple / buffered, which save decompressed size needed by simple decompression mode
  • streaming, which doesn't, and thus requires stream decompression mode

Looks like the issue is that Zstdnet doesn't support streaming mode.

P.S. notice: this extension simply provides bindings on the libzstd, and implement most of its features

remicollet avatar Apr 27 '20 05:04 remicollet

From http://facebook.github.io/zstd/zstd_manual.html (for ZSTD_getFrameContentSize)

note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode. When return==ZSTD_CONTENTSIZE_UNKNOWN, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data.

remicollet avatar Apr 27 '20 06:04 remicollet

From http://facebook.github.io/zstd/zstd_manual.html (for ZSTD_getFrameContentSize)

note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode. When return==ZSTD_CONTENTSIZE_UNKNOWN, data to decompress could be any size. In which case, it's necessary to use streaming mode to decompress data.

This extension support the various Zstd compression modes

  • simple / buffered, which save decompressed size needed by simple decompression mode
  • streaming, which doesn't, and thus requires stream decompression mode

Looks like the issue is that Zstdnet doesn't support streaming mode.

P.S. notice: this extension simply provides bindings on the libzstd, and implement most of its features

Thank you for the information. However, we were using non-stream based compression and still lacking the decompressed size. It would be nice to add it in for non-stream based compression.

TheBlackPlague avatar May 01 '20 11:05 TheBlackPlague