Add ztstandard compression format

[ross-spencer]

Format name

Zstandard.

Alias: zstd.

Version number

It is unclear what would differentiate a version. Because of the standardization of its magic number since 0.8.0 (now on 1.5.7) it may only be feasible to identify zstd broadly. See magic numbers below for more information, but previous magic numbers might also be used in the same generic record, or used to create up to seven other records (I'd plump for generic).

Extensions

zst

MIME/Media Type

application/zstd

Description

Via: http://fileformats.archiveteam.org/wiki/Zstd

Zstandard, or Zstd, is a general-purpose compression algorithm and file format. It is designed to have a good balance between compression ratio, compression speed, and decompression speed.

Format type

Aggregate.

Vendor

Facebook.

File format identification signatures

Its magic number is: 28 b5 2f fd (see: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/zstd.h#L142) it is unclear if the little-endian version 0xFD2FB528 described in code and in the spec (https://datatracker.ietf.org/doc/html/rfc8878#name-frames) would appear on-disk on any platform. https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/zstd.h#L142

The source repository for zstandard describes legacy magic numbers in their various .h files, e.g.

0.1.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v01.h#L86-L87 0.2.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v02.h#L86 0.3.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v03.h#L86 0.4.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v04.h#L135 0.5.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v05.h#L153 0.6.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v06.h#L164 0.7.0: https://github.com/facebook/zstd/blob/6e58e9a2c42ea344708b72a14052b3bd4581e118/lib/legacy/zstd_v07.h#L180

Relevant links, documentation, extra information

• https://github.com/facebook/zstd
• https://datatracker.ietf.org/doc/html/rfc8478
• http://fileformats.archiveteam.org/wiki/Zstd

Credit

Ross Spencer. (No affiliation).

Example

Using zstd <filename> on Linux we can get a simple example:

00000000  28 b5 2f fd 24 0d 69 00  00 68 65 6c 6c 6f 20 50  |(./.$.i..hello P|
00000010  52 4f 4e 4f 4d 0a 1f 55  f8 3a                    |RONOM..U.:|
0000001a

Other examples should be easy enough to create although I'm not sure what tools are available on Windows.