z-run icon indicating copy to clipboard operation
z-run copied to clipboard

Add support for compressing scriptlets bodies

Open cipriancraciun opened this issue 5 years ago • 1 comments

At the moment compiled scriptlets libraries are stored in CDB with the body of the scriptlets uncompressed. (Also for the SSH feature they are also transmitted over the network without compression.)

In some cases, where the scriptlet body is above a certain threshold (say 4 KiB), it would be useful to use a quick compression mechanism such as zstd.

(However the usage of a compression library shouldn't add any dynamic libraries to the executable.)

cipriancraciun avatar Apr 20 '20 07:04 cipriancraciun

Quick experimentation with gzip based compression yields around 50% compression at almost negligible performance impact. (I.e. a 40 MiB CDB library is compressed to around 24 MiB.)

However, in this experiment compression is applied for each individual value larger than 1 KiB, which means that the compression is not very efficient. (Compressing the whole 40 MiB file with gzip --fast yields ~6.4 MiB; but this wouldn't be applicable in our case...)

Given that this optimization is worth only for large libraries (the 40 MiB test library has ~16K snippets), I think it's safe to postpone this feature.

cipriancraciun avatar Jun 18 '20 19:06 cipriancraciun