Replace zlib with a faster implementation and remove sources from repo

Open flsobral opened this issue 4 years ago • 1 comments

HELP WANTED

Help us by suggesting other libraries to consider or making the build and the benchmark.

Currently we use the version 1.2.8 (possibly customized for usage with the tcvm - have yet to check) The objective is to replace the current implementation with another one with improved performance and COMPATIBLE API.

This library is responsible for the deflate, zlib and gzip algorithms, and using a faster implementations leads to better performance for HTTP connections, faster loading of PNG files in general, and faster loading of resources included in tcz files.

Our main options are:

czlib (chromium's zlib) - https://source.chromium.org/chromium/chromium/src/+/master:third_party/zlib/
cloudflare-zlib - https://github.com/cloudflare/zlib
original zlib - https://github.com/madler/zlib

zlib-ng or any other library with incompatible API is NOT an option, the main objective is to improve performance of features that already rely on zlib API. The usage of alternative compression algorithms will be discussed later in another issue.

Relevant stuff:

Tasks:

[ ] Build all the previous listed options for benchmarking, preferably for arm32v7 (please, include a link for a repo with complete build)
[ ] Benchmark (use one of the many available benchmarks out there for zlib or make your own), preferably using stuff from Silesia compression corpus, like this tar used to benchmark Lizard
[ ] Create patches for custom tcvm code if necessary (hopefully not)
[ ] Build it automatically as a static library using CMake with SIMD enabled
[ ] Remove zlib sources from our repo
Test new build on every target
- [ ] Linux x64
- [ ] Linux arm32v7
- [ ] Linux arm64
- [ ] Android
- [ ] iOS
- [ ] Win32
- [ ] WinCE
- [ ] MacOS (only used for debugging, but that's important for development)

Mar 19 '21 18:03 flsobral