powturbo

Testing [TurboPFor](https://github.com/powturbo/TurboPFor), I've made some experiments with nanopore signal data. File: all 16-bits signal data extracted from [multi_fast5_zip.fast5](https://github.com/nanoporetech/vbz_compression/tree/master/test_data) ./icapp sig.u16 -Elzturbo,39 -Fs -e83,77,11,39,51,30,76 file: max bits histogram: 04: 0.000% 06:...

TurboPFor: IcApp Integer Compression Benchmark ARM A73-ODROID-N2 1.8GHz [Inverted index testdata](https://github.com/ot/partitioned_elias_fano/tree/master/test/test_data) docs: Document Ids ./icapp test_collection.docs file: max bits histogram: 00: 0.002% 01:## 1.7% 02:# 0.7% 03: 0.4% 04: 0.2%...

TurboPFor: IcApp Integer Compression Benchmark - Skylake i7-6700 3.4GHz [Inverted index testdata](https://github.com/ot/partitioned_elias_fano/tree/master/test/test_data) docs: Document Ids ./icapp test_collection.docs file: max bits histogram: 00: 0.002% 01:## 1.7% 02:# 0.7% 03: 0.4% 04:...

TurboBench: Maximum Compression Benchmark - Skylake i7-6700 3.4GHz

3

Latest update: 2019.08.09 Binary:mingw-w64_x86_64-6.3.0-posix-seh-rt_v5-rev2.tar C Size ratio% C MB/s D MB/s Name (bold = pareto) MB=1.000.0000 42179292 9.1 0.91 189.42 lzturbo 49 42457543 9.1 2.07 186.33 lzma 9d29:fb273:mf=bt4 42988097 9.2...

[Quantile Compression/PCodec](https://github.com/mwlon/quantile-compression) is claiming 35%-71% better compression than zstd. I've integrated the rust library into TurboPFor using the ffi bindings for comparison purpose. We use the synthetic [dataset](https://github.com/mwlon/pcodec/tree/main/quantile-compression/q_compress/assets) provided in...

Most of time series databases (ex. DuckDB) are storing floating point data as 64 bits. They are reporting some [extraordinary](https://github.com/panagiotisl/chimp/issues/2) compression ratio by using a gorilla/chimp like algorithm. However as...

> icapp ZIPF zipf alpha=1.50 range[0..255].n=1000000 bits histogram: 00:######################################## 40% 01:############## 14% 02:############# 13% 03:########## 10% 04:######## 7.7% 05:###### 5.7% 06:#### 4.1% 07:### 2.9% 08:## 2.1% size ratio E MB/s...

- 2D/3D datasets: +[Float Compression dataset](https://github.com/aras-p/float_compr_tester/tree/main/data) : [description](https://aras-p.info/blog/2023/01/29/Float-Compression-0-Intro/) +[Datasets for Benchmarking Floating-Point Compressors](https://dps.uibk.ac.at/~fabian/datasets/) - Lossless floating point compression: icapp sq1024x1024x4.f32 -R0 -Ff -I15 -J15 -e105,143,80,102 -Ezstd,15 option -R0 = automatic...

TurboBitByte (=TurboByte+TurboPack) compress considerably better and can be more than 3 times faster than streamvbyte. icapp ZIPF -I15 -J15 -v0 -e45,50,55,130 zipf alpha=1.50 range[0..255].n=1000000 E MB/s size ratio D MB/s...

Quantization of the floating point mantissa by zeroing the rightmost bits according to a maximum allowed pointwise relative error. - Floating point datasets: +[Float Compression dataset](https://github.com/aras-p/float_compr_tester/tree/main/data) : [description](https://aras-p.info/blog/2023/01/29/Float-Compression-0-Intro/) +[Datasets for...