vortex
vortex copied to clipboard
Chunked array statistical compression
As a chunked array loops over its chunks, the compressor could pick the most recently used compression scheme provided the resulting compression ratio remains within some bound.
This would short-cut sampling, but hopefully in a way that allows the compressor to bail out and re-sample if the patterns in the data change. Could also periodically re-sample in the background.