flow-dps icon indicating copy to clipboard operation
flow-dps copied to clipboard

Regenerate dictionaries using mainnet checkpoint for samples

Open Ullaakut opened this issue 3 years ago • 0 comments

Description

The current dictionaries were generated by sampling data against our snapshot, which contains realistic data but of less variety than a proper mainnet checkpoint would.

It would probably improve compression further to index a mainnet checkpoint completely and then regenerate the dictionaries, or to change the sample generation for dictionaries to read from checkpoints instead of reading from our index.

Ullaakut avatar Dec 10 '21 06:12 Ullaakut