tools
tools copied to clipboard
WIP (do not merge): what if we went completely insane with bitmap formats
executive summary: gosh, this container thing is a cool idea. What if it was RECURSIVE? What if each 16-bit space was treated as 256 8-bit spaces? Actually... that sorta works. Sorta.
note that the in-memory representation is larger than the on-disk representation. The intent here is to have a thing which is extra squished for disk storage.
On sample/test fields (where nXXX means XXX/1000 bits are set, randomly):
n001 flaggy naive omitting roaring
array 160: 25125 56402 24805 20644
n004 flaggy naive omitting roaring
array 160: 73555 87656 73239 83152
n050 flaggy naive omitting roaring
array 160: 564057 568537 568857 1044914
n100 flaggy naive omitting roaring
bitmap 160: 1079106 1083586 1083906 1310720
n200 flaggy naive omitting roaring
bitmap 160: 1352279 1356759 1357079 1310720
n500 flaggy naive omitting roaring
bitmap 160: 1352320 1356800 1357120 1310720
n900 flaggy naive omitting roaring
bitmap 160: 1079912 1084392 1084712 1310720
n950 flaggy naive omitting roaring
bitmap 160: 563678 568158 568478 1310720
n996 flaggy naive omitting roaring
run 160: 83479 87959 88279 167876
n999 flaggy naive omitting roaring
run 160: 52194 56674 56994 43276
total flaggy naive omitting roaring
run 320: 135673 144633 145273 211152
array 480: 662737 712595 666901 1148710
bitmap 800: 5427295 5449695 5451295 6553600
total 1600: 6225705 6306923 6263469 7913462