tools icon indicating copy to clipboard operation
tools copied to clipboard

WIP (do not merge): what if we went completely insane with bitmap formats

Open seebs opened this issue 5 years ago • 0 comments

executive summary: gosh, this container thing is a cool idea. What if it was RECURSIVE? What if each 16-bit space was treated as 256 8-bit spaces? Actually... that sorta works. Sorta.

note that the in-memory representation is larger than the on-disk representation. The intent here is to have a thing which is extra squished for disk storage.

On sample/test fields (where nXXX means XXX/1000 bits are set, randomly):

n001                 flaggy      naive   omitting    roaring 
array       160:      25125      56402      24805      20644 
n004                 flaggy      naive   omitting    roaring 
array       160:      73555      87656      73239      83152 
n050                 flaggy      naive   omitting    roaring 
array       160:     564057     568537     568857    1044914 
n100                 flaggy      naive   omitting    roaring 
bitmap      160:    1079106    1083586    1083906    1310720 
n200                 flaggy      naive   omitting    roaring 
bitmap      160:    1352279    1356759    1357079    1310720 
n500                 flaggy      naive   omitting    roaring 
bitmap      160:    1352320    1356800    1357120    1310720 
n900                 flaggy      naive   omitting    roaring 
bitmap      160:    1079912    1084392    1084712    1310720 
n950                 flaggy      naive   omitting    roaring 
bitmap      160:     563678     568158     568478    1310720 
n996                 flaggy      naive   omitting    roaring 
run         160:      83479      87959      88279     167876 
n999                 flaggy      naive   omitting    roaring 
run         160:      52194      56674      56994      43276 
total                flaggy      naive   omitting    roaring 
run         320:     135673     144633     145273     211152 
array       480:     662737     712595     666901    1148710 
bitmap      800:    5427295    5449695    5451295    6553600 
total      1600:    6225705    6306923    6263469    7913462 

seebs avatar Jun 16 '19 14:06 seebs