okayzed

tried out DeDiS protobuf, actually slower. so... not sure what that means, but look in https://github.com/logv/sybil/tree/protobuf for the diffs. compared 2million timestamps, 300ms load for gob, 400 - 600ms for...

look into https://github.com/andyleap/gencode

evan suggests we can do this using bloom filters and hierarchical bloom filters. this will work for equality but not regex, as far as i can tell

An initial implementation now exists for simple equality on strings on a per block basis, but it is not using bloom filters

I'm conflicted on this. On the one hand, timestamps are not required (or are not required to be named "time") but they are generally useful to have in most datasets....

Steps that we've taken to get to distributed queries * lower mem usage during aggregation * switch to using loglogbeta for count distinct queries (definitely saves RAM) * add encoder...

my first attempt was 1) slower than filtering after loading and 2) giving wrong answers. :cry:

the gRPC service essentially was a new flag on the sybil binary that allows you to specify a remote host to connect to and execute a sybil query on. the...

is there a particular reason you want this change / architecture? sybil is written as a single invocation binary for a given operation instead of a server process. you'll find...

id also gladly accept a library for sybil that wraps sybil cli calls so it can be used in the way you describe.