Benchmarks?

[e271828-]

Are you planning to publish any basic benchmarks?

Also, nice project! The planned "advanced client-side sync" is also something we'd like to see.

[kristoff-it]

Hi, benchmarks are a bit of a tricky subject, as AFAIK this is the only Cuckoo Filter implementation that delegates hashing to the client. Any benchmark would not be directly comparable to other libraries.

When coding the library I created the various bucket size configurations so that they would be word-aligned, in order to ensure that a search requires a maximum of 2 reads from memory.

Additionally, the code doesn't have superfluous runtime branching thanks to Zig's compile-time branch elision, like in the following piece of code

https://github.com/kristoff-it/redis-cuckoofilter/blob/master/src/lib/zig-cuckoofilter.zig#L235-L245

The quoted loop is resolved as 2 or 4 statements (depending on the bucket size configuration) and the switch case is elided too, as it depends on a compile-time known value.

If you want to know more about Zig's comptime, I wrote an article about it.

So, going back to the benchmark idea, the code is very straightforward, Zig has D-style compile-time facilities, and it uses LLVM to compile, all of which, combined with the fact that a benchmark would not be directly comparable with any other implementation, made me avoid the subject entirely.

If you are interested in doing some benchmarking, I'll be happy to quote (and properly attribute) any findings in the readme.

About the client-side sync idea, to make it worthwhile, first we need client-side implementations of this cuckoo filter library. I started the work by separating the business logic in https://github.com/kristoff-it/zig-cuckoofilter and I've also included in the c-abi-examples directory how to embed it in a few popular languages. The next step would be to publish each as a real package for each language.

Once that's done, it shouldn't be too hard to wire everything together using a Redis Stream as a "commit log", for example. Unfortunately my attention is currently elsewhere so I don't have a real ETA on if / when I'll do it. That said, I'd be happy to see somebody else maintaining a language package for this cuckoo filter library.

Sorry, I realize this is a willingness to help with 0/2 issues, but if you do start using the library and encounter any problem, I'll be happy to help you diagnose it.