Benchmarks for binary vectors?

[thvasilo]

FAISS and I assume some other libraries include optimized algorithms for binary features (both exact like Multi-Index Hashing and approximate).

I was wondering if there are any plans to include such specializations? I assume SIFT features would be good candidates for data.

[maumueller]

This already exists with two datasets at http://ann-benchmarks.com/ (Distance: Hamming). In particular, MIH is supported already.

Would be nice to have the FAISS binary variants, though. Feel free to add it via a PR. :-)

[thvasilo]

Gave it a shot today, but I think I want to clarify a couple of things. Is it OK to open a WIP PR to ask for help fixing a couple of potential issues?

[maumueller]

Sorry for the late answer. Please submit a [WIP] PR, yes.

[tim-hilt]

This already exists with two datasets at http://ann-benchmarks.com/ (Distance: Hamming)

@maumueller i couldn't find it when i looked there today! Were binary benchmarks excluded again?

[maumueller]

Yes, I think the website currently reflects only the latest run of @erikbern. We could put it into the results manually, but this will mix up different versions of implementations. :-(

[erikbern]

I can include it next time I run the benchmarks!

[tim-hilt]

@erikbern that would be great, thanks! :+1:

[tim-hilt]

@erikbern Any progress on this?

[erikbern]

Sorry, haven't had time to work on it. I hope to have plenty of free time in March

[tim-hilt]

Thanks for the feedback @erikbern :) Great to here, that it's not forgotten!

[Charlyo]

Any new regarding this? Would appreciate having binary vectors benchmarks :)