ann-benchmarks
ann-benchmarks copied to clipboard
erikbern
Add LOPQ
https://github.com/yahoo/lopq
writeup from Flickr: http://code.flickr.net/2017/03/07/introducing-similarity-search-at-flickr/
Hi. I took a look into adding lopq (https://github.com/yahoo/lopq) , however currently it only supports python 2.7. I did some PR on the library but it seems it has no active maintainer for now. When someone merges my PR, I'll port it to Python 3.6 and add it to this benchmark.
@erikbern Is it possible to use the current python 2.7 version of lopq within the benchmark framework? Thanks.
@mohamed-ali That's no problem. In fact, faiss is still run on Python 2.7. Just see https://github.com/erikbern/ann-benchmarks/blob/master/install/Dockerfile.faiss. (Nothing has to be done on the algorithm wrapper put in ann-benchmarks/algorithms/....)
@maumueller Thanks for clarifying that (it saves me some time). I'll attempt to add LOPQ in here then.
@erikbern @maumueller In readme, section running there is only an example on how to run the benchmark in a production setup (if that's correct it to say), however I don't see anything about running for local testing/development. It would be useful to have also an example on how to run when developing(integrating a new algorithm for instance).
Regarding adding LOPQ, I made some progress, however, it seems there is no clear way to run the tests locally. For example, trying to use the docker ann-benchmarks doesn't work because it not accessible:
$ docker build -f install/Dockerfile.lopq -t ann-lopq .
Sending build context to Docker daemon 16.07MB
Step 1/6 : FROM ann-benchmarks
pull access denied for ann-benchmarks, repository does not exist or may require 'docker login'
I can keep debugging, however, if you have quick guidelines on how to run locally that would save me some time. Thanks.
Please use python3 install.py --algorithm lopq in the root. This script takes care of building the base image and lopq.
if my memory serves me right, i believe you can just do
python install.py --algorithm lopq
python run.py --algorithm lopq
does that work? we can add it to the documentation