elastik-nearest-neighbors icon indicating copy to clipboard operation
elastik-nearest-neighbors copied to clipboard

Create docker image for easier demoing

Open henrywallace opened this issue 6 years ago • 3 comments

I think learning how to use the plugin could be facilitated by providing a docker image that one just docker run, and then start trying out the plugin.

So far, I've done the following:

$ cat setup.sh
#!/bin/sh

gradle clean build -x integTestRunner -x test
mv build/distributions/elasticsearch-aknn-0.0.1-SNAPSHOT.zip aknn.zip

With Dockerfile

FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.4

COPY aknn.zip /
RUN /usr/share/elasticsearch/bin/elasticsearch-plugin install file:///aknn.zip

henrywallace avatar Jun 20 '18 01:06 henrywallace

I agree this would be nice to have for simple demos. In the meantime, there is a script here which should contain all the steps necessary to setup a multi-node Elasticsearch cluster on AWS EC2 ubuntu instances: https://github.com/alexklibisz/elastik-nearest-neighbors/blob/master/demo/pipeline/ec2_es_setup.sh

alexklibisz avatar Jun 21 '18 02:06 alexklibisz

I am using the following Dockerfile for ES + aknn plugin, in the elasticsearch-aknn folder.

FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.4

ADD . /aknn
WORKDIR /aknn


# Install Java 10
# Before building this image, download the .rpm files to elasticsearch-aknn directory
#     from http://www.oracle.com/technetwork/java/javase/downloads/index.html

RUN yum -y install jdk-10.0.2_linux-x64_bin.rpm
RUN yum -y install jre-10.0.2_linux-x64_bin.rpm
ENV JAVA_HOME=/usr/java/jdk-10.0.2/


# Install gradle 4.9

RUN wget https://services.gradle.org/distributions/gradle-4.9-bin.zip
RUN mkdir /opt/gradle
RUN unzip -d /opt/gradle gradle-4.9-bin.zip
ENV PATH=$PATH:/opt/gradle/gradle-4.9/bin


# Build & install the plugin

RUN gradle clean build -x integTestRunner -x test
RUN elasticsearch-plugin install -b file:build/distributions/elasticsearch-aknn-0.0.1-SNAPSHOT.zip

# Configure ElasticSearch
ENV ES_JAVA_OPTS="-Xms10g -Xmx10g"

mattiasarro avatar Jul 26 '18 13:07 mattiasarro

There's a demo based on mnist with docker image here: https://github.com/sinemetu1/elastik-nearest-neighbors/blob/add-docker/Dockerfile and this makefile: https://github.com/sinemetu1/elastik-nearest-neighbors/blob/add-docker/Makefile

cd elastik-nearest-neighbors/ && make demo

then query for documents and use the plugin:

curl "http://localhost:9201/mnist_images/_search?size=1" \
  | jq -r '.hits.hits[] ._source.label, .hits.hits[] ._id'

curl "http://localhost:9201/mnist_images/mnist_images/1001/_aknn_search?k1=50&k2=10&pretty" \
  | jq -r '.hits .hits[0:3]'

sinemetu1 avatar Feb 24 '19 14:02 sinemetu1