elastalert icon indicating copy to clipboard operation
elastalert copied to clipboard

Setting the Version of Elastalert still uses latest elasticsearch python package

Open surfer190 opened this issue 5 years ago • 2 comments

After running make v=v0.1.39 and running the container I ge the following error message:

So I want to use elasticsearch 6.

11:19:49.314Z ERROR elastalert-server:
    ProcessController:      client.start()
      File "/opt/elastalert/elastalert/elastalert.py", line 1106, in start
        self.run_all_rules()
      File "/opt/elastalert/elastalert/elastalert.py", line 1158, in run_all_rules
        self.send_pending_alerts()
      File "/opt/elastalert/elastalert/elastalert.py", line 1534, in send_pending_alerts
        pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
      File "/opt/elastalert/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts
        size=1000)
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
        return func(*args, params=params, **kwargs)
    TypeError: search() got an unexpected keyword argument 'doc_type'

Looking at the log output from step 10 in the dockerfile, it downloads the latest elaticsearch:

Step 10/29 : RUN sed -i 's/jira>=1.0.10/jira>=1.0.10,<1.0.15/g' setup.py &&     python setup.py install &&     pip install -r requirements.txt
 ---> Running in 8a4eeb71b777
running install
...
Installed /usr/lib/python2.7/site-packages/envparse-0.2.0-py2.7.egg
Searching for elasticsearch
Reading https://pypi.org/simple/elasticsearch/
Downloading https://files.pythonhosted.org/packages/ae/43/38329621bcca6f0b97e1cc36fb3cef889414a1960fcdc83a41e26b496634/elasticsearch-7.0.2-py2.py3-none-any.whl#sha256=d1b176b87a7fb75dca82978c82a4023e8b21cbc98f4018cb51190fb0b8b43764
Best match: elasticsearch 7.0.2
Processing elasticsearch-7.0.2-py2.py3-none-any.whl
Installing elasticsearch-7.0.2-py2.py3-none-any.whl to /usr/lib/python2.7/site-packages
writing requirements to /usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/EGG-INFO/requires.txt
Adding elasticsearch 7.0.2 to easy-install.pth file

surfer190 avatar May 29 '19 11:05 surfer190

This might not be your problem though, I think the requirements file in elastalert may not be locked down to a specific version. Eg pip install elasticsearch==6.3.1

Yes, not your problem as can be seen: https://github.com/Yelp/elastalert/blob/v0.1.39/requirements.txt

The way I got it to work was by editing the DockerFile:

RUN sed -i 's/jira>=1.0.10/jira>=1.0.10,<1.0.15/g' setup.py && \
    python setup.py install && \
    pip install elasticsearch==6.3.1 && \
    pip install -r requirements.txt

surfer190 avatar May 29 '19 11:05 surfer190

vi Dockerfile

FROM alpine:latest as py-ea
ARG ELASTALERT_VERSION=v0.1.39
#ARG ELASTALERT_VERSION=v0.2.0b2
...
RUN sed -i 's/jira>=1.0.10/jira>=1.0.10,<1.0.15/g' setup.py && \
    sed -i 's/elasticsearch/elasticsearch==6.4.0/g' setup.py && \  
    python setup.py install && \
    pip install -r requirements.txt
...

vi Makefile

v ?= v0.1.39
#v ?= v0.2.0b2
...

build

docker pull alpine:latest && docker pull node:alpine
docker build -t elastalert .

run

docker run -d --rm -p 3030:3030 --net="host" elastalert:latest

juhwankim avatar May 30 '19 01:05 juhwankim