TransportError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on')

[anjan-cele]

Hi @Qmando

When I am running example_frequency.yaml rule. I am getting following error:

ERROR:root:Error finding recent pending alerts: TransportError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on') {'sort': {'alert_time': {'order': 'asc'}}, 'query': {'bool': {'filter': {'range': {'alert_time': {'to': '2017-07-25T15:43:52.149157Z', 'from': '2017-07-23T15:43:52.149125Z'}}}, 'must': {'query_string': {'query': '!exists:aggregate_id AND alert_sent:false'}}}}} Traceback (most recent call last): File "/home/watchman/elastalert/elastalert/elastalert.py", line 1387, in find_recent_pending_alerts size=1000) File "build/bdist.linux-x86_64/egg/elasticsearch/client/utils.py", line 73, in _wrapped return func(*args, params=params, **kwargs) File "build/bdist.linux-x86_64/egg/elasticsearch/client/init.py", line 623, in search doc_type, '_search'), params=params, body=body) File "build/bdist.linux-x86_64/egg/elasticsearch/transport.py", line 312, in perform_request status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) File "build/bdist.linux-x86_64/egg/elasticsearch/connection/http_requests.py", line 90, in perform_request self._raise_error(response.status_code, raw_data) File "build/bdist.linux-x86_64/egg/elasticsearch/connection/base.py", line 125, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info) RequestError: TransportError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on')

When I am running my elastalert_status thru following command, it is giving output as : curl amdc2webl24:9200/elastalert_status/_mapping

{"elastalert_status":{"mappings":{"elastalert_error":{"properties":{"@timestamp":{"type":"date"},"data":{"properties":{"rule":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"message":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"traceback":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}}}},"elastalert_status":{"properties":{"@timestamp":{"type":"date"},"endtime":{"type":"date"},"hits":{"type":"long"},"matches":{"type":"long"},"rule_name":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":256}}},"starttime":{"type":"date"},"time_taken":{"type":"float"}}}}}}

If I am trying to recreate the index by - elastalert-create-index then getting following error 👍

Traceback (most recent call last): File "/usr/local/bin/elastalert-create-index", line 11, in load_entry_point('elastalert', 'console_scripts', 'elastalert-create-index')() File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 563, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2651, in load_entry_point return ep.load() File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2305, in load return self.resolve() File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 2311, in resolve module = import(self.module_name, fromlist=['name'], level=0) File "/home/watchman/elastalert/elastalert/create_index.py", line 12, in from auth import Auth ImportError: No module named 'auth'

I think index is creating for Python 3.5, If so then how can I map alert_time in index i.e elastalert_status for Python 2.7. Do I need to uninstall Python 3.5 which I can't because other applications need Python 3.5.

Please suggest. Thank you.

[1543471767]

I have this bug also

[Qmando]

The mapping occasionally fails to apply cleanly, I'm not entirely sure why. The fix for this is to delete the elastalert_status index and try running elastalert-create-index again.

Also, elastalert only runs on python2. If you have python3 as your default, you need to explicitly specify python2 setup.py install or pip2 install elastalert rather than just type "python"

[Cas-pian]

I met this bug too, es5.4.3 and elastalert 0.1.21. And recreate elastalert_status worked for me. Hope it's fixed in furture release.Thanks!

[pzelnip]

Same issue for us, and happened multiple times. Elastalert 0.1.21 and ES 5.1.1

Deleting & recreating the index fixed it, though the first time we did it it didn't resolve the problem (ie had to delete & recreate multiple times)

[pastechecker]

What if I do not have @timestamp mappings in my data because I have chosen to index using a custom field and it is not called timestamp but "mytime"?

I am running Kibana + Docker on Elastic/Kibana 6.4.2 elastalert_error - {'message': "Error running query: RequestError(400, u'search_phase_execution_exception', u'No mapping found for [@timestamp] in order to sort on')

[Qmando]

From the documentation: "All documents must have a timestamp field. ElastAlert will try to use @timestamp by default, but this can be changed with the timestamp_field option"

[miff2000]

In my case, using ES 5.5, I updated the mappings on the index to include one for alert_time (as it was missing) using this:

curl -X PUT https://my.elasticsearch.server/es/elastalert_status/_mapping/elastalert_status -d '{
    "properties": {
        "@timestamp": {
            "type": "date"
        },
        "alert_time": {
            "type": "date",
            "format": "dateOptionalTime"
        },
        "endtime": {
            "type": "date"
        },
        "hits": {
            "type": "long"
        },
        "matches": {
            "type": "long"
        },
        "rule_name": {
            "type": "text",
            "fields": {
                "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                }
            }
        },
        "starttime": {
            "type": "date"
        },
        "time_taken": {
            "type": "float"
        }
    }
}'

[loGit123]

Same issue for us, and happened multiple times. Elastalert 0.1.21 and ES 5.1.1

Deleting & recreating the index fixed it, though the first time we did it it didn't resolve the problem (ie had to delete & recreate multiple times)

How did you recreate the index?