intelmq

intelmq copied to clipboard

Bot has found a problem.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/intelmq/lib/bot.py", line 272, in start
    self.process()
  File "/usr/lib/python3/dist-packages/intelmq/bots/outputs/elasticsearch/output.py", line 126, in process
    self.es.index(index=self.get_index(event_dict, default_date=datetime.today().date()),
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/client/utils.py", line 168, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/client/__init__.py", line 406, in index
    return self.transport.perform_request(
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/transport.py", line 458, in perform_request
    raise e
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/transport.py", line 419, in perform_request
    status, headers_response, data = connection.perform_request(
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/connection/http_urllib3.py", line 277, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python3.8/dist-packages/elasticsearch/connection/base.py", line 330, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Could not dynamically add mapping for field [extra.source.sector]. Existing mapping for [extra.source] must be of type object but found [text].')

I believe this is because one of your feeds has mapped extra.source to a text value, while others like Shadowserver and CTIP use values like extra.source.sector and extra.source.connection_type, which expect extra.source to be an object with subvalues. I have not noticed such an issue before, and no bot in this repo seems to contain any explicit mapping to extra.source. Perhaps you could look at the entries in your Elasticsearch database in order to figure out where they came from? Maybe you have a custom bot that added this field?