Elasticsearch type + Doctrine channel throwing errors

[DavidGarciaCat]

Good day,

For some reason, my Monolog's configuration with Elasticsearch just works for every channel but for Doctrine.

Other channels seem to work without errors.

This doctrine channel error was thrown when running Doctrine Migrations, but after disabling it for the migrations I have confirmed it also fails for Insert, Select and Update operations:

In ElasticaHandler.php line 120:

  Error sending messages to Elasticsearch


In Bulk.php line 337:

  Error in one or more bulk request actions:

  index: /monolog/_doc/7i8l-HcBRIsMsJkcYKtx caused object mapping for [context]
  tried to parse field [null] as object, but found a concrete value

And these are my configuration details:

# docker-compose.yml

version: '3.1'

services:

  # ...

  elasticsearch:
    container_name: myproject-elasticsearch
    image: elasticsearch:7.10.1
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      - discovery.type=single-node

  # ...

# config/packages/dev/monolog.yaml

monolog:
    handlers:
        main:
            type: stream
            path: "%kernel.logs_dir%/%kernel.environment%.log"
            level: debug
            handler: elasticsearch
        console:
            type: console
            process_psr_3_messages: false
            channels: ["!event", "!doctrine", "!console"]
        elasticsearch:
            type: elasticsearch
            elasticsearch:
                host: '%env(ELASTICSEARCH_HOST)%'
                port: '%env(ELASTICSEARCH_PORT)%'
#            channels: ["!doctrine"]

I have checked (connecting to Elaticsearch through a GUI) that other logs are stored as expected, including request, security and events channels among others.

Any suggestions to deal with this problem, or is this actually a bug?

Cheers,

[jderusse]

Are you sure Doctrine is pushing logs with a level High enough to trigger the handler?

[DavidGarciaCat]

If that was not the case, then would be possible getting errors that stop the execution?

[ProfChene]

Hi there,

We got the same issue in our project. I noticed that in the document sent to Elasticsearch, the context value for doctrine channel is an array of int instead of an object, as it is in other channels. This leads to mapping errors in Elasticsearch, thus it responds with an error to the curl call made by Elastica.

Here is an example of document for the event channel :

{
  "message": "Notified event \"{event}\" to listener \"{listener}\".",
  "context": {
    "event": "debug.security.authorization.vote",
    "listener": "Symfony\\Bundle\\SecurityBundle\\EventListener\\VoteListener::onVoterVote"
  },
  "level": 100,
  "level_name": "DEBUG",
  "channel": "event",
  "datetime": "2021-11-15T10:30:01.720644+00:00",
  "extra": []
}

And here is an example for the doctrine channel :

{
  "message": "ANY SQL REQUEST HERE",
  "context": [
    1
  ],
  "level": 100,
  "level_name": "DEBUG",
  "channel": "doctrine",
  "datetime": "2021-11-15T10:30:01.723233+00:00",
  "extra": []
}

As I understand it (am an Elasticsearch noob), all values should have the same format for a given index, otherwise Elasticsearch will have mapping errors. The context here have 2 different formats : object and array, so it doesn't work.

[derrabus]

Context should always be a JSON object, as far as I can tell. Do you want to work on a fix?

[DavidGarciaCat]

Yes, it must always be a JSON formatted value.

I'm not familiar with the source code of the bundle. Although I can take a look, it might take some time for me. Unless someone else wants to work on the bugfix

[derrabus]

No worries, take your time. If nobody has picked this up since March, I guess we don't need to rush anything now. 🙂

[DavidGarciaCat]

Well... I didn't know what happened because I am not familiar either with this bundle or the monolog library itself...

But I would definitively like to store my logs into elasticsearch and I can't as doctrine fails