[BUG] Opensearch Alerting: Per Document Monitor not including document field(s) in alert message

[ashishkumar256]

I am newbie to Opensearch. Apologies in advance in case wrong terminologies are used. We recently started to use OpenSearch as logging database.

We are trying to setup Alerting in Opensearch. As we are using Opensearch for storing application logs, therefore we more interested to go ahead with Per document monitor in order to apply more complex query based on business needs.

We have configured Per document monitor with below extraction query snippet -

{
    "description": "",
    "queries": [
        {
            "id": "log-level",
            "name": "log-level",
            "query": "status:\"error\"",
            "tags": []
        }
    ]
}

Its working well. Now the requirement is to pipe app_log.message in alerts message (to slack) -

"_source": {
   "app_log": {
       "message": "[No Bank Account id found with XX-YY-ZZ]",
       "caller": "/app/internal/usecase/implementation/xyz.go:1801",
       "level": "error"
   },
   "status": "error"
}

We tried below snippet but no luck worked-

{{#ctx.results.0.hits.hits}} 
Message: {{_source.app_log.message}}
{{/ctx.results.0.hits.hits}}

We already tried with couple of blogs available on internet -

1. https://forum.opensearch.org/t/how-do-i-include-message-fields-in-the-alert-action-message-for-per-document-monitor/14428
2. https://forum.opensearch.org/t/how-do-i-include-message-fields-in-the-alert-action-message/8167/12
3. https://github.com/opensearch-project/alerting/issues/479

Apart from that, there is one more issue want to highlight is Per document monitor does work with nested fields ("query": "app_log.level:\"error\"") -

{
    "description": "",
    "queries": [
        {
            "id": "log-level-error",
            "name": "log-level-error",
            "query": "app_log.level:\"error\"",
            "tags": []
        }
    ]
}

Let me know if more information is required.

[engechas]

Hi - we added sample docs to the ctx variable in #1450. Could you check if this meets your use case?