kibana icon indicating copy to clipboard operation
kibana copied to clipboard

[Security Solution][Alerts] Exceptions on nested fields cause rule failure if nested field isn't mapped

Open marshallmain opened this issue 3 years ago • 3 comments

https://github.com/elastic/kibana/issues/121387

The error message in this case is very verbose and hard to read. A quick enhancement would be to format the error message better and/or detect that the error is a duplicate repeated for each index and display a more readable summary.

It may be possible to detect exceptions that will cause failures like this and automatically remove them (and display an appropriate warning that some exceptions were not compatible when executing the rule). However, implementing a further enhancement like that would be out of scope of this issue. It would be nice to do initial research on possible further enhancements while creating a more readable error message though.

marshallmain avatar Mar 01 '22 00:03 marshallmain

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine avatar Mar 01 '22 00:03 elasticmachine

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine avatar Mar 01 '22 06:03 elasticmachine

The buildEsQuery API has a new option, nestedIgnoreUnmapped, for ignoring unmapped nested fields via https://github.com/elastic/kibana/pull/134580

Here's an example PR where the new option is used:

buildEsQuery(
  indexPattern,
  queries,
  filters.filter((f) => f.meta.disabled === false),
  {
    nestedIgnoreUnmapped: true, // by default, prevent shard failures when unmapped `nested` fields are queried: https://github.com/elastic/kibana/issues/130340
    ...config,
    dateFormatTZ: undefined,
  }
)

cc @marshallmain @yctercero

andrew-goldstein avatar Jun 23 '22 16:06 andrew-goldstein