Improve ignoreFilterIfFieldNotInIndex functionality for multi-field query string filters

[neodescis]

trafficstars

Is your feature request related to a problem? Please describe.

Our users need to view data from multiple, disparate index patterns on a dashboard. They also need to filter the data they are viewing, and they need only the applicable filters for each index to be applied. As such, we have ignoreFilterIfFieldNotInIndex turned on. This works fine for simple filters (specifically, the ones where filter.meta.key points to a single field being filtered on), but it does not work at all for QueryStringFilters. We have a plugin visualization that dynamically constructs a QueryStringFilter, and the filters it creates are incorrectly ignored when the setting is enabled. The filter also adds a "Warning" to the filter's pill in the filter bar.

Describe the solution you'd like

Currently we are running a fork of Kibana that parses the filter's query string to determine if the filter matches an index. We are using the lucene npm package to parse the query string. The filterMatchesIndex() function is updated to use this list of fields when checking a QueryStringFilter against an index pattern. To address the filter pill warning, we could either turn it off entirely for QueryStringFilters, or we could call filterMatchesIndex() to determine whether to show the warning.

Describe alternatives you've considered

We have considered creating a different type of filter, but no filters seem to work correctly with multiple fields and ignoreFilterIfFieldNotInIndex enabled. We also briefly considered disabling ignoreFilterIfFieldNotInIndex, but it is not a viable solution for us given the data we want to display.

Additional context

The lucene npm package mentioned would be a new dependency.

[brijos]

Thanks @neodescis for this submission. We think this will be helpful, but want to set expectations that it is unlikely that we will get to it in the next few months. Please update this issue in the event any requirements change. Cheers!

[neodescis]

Actually, we plan to get a pull request together for this in the next week or so. We've been running with the change for a couple of years, but now that we're migrating to OpenSearch, we'd love to get it merged into the official repository.