elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

ESQL: push down "[text_field] is not null"

Open astefan opened this issue 1 year ago • 2 comments

Addresses https://github.com/elastic/elasticsearch/issues/102786.

I have noticed an improvement in response times for the following setup:

  • nyc_taxis index with 1mil documents in it. 1 node. 5 shards, no replicas
  • one additional message field of type text. All 1 mil documents above do not have this field
  • indexed 10 more documents with a concrete value for message field

Query with filter executes in 6-8ms: "query": "from nyc_taxis | stats count(message)", "filter": {"exists": {"field":"message"}} Query with filter inside it used to execute in 10-12ms: "query": "from nyc_taxis | where message is not null | stats count(message)" With this PR the second query executes in the same amount of time as the first one. Tests executed manually.

astefan avatar Feb 16 '24 15:02 astefan

Hi @astefan, I've created a changelog YAML for you.

elasticsearchmachine avatar Feb 16 '24 15:02 elasticsearchmachine

Pinging @elastic/es-analytical-engine (Team:Analytics)

elasticsearchmachine avatar Feb 16 '24 15:02 elasticsearchmachine

@elasticmachine update branch

astefan avatar Feb 24 '24 09:02 astefan

💚 Backport successful

Status Branch Result
8.13

elasticsearchmachine avatar Feb 24 '24 10:02 elasticsearchmachine