elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

ESQL: INLINESTATS followup

Open nik9000 opened this issue 7 months ago • 3 comments

Description

https://github.com/elastic/elasticsearch/pull/109583 will add support for INLINESTATS, a command to run a STATS and then merge the results into the stream of results. This issue tracks follow up work:

Before GA

  • [x] Run logical plan optimization before splitting the phases. See https://github.com/elastic/elasticsearch/pull/109583#discussion_r1683084852
  • [x] Allow functions in the grouping position (+union types test with conversion in the grouping position // TODO once inlinestats supports expressions in groups we'll likely need the same sort of extraction here)
  • [ ] Address left over comments on https://github.com/elastic/elasticsearch/pull/111690
  • [ ] Fix union fields in INLINESTATS - they are allowed now but they produce "fun" error messages and don't work
  • [x] Add all phases to the result of profile
  • [ ] Allow push down for conditions coming from the EVAL based join. So | INLINESTATS a=AVG(foo) | WHERE foo > a should be able to push the foo > a bit in the second phase. It can't now.
  • [ ] Track intermediate request memory usage and lift the 1mb limit.
  • [ ] Keep readers open for the second round.
  • [ ] Fix the test labeled brokenWhy-Ignore, byConstant-Ignored
  • [ ] More shadowing tests
  • [ ] Fix INLINESTATS x=MAX(a), x=MIN(a) - shadowingInternal-Ignored
  • [ ] Fix shadowingSelfBySelf
  • [ ] Fix INLINESTATS in CCS
  • [ ] Once we have the above - we should look into pushing the Phased stuff further into physical planning. It'd be nice to, for example, and a SubqueryExec plan that runs like Phased does here. Not sure if physical or logical - but physical feels better. We're doing logical now though.
  • [ ] Test with the BUCKET function. Sounds like it doesn't work at the moment.
  • [ ] More than one INLINESTATS. see. Note that this might have to do with multiple lookups - that's tracked in https://github.com/elastic/elasticsearch/issues/109353

Evantually

  • [ ] Some https://github.com/elastic/elasticsearch/issues/109353

nik9000 avatar Jul 16 '24 13:07 nik9000