elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

ESQL: FoldNull folding aggs into literals, raising an "unknown agg" error

Open ivancea opened this issue 1 year ago • 6 comments

When an aggregation is folded by FoldNull, it's converted to something like STATS null, which leads to an error.

Reproduced by using adding a null param to any agg that won't surrogate based on that param being foldable. Example:

ROW a = 5
| STATS percentile(a, NULL)

Result:

{
  "type": "esql_illegal_argument_exception",
  "reason": "unknown agg: class org.elasticsearch.xpack.esql.core.expression.Literal: null"
}

ivancea avatar Jun 28 '24 11:06 ivancea

Pinging @elastic/es-analytical-engine (Team:Analytics)

elasticsearchmachine avatar Jun 28 '24 11:06 elasticsearchmachine

Related with https://github.com/elastic/elasticsearch/issues/100634 I'm closing as duplicate. It's not exactly the same issue I think, but it's part of it for sure

ivancea avatar Jun 28 '24 14:06 ivancea

I'm not sure we should close this, https://github.com/elastic/elasticsearch/issues/100634 is a request for enhancements while this one is an actual bug (error 500). Waiting for https://github.com/elastic/elasticsearch/issues/100634 to be fully supported, we should at least add some validation for these cases.

Interestingly, median(null) works just fine, while percentile(null, 50) throws an error

luigidellaquila avatar Jun 28 '24 15:06 luigidellaquila

median(const) is already implemented, while percentile still needs a corresponding MV_... function to act as surrogate.

Not sure it's worth adding a validation for this, as implementing something like MV_PERCENTILE (but not exposing it to users, internal usage only) may not be a whole lot more work.

alex-spies avatar Jun 28 '24 15:06 alex-spies

I'm 50/50 here really. The aggs that don't fail, is because they have the surrogate, indeed. However, any agg with multiple parameters, that have a null in one of them but a non-foldable in the other, will fail.

For example, percentile can't really be fixed. We can't convert PERCENTILE(field, null) to MV_PERCENTILE, as field is not foldable (At least not with the current methods, afaik). And it fails too.

So, we can reopen this if you think it's interesting

ivancea avatar Jun 28 '24 16:06 ivancea

Ah, I was thinking about percentile(null, 50), but this is indeed a different problem than percentile(some_field, null). The latter indeed should end up with an error thrown at validation time.

Re-opening, but I think we need to make the wording of this issue more precise.

alex-spies avatar Jun 28 '24 17:06 alex-spies

Probably same issue:

ROW x = null | STATS VALUES(x) 

returns a 500 INTERNAL SERVICE ERROR

[esql] > Unexpected error from Elasticsearch: esql_illegal_argument_exception - unknown agg: class org.elasticsearch.xpack.esql.core.expression.Literal: null 

jan-elastic avatar Apr 29 '25 11:04 jan-elastic