datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard

We do not respect ignoreNulls in first_value / last_value aggregates

Open andygrove opened this issue 8 months ago • 3 comments

Describe the bug

When creating the native plan for first_value / last_value aggregates, we currently hard-code ignoreNulls instead of using the value from the protobuf encoding:

                AggregateExprBuilder::new(Arc::new(func), vec![child])
                    .schema(schema)
                    .alias("first")
                    .with_ignore_nulls(false)  <-- this should not be hard-coded!

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

andygrove avatar Apr 09 '25 17:04 andygrove

I will take a look at this one.

anuragmantri avatar Apr 15 '25 20:04 anuragmantri

I will take a look at this one.

Thanks @anuragmantri. There is a closely related issue https://github.com/apache/datafusion-comet/issues/1646 that you may want to look at as well.

andygrove avatar Apr 15 '25 21:04 andygrove

Althtough this is a bug and a correctness issue, it is not a regression from 0.7.0 behavior, so I am changing the milestone to 0.9.0

andygrove avatar Apr 18 '25 03:04 andygrove

This issues seems to addressed. Should we close this @andygrove

dharanad avatar Jun 26 '25 14:06 dharanad

take

hsiang-c avatar Sep 09 '25 17:09 hsiang-c