Andy Grove

I created a separate issue for improving the "native memory management" approach. https://github.com/apache/datafusion-comet/issues/996

> The downside (to using unified memory) is that comet operators cannot trigger the spilling of other memory consumers, which makes it easy to run into issues similar to https://github.com/apache/datafusion-comet/issues/886...

Thanks for the PR @tlm365. Could I suggest that you mark the tests with `ignore` and add a link to the issue https://github.com/apache/datafusion-comet/issues/1729 and then we can merge this PR...

It looks like this error is happening in Spark code and not in Comet code? It is difficult to know how to help with this since you have a custom...

> @andygrove how can we test this with Comet? Can I just pin to a datafusion version? Yes, assuming that there are no breaking API changes in DataFusion since 48...

> > @andygrove how can we test this with Comet? Can I just pin to a datafusion version? > > Yes, assuming that there are no breaking API changes in...

> [apache/datafusion-comet#1913](https://github.com/apache/datafusion-comet/pull/1913) @andygrove CI seems to have passed.🎉 Yes, I confirmed that the test passes now: ``` 2025-06-19T18:36:27.5009047Z [info] - postgreSQL/window_part1.sql (12 seconds, 530 milliseconds) ``` Thanks, @suibianwanwank, for backporting...

Hi @irenjj. Could you explain in the PR description how these functions differ from the standard DataFusion implementation (i.e., what is Spark-specific about them)? That will help with reviews.

@parthchandra @huaxingao @comphead @mbutrovich @kazuyukitanimura, FYI, in case you want to review. In Comet, we are currently using DataFusion's ceil and floor functions and have not specialized for Spark.

@Kontinuation @viirya I am trying to reproduce this issue now, but I am not sure if it is already resolved by https://github.com/apache/datafusion-comet/pull/988?