datafusion-comet icon indicating copy to clipboard operation
datafusion-comet copied to clipboard
verified publisher icon apache

Require Comet 0.6 Docker image for Spark 3.5.5

Open RaghavendraGanesh opened this issue 9 months ago • 4 comments

What is the problem the feature request solves?

The current image only supports 0.5 on spark 3.4

Describe the potential solution

No response

Additional context

No response

RaghavendraGanesh avatar Mar 12 '25 02:03 RaghavendraGanesh

We support Spark 3.5.4 but not 3.5.5 yet due to https://github.com/apache/datafusion-comet/issues/1461 (it should not be too hard to fix).

We plan on releasing 0.7.0 in the next week. It would make sense to update our current Docker image to use Spark 3.5.4 instead of Spark 3.4.x.

andygrove avatar Mar 12 '25 04:03 andygrove

@RaghavendraGanesh We now have a 0.7.0 image for Spark 3.5.4. Perhaps that helps?

https://hub.docker.com/r/apache/datafusion-comet/tags

andygrove avatar Mar 18 '25 00:03 andygrove

Thanks @andygrove , will give it a try.

RaghavendraGanesh avatar Mar 18 '25 12:03 RaghavendraGanesh

Hey @RaghavendraGanesh, my PR #1565 was merged - we'll have version 0.8.0 aligned with Spark 3.5.5.

YanivKunda avatar Mar 24 '25 21:03 YanivKunda