Oleks V
Oleks V
Thats true, thanks for raising this. Spark Kubernetes pods respect Spark params but has no any idea about Comet params for now. Once pod memory allocated its not possible to...
I think what might happen is the executor pod already started by the time Comet tries to tweak the memory and once the pod is up, its not possible to...
I'll run some tests on this today
Depends on https://github.com/apache/datafusion-comet/issues/643
Depends on #689
Some tests doesn't pass, its good to check them in DF, SMJ joined filter is still in progress
just checked DF on one of the failed queries ``` #[tokio::test] async fn test_smj_left_filtered() -> Result { let ctx: SessionContext = SessionContext::new(); let sql = "set datafusion.optimizer.prefer_hash_join = false;"; let...
looks like Comet produces duplicates
Right join fails in DF ``` #[tokio::test] async fn test_smj_left_filtered() -> Result { let ctx: SessionContext = SessionContext::new(); let sql = "set datafusion.optimizer.prefer_hash_join = false;"; let _ = ctx.sql(sql).await?.collect().await?; let...
Filed https://github.com/apache/datafusion/issues/10882