Surbhi-Vijay
Surbhi-Vijay
`df.collect()` works as expected for `map_concat` function. (converts to velox plan) `df.show()` is still executing in Spark due to the fallback. The reason of fallback is, Spark adds limit of...
@rui-mo can you please check this patch again. I have added few comments and rewritten the spark test case since there is no exception thrown for duplicate values. Velox already...
> Hi @Surbhi-Vijay, I notice `MapKeyDedupPolicy.EXCEPTION` seems to be the default behavior of Spark, is that right? In your opinion, do we need to customize the map_concact function in Velox...
> > Hi @Surbhi-Vijay, I notice `MapKeyDedupPolicy.EXCEPTION` seems to be the default behavior of Spark, is that right? In your opinion, do we need to customize the map_concact function in...
`nullif` is out of the box supported. Spark send the converted expression as `If` expression and it is supported in Gluten.
I would like to give `printf` a try.
cc @zhztheplayer @zhli1142015
Thanks @FelixYBW for explaining. I am trying to come up with a minimal query to showcase the impact. If I do not achieve the same then will post the detailed...
We investigated the Regressing queries. We see that the regression is not directly caused by "VeloxAppendBatches" but rather due to plan changes when data size is reduced due to this...
> @Surbhi-Vijay Could you share the metrics details of `ColumnarExchange` with "VeloxAppendBatches" enabled/disabled? Below "ColumnarExchange" is for join (store_sales join customer) which converted to BHJ from SHJ in q24b when...