Wenchen Fan

icon indicating an email [email protected]

icon company Databricks icon location Hangzhou, China

Results 245 comments of Wenchen Fan

[SPARK-39976][SQL] ArrayIntersect should handle null in left expression correctly

@AngersZhuuuu can you create 2 more PRs for 3.2 and 3.1? thanks!

[SPARK-39976][SQL] ArrayIntersect should handle null in left expression correctly

@AngersZhuuuu do you know which refactor PR caused this bug?

[SPARK-39883][SQL][TESTS] Add DataFrame function parity check

looks fine to me, cc @HyukjinKwon @viirya

[SPARK-39825][SQL] Fix PushDownLeftSemiAntiJoin push through project

cc @sigmod

[SPARK-39825][SQL] Fix PushDownLeftSemiAntiJoin push through project

BTW, does filter pushdown have the same issue?

[SPARK-39887][SQL] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique

> Fixes AliasAwareOutputPartitioning and AliasAwareOutputOrdering as testing discovered that ... Can we create a new PR for this?

[SPARK-39887][SQL] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique

cc @sigmod

[SPARK-39887][SQL] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique

@peter-toth do you know which Spark version starts to have this issue?

[SPARK-39887][SQL] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique

The GA failure is unrelated, thanks, merging to master! Since there are conflicts, can you create backport PRs for 3.3? thanks!

[SPARK-39950][SQL] It's unnecessary to materialize BroadcastQueryStage firstly, because the BroadcastQueryStage does not timeout in AQE.

looks fine, but let's think a bit more and see if there is any benefit to submit broadcast jobs first. cc @yaooqinn @maryannxue