[SPARK-39950][SQL] It's unnecessary to materialize BroadcastQueryStage firstly, because the BroadcastQueryStage does not timeout in AQE.
What changes were proposed in this pull request?
In the https://issues.apache.org/jira/browse/SPARK-33933, it materializes BroadcastQueryStage firstly to try to avoid broadcast timeout in AQE, but the BroadcastQueryStage does not timeout in AQE any more, so we should not materialize BroadcastQueryStage firstly.
Why are the changes needed?
It's unnecessary to materialize BroadcastQueryStage firstly in AQE.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass CI.
Can one of the admins verify this patch?
but the BroadcastQueryStage does not timeout in AQE any more
@weixiuli Could you point out the timeout issue no longer exists after which pr in the pr description?
cc @cloud-fan @viirya
Additionally, does this pr just revert SPARK-33933?
looks fine, but let's think a bit more and see if there is any benefit to submit broadcast jobs first. cc @yaooqinn @maryannxue
@weixiuli Could you point out the timeout issue no longer exists after which pr in the pr description? cc @cloud-fan @viirya
@weixiuli @cloud-fan Hey, same questions with @LuciferYang , could you please point out which pr completely fix this issue?
here you are: https://github.com/apache/spark/commit/0c94e47aecab0a8c346e1a004686d1496a9f2b07
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!