YangJie comments

Results 132 comments of


                                            YangJie

[SPARK-39950][SQL] It's unnecessary to materialize BroadcastQueryStage firstly, because the BroadcastQueryStage does not timeout in AQE.

> but the BroadcastQueryStage does not timeout in AQE any more @weixiuli Could you point out the timeout issue no longer exists after which pr in the pr description? cc...

[SPARK-39950][SQL] It's unnecessary to materialize BroadcastQueryStage firstly, because the BroadcastQueryStage does not timeout in AQE.

Additionally, does this pr just revert SPARK-33933?

[SPARK-39927][BUILD] Upgrade to Avro 1.11.1

seems `support user provided non-nullable avro schema for nullable catalyst schema without any null record *** FAILED *** (181 milliseconds)` still failed

[SPARK-39866][SQL] Memory leak when closing a session of Spark ThriftServer

> jenkins, test this please no jenkins now :)

[SPARK-40030][BUILD] Upgrade scala-maven-plugin to 4.7.1

cc @HyukjinKwon @pan3793

[SPARK-40030][BUILD] Upgrade scala-maven-plugin to 4.7.1

thanks @HyukjinKwon @pan3793

[SPARK-38909][CORE][YARN] Encapsulate `LevelDB` used to store remote/external shuffle state as `DB`

thanks @dongjoon-hyun

[SPARK-38909][CORE][YARN] Encapsulate `LevelDB` used to store remote/external shuffle state as `DB`

I found `RemoteBlockPushResolver` begin to use `LevelDB`, more refactor work is required before reopen this pr

[SPARK-38909][CORE][YARN] Encapsulate `LevelDB` used to store remote/external shuffle state as `DB`

Merge with master and re-trigger GA

[SPARK-38909][CORE][YARN] Encapsulate `LevelDB` used to store remote/external shuffle state as `DB`

> This is a recent change, to add missing functionality for preserving merge data across restarts. Please tag me when you have an updated PR @LuciferYang, will help review it....