Mridul M

icon company LinkedIn icon location Santa Clara, CA icon location PMC Apache Spark, Apache Celeborn

Results 143 comments of Mridul M

[SPARK-40407][SQL] Fix the potential data skew caused by df.repartition

Or, we can use what `RDD.coalesce` does currently.

[SPARK-25050][SQL] Avro: writing complex unions

+CC @dongjoon-hyun, @HyukjinKwon who might be able to review this better than me.

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

+CC @zhouyejoe

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

Please don't force push the code, unless it is to resolve conflicts/etc which can be handled otherwise - it makes reviewing what changed much harder

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

Can you take a look at the test failures @thejdeep ? They look relevant.

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

I am not sure what has changed since last review due to the force pushes - please do not force push, particularly for large PR's.

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

Looks good to me, will wait for successful build before merging.

[SPARK-36620][SHUFFLE] Add Push Based Shuffle client side metrics

Merged to master. Thanks for working on this @thejdeep ! Thanks for the reviews @otterc, @zhouyejoe, @gengliangwang :-)

[SPARK-46920][YARN] Improve executor exit error message on YARN

+CC @tgravescs

[WIP][SPARK-46947][CORE] Delay memory manager initialization until Driver plugin is loaded

+CC @dongjoon-hyun and @tgravescs