Mridul M comments

Results 143 comments of


                                            Mridul M

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

@wankunde, please ping me when you are done with the updates to the PR. Thanks

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

I just merged #39316 to unblock this PR, can you update to latest please ? Thx

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

The tests are failing @wankunde, though I dont think it is due to your PR. Can you please take a look ? And retrigger it if it is unrelated ?...

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

Merged to master. Thanks for fixing this @wankunde ! Thanks for the reviews @otterc, @akpatnam25, @yabola :-)

[SPARK-39399] [CORE] [K8S]: Fix proxy-user authentication for Spark on k8s in cluster deploy mode

+CC @Ngone51 , @HyukjinKwon

[SPARK-40045][SQL]Optimize the order of filtering predicates

QQ: Why is this PR targeting 3.3 and not master ?

[SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

Can you fix the linter error @rmcyang ? ` src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java:[1281] (sizes) LineLength: Line is longer than 100 characters (found 104).`

[SPARK-33573][SHUFFLE][YARN] Shuffle server side metrics for Push-based shuffle

Merged to master. Thanks for working on this @rmcyang ! Thanks for the reviews @zhouyejoe, @otterc :-)

IndexedRDD for accelerated joins

I have not looked into this in great detail, but shouldn't cogroup, etc (anything using multiple rdd's) validate the partitioners are equal before comparing partition index, etc ? Or is...

IndexedRDD for accelerated joins

To give an example of what I was referring to - take a look at cogroup in IndexedRDDFunctions, self and other IndexedRDDFunctions need not be partitioned by the same partitioner...