wankun

https://blog.csdn.net/wankunde [email protected]

free 上海

Results 30 comments of


                                            wankun

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

https://github.com/apache/spark/pull/37922#discussion_r1054110064 For the `AppShufflePartitionInfo` that is processing the first pushed shuffle block, `partition.mapTracker.getCardinality() == 0`, and won't put this reduceId to reduceIds, and then the shuffle files will not be...

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

> Could you add some unit test for this? I wrote [unit test](https://github.com/apache/spark/pull/38560/files#diff-955b4eb8f495555d5e188ed93a4acf5393e1d9c7855dffddedb5e9201eebdef2R1213) before. You can have a look or write new UT. Thanks, backport the UT

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

@mridulm I rebase the code, could you help to review this PR again? Thanks

[WIP][SPARK-40480][SHUFFLE] Remove push-based shuffle data after query finished

@otterc Thanks for your review. Format the code.

PARQUET-2415: Reuse hadoop file status and footer in ParquetRecordReader

I can not reproduce the failed UT: ``` 2023-12-22T04:25:27.6546055Z [INFO] Running org.apache.parquet.cli.commands.ShowFooterCommandTest 2023-12-22T04:25:27.8426114Z [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.179 s javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]->javax.security.auth.Subject["principals"]->java.util.Collections$SynchronizedSet[5]->org.apache.hadoop.security.User["login"]->javax.security.auth.login.LoginContext["subject"]) 2023-12-22T04:25:27.8914504Z Caused...

PARQUET-2415: Reuse hadoop file status and footer in ParquetRecordReader

Hi, @wgtmac I have fixed this issue in cli module. Thanks

PARQUET-2415: Reuse hadoop file status and footer in ParquetRecordReader

Hi, @wgtmac @amousavigourabi is there any concern about this PR ?

[WIP] Add spark 35

Some class and function has changed in spark 3.5, maybe we need some more shim classes to support both spark 3.5 and the versions before. I has a similar commit...

[SPARK-52103][SQL] Fallback complex expression whole-stage codegen

Hi, @panbingkun do you have any idea about this codegen JIT fail issue?

Migrate to JDK25

Hi, should we also update the checkJavaVersion method to support JDK 25 here? https://github.com/apache/datasketches-memory/blob/main/src/main/java/org/apache/datasketches/memory/internal/ResourceImpl.java#L165-L170