YangJie
YangJie
Merged into master. Thanks @rickyma
Tested this pr using 10TB TPC-DS, the latency of q9 has been reduced by 83.39% in my production environment. | | Master | SPARK-40193 | Percentage| |---------------|-----------|-------------------------|--------------------| | q9 |...
Will adding the `- Dmaven.resolver.transport=wagon` in `MAVEN_OPTS` have any effect? The default implementation of Maven 3.9 pieces of resolver has changed from wagon to httpclient
Thank you for your efforts on this. @pan3793
should we test 33.2.0?
> > should we test 33.2.0? > > `33.1.0-jre` is chosen because Spark uses `33.1.0-jre` in other places, and after this change, we can unify all Guava versions. OK
Is there a possibility that upgrading Guava could carry similar risks as described in https://github.com/apache/spark/pull/46528 ? Could there be legacy Hive UDF jars that are bound to Guava 14.0.1? WDYT...
Or how about having these modules depend on the `common/utils` module? `common/utils` doesn't seem to be a heavyweight module, in this way, the existing cases can be fixed.
MIMA took 8 minutes and 6 seconds, and there doesn't seem to be a significant performance difference. I think this is feasible. Could you have it run a few more...
This is fine for me, and the time is about the same as using the maven local cache. But I suggest splitting `disable maven local cache` and `upgrade to sbt...