incubator-uniffle
incubator-uniffle copied to clipboard
[Bug] Duplicated shaded-grpc jar cause task fail
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
Search before asking
- [X] I have searched in the issues and found no similar issues.
Describe the bug
There is shaded-grpc jar in client-tez, When there is also shaded-grpc jar in user UDF, then, there will be sometimes task fail.
Affects Version(s)
0.7.0
Uniffle Server Log Output
No response
Uniffle Engine Log Output
###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||RssDAGAppMaster--->Registering coordinators 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||CoordinatorClientFactory--->Start to create coordinator clients from 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:42.013|||ERROR|||-|||-|||main|||-|||RssDAGAppMaster--->Error starting RssDAGAppMaster
java.lang.NoSuchFieldError: EPOLL_DOMAIN_CLIENT_CHANNEL_TYPE
at io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider.isAvailable(UdsNettyChannelProvider.java:34)
at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:172)
at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:168)
at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:68)
at io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:96)
at io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:41)
at io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:39)
at org.apache.uniffle.client.impl.grpc.GrpcClient.<init>(GrpcClient.java:43)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:91)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:87)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.<init>(CoordinatorGrpcClient.java:83)
at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:43)
at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:69)
at org.apache.uniffle.client.impl.ShuffleWriteClientImpl.registerCoordinators(ShuffleWriteClientImpl.java:431)
at org.apache.tez.dag.app.RssDAGAppMaster.initAndStartRSSClient(RssDAGAppMaster.java:144)
at org.apache.tez.dag.app.RssDAGAppMaster.serviceInit(RssDAGAppMaster.java:107)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.RssDAGAppMaster.main(RssDAGAppMaster.java:348)
Uniffle Server Configurations
No response
Uniffle Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
In tez-client:
There is no "EPOLL_DOMAIN_CLIENT_CHANNEL_TYPE" field in user shaded-grpc jar:
Could you help fix this? @lifeSo
Could you help fix this? @lifeSo
OK, I could try, remove or rename the shaded jar name to other, to not conflict user's jar.