dubbo icon indicating copy to clipboard operation
dubbo copied to clipboard

Dubbo关闭时报Exception in thread "DubboShutdownHook" java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError

Open xiangzz159 opened this issue 3 years ago • 4 comments

  • [ ] I have searched the issues of this repository and believe that this is not a duplicate.

Environment

  • Dubbo version: 3.0.10
  • Operating System version: CentOS Linux release 7.9.2009
  • Java version: 1.8.0_151-b12

Steps to reproduce this issue

使用kill -15 pid的时候应用优雅关闭,错误日志中输出如下错误

[2022-10-11T17:04:38.048+08:00] [WARN] [DubboShutdownHook-33] [org.apache.dubbo.registry.support.RegistryManager] [PRO-xsyx-item-library-base-core-9-124] [10.32.9.124] [] [] [] [] [RegistryManager.java,107,org.apache.dubbo.registry.support.RegistryManager,destroyAll] [ [DUBBO] java.lang.NoClassDefFoundError: org/apache/dubbo/common/function/ThrowableAction, dubbo version: 3.0.10, current host: 10.32.9.124] ## ' java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError: org/apache/dubbo/common/function/ThrowableAction
        at org.apache.dubbo.registry.client.ServiceDiscoveryRegistry.destroy(ServiceDiscoveryRegistry.java:284)
        at org.apache.dubbo.registry.support.RegistryManager.destroyAll(RegistryManager.java:105)
        at org.apache.dubbo.config.deploy.DefaultApplicationDeployer.destroyRegistries(DefaultApplicationDeployer.java:1019)
        at org.apache.dubbo.config.deploy.DefaultApplicationDeployer.preDestroy(DefaultApplicationDeployer.java:767)
        at org.apache.dubbo.rpc.model.ApplicationModel.onDestroy(ApplicationModel.java:244)
        at org.apache.dubbo.rpc.model.ScopeModel.destroy(ScopeModel.java:110)
        at org.apache.dubbo.config.DubboShutdownHook.doDestroy(DubboShutdownHook.java:78)
        at org.apache.dubbo.config.DubboShutdownHook.run(DubboShutdownHook.java:73)
Caused by: java.lang.NoClassDefFoundError: org/apache/dubbo/common/function/ThrowableAction
        ... 8 more
Caused by: java.lang.ClassNotFoundException: org.apache.dubbo.common.function.ThrowableAction
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:93)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 8 more

应用启动参数为:

java -DappName=demo -server -Xmx5144M -Xms5144M -Xmn2048M -Xss1024k -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:-CMSParallelRemarkEnabled -XX:-CMSClassUnloadingEnabled -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -XX:+UseG1GC -Dapp.env=prod -Ddk.class.preload.timeout=140 -Ddk.logging.path=/home -Dfastjson.parser.safeMode=true -jar demo.jar

Pls. provide [GitHub address] to reproduce this issue.

Expected Behavior

Actual Behavior

If there is an exception, please attach the exception trace:

Just put your stack trace here!

xiangzz159 avatar Oct 12 '22 06:10 xiangzz159

在本地和容器中无法复现,但是在生产环境会有这个问题,导致优雅下线失败

xiangzz159 avatar Oct 13 '22 01:10 xiangzz159

在本地和容器中无法复现,但是在生产环境会有这个问题,导致优雅下线失败

这个应该是在 kill 的过程中外部的由 spring 管理的 LaunchedURLClassLoader 已经关闭了,导致 Dubbo 无法加载新的下线相关的类导致的

AlbumenJ avatar Oct 13 '22 02:10 AlbumenJ

这个问题不是必现的,2台机器,第一次重新部署的时候只有其中一台机器会出现优雅下线失败的问题,第二次就是2台机器都出现了

xiangzz159 avatar Oct 13 '22 02:10 xiangzz159

@xiangzz159 问题根源请参考 Albumen 给的思路,Spring 和 Dubbo 的 JVM shutdown hook 同时在执行可能会导致这个问题。

Dubbo 这边在 Spring 场景下,应该关闭自身的hook并将注销流程绑定到 Spring hook 上以确保被顺序调度。related to https://github.com/apache/dubbo/pull/10730

chickenlj avatar Oct 13 '22 06:10 chickenlj

@xiangzz159 暂时先关闭这个issue,如果还有问题可以重新打开这个issues。

CrazyHZM avatar Nov 15 '22 01:11 CrazyHZM