shardingsphere-elasticjob icon indicating copy to clipboard operation
shardingsphere-elasticjob copied to clipboard

How can I gracefully exit a scheduled task?

Open yujun-8848 opened this issue 3 years ago • 2 comments

when i stop the applicaiton,the job will trow some exceptions. How can I gracefully exit a scheduled task?

2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Destruction of the end
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.core.ServerListManager do shutdown stop
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.http.NamingHttpClientProxy do shutdown begin
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.beat.BeatReactor do shutdown begin
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.beat.BeatReactor do shutdown stop
2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Start destroying NacosRestTemplate
2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Destruction of the end
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] c.a.n.client.identify.CredentialWatcher  : [null] CredentialWatcher is stopped
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] c.a.n.client.identify.CredentialService  : [null] CredentialService is freed
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.http.NamingHttpClientProxy do shutdown stop
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Shutdown rpc client ,set status to shutdown
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Shutdown  client event executor java.util.concurrent.ScheduledThreadPoolExecutor@2b76189e[Running, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0]
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Close current connection 1631495343812_172.16.3.57_53198
2021-09-13 09:12:07.362  INFO 61205 --- [ionShutdownHook] c.a.n.c.remote.client.grpc.GrpcClient    : Shutdown grpc executor java.util.concurrent.ThreadPoolExecutor@28928814[Running, pool size = 11, active threads = 0, queued tasks = 0, completed tasks = 83]
2021-09-13 09:12:07.363  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.NamingClientProxyDelegate do shutdown stop
2021-09-13 09:12:07.366  WARN 61205 --- [ionShutdownHook] o.s.b.f.support.DisposableBeanAdapter    : Destroy method 'close' on bean with name 'nacosServiceRegistry' threw an exception: java.lang.NullPointerException
2021-09-13 09:12:07.392  INFO 61205 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-09-13 09:12:07.419  INFO 61205 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
2021-09-13 09:12:07.936  INFO 61205 --- [tor-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl        : backgroundOperationsLoop exiting
2021-09-13 09:12:08.056  INFO 61205 --- [ain-EventThread] org.apache.zookeeper.ClientCnxn          : EventThread shut down for session: 0x1002dbd6db30010
2021-09-13 09:12:08.056  INFO 61205 --- [ionShutdownHook] org.apache.zookeeper.ZooKeeper           : Session: 0x1002dbd6db30010 closed
Exception in thread "Quartz Shutdown-Hook secondJob" org.apache.shardingsphere.elasticjob.reg.exception.RegException: java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
	at org.apache.shardingsphere.elasticjob.reg.exception.RegExceptionHandler.handleException(RegExceptionHandler.java:48)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.getDirectly(ZookeeperRegistryCenter.java:162)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.get(ZookeeperRegistryCenter.java:143)
	at org.apache.shardingsphere.elasticjob.lite.internal.storage.JobNodeStorage.getJobNodeData(JobNodeStorage.java:77)
	at org.apache.shardingsphere.elasticjob.lite.internal.election.LeaderService.isLeader(LeaderService.java:82)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobShutdownHookPlugin.shutdown(JobShutdownHookPlugin.java:68)
	at org.quartz.core.QuartzScheduler.shutdownPlugins(QuartzScheduler.java:2381)
	at org.quartz.core.QuartzScheduler.shutdown(QuartzScheduler.java:730)
	at org.quartz.impl.StdScheduler.shutdown(StdScheduler.java:206)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobScheduleController.shutdown(JobScheduleController.java:173)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobShutdownHookPlugin$1.run(JobShutdownHookPlugin.java:82)
Caused by: java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
	at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:823)
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkState(CuratorFrameworkImpl.java:423)
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:457)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.getDirectly(ZookeeperRegistryCenter.java:158)
	... 9 more

yujun-8848 avatar Sep 13 '21 01:09 yujun-8848

According to your log, the ZooKeeper client was shutdown before the job shutdown, which may be the cause of the exception. We can discuss how.

TeslaCN avatar Sep 16 '21 16:09 TeslaCN

when i stop the applicaiton,the job will trow some exceptions. How can I gracefully exit a scheduled task?

2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Destruction of the end
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.core.ServerListManager do shutdown stop
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.http.NamingHttpClientProxy do shutdown begin
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.beat.BeatReactor do shutdown begin
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.beat.BeatReactor do shutdown stop
2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Start destroying NacosRestTemplate
2021-09-13 09:12:07.359  WARN 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : [NamingHttpClientManager] Destruction of the end
2021-09-13 09:12:07.359  INFO 61205 --- [ionShutdownHook] c.a.n.client.identify.CredentialWatcher  : [null] CredentialWatcher is stopped
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] c.a.n.client.identify.CredentialService  : [null] CredentialService is freed
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.http.NamingHttpClientProxy do shutdown stop
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Shutdown rpc client ,set status to shutdown
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Shutdown  client event executor java.util.concurrent.ScheduledThreadPoolExecutor@2b76189e[Running, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 0]
2021-09-13 09:12:07.360  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.common.remote.client   : Close current connection 1631495343812_172.16.3.57_53198
2021-09-13 09:12:07.362  INFO 61205 --- [ionShutdownHook] c.a.n.c.remote.client.grpc.GrpcClient    : Shutdown grpc executor java.util.concurrent.ThreadPoolExecutor@28928814[Running, pool size = 11, active threads = 0, queued tasks = 0, completed tasks = 83]
2021-09-13 09:12:07.363  INFO 61205 --- [ionShutdownHook] com.alibaba.nacos.client.naming          : com.alibaba.nacos.client.naming.remote.NamingClientProxyDelegate do shutdown stop
2021-09-13 09:12:07.366  WARN 61205 --- [ionShutdownHook] o.s.b.f.support.DisposableBeanAdapter    : Destroy method 'close' on bean with name 'nacosServiceRegistry' threw an exception: java.lang.NullPointerException
2021-09-13 09:12:07.392  INFO 61205 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2021-09-13 09:12:07.419  INFO 61205 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
2021-09-13 09:12:07.936  INFO 61205 --- [tor-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl        : backgroundOperationsLoop exiting
2021-09-13 09:12:08.056  INFO 61205 --- [ain-EventThread] org.apache.zookeeper.ClientCnxn          : EventThread shut down for session: 0x1002dbd6db30010
2021-09-13 09:12:08.056  INFO 61205 --- [ionShutdownHook] org.apache.zookeeper.ZooKeeper           : Session: 0x1002dbd6db30010 closed
Exception in thread "Quartz Shutdown-Hook secondJob" org.apache.shardingsphere.elasticjob.reg.exception.RegException: java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
	at org.apache.shardingsphere.elasticjob.reg.exception.RegExceptionHandler.handleException(RegExceptionHandler.java:48)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.getDirectly(ZookeeperRegistryCenter.java:162)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.get(ZookeeperRegistryCenter.java:143)
	at org.apache.shardingsphere.elasticjob.lite.internal.storage.JobNodeStorage.getJobNodeData(JobNodeStorage.java:77)
	at org.apache.shardingsphere.elasticjob.lite.internal.election.LeaderService.isLeader(LeaderService.java:82)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobShutdownHookPlugin.shutdown(JobShutdownHookPlugin.java:68)
	at org.quartz.core.QuartzScheduler.shutdownPlugins(QuartzScheduler.java:2381)
	at org.quartz.core.QuartzScheduler.shutdown(QuartzScheduler.java:730)
	at org.quartz.impl.StdScheduler.shutdown(StdScheduler.java:206)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobScheduleController.shutdown(JobScheduleController.java:173)
	at org.apache.shardingsphere.elasticjob.lite.internal.schedule.JobShutdownHookPlugin$1.run(JobShutdownHookPlugin.java:82)
Caused by: java.lang.IllegalStateException: Expected state [STARTED] was [STOPPED]
	at org.apache.curator.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:823)
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.checkState(CuratorFrameworkImpl.java:423)
	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getData(CuratorFrameworkImpl.java:457)
	at org.apache.shardingsphere.elasticjob.reg.zookeeper.ZookeeperRegistryCenter.getDirectly(ZookeeperRegistryCenter.java:158)
	... 9 more

Whether the problem is solved?

x92811x avatar Oct 21 '22 06:10 x92811x