Metamorphosis icon indicating copy to clipboard operation
Metamorphosis copied to clipboard

MetaQ接收数据客户端进入死循环

Open yantzu opened this issue 7 years ago • 2 comments

情况偶现,症状就是程序不退出,进入死循环,CPU使用率100%,关键的堆栈如下:

"Thread-48" daemon prio=10 tid=0x00007f47a8048000 nid=0x73d2 in Object.wait() [0x00007f4796dec000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1289) - locked <0x000000077858ab88> (a java.lang.Thread) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.interruptRunners(SimpleFetchManager.java:146) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.stopFetchRunner(SimpleFetchManager.java:128) at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.shutdown(SimpleMessageConsumer.java:162) - locked <0x0000000776bdde10> (a com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer) at com.taobao.metamorphosis.client.MetaMessageSessionFactory.shutdown(MetaMessageSessionFactory.java:336) - locked <0x0000000776bdde58> (a com.taobao.metamorphosis.client.MetaMessageSessionFactory) at com.taobao.metamorphosis.client.MetaMessageSessionFactory$2.run(MetaMessageSessionFactory.java:244)

"fetch-Runner-0" daemon prio=10 tid=0x00007f4898002800 nid=0x6bdd runnable [0x00007f49294c5000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.fillInStackTrace(Throwable.java:783) - locked <0x000000073760cf20> (a java.lang.InterruptedException) at java.lang.Throwable.(Throwable.java:250) at java.lang.Exception.(Exception.java:54) at java.lang.InterruptedException.(InterruptedException.java:57) at java.lang.Object.wait(Native Method) at com.taobao.gecko.core.nio.impl.SelectorManager.awaitReady(SelectorManager.java:147) - locked <0x00000007702ecd40> (a com.taobao.gecko.core.nio.impl.SelectorManager) at com.taobao.gecko.core.nio.impl.SelectorManager.registerChannel(SelectorManager.java:122) at com.taobao.gecko.core.extension.GeckoTCPConnectorController.connect(GeckoTCPConnectorController.java:75) at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:120) at com.taobao.gecko.service.impl.DefaultRemotingClient.connect(DefaultRemotingClient.java:148) - locked <0x0000000776b9dbc8> (a com.taobao.gecko.service.impl.DefaultRemotingClient) at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:156) - locked <0x0000000770384cb8> (a java.util.HashSet) at com.taobao.metamorphosis.client.RemotingClientWrapper.connectWithRef(RemotingClientWrapper.java:192) at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.fetch(SimpleMessageConsumer.java:330) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.processRequest(SimpleFetchManager.java:227) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager$FetchRequestRunner.run(SimpleFetchManager.java:215) at java.lang.Thread.run(Thread.java:745)

yantzu avatar Feb 07 '17 09:02 yantzu

在程序直接调用System.exit()退出的时候,会调用metaq注册的shutdownhook,shutdownhook会调用SimpleFetchManager.stopFetchRunner()方法, 这个方法会判断条件this.requestQueue.size() < this.fetchRequestCount, 如果满足就会进入一个循环不停的等待。这个条件的退出,依赖于fetch线程退出,将requestQueue加回,但是此时fetch线程正好执行到代码SelectorManager.awaitReady()方法,SelectorManager此时已经被执行stop(), 所以条件一直被满足,进入一个死循环。结果就是fetch线程不退出,而shutdownhook等待fetch线程退出,jvm等待shutdownhook退出

yantzu avatar Feb 14 '17 07:02 yantzu

提交了一个gecko的PR,https://github.com/killme2008/gecko/pull/5

yantzu avatar Feb 14 '17 08:02 yantzu