Metamorphosis
Metamorphosis copied to clipboard
MetaQ接收数据客户端进入死循环
情况偶现,症状就是程序不退出,进入死循环,CPU使用率100%,关键的堆栈如下:
"Thread-48" daemon prio=10 tid=0x00007f47a8048000 nid=0x73d2 in Object.wait() [0x00007f4796dec000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1289) - locked <0x000000077858ab88> (a java.lang.Thread) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.interruptRunners(SimpleFetchManager.java:146) at com.taobao.metamorphosis.client.consumer.SimpleFetchManager.stopFetchRunner(SimpleFetchManager.java:128) at com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer.shutdown(SimpleMessageConsumer.java:162) - locked <0x0000000776bdde10> (a com.taobao.metamorphosis.client.consumer.SimpleMessageConsumer) at com.taobao.metamorphosis.client.MetaMessageSessionFactory.shutdown(MetaMessageSessionFactory.java:336) - locked <0x0000000776bdde58> (a com.taobao.metamorphosis.client.MetaMessageSessionFactory) at com.taobao.metamorphosis.client.MetaMessageSessionFactory$2.run(MetaMessageSessionFactory.java:244)
"fetch-Runner-0" daemon prio=10 tid=0x00007f4898002800 nid=0x6bdd runnable [0x00007f49294c5000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
- locked <0x000000073760cf20> (a java.lang.InterruptedException)
at java.lang.Throwable.
在程序直接调用System.exit()退出的时候,会调用metaq注册的shutdownhook,shutdownhook会调用SimpleFetchManager.stopFetchRunner()方法, 这个方法会判断条件this.requestQueue.size() < this.fetchRequestCount, 如果满足就会进入一个循环不停的等待。这个条件的退出,依赖于fetch线程退出,将requestQueue加回,但是此时fetch线程正好执行到代码SelectorManager.awaitReady()方法,SelectorManager此时已经被执行stop(), 所以条件一直被满足,进入一个死循环。结果就是fetch线程不退出,而shutdownhook等待fetch线程退出,jvm等待shutdownhook退出
提交了一个gecko的PR,https://github.com/killme2008/gecko/pull/5