kairosdb icon indicating copy to clipboard operation
kairosdb copied to clipboard

Published 20 hours ago • verified publisher icon kairosdb

KairosDB gets suspended after inserting large amount of data.

Open ghost opened this issue 10 years ago • 8 comments

I set up KairosDB on a single node (12 cores x 2 threads per core, 12 GB of RAM). I used another server to send data to KairosDB via telnet. There were 10,000 metrics (from TAG_1 to TAG_10000) and time resolution was 100 ms => 100,000 data points per second. I intended to run for 200,000 seconds but unfortunately the KairosDB got suspended after around 86,000 seconds. Querying data via http endpoint just returned {"errors":["java.lang.NullPointerException"]}, and the Kairosdb kept printing lots of followings in the kairosdb.log:

07-22|14:30:23.072 [New I/O worker #30] WARN [TelnetServer.java:156] - Message: 'putm TAG_8471 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #27] WARN [TelnetServer.java:156] - Message: 'putm TAG_3071 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6822 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3071 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6822 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #27] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3071 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #30] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_8471 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #29] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3071 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6823 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #27] WARN [TelnetServer.java:156] - Message: 'putm TAG_3072 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6823 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3072 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #30] WARN [TelnetServer.java:156] - Message: 'putm TAG_8472 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6824 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3072 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #27] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3072 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3073 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6824 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #30] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_8472 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.076 [New I/O worker #28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6825 1300189252100 4.045085 stt=good ' From: 192.168.20.14

And the log file size increased dramatically, up to 4 GB.

I stopped KairosDB then restarted it and it worked well again.

ghost avatar Jul 22 '15 06:07 ghost

Are you using h2 or cassandra? On Jul 22, 2015 12:43 AM, "duong2179" [email protected] wrote:

I set up KairosDB on a single node (12 cores x 2 threads per core, 12 GB of RAM). I used another server to send data to KairosDB via telnet. There were 10,000 metrics (from TAG_1 to TAG_10000) and time resolution was 100 ms => 100,000 data points per second. I intended to run for 200,000 seconds but unfortunately the KairosDB got suspended after around 86,000 seconds. Querying data via http endpoint just returned {"errors":["java.lang.NullPointerException"]}, and the Kairosdb kept printing lots of followings in the kairosdb.log:

07-22|14:30:23.072 [New I/O worker #30 https://github.com/kairosdb/kairosdb/pull/30] WARN [TelnetServer.java:156] - Message: 'putm TAG_8471 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #27 https://github.com/kairosdb/kairosdb/pull/27] WARN [TelnetServer.java:156] - Message: 'putm TAG_3071 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6822 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.072 [New I/O worker #29 https://github.com/kairosdb/kairosdb/pull/29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3071 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6822 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #27 https://github.com/kairosdb/kairosdb/pull/27] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3071 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #30 https://github.com/kairosdb/kairosdb/pull/30] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_8471 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #29 https://github.com/kairosdb/kairosdb/pull/29] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3071 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6823 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.074 [New I/O worker #27 https://github.com/kairosdb/kairosdb/pull/27] WARN [TelnetServer.java:156] - Message: 'putm TAG_3072 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6823 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29 https://github.com/kairosdb/kairosdb/pull/29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3072 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #30 https://github.com/kairosdb/kairosdb/pull/30] WARN [TelnetServer.java:156] - Message: 'putm TAG_8472 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6824 1300189252100 4.045085 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29 https://github.com/kairosdb/kairosdb/pull/29] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3072 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #27 https://github.com/kairosdb/kairosdb/pull/27] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_3072 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #29 https://github.com/kairosdb/kairosdb/pull/29] WARN [TelnetServer.java:156] - Message: 'putm TAG_3073 1300189253000 5.000000 stt=good ' From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_6824 1300189252100 4.045085 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.075 [New I/O worker #30 https://github.com/kairosdb/kairosdb/pull/30] WARN [TelnetServer.java:156] - Failed to execute command: putm TAG_8472 1300189253000 5.000000 stt=good Reason: java.lang.NullPointerException From: 192.168.20.14 07-22|14:30:23.076 [New I/O worker #28 https://github.com/kairosdb/kairosdb/pull/28] WARN [TelnetServer.java:156] - Message: 'putm TAG_6825 1300189252100 4.045085 stt=good ' From: 192.168.20.14

And the log file size increased dramatically, up to 4 GB.

I stopped KairosDB then restarted it and it worked well again.

— Reply to this email directly or view it on GitHub https://github.com/kairosdb/kairosdb/issues/197.

brianhks avatar Jul 22 '15 14:07 brianhks

I am using Cassandra 2.1.6.

Update: After I restarted KairosDB, I continued to insert data (from the second of ~86,0000) and the KairosDB got suspended again at the second of ~180,000)

ghost avatar Jul 22 '15 14:07 ghost

Could you guys please confirm if this is a bug and help to fix it? I really really need KairosDB now.

ghost avatar Jul 24 '15 02:07 ghost

When you say "KairosDB got suspended again at the second of ~180,000" do you mean that when you were pushing at the rate of ~180,000/sec it failed?

I havn't seen this before, if you are trying to push data really fast it is usually Cassandra that fails.

Try running it again but turn on debug in the logback.xml file with

<logger name="org.kairosdb.core.telnet.TelnetServer" level="DEBUG"/>

brianhks avatar Jul 24 '15 02:07 brianhks

What I mean is that I sent data at rate of 100,000 data points per second (10000 metrics, 10 data points per metric per second). I started with data from the timestamp of 1300000000000. The KairosDB first got suspended after ~86,000 seconds meaning that at the timestamp of ~1300086000000. I killed KairosDB and started it again. Then I continued to send data from the timestamp of ~1300086000000. It worked fine for a while before it got suspended again at the timestamp of ~1300180000000.

ghost avatar Jul 24 '15 03:07 ghost

I see. We are running this in our production environment and push billions of data points a month through our kairos nodes without problems. We are using Cassandra 2.0.x and have multiple Kairos nodes so the individual load is not as great.

Run it with the debug option on so we can get greater detail into what happened.

brianhks avatar Jul 24 '15 03:07 brianhks

Today I got this error again, here is some logs with Telnet debug enabled. Please help to figure this out!

08-06|22:24:15.736 [New I/O worker #44] DEBUG [TelnetServer.java:153] - Message: 'putm TAG_8619 1300063533400 -4.045085 stt=good ' From: 10.0.0.12 08-06|22:24:15.735 [New I/O worker #19] DEBUG [TelnetServer.java:153] - Message: 'putm TAG_6726 1300064040000 5.000000 stt=good ' From: 10.0.0.12 08-06|22:24:15.735 [New I/O worker #23] DEBUG [TelnetServer.java:153] - Message: 'putm TAG_9313 1300064114100 4.045085 stt=good ' From: 10.0.0.12 08-06|22:24:15.735 [New I/O worker #38] DEBUG [TelnetServer.java:151] - Failed to execute command: putm TAG_4823 1300063618200 1.545085 stt=good Reason: java.lang.NullPointerException From: 10.0.0.12 org.kairosdb.core.exception.DatastoreException: java.lang.NullPointerException at org.kairosdb.datastore.cassandra.CassandraDatastore.putDataPoint(CassandraDatastore.java:379) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.datastore.KairosDatastore.putDataPoint(KairosDatastore.java:182) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:105) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:66) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.TelnetServer.messageReceived(TelnetServer.java:120) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.5.Final.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] java.lang.NullPointerException: null 08-06|22:24:15.750 [New I/O worker #19] DEBUG [TelnetServer.java:151] - Failed to execute command: putm TAG_6726 1300064040000 5.000000 stt=good Reason: java.lang.NullPointerException From: 10.0.0.12 org.kairosdb.core.exception.DatastoreException: java.lang.NullPointerException at org.kairosdb.datastore.cassandra.CassandraDatastore.putDataPoint(CassandraDatastore.java:379) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.datastore.KairosDatastore.putDataPoint(KairosDatastore.java:182) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:105) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:66) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.TelnetServer.messageReceived(TelnetServer.java:120) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.5.Final.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] java.lang.NullPointerException: null 08-06|22:24:15.735 [New I/O worker #28] DEBUG [TelnetServer.java:151] - Failed to execute command: putm TAG_2081 1300063756200 1.545085 stt=good Reason: java.lang.NullPointerException From: 10.0.0.12 org.kairosdb.core.exception.DatastoreException: java.lang.NullPointerException at org.kairosdb.datastore.cassandra.CassandraDatastore.putDataPoint(CassandraDatastore.java:379) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.datastore.KairosDatastore.putDataPoint(KairosDatastore.java:182) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:105) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.PutMillisecondCommand.execute(PutMillisecondCommand.java:66) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.kairosdb.core.telnet.TelnetServer.messageReceived(TelnetServer.java:120) ~[kairosdb-1.0.0-1.jar:1.0.0-1.20150604225857] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.5.Final.jar:na] at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.5.Final.jar:na] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.5.Final.jar:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.5.Final.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] java.lang.NullPointerException: null

ghost avatar Aug 07 '15 00:08 ghost

I had a bunch of very nasty errors if I got something silly with the TTL, IE, in the past or something (like setting it to timestamp and overflowing rather than amount of time) with 2.1. I'm not sure if it was this error though but some bad TTLs maybe making it delete immediately would cause KDB to get stuff in an error loop and no longer be able to insert.

joeyhub avatar Jan 17 '20 16:01 joeyhub