processing_statistics_logger.cc throwing NaN value

[infynyxx]

INFO  [2017-08-03 21:25:53,519] com.amazonaws.services.kinesis.producer.LogInputStreamReader: [2017-08-03 21:25:53.518979] [0x00000289][0x00007f35f1dc9700] [info] [processing_statistics_logger.cc:109] Stage 1 Triggers: { stream: '***', manual: 0, count: 0, size: 0, matches: 0, timed: 0, UserRecords: 0, KinesisRecords: 0 }
INFO  [2017-08-03 21:25:53,519] com.amazonaws.services.kinesis.producer.LogInputStreamReader: [2017-08-03 21:25:53.519060] [0x00000289][0x00007f35f1dc9700] [info] [processing_statistics_logger.cc:112] Stage 2 Triggers: { stream: '***', manual: 0, count: 0, size: 0, matches: 0, timed: 0, KinesisRecords: 0, PutRecords: 0 }
INFO  [2017-08-03 21:25:53,519] com.amazonaws.services.kinesis.producer.LogInputStreamReader: [2017-08-03 21:25:53.519077] [0x00000289][0x00007f35f1dc9700] [info] [processing_statistics_logger.cc:127] (segments) Average Processing Time: -nan ms

Version: 0.12.5

[sahilpalvia]

Thank you for the feedback. We will prioritize it accordingly against the other customer requests.

[bowenli86]

Hi @infynyxx @sahilpalvia , I ran into this too. What does it mean? Does it indicate that KPL is losing data, or just a small bug in logging? I'm not a c++ guy, and would appreciate your insights.

Thanks!

[infynyxx]

Yeah, seems like a small logging bug coming from https://github.com/awslabs/amazon-kinesis-producer/blob/v0.12.5/aws/utils/processing_statistics_logger.cc#L127

[pfifer]

This means the KPL isn't receiving any work, but it will still publish trigger information. If you're not currently pushing data to the stream the NAN values can safely be ignored. If you are sending data to the stream, this could indicate a problem.

[pooja84k]

I ran into this issue too. Does it mean that the data sent to the stream is lost?

[jef]

I also get InvalidClientTokenId over time.

[rajendrapallala]

We are getting similar issue. We are running KPL from onprem linux servers. We tested with KPL versions (0.13.1) and (0.12.11). Below are message we are seeing. Metric upload failing. This is happening while processing data. KPL successfully placing messages to Kinesis stream but cloudwatch not writing KPL Metrics.

[2020-06-04 14:55:06.579413] [0x00002aa4] [info] [processing_statistics_logger.cc:109] Stage 1 Triggers: { stream: 'stream-name-removed', manual: 0, count: 0, size: 0, matches: 0, timed: 0, UserRecords: 0, KinesisRecords: 0 } [2020-06-04 14:55:06.579413] [0x00002aa4] [info] [processing_statistics_logger.cc:112] Stage 2 Triggers: { stream: 'stream-name-removed', manual: 0, count: 0, size: 0, matches: 0, timed: 0, KinesisRecords: 0, PutRecords: 0 } [2020-06-04 14:55:06.579413] [0x00002aa4] [info] [processing_statistics_logger.cc:127] (stream-name-removed) Average Processing Time: -nan(ind) ms [2020-06-04 14:55:28.708604] [0x00001e50] [warning] [metrics_manager.cc:147] Metrics upload failed, but will retry. | Code: | Message: (Code: and Message: are actually empty not removed by me)