bottledwater-pg icon indicating copy to clipboard operation
bottledwater-pg copied to clipboard

Extracted timestamp value is negative, which is not allowed.

Open machty opened this issue 8 years ago • 6 comments

I'm trying to use Kafka-Streams to parse the data I've loaded into Kafka using Postgres and I'm getting

Exception in thread "StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Extracted timestamp value is negative, which is not allowed.
        at org.apache.kafka.streams.processor.internals.RecordQueue.addRawRecords(RecordQueue.java:111)
        at org.apache.kafka.streams.processor.internals.PartitionGroup.addRawRecords(PartitionGroup.java:117)
        at org.apache.kafka.streams.processor.internals.StreamTask.addRecords(StreamTask.java:143)
        at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:415)
        at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:242)

I found the following link http://docs.confluent.io/3.1.0/streams/faq.html#invalid-timestamp-exception

Am I possibly seeing this error because bottlewater was built for 0.9 and doesn't have proper timestamps for 0.10?

machty avatar Feb 14 '17 02:02 machty

FYI I "fixed" this by specifying a TIMESTAMP_EXTRACTOR_CLASS_CONFIG of 'org.apache.kafka.streams.processor.WallclockTimestampExtractor' in the consumer.

That said I'd still like to figure out what could/should change about BW to include timestamp information, otherwise we're limited to at-processing timestamps only (which is what the setting above accomplishes).

machty avatar Feb 15 '17 13:02 machty

Thanks @machty - this solution worked for me also.

mmaia avatar Jul 31 '17 11:07 mmaia

Thanks @machty - Above solution also worked for me.

Asiddiki avatar Aug 05 '17 10:08 Asiddiki

Which config file do you add this to and what's the exact line?

postelrich avatar Apr 25 '18 19:04 postelrich

Honestly I have no idea, never really stuck with this learning Kafka. I'd recommend you check out debezium rather than bottledwater though.

machty avatar Apr 25 '18 19:04 machty

Thanks @machty. Above solution worked for me as well.

parth22594 avatar Dec 06 '18 12:12 parth22594