Cannot read property offset

[kroyhpe]

Hi ,

We have been using no-kafka since 6 months in production . We run multiple instances of no-kafka code running as consumer. We have been observing that sometimes the consumer code will start logging this error . The error logs seems to be as if code is stuck for a specific message and it runs in a loop trying for the same message . Also as this happens the Kafka Bytes out JMX Metrics slowly ramps up in MBs whereas Bytes in is still in KBs . Attached the screenshot .

{"log":"[at apply (/home/addison/node_modules/no-kafka/node_modules/lodash/lodash.js:484:17)]\n","stream":"stderr","time":"2017-09-29T22:51:00.471242722Z"}

{"log":"2017-09-29T22:51:00.491Z WARN no-kafka-client Handler for entmonTxn_Entmon_hpit-cld-ptmonitor:0 failed with TypeError: Cannot read property 'offset' of undefined\n","stream":"stdout","time":"2017-09-29T22:51:00.492837884Z"}

{"log":" at /home/addison/src/plugin/msg-sys.js:276:28\n","stream":"stdout","time":"2017-09-29T22:51:00.492915616Z"}

{"log":" at Array.forEach (native)\n","stream":"stdout","time":"2017-09-29T22:51:00.492927214Z"}

{"log":" at Object.dataHandler (/home/addison/src/plugin/msg-sys.js:272:20)\n","stream":"stdout","time":"2017-09-29T22:51:00.492935456Z"}

{"log":" at Object.tryCatcher (/home/addison/node_modules/bluebird/js/release/util.js:16:23)\n","stream":"stdout","time":"2017-09-29T22:51:00.492944657Z"}

{"log":" at Object.handler (/home/addison/node_modules/bluebird/js/release/method.js:15:34)\n","stream":"stdout","time":"2017-09-29T22:51:00.492953261Z"}

{"log":" at self.client.fetchRequest.map.concurrency (/home/addison/node_modules/no-kafka/lib/base_consumer.js:66:26)\n","stream":"stdout","time":"2017-09-29T22:51:00.492960986Z"}

{"log":" at bound (domain.js:280:14)\n","stream":"stdout","time":"2017-09-29T22:51:00.493104566Z"}

{"log":" at runBound (domain.js:293:12)\n","stream":"stdout","time":"2017-09-29T22:51:00.494278963Z"}

{"log":" at tryCatcher (/home/addison/node_modules/bluebird/js/release/util.js:16:23)\n","stream":"stdout","time":"2017-09-29T22:51:00.494317597Z"}

{"log":" at MappingPromiseArray._promiseFulfilled (/home/addison/node_modules/bluebird/js/release/map.js:61:38)\n","stream":"stdout","time":"2017-09-29T22:51:00.494346361Z"}

{"log":" at MappingPromiseArray.PromiseArray._iterate (/home/addison/node_modules/bluebird/js/release/promise_array.js:114:31)\n","stream":"stdout","time":"2017-09-29T22:51:00.494377258Z"}

{"log":" at MappingPromiseArray.init (/home/addison/node_modules/bluebird/js/release/promise_array.js:78:10)\n","stream":"stdout","time":"2017-09-29T22:51:00.494403345Z"}

{"log":" at bound (domain.js:280:14)\n","stream":"stdout","time":"2017-09-29T22:51:00.494425892Z"}

{"log":" at MappingPromiseArray.runBound (domain.js:293:12)\n","stream":"stdout","time":"2017-09-29T22:51:00.494451468Z"}

{"log":" at Promise._settlePromise (/home/addison/node_modules/bluebird/js/release/promise.js:566:21)\n","stream":"stdout","time":"2017-09-29T22:51:00.494479628Z"}

{"log":" at Promise._settlePromise0 (/home/addison/node_modules/bluebird/js/release/promise.js:614:10)\n","stream":"stdout","time":"2017-09-29T22:51:00.494501482Z"}

[oleksiyk]

I think the error is being thrown in your handler, not in no-kafka:

Handler for entmonTxn_Entmon_hpit-cld-ptmonitor:0 failed with TypeError

[kroyhpe]

Please refer to #214 on the actual root cause when we see this error

[oleksiyk]

I don't think wrong producer payload can be the reason for consumer handler error. If the produced payload can't be deserialised by Kafka server then it won't be sent to the consumers in the first place. So #214 definitely has no relation to this issue.

[kroyhpe]

Hi , Based on what we saw when trying to consume these messages - attached the observation . 

  From: Oleksiy Kryvoshey <[email protected]>

To: oleksiyk/kafka [email protected] Cc: kroyhpe [email protected]; Author [email protected] Sent: Monday, November 13, 2017 11:11 PM Subject: Re: [oleksiyk/kafka] Cannot read property offset (#207)

I don't think wrong producer payload can be the reason for consumer handler error. If the produced payload can't be deserialised by Kafka server then it won't be sent to the consumers in the first place. So #214 definitely has no relation to this issue.— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.