node-sinek icon indicating copy to clipboard operation
node-sinek copied to clipboard

Error when kafka topic has more than one partition

Open cferrera opened this issue 3 years ago • 0 comments

Hello!

I'm facing the following error when I try to produce a message to a kafka topic that has more than one partition. I've tried the exactly code on topics with 1, 2, 3 and 4 partitions and it only works for topics with 1 partition.

TypeError: Cannot read property 'v3' of undefined
    at JSProducer._murmur (~/node_modules/sinek/dist/lib/kafkajs/JSProducer.js:108:77)
    at JSProducer._getPartitionForKey (~/node_modules/sinek/dist/lib/kafkajs/JSProducer.js:206:21)
    at JSProducer.<anonymous> (~/node_modules/sinek/dist/lib/kafkajs/JSProducer.js:254:34)
    at Generator.next (<anonymous>)
    at fulfilled (~/node_modules/sinek/dist/lib/kafkajs/JSProducer.js:5:58)

I was able to track the issue a little bit:

This is the piece where the code breaks:

case "3":
  this._murmur = (key, partitionCount) => murmurhash_1.murmur.v3(key) % partitionCount;
  break;

it states that "murmur" is undefined. Removing "murmur" from the call - leaving only murmurhash_1.v3(key) - it works just fine. 👌

Going into a little further into murmurhash package, we can find this, a conditional on how the value will be exported.

I don't know when a module has type undefined but on every test that I did it was always a typeof object.

Going back to JSProducer, replacing this: https://github.com/nodefluent/node-sinek/blob/48e84257a7157ece410e22ba6a47460c9df7aac3/src/lib/kafkajs/JSProducer.ts#L5 with this

 import * as murmur from "murmurhash"; 

worked for me.

Not sure if anyone else has the same issue or if I'm doing something wrong but please let me know, other than that, if this replace sounds correct, will be a pleasure to contribute with a PR.

Thanks in advance. ✌️

cferrera avatar Feb 21 '22 16:02 cferrera