kafka
kafka copied to clipboard
Partition required when sending Keyed Messages
Why is a partition required when I send keyed messages? I'm a little new to Kafka, but I thought I understood that when using keys, "producer guarantees that all messages with the same key will arrive to the same partition". If I call
producer.send({ topic: 'test', message: { key: '1001', message: 'test' })
it complains about a missing partition parameter.
If you are using 2.9.2, you must provide a partitioner function on the payload, otherwise you should provide the partition ID.
From the README: "The partitioner function receives 3 arguments: the topic name, an array with topic partitions, and the message"
The return of the partitioner function should be the desired partition Id.
However, if you are using 3.0, partitioner should be an instance of Kafka.DefaultPartitioner or an instance that inherits from that prototype.