kafka icon indicating copy to clipboard operation
kafka copied to clipboard

Partition required when sending Keyed Messages

Open peebles opened this issue 9 years ago • 1 comments
trafficstars

Why is a partition required when I send keyed messages? I'm a little new to Kafka, but I thought I understood that when using keys, "producer guarantees that all messages with the same key will arrive to the same partition". If I call

producer.send({ topic: 'test', message: { key: '1001', message: 'test' })

it complains about a missing partition parameter.

peebles avatar Oct 30 '16 17:10 peebles

If you are using 2.9.2, you must provide a partitioner function on the payload, otherwise you should provide the partition ID.

From the README: "The partitioner function receives 3 arguments: the topic name, an array with topic partitions, and the message"

The return of the partitioner function should be the desired partition Id.

However, if you are using 3.0, partitioner should be an instance of Kafka.DefaultPartitioner or an instance that inherits from that prototype.

ghermeto avatar Nov 16 '16 03:11 ghermeto