sarama icon indicating copy to clipboard operation
sarama copied to clipboard

How do I specify partitions for production messages

Open gopherhiro opened this issue 11 months ago • 7 comments

gopherhiro avatar Mar 30 '24 13:03 gopherhiro

	msg := &sarama.ProducerMessage{
		Topic:     "example-topic",
		Partition: 0,
		Value:     sarama.StringEncoder("Hello, Kafka! - 1, i: " + string(i)),
	}

I specified it explicitly, but it didn't work

gopherhiro avatar Mar 30 '24 13:03 gopherhiro

2024/03/30 21:54:46 Message sent to partition 0 at offset 28640 2024/03/30 21:54:47 Message sent to partition 0 at offset 28642 2024/03/30 21:54:48 Message sent to partition 1 at offset 15024 2024/03/30 21:54:49 Message sent to partition 1 at offset 15025 2024/03/30 21:54:50 Message sent to partition 1 at offset 15027

gopherhiro avatar Mar 30 '24 13:03 gopherhiro

@gopherhiro Can you confirm whether you set config.Producer.Partitioner = NewManualPartitioner when creating producer ? Setting this partitioner will use the value from msg.Partition to decide partition for produced message.

abhipranay avatar May 12 '24 05:05 abhipranay

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

Mengsen-W avatar Jun 26 '24 12:06 Mengsen-W

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

Check struct sarama.ProducerMessage. You can set Partition field at runtime. The value of this field will be considered only when you set config.Producer.Partitioner = NewManualPartitioner

abhipranay avatar Jun 26 '24 12:06 abhipranay

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

Check struct sarama.ProducerMessage. You can set Partition field at runtime. The value of this field will be considered only when you set config.Producer.Partitioner = NewManualPartitioner

Greate Help ! I think the comment for this parameter should be rewritten. That way I know straight away how to set up the partition.

// ProducerMessage is the collection of elements passed to the Producer in order to send a message.
type ProducerMessage struct {
	...
        // ------------- !!!THIS PARAMETER COMMENT!!! -------------- //
	// Partition is the partition that the message was sent to. This is only
	// guaranteed to be defined if the message was successfully delivered.
	Partition int32
}

Mengsen-W avatar Jun 26 '24 12:06 Mengsen-W

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

github-actions[bot] avatar Sep 24 '24 14:09 github-actions[bot]