kafka-go icon indicating copy to clipboard operation
kafka-go copied to clipboard

Tombstones messages from compaction are not null

Open KarolineKarkosch opened this issue 3 years ago • 0 comments

Describe the bug When subscribing to a compacted topic that contains tombstone messages, these messages are not received with null as the payload but instead the subject is always 4 bytes long: 00 00 00 01 and the value is always 6 bytes long. Sometimes the value is all zeroes but sometimes (for some unexplicable reason) the last byte is 04.

Kafka Version 2.2.1

To Reproduce Subscribe to a topic that has compacted messages with a partition reader:

r := kafka.NewReader(kafka.ReaderConfig{
		Brokers:   brokers,
		Partition: partition,
		Topic:     topic,
		Dialer:    d,
		MaxWait:   1 * time.Second,
	})

Expected behavior I expect to get nul in the value of the event (all zero bytes)

Additional context What is also weird is that there is always an offset skipped. So I get events for offset 667, 669, 671 etc, but not for even numbers. Also there are always around 5-10 Events with the 4 in the value and then 5-10 events without the 4 in the value alternating.

KarolineKarkosch avatar May 11 '21 11:05 KarolineKarkosch