kafka-go
kafka-go copied to clipboard
Tombstones messages from compaction are not null
Describe the bug When subscribing to a compacted topic that contains tombstone messages, these messages are not received with null as the payload but instead the subject is always 4 bytes long: 00 00 00 01 and the value is always 6 bytes long. Sometimes the value is all zeroes but sometimes (for some unexplicable reason) the last byte is 04.
Kafka Version 2.2.1
To Reproduce Subscribe to a topic that has compacted messages with a partition reader:
r := kafka.NewReader(kafka.ReaderConfig{
Brokers: brokers,
Partition: partition,
Topic: topic,
Dialer: d,
MaxWait: 1 * time.Second,
})
Expected behavior I expect to get nul in the value of the event (all zero bytes)
Additional context What is also weird is that there is always an offset skipped. So I get events for offset 667, 669, 671 etc, but not for even numbers. Also there are always around 5-10 Events with the 4 in the value and then 5-10 events without the 4 in the value alternating.