franz-go icon indicating copy to clipboard operation
franz-go copied to clipboard

sarama-compatible hasher

Open C-Pro opened this issue 7 months ago • 2 comments

Looks like SaramaHasher is not exactly compatible with sarama. I switched from segmentio/kafka-go to franz-go and noticed that now my keys map to different partition (this is a no-go for my usecase).

I have created a test to check key->partiton mapping consistency between sarama, segmentio and franz. kgo.StickyKeyPartitioner(SaramaCompatHasher(fnv32a)) returns different partition number.

Test output shows franz yields different partition numbers.

So I added new SaramaCompatHasher that behaves exactly like sarama's and segmentio ones (uses int32 instead of int). I opted to create a new instead of changing the existing SaramaHasher to maintain compatibility with old library versions.

C-Pro avatar Nov 22 '23 11:11 C-Pro