sarama-compatible hasher

[C-Pro]

Looks like SaramaHasher is not exactly compatible with sarama. I switched from segmentio/kafka-go to franz-go and noticed that now my keys map to different partition (this is a no-go for my usecase).

I have created a test to check key->partiton mapping consistency between sarama, segmentio and franz. kgo.StickyKeyPartitioner(SaramaCompatHasher(fnv32a)) returns different partition number.

Test output shows franz yields different partition numbers.

So I added new SaramaCompatHasher that behaves exactly like sarama's and segmentio ones (uses int32 instead of int). I opted to create a new instead of changing the existing SaramaHasher to maintain compatibility with old library versions.

[twmb]

Good find, I'm not sure why I didn't think of the importance of the int32-ness for wrapping, considering I implemented this by having the Sarama code up at the same time.

That said, given a code search: https://github.com/search?q=kgo.SaramaHasher&type=code The only public uses of this function are in forks, and one line in your test that found this bug.

I doubt the Sarama hasher is used by anybody. If anybody ported from Sarama to franz-go, they'd have broken hashing to begin with and would have created a bug report a long time ago. What do you think about fixing the function and not introducing a second compatibility layer?

[C-Pro]

It is ok for me, but scary still. There can be some uses in private repos. Even if you make it a major version bump, it can hit someone who upgraded without reading release notes.