rdkafka-ruby
rdkafka-ruby copied to clipboard
Support for String partitioner config
I am using racecar version 2.8.2, rdkafka 0.12.0, and kafka 3.2.1. I am producing messages to the same topic as a Java service, and I need to set the producer partitioner_name to murmur2 so that they hash the partition keys in the same way and events with the same partition key are sent to the same partition. I have set my config in racecar.yaml to include the partitioner like this:
# These config values will be shared by all environments but can be overridden.
common: &common
client_id: "my-client-id"
producer:
- partitioner=murmur2
Racecar is then pulling that partitioner value into a hash and configuring an rdkafka producer: racecar/lib/racecar/config.rb
however, when rdkafka is configuring the consumer, it expects the partitioner configuration to be a symbol rather than a String: https://github.com/appsignal/rdkafka-ruby/blob/c02f217a189b71e3e253fe4ad59286b5fc9d4034/lib/rdkafka/config.rb#L184
I've raised a PR in rdkafka to support both string and symbol keys for the partitioner
config: https://github.com/appsignal/rdkafka-ruby/pull/213