hermann icon indicating copy to clipboard operation
hermann copied to clipboard

Update producer default settings

Open jamescway opened this issue 9 years ago • 1 comments

#Current defaults
      DEFAULTS = {
                    'partitioner.class'     => 'kafka.producer.DefaultPartitioner',
                    'request.required.acks' => '1',
                    'message.send.max.retries' => '0'
                  }.freeze

I'm thinking we should make: request.required.acks = -1 (needs ack from all ISR's, best durability) message.send.max = 3 (to made the tradeoff towards reliability, when the cost is possibility of dups)

jamescway avatar Jun 15 '15 19:06 jamescway

I spoke briefly with @polynomial about this and there is definitely some trade-off to be made here. Before we change the defaults, we should probably do a lot more experimentation to take a simulated load and execute it against a Kafka cluster with/without the minimum required acks.

There's also some questions I have around the use/exposure of the min.insync.replicas setting as it relates to Hermann.

rtyler avatar Jun 15 '15 21:06 rtyler