ruby-kafka

ruby-kafka copied to clipboard

Push compression metrics to StatsD/Datadog/Prometheus

3

The [Compressor](https://github.com/zendesk/ruby-kafka/blob/d3796daf8433611043f6680412b27355317896e1/lib/kafka/compressor.rb#L78-L81) class is pushing ActiveSupport notifications with metrics regarding the messages getting compressed. However these don't get pushed to StatsD/Datadog, since there's no subscriber for them. This PR adds...

ellimist

Max_wait_time same as max.interval.poll.ms

1

Few Partitions of the topic are very slow intermittently. Lag for 2-3 partitions will keep increasing and not decreasing. Once we restart the service everything looks normal. All Lag is...

Ravinderbaid

Is there a way to set what ciphers should be used when connecting to the kafka brokers?

mdost

MessageSizeTooLargeException causes infinite delivery loop in async producer

1

If this is a bug report, please fill out the following: * Version of Ruby: 2.6 * Version of Kafka: do not know * Version of ruby-kafka: 1.3.0 I have...

astudnev

Consuming from topic with zstd compression

2

I'm surprised this hasn't come up before because I'm not sure if this ever worked. It appears we need to use v10+ of the fetch protocol to get zstd support...

rkrage

Bump DescribeConfigs API to version 1

2

Version 1 of this API exposes `config_source` which describes the origin of a particular config (topic, broker, default value, etc). We have tooling that manages topic-level configs so being able...

rkrage

python: rdkafka_roundrobin_assignor.c:97: rd_kafka_roundrobin_assignor_assign_cb: Assertion `next < rd_list_cnt(&eligible_topic-> members)' failed

1

* Version of Ruby: 2.5.5 * Version of Kafka: * Version of ruby-kafka: 1.3.0 Hi Everyone. As the title says, I get the error python: rdkafka_roundrobin_assignor.c:97: rd_kafka_roundrobin_assignor_assign_cb: Assertion `next <...

benjaminelkrieff

bottleneck in producing messages?

4

I'm using newrelic to trace where the slowest part of my code is coming and it's pointing me to the `produce` method. I tried using sync and async producers but...

katpadi

Recover from Kafka::MessageSizeTooLarge exception in async producer.

2

You can get into a bad unrecoverable state when a Kafka::MessageSizeTooLarge exception is raised. When we receive this exception, if we're using the sync producer, then we simply re-raise this...

RyanAmos

Prefer `IO#wait_readable` and `IO#wait_writable` where possible.

1

There are several instances of `IO.select` with a single file descriptor. It is better to use `IO#wait_readable` or IO#wait_writable` because it is more efficient internally. https://github.com/zendesk/ruby-kafka/blob/62a0cf9c44b0c05a4f98c4ec073ae5f04f4194e0/lib/kafka/socket_with_timeout.rb#L40

samuel-williams-shopify