ruby-kafka
ruby-kafka copied to clipboard
zendesk
A Ruby client library for Apache Kafka
#### Description - The existing implementation doesn't support temporary IAM credentials. A user has to create an IAM user, get `iam_access_key` and `iam_secret_key` and then use those values to create...
* Version of Ruby: unrelated * Version of Kafka: unrelated * Version of ruby-kafka: unknown? ###### Steps to reproduce I don't have the steps to reproduce. I tried in https://github.com/zendesk/ruby-kafka/pull/955...
I am trying to replicate idempotent async producer behavior when no partitioning info is provided so that messages are routed to random partitions. I think it leads to incorrect behavior.
* Version of Ruby: 2.6.5 * Version of Kafka: 2.7 * Version of ruby-kafka: 1.3.0 Often times, the @braze-inc platform is producing messages at a rate such that if the...
Maybe I'm doing something wrong; or maybe there is an issue? * Version of Ruby: ``` $ ruby -v ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux] ``` * Version of Kafka:...
Hi, MSK IAM auth support was added recently https://github.com/zendesk/ruby-kafka/pull/937, but it doesn't support temp credentials/ assume role refreshes. This feature was initially mentioned by @garrett528 in a thread https://github.com/zendesk/ruby-kafka/pull/907#issuecomment-978146627 I...
The example "consuming messages using the batch API" sometimes gets stuck as follows, and the reason might be `Queue#pop` waits for a message unlimitedly. * https://app.circleci.com/pipelines/github/zendesk/ruby-kafka/233/workflows/b80d0db8-1334-49e2-abf9-9d51580c42bc/jobs/6931 * https://app.circleci.com/pipelines/github/zendesk/ruby-kafka/232/workflows/a612607f-f20d-4bb0-b6b4-47e184a26474/jobs/6920 * https://app.circleci.com/pipelines/github/zendesk/ruby-kafka/230/workflows/fedf26e8-22af-4deb-878d-aa6c5fc5ae6f/jobs/6895
CI sometimes times out due to "log writing failed. deadlock; recursive locking" as follows, and it might be related to https://bugs.ruby-lang.org/issues/15360. If so, using the image "cimg/ruby:2.5.9" resolves the problem....
This PR prevents the example "create partitions" from failing like https://app.circleci.com/pipelines/github/zendesk/ruby-kafka/221/workflows/7d51470f-c102-4b30-b2e0-68c618eeb1d2/jobs/6799?invite=true#step-104-122. As there is a time lag for Kafka's metadata to be updated, `Kafka::Client#partitions_for` is needed to be executed until...
This PR resolves https://github.com/zendesk/ruby-kafka/issues/941. As the coordinator waits for each member to rejoin when rebalancing the group, the timeout of join request should be greater than rebalance_timeout. > rebalance_timeout_ms :...