arroyo
arroyo copied to clipboard
A library to build streaming applications that consume from and produce to Kafka.
### Overview Change the signature for the accumulator in reduce step so it can generate backpressure. ### Details In the rust snuba metrics processor, we're using an unbounded queue (defined...
Attempted fix for #357 [DACI](https://www.notion.so/DACI-Improved-kafka-consumer-metrics-a3828b53d0554ddcb2ed67f4d0c89f83) Starting with the metrics most important for understanding consumer balancing across a group, but it may make sense to include this tag on other consumer-related...
We currently have a bunch of metrics in the `arroyo` namespace, however they are only tagged with VM / host information, rather than consumer, so it is hard (or impossible)...
There were a number of incidents recently (e.g. inc-706) where consumers were not shutdown cleanly as they were unable to finish processing all pending messages before they get kicked out...
👋 Hi there! This PR was automatically generated 🤖 Triggered by [email protected] Fixes [SNUBA-4KJ](https://sentry.io/organizations/sentry/issues/5106176792/) The Kafka consumer encounters an OffsetOutOfRange error, indicating the requested offset is not available on the...
For INC-599, in followup SNS-2603, add a test that ensures that the consumer reliably crashes when a broker dies. I will work a bit more on the CI setup, but...
This is a breaking change since we need to update Snuba to use `FxHashMap` instead of `HashMap`. I believe the performance benefits makes this justifiable, but I'm not sure what...