arroyo

arroyo copied to clipboard

ref: Update accumulator sig to return Result<TResult> instead of TResult

3

### Overview Change the signature for the accumulator in reduce step so it can generate backpressure. ### Details In the rust snuba metrics processor, we're using an unbounded queue (defined...

john-z-yang

add consumer_member_id tag to partitions assign / revoked metrics

2

Attempted fix for #357 [DACI](https://www.notion.so/DACI-Improved-kafka-consumer-metrics-a3828b53d0554ddcb2ed67f4d0c89f83) Starting with the metrics most important for understanding consumer balancing across a group, but it may make sense to include this tag on other consumer-related...

mwarkentin

Record consumer / client ID on metrics

1

We currently have a bunch of metrics in the `arroyo` namespace, however they are only tagged with VM / host information, rather than consumer, so it is hard (or impossible)...

mwarkentin

ayirr7

feat: A better API for abandoning messages on shutdown

1

There were a number of incidents recently (e.g. inc-706) where consumers were not shutdown cleanly as they were unable to finish processing all pending messages before they get kicked out...

lynnagara

metric(multiprocessing): Add metric to track time it takes to prepare and submit batch

1

ayirr7

👋 Hi there! This PR was automatically generated 🤖 Triggered by [email protected] Fixes [SNUBA-4KJ](https://sentry.io/organizations/sentry/issues/5106176792/) The Kafka consumer encounters an OffsetOutOfRange error, indicating the requested offset is not available on the...

sentry-autofix-experimental[bot]

For INC-599, in followup SNS-2603, add a test that ensures that the consumer reliably crashes when a broker dies. I will work a bit more on the CI setup, but...

untitaker

This is a breaking change since we need to update Snuba to use `FxHashMap` instead of `HashMap`. I believe the performance benefits makes this justifiable, but I'm not sure what...

anonrig

fix(rust): Add required metadata for crates.io

1

untitaker