Uday Sagar Shiramshetty

Impact of Cost on Concurrency limit of a Partition

2

Hi, I would like to discuss a feature I need. I am not entirely sure if it fits well with concurrency limiting logic. Problem: The cost (latency or resource usage)...

GradientLimit is very similar to Gradient2Limit except for few minor things and looks like GradientLimit needs some fixes, such as: ``` - estimatedLimit = Math.max(minLimit, queueSize); + estimatedLimit = Math.max(minLimit,...

Add PartitionRegistry

1

Hi, do you think it is a good idea to have a PartitionRegistry that is capable of adding or removing Partitions during runtime? I can submit a PR based on...

Thrift integration

1

Hi @elandau, what's your input on integrating this library with Thrift? I would like to experiment on that. As far as I know, Thrift doesn't have flexible interceptors to tap...

Hi @elandau, thanks for providing this library. I have a question around partition based limiters. From the code, the partition limits are calculated as newTotalLimit * Partition#percent. But the README...

[PEP] stream plugin that supports object-storage micro-batches with optional inline batch support

8

## Summary: Introduce a new stream plugin that can handle external files or inline batches for real-time ingestion. ## Motivation: - Reduce Kafka cross-AZ replication, production, and consumption network costs....