numaflow
numaflow copied to clipboard
numaproj
Kubernetes-native platform to run massively parallel data/streaming jobs
```[tasklist] ### Tasks - [ ] https://github.com/numaproj/numaflow/pull/1904 - [ ] https://github.com/numaproj/numaflow/issues/1908 - [ ] emit basic metrics - [x] lag reader and pending - [x] integrate with soucer-sinker numaflow-model -...
# Summary What change needs making? # Use Cases When would you use this? --- **Message from the maintainers**: If you wish to see this enhancement implemented please add a...
Run an endurance test such that we won't have an OOM for the following distribution of the payload. TPS = 20K (5 partitions) 1% 64MB 2% 32MB 2% 48MB 15%...
Run an endurance test such that we won't have an OOM for the following distribution of the payload. TPS = 20K (5 partitions) 1% 64MB 2% 32MB 2% 48MB 15%...
Highlighting observations in comments as I'm progressing
Even with `Redelivered Messages: 0`, why are we seeing the difference? https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/consumers#stream-vs-consumer-sequence-numbers ``` Configuration: Name: oss-analytics-xxx-usw2-prd-fci-blackhole-pipeline-2-blackhole-sink-1 Pull Mode: true Filter Subject: oss-analytics-xxx-usw2-prd-fci-blackhole-pipeline-2-blackhole-sink-1 Deliver Policy: All Ack Policy: Explicit Ack Wait:...
**Describe the bug** While applying an containerTemplate to a vertex, it gets partially propagated to the initContainer spec as well causing a mismatch in spec and issues in reconciler. **To...
**Describe the bug** Unprocessed == 0 while Stream seq in the stream metadata is moving ahead of the consumer seq number. ### Consumer Metadata ``` Last Delivered Message: Consumer sequence:...