numaflow
numaflow copied to clipboard
numaproj
Kubernetes-native platform to run massively parallel data/streaming jobs
# Summary We need to enable version selection on the documentation website, allowing users to read the documentation across different versions. # Use Cases When would you use this? -...
**Describe the bug**  **To Reproduce** Steps to reproduce the behaviour: 1. IMPORTANT: choose a non-prod cluster which has minimal numaflow users. This is to ensure we don't block others...
# Summary - [ ] update version compatibility matrix - [ ] release all SDKs - [ ] enable `WorkQueue` policy on JetStream - [ ] endurance test - [...
There will be other extensions that will be written in Rust, so we should make the top level dir `ext` and serving be a workspace member.
# Summary Today, if we want to restart a particular vertex (restart all the vertex pods) after changing some spec (like max payload size for jetstream), we need to restart...
# Summary We need to build a serving layer (supporting HTTP 1 and 2) on top of the Numaflow engine to support both sync/async endpoints. The serving layer can also...
**Describe the bug** watermark updates get linearly slow with an increase in partition count, causing `forward-a-chunk` to block. we noticed this issue when reading from Kafka with partition counts ~...
# Summary ISBs should auto-heal and send alerts and events - [ ] auto-balancing of streams - [ ] scripts to recreate a irrecoverable or bad ISB - [ ]...
``` {"level":"error","ts":"2024-07-27T15:13:08.068429208Z","logger":"numaflow.Sink-processor","caller":"fetch/processor_manager.go:269","msg":"Unable to find the processor","pipeline":"dummy-ceres-transaction-pipeline","vertex":"out","processorEntity":"dummy-ceres-transaction-pipeline-input-318","stacktrace":"github.com/numaproj/numaflow/pkg/watermark/fetch.(*processorManager).startTimeLineWatcher\n\t/home/runner/work/numaflow/numaflow/pkg/watermark/fetch/processor_manager.go:269"} {"level":"error","ts":"2024-07-27T15:13:08.068445148Z","logger":"numaflow.Sink-processor","caller":"fetch/processor_manager.go:269","msg":"Unable to find the processor","pipeline":"dummy-ceres-transaction-pipeline","vertex":"out","processorEntity":"dummy-ceres-transaction-pipeline-input-353","stacktrace":"github.com/numaproj/numaflow/pkg/watermark/fetch.(*processorManager).startTimeLineWatcher\n\t/home/runner/work/numaflow/numaflow/pkg/watermark/fetch/processor_manager.go:269"} panic: runtime error: index out of range [4] with length 1 goroutine 208 [running]: github.com/numaproj/numaflow/pkg/watermark/fetch.(*processorManager).startTimeLineWatcher(0xc0004cc300) /home/runner/work/numaflow/numaflow/pkg/watermark/fetch/processor_manager.go:296 +0xc70 created by...
