numaflow
numaflow copied to clipboard
numaproj
Kubernetes-native platform to run massively parallel data/streaming jobs
## Problem Today, we cannot start the numaflow pipeline at a particular source-state. We always start from a relative state (e.g., latest, oldest offset); we cannot say start at offset...
https://github.com/numaproj/numaflow/actions/runs/8195555528/job/22415020803
# Summary Today, we have to have at least a Source and a Sink. This means we need to write the message from the Source vertex to the ISB so...
**Describe the bug** A clear and concise description of what the bug is. Edges: ```yaml edges: - from: input to: window - from: input to: input-output - from: window to:...
The current logger doesn't support TRACE-level logging and debugging is getting trickier and harder. [zerolog](https://github.com/rs/zerolog) supports `TRACE` and is also more efficient.
- [x] #985 - [ ] introduce Used Defined Sink Transformers - [ ] Update documentation
https://go.dev/blog/deadcode Run the tool under the `cmd` folder to identify. Look for opportunities to clean up.
For the activeToPartitions check in the forwarders, replace for-loop with bitops.
# Summary From Vigith: > We should also think how we can get the SDK version from the SDK so we can have the numaflow to SDK compatibility matrix.. this...
Add a new Terraform blueprint for [Numaflow](https://numaflow.numaproj.io/) under streaming https://github.com/awslabs/data-on-eks/tree/main/streaming Ref: https://github.com/awslabs/data-on-eks/issues/382 ```[tasklist] ### Tasks ```