numaflow
numaflow copied to clipboard
numaproj
Kubernetes-native platform to run massively parallel data/streaming jobs
Latency should be minimal since it is only introduced during the testing of one component, and each component is run in parallel. I'll give it some thought and create a...
A follow-up doc for JetStream source is needed. _Originally posted by @whynowy in https://github.com/numaproj/numaflow/pull/1723#pullrequestreview-2053330186_
The path to the `namespaced-controller-wo-crds.yaml` was bringing up the GitLab human-readable file, but I suspect we would prefer to point to the raw YAML This is just a minor documentation...
# Summary I'mm looking for the ability for a mapper and sink to take a batch of items at once. Today these take single item at once, regardless of `readBatchSize`...
reduce-two-e2e - https://github.com/numaproj/numaflow/actions/runs/9034959063/job/24828692966?pr=1723 idle-source-e2e - https://github.com/numaproj/numaflow/actions/runs/9026274446/job/24803315933
https://github.com/numaproj/numaflow/blob/3225b730d8cdd5c855da9b119cb27fe9f8112f5b/pkg/sinks/udsink/sink.go#L73 https://github.com/numaproj/numaflow/blob/3225b730d8cdd5c855da9b119cb27fe9f8112f5b/pkg/sinks/forward/forward.go#L258
## Performance testing strategy - latency of pipeline with source, map, reduce and sink vertices - number of records that can be sent from the source with no sustained back...
Update numaflow controller from `rc1` to `rc4`. I see this error message in the sink vertex. Sink Pods remained in Running State. Error in numa container ``` {"level":"error","ts":"2024-04-08T18:38:21.170233545Z","logger":"numaflow.Sink-processor","caller":"forward/forward.go:415","msg":"Retrying failed messages","pipeline":"kafka-test-pipeline-1","vertex":"custom-out","errors":{"gRPC...
related to https://github.com/numaproj/numaflow/issues/1592