numaflow
numaflow copied to clipboard
numaproj
Kubernetes-native platform to run massively parallel data/streaming jobs
# Summary We use [encoding/binary](https://pkg.go.dev/encoding/binary) for serde. We need to standardize on a wire format that is scalable and also supports backward compatibility. `encoding/binary` encourages using `gob` or `protobuf`; I...
# Summary Some kafka clients allows regexp patterns for the topic name to consume from multiple topics. Is this possible with the sarama client? Or can this be supported? Here...
# Summary When a pipeline is created, a job is generated to create buffers in ISBService, similarly, when a pipeline is deleted, another job is generated to clean up buffers...
# Summary Higher test coverage is one of the metrics that could help improve the code quality, our ultimate goal is to improve [code coverage](https://app.codecov.io/gh/numaproj/numaflow) to 85%. - [ ]...
**Describe the bug** In prometheus.yml i am using remote write to send metrics to a source vertex of http type. and getting this error ERROR 2024-06-19 21:54:43 {"level":"error","ts":1718814283.906287,"logger":"numaflow.Source-processor","caller":"forward/forward.go:473","msg":"Retrying failed msgs","vertex":"simple-pipeline-in","errors":{"expected...
# Summary The Pipeline K8s Events table only use half page height. Pagination is fixed at 6 events. Lots of space below.  # Use Cases Prevent excessive page scrolling....
# Summary In the current system we do have multiple things that the user can see, but we need to add more metrics which can help for better observability and...
Previously numaflow UI was supporting routes for namespaces and pipelines. Due to numaflow package requirements, this had to be converted into url parameters. This issue is support both url parameters...
In SDK implementations and event time filter examples there exists [`MessageToDrop(...)`](https://github.com/numaproj/numaflow-go/blob/main/pkg/sourcetransformer/examples/event_time_filter/impl/filter.go#L13C71-L13C84) (go) and [`to_drop(...)`](https://github.com/numaproj/numaflow-python/blob/20aa1fd009a3686de0dfdb65dc75248f480afbed/examples/sourcetransform/event_time_filter/example.py#L27C33-L27C40) (python). It is not obvious why dropping messages must be done by adding dropped message with...
# Summary Argo CD supports to define custom health check scripts for CRD objects, which is located at https://github.com/argoproj/argo-cd/tree/master/resource_customizations, check the [doc](https://argo-cd.readthedocs.io/en/stable/operator-manual/health/#way-2-contribute-a-custom-health-check). We need to define health check scripts for...