numaflow icon indicating copy to clipboard operation
numaflow copied to clipboard

move to bi-directional streaming for source, map, transformer, and sink for better performance

Open vigith opened this issue 5 months ago • 0 comments

Summary

The integration between numa and SDK server should be via gRPC Streaming and not unary for better performance. We should even move half-duplex to bi-di.

Use Cases

Getting Numaflow ready for future performance improvements.

### Tasks
- [ ] https://github.com/numaproj/numaflow/issues/2045
- [ ] Add Stop and Stopped to Source gRPC (no-op)
- [ ] https://github.com/numaproj/numaflow/issues/2046
- [ ] https://github.com/numaproj/numaflow/issues/2092
- [ ] https://github.com/numaproj/numaflow/issues/2091

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

vigith avatar Sep 06 '24 04:09 vigith