numaflow-python icon indicating copy to clipboard operation
numaflow-python copied to clipboard

chore: streaming source

Open kohlisid opened this issue 4 months ago • 2 comments

PR to make source streaming in nature. We have removed the sync source from the implementation Refer: https://grpc.io/docs/guides/performance/#python

Streaming RPCs create extra threads for receiving and possibly sending the messages, which makes streaming RPCs much slower than unary RPCs in gRPC Python, unlike the other languages supported by gRPC.

Using asyncio could improve performance.

kohlisid avatar Oct 01 '24 16:10 kohlisid