Easier way to test a Python UDF locally

[syayi]

Summary

Have a python UDF that does sentiment analysis on a stream of text messages. It was hard to test the UDF locally to see if it works or not without creating a image and deploy the pipeline, alternate option for me is to create a 'Datum' object with test data. Is there a better way to test without creating a pipeline?

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[tczhao]

I'm wondering how about having the numaflow python grpc client code as well, rather than just the server

so that people can test their UDFs/UDSinks using the grpc client first without building the image and test in numaflow pipeline.

[whynowy]

I'm wondering how about having the numaflow python grpc client code as well, rather than just the server

so that people can test their UDFs/UDSinks using the grpc client first without building the image and test in numaflow pipeline.

Some thoughts from me:

1. The UDxxx code (gRPC server) should be able to start from local;
2. Run a CLI command as client send data to the local gRPC for testing.

Another similar approach - run the UDxxx code on the local in HTTP mode, so that we can post data for testing. But need to think about if it's possible to test functions like reduce (gRPC streaming).