Adding concurrency and Functor conveniences

[kpwelsh]

Goals

Pyro5 is a great library to accelerate development for python IPC projects. However, it lacks two things that are necessary for our use cases.

Firstly, it doesn't have convenient concurrent communication support. Only 1 thread on the client can use a Proxy at once. The stated workaround for this is to pass around the URI instead of the proxy, but this has 2 drawbacks:

1. It is inconvenient and some of the transparency that is gained from having a proxy in the first place is lost. Consumers of the proxy now need to give it special treatment.
2. It doesn't address the issue of proxies unknowingly crossing thread boundaries. meaning that no proxy can safely be passed around if there is a chance that it will be put onto another thread.

Secondly, it is inconvenient to not be able to auto-proxy methods with un-exposed classes. The two biggest occurrences of this are: giving callbacks to the server that are regular functions or lambdas (base class e.g.: builtins.function) or using a method of a normal object as a callback (i.e. my_queue.put).

Implementation

Concurrency

There are some challenges to implementing socket-level concurrency in Pyro. The protocol expects messages to come in the right order, so un-collating a concurrent message stream would be a bit annoying. So with the stated goals in mind, I decided to create a new class, ConcurrentProxy that will perform the intended workflow, but automatically.

If you are not careful with this, it is possible to cause significant memory growth that is unnecessary as well as unnecessary chatter.

Function Calling

I created a new class, Functor that can be used to wrap and expose methods that otherwise aren't exposed by their original owning class. To make this work more conveniently, I also removed __call__ from the dunder method private list.

Now you can wrap functions or lambdas with Pyro5.api.Functor and call them like functions on the server side.

Feedback Request

I am sure there is a better way to correctly manage concurrent proxies. So I am interested in feedback there. Also, I am not totally sure why __call__ was on the private list to begin with. If there is some reason I am missing, I would be interested to know.

[irmen]

Interesting, I'll have a look when time permits.

__call__ is discussed in #95