aio-pika icon indicating copy to clipboard operation
aio-pika copied to clipboard

RPC interoperability with non-Python clients

Open cloud-rocket opened this issue 2 years ago • 3 comments

Adding this issue to properly document the PR:

To make the RPC communication work properly with non-Python libraries, the serialization should be done with JSON (or another non-Python proprietary) and not with Python Pickle.

The existing JsonRPC serializer is having the following issues:

  • It's not working with Python 3+
  • The exceptions are not properly unserialized.
  • JsonRPC exception serializer does not include Python module which is important for dynamic class creation

The fix to both of those issues is available here - https://github.com/mosquito/aio-pika/pull/418

Exception object serialization is especially challenging, since they need to be recreated dynamically when the class is known or not known on the client size.

  • When the class is registered (i.e. standard Python exceptions, asserts) it is handled with pydoc.locate standard function.
  • When the class is custom (i.e. custom derivative of Exception which is not registered on client side. The dynamic creation is done with type construction.

cloud-rocket avatar Oct 26 '21 17:10 cloud-rocket

Hello @cloud-rocket, I hope you've not given up on completing the MR. I'm experiencing the same issues you have found and currently using very hacky work-arounds. It seems that only the dynamic creation of Exception is left for your MR to be merged.

skrech avatar Nov 23 '21 12:11 skrech

@skrech - yes, I'll try to finish this out shortly

cloud-rocket avatar Nov 23 '21 16:11 cloud-rocket

@skrech @mosquito - please review. The PR fixed based on @mosquito recommendation

cloud-rocket avatar Nov 23 '21 18:11 cloud-rocket