nats.py

nats.py copied to clipboard

This issue is here to track the possible work of implementing a synchronous client in nats.py that can be used instead of the asyncio client that this library supports today.

Some projects might do their own worker implementations and need a client library that does not do asyncio and only does blocking operations.

Based on a older version of nats.py I've forked it and implemented a simple PoC for that functionality here https://github.com/tobias-urdin/nats.py/commits/sync-client that is missing a lot of error checking and testing.

• Moving common code, constants, abstract classes etc from the aio client to a new nats.common module
• Implement a new synchronous TcpTransport
• Implement a copy of the Parser as SyncParser since it needs to call functions and not await coroutines
• Implementing a new nats.sync module that implements SyncClient class that does blocking operations instead of using asyncio
• Some caveats is that the PoC does a lot of code duplication for Msg, Subscription etc since we need to call a synchronous function in there

This fork has been used for implement a PoC oslo.messaging driver so that NATS can be used as the messaging bus for an OpenStack cloud https://review.opendev.org/c/openstack/oslo.messaging/+/878588 where the Devstack job is passing.

Looking for feedback on a approach that works for you, any potential help and anything else that might help push it forward.

Thanks @tobias-urdin. Looping in @wallyqs to provide guidance.

I would also be very interested in this and it would unblock the migration of our project entirely towards NATS.

Waiting for this changes too! Also ready to help with some parts of implementation to help release sync version soon

@wallyqs Any feedback on this? It's basically what we talked about quite a few months back now but need some direction for moving forward.

Hi @tobias-urdin, Sorry to bug you. I am new to writing asynchronous code.

Do you think this issue I have posted resonate with what you are saying ? https://github.com/nats-io/nats.py/issues/482

Because I am unable to run multiple worker processes in parallel with nats.

+1 on this change. Will help to user celery with nats.

Is this still going on? It seems quite people is interested, should we organize somehow?

I'm not actively working on this because of priorities but is still something I want to do.

It would be awesome if more people would like to coordinate this work but it's my opinion that we would need some directions from @bruth and @wallyqs on how they want this implemented for it to be acceptable, we wouldn't want to do all this work and afterwards get feedback on the approach.

Hi everyone, friendly ping 😊: is there anything we can do to get the ball rolling on this?

For what it's worth, I tried to come up with a Sans-IO protocol implementation in python. This article from the author of the websocket library really rang a bell while I read this thread so I tried to come up with a similar solution for NATS.

The code is available on github: https://github.com/charbonnierg/natsproto, it's far from being robust or correct at the moment, but I believe that this is a very promising approach for supporting both a synchronous and an asynchronous client without reiventing the wheel. Any feedback would be greatly appreciated.

I also tried in this repository to write an async client without relying on asyncio, which would make it possible to run the client using trio (assuming a transport implemented on top of anyio also exists). TBH I was quite lost at the beginning, especially regarding cancellation 😅 but in the end I believe that writing code using structured concurrency rather than future-based approach + callbacks is easier to read and understand.

So to conclude there are 3 aspects in this proof of concept:

• a sans IO protocol
• an async client
• a sync client (poorly written)

I'm willing to work on my free time on the sans IO protocol up until there is a 100% coverage and more features such as validating subjects, validating max payload size, validating URL schemes, support for verbose mode, etc..., assuming that you're interested in this @wallyqs. This could take from a few weeks to a month. Biggest effort would be porting all existing protocol tests I guess. We could then come up with a very simple sync client and refactor the existing async client to use the sans-IO protocol. Only then would I consider introducing breaking changes to the async client to use anyio instead of asyncio, if you want this.

I think the best approach right now is to use https://github.com/airtai/faststream/ - yep its an additional layer of code but it works out of the box.

I think the best approach right now is to use https://github.com/airtai/faststream/ - yep its an additional layer of code but it works out of the box.

does it support sync operations? It seems to me like it is still async-only.

I think the best approach right now is to use https://github.com/airtai/faststream/ - yep its an additional layer of code but it works out of the box.

does it support sync operations? It seems to me like it is still async-only.

It works both ways