meadow icon indicating copy to clipboard operation
meadow copied to clipboard

Conflict when calling meadow from another runtime

Open fgadaleta opened this issue 2 years ago • 5 comments

block_on might be an issue for integration of meadow in tokio-based projects

fgadaleta avatar Aug 14 '23 05:08 fgadaleta

I agree. I was also thinking about, at least temporarily, trying to replace this with spawn_blocking per this thread to see how that goes, although I'm not sure it will buy much in this situation.

Frankly, Meadow's design around async inside of Nodes bothers me a little bit to begin with anyway; I don't like that every Node needs to have its own runtime, and I've found myself becoming interested in possibly differentiating Nodes into sync and async versions anyway, so that it would be nice to share a single runtime between multiple Nodes. As you said, I expect this would make integration into other Tokio-based project a bit easier. If you have thoughts on how to implement this, I'm certainly open to suggestions; otherwise, there will probably be one or two slow-moving branches looking in that direction that appears over the next few weeks/months.

quietlychris avatar Aug 15 '23 02:08 quietlychris

Starting to play around with different approaches for this in the async_split branch

quietlychris avatar Sep 05 '23 00:09 quietlychris

So it turns out that splitting async and sync code, and patterns for doing so, is not some kind of novel problem. Per this Stack Overflow thread libraries like reqwest do this sort of same thing, and recommends encapsulating most of the packet-level logic into stand-alone functions that can be called in the respective async/sync versions, resulting in less significant code duplication. I'm inclined to agree, but this will probably require some non-trivial re-structuring of meadow. This can actually probably be done bit by bit in main until pub/sub functions are more simple to reproduce before reaching the (current) block_on() calls that are being made in Node to run the I/O.

quietlychris avatar Sep 30 '23 21:09 quietlychris

@fgadaleta This doesn't necessarily address your block_on issue, but I've been toying around with trying to solve some potential Tokio conflicts by making it optional for Nodes to own their own runtime vs. being passed a handle to an external one; a link to an example demonstrating some of those experiments in the node_runtime branch is below. Certainly no pressure, but if you're willing to take a look or have any feedback from when you looked into this, I'd appreciate it :)

https://github.com/quietlychris/meadow/blob/93a0c2b7efdb7250cc12fcaeeb4bff3323d13320/examples/external_runtime.rs#L37

quietlychris avatar Jan 13 '24 04:01 quietlychris

This is being worked on in #64 via the async branch

quietlychris avatar May 25 '24 21:05 quietlychris