Discuss: general case for Async calls into Rust.

[jhugman]

This came from the discussion on delegates/decorators in #1051. That supports specializing method calls, perhaps with an async dispatch, but does not model every case.

The problem statement is relatively simple: the app would like to call into a Rust library via uniffi. The library performs some kind of computation, and returns the result back to the app (again, via uniffi).

The complexity/things to nail down is all the variations available, and/or we need to support:

• variation on async models: e.g. should the result be returned by callback, or by some kind of Future/Promise
• variation on who owns the threading: Rust or the foreign language side. Some foreign languages' default configurations don't support threading
• variation on who owns the thread pool: Fenix definitely wants ownership, Firefox for iOS is less concerned.
• variation of available implementations in each library: some rely on language features (async/await in Rust/Swift/JS/C#, suspend in Kotlin), some have specialist types: e.g. Promises

All of these variations have different popularity and maturity. It's clear we need to support some variation by library owners and app owners; it's not clear we can pick winners in most cases.

Post your strawfox proposals here, and point any discussions of async uniffi here.

┆Issue is synchronized with this Jira Task ┆Issue Number: UNIFFI-89

[mhammond]

Thanks James. I haven't thought too deeply about this, but some 1/2-baked thoughts:

• A bit like the wrapping discussions, I suspect that where policies must be dictated, they should be dictated by the foreign side. As you mention, a-c clearly wants to own the threading model and I suspect any sufficiently large and complex project will end up wanting this too.

• Alot of the patterns used in application-services were developed before async/await were available in practice - but I suspect the above considerations would have prevented us using them if they were. async/await/promises etc obviously don't exist in a vacuum - there's got to be an event-loop and background threads for when blocking the event-loop would otherwise be necessary, and having the rust component "own" these would probably not be satisfactory for some consumers (like Fenix).

This seems to imply that this problem lives in the domain of the foreign binding generators?

[jhugman]

This seems to imply that this problem lives in the domain of the foreign binding generators?

I think to a first approximation this analysis is correct; though I'm still not sure what happens in the Javascript or Python cases.

Perhaps a good place to start with this would be to gather examples of what we're doing by hand, and what concurrency patterns and practices are common in the each language community; then hopefully concrete the cowpaths.

[ShadowJonathan]

As an example of interop, i'd like to mention pyo3-asyncio here as an interesting case, they chose to have 2 event loops running on different threads; a rust one, and the python one.

Instead, async semantics are handled by suspending and passing callback messages between the two barriers using the help of some synchronisation primitives.

There is an issue open that talks about in-event-loop rust-future awaiting (https://github.com/awestlake87/pyo3-asyncio/issues/59), but this is a design which requires a ton of thought.

In general, i think above design (different event loops that synchronise eachother) could work for a majority of usecases.

though I'm still not sure what happens in the Javascript or Python cases.

I'm not sure what napi does, but wasm-bindgen entangles its event loop with Promises, and hands it over to the browser to schedule/ready these.

Ultimately, the problem exists that an event loop implementation must be aware of the foreign loop to properly work with it, as it assumes its domineering in that aspect, that it has total control over the thread.

---

In my opinion, above pattern (seperate event loops on different threads while they synchronise) could work for a majority of usecases, and could be a good stepping stone for other (more difficult) patterns.

[mhammond]

In general, i think above design (different event loops that synchronise eachother) could work for a majority of usecases.

While I think that's true, it does assume that usecases have flexibility about how to implement event loops. If you consider desktop Firefox or Android apps, it's probably not going to be reasonable to assume that UniFFI-based projects can influence the event loop implementations.

[bendk]

I'd like to revive this discussion now that we're hoping to add better async/callback support for Desktop JS in order to support Nimbus and viaduct. We've discussed several use cases that we think we need and features that would enable those use cases. However, I've never felt like we've completely mapped out the use cases and wondered if we're missing something.

I'm hoping that we can group all uses cases into a table with 4 dimensions:

• What kind of call is this? A normal call from rust into the foreign language or a call the other way using a CallbackInterface
• Who created/controls the thread the call is coming from? Right now we assume the thread is "owned" by the foreign language, but we want to add support for threads "owned" by Rust, for example calls that come from a tokio event loop.
• Do we want to switch the thread? I.e. do we want to a call from a foreign controlled thread to run on a rust controlled thread or vice-versa?
• Is the call blocking or asynchronous?

Based on those dimensions, let's try to actually map out all the possibilities:

Call type	Calling thread	Thread switch?	Blocking	Notes/Examples
Normal	Foreign	Same thread	Blocking	Normal use, already supported. This also includes the main way we achieve parallelism now: creating a thread pool on the foreign side, calling into rust from one of those threads and awaiting the result
Normal	Foreign	Same thread	Async	Async calls, new functionality. This only makes sense is if the Rust code then awaits an async call back into the foreign code using a CallbackInterface. But this use case seems very valid to me, for example the foreign code makes an async call to Nimbus.init() which then makes async calls to Jexl.eval().
Normal	Foreign	Thread switch	Blocking	Foreign code blocking on a Rust tokio-based network request. I think this is possible right now by making a normal blocking call, which then awaits the async call.
Normal	Foreign	Thread switch	Async	Foreign code awaiting a Rust tokio network request. This is new functionality, but I think it's the same work as the non-thread switch case.
Normal	Rust	*	*	X - In general, it's not safe for the foreign code to run on a Rust thread (although some languages might support this)
CallbackInterface	Foreign	Same thread	Blocking	Normal callback, already supported
CallbackInterface	Foreign	Same thread	Async	Async callback, new functionality. One question here is what the Rust thread then returns, since it's running on a foreign thread. I think there are 2 possibilities: returns a Future back that depends on the CallbackInterface future (i.e. this is the twin case of Normal / Foreign / Same thread / Async) or returns void (which I believe is the current plan for desktop Nimbus, the JS code calls Nimbus.init(), that starts up an async rust function that makes async calls back into JS to evaluate the JEXL, then Nimbus.init() returns None, handing the thread back to JS.)
CallbackInterface	Foreign	Thread switch	*	X - Not safe for the foreign code to run on a Rust thread
CallbackInterface	Rust	Same thread	*	X - Not safe for foreign code to run on a Rust thread
CallbackInterface	Rust	Thread switch	Blocking	Rust thread making a blocking call into the foreign code, which would require switching threads
CallbackInterface	Rust	Thread switch	Async	Rust thread making an async call into the foreign code, which would require switching threads

Based on that, I believe there are 2 features that we need to add:

• Rust code making an async call into a CallbackInterface / Foreign code making an async into Rust. I think this could be implemented by having UniFFI generate something like the hand-written demo from #1252.
• Handling CallbackInterface calls from a Rust-based thread. I'm not exactly sure how this would work, there's at least 2 options here:
  • Handle everything on the foreign side. In the generate code we currently register a callback to invoke a CallbackInterface call. We might be able to update that code so that it schedules the call to run on the correct thread. But this assumes that it's safe to call that callback on the Rust thread. Is that true for all of our current languages? Are we okay with adding this requirement for future languages?
  • Use a queue plus a waker. Push the CallbackInterface call to a queue, signal the foreign side (maybe writing a byte to a pipe or socket), then the foreign side would wake up and try to read from the queue. This seems more complicated than the first system, but might let us support more foreign languages.

I think the CallbackInterface calling mechanism should block, since we can still add async functionality on top of it using the first feature. I.e. the call would return a Future, which we then wrap and turn into an async call.

Does this analysis make sense? Is there any use cases that aren't captured by the table above?

[mhammond]

Thanks for continuing to push on this Ben! I need more time to think about this because it's quite complicated 😅 . I'm also struggling to match up actual use-cases and what some of the table items mean in practice - eg, "Callback/Foreign/SameThread/Async" means, IIUC:

• Foreign thread started, blocking "normal" call made (otherwise we wouldn't be able to make the callback on the same thread?)
• We make an async callback, back to the foreign code - this returns a promise/future
• Rust code, still on that foreign thread wants the result - how does it "await" on that foreign thread exactly? Or maybe the expectation is that it would not await, but instead the callback promise is returned back to that initial, blocking "normal" call? Or maybe I'm missing something?

(That's just one example I'm noting more to highlight that some bits aren't quite clear and could maybe do with some expansion.

Thinking about viaduct, we currently use callbacks, but that's only because we don't have async, right? ie, if we had some async support for the "Normal/Foreign/?/Async" case, Viaduct wouldn't use callbacks at all, right?

I guess I'm wondering if it's worth fleshing out some realistic real-world use cases along with the table, including their UDL? Similarly for #1252, I'm struggling to draw the line between "this part is hand-written because UniFFI doesn't support it yet" vs "this is what it would look like if UniFFI did support it". I'm sure we can come up with contrived examples for most of the entries, but are they actually realistic? I'm also struggling a little with what the "thread switch" column means in practice - I know what it means, but don't quite understand all the use-cases, nor how it fits with a single-threaded executor.

These are just my initial thoughts and might not make sense once I've thought this through a little more - I'll come back to this and will try and do some of my suggestions re use-cases etc.

(Another random thought is that it's tricky to add comments/questions to the comment above - there were a few things I didn't fully understand or had minor suggestions for clarity. I wonder if this should be a PR to a document, so that we can add comments/suggestions to individual lines and/or entries in the table etc?)

[bendk]

@mhammond I totally agree with all of that. I reformatted this into a larger doc, tried to add some comments, and opened #1306 to discuss it more.