Investigate async Rust on Android and iOS

[str4d]

The question I have is primarily how we connect things like Android coroutines into the Rust async ecosystem, and whether that is an effective way to manage the necessary work (vs just having Rust do its own thing).

[str4d]

I had a look at the current tokio runtime impl. I think if we wanted to have the runtime integrate with e.g. Kotlin coroutines, we'd need to effectively implement the tokio runtime within Android or iOS directly.

I think the optimal strategy will be to just run the tokio runtime inside a background thread (either reusing an existing IO thread or making a separate one). The Rust side will spawn its own threads for IO and blocking work (the number of which can be configured at runtime creation). Then we just rely on the OS to handle thread scheduling.

[pacu]

I think the optimal strategy will be to just run the tokio runtime inside a background thread (either reusing an existing IO thread or making a separate one).

@str4d can you point me to some docs?

[str4d]

See the tokio::runtime docs. What I said above would correspond to something like:

/// FFI function to start the reactor.
/// This would need to be called before any FFI function that requires it.
fn start_reactor() {
    let runtime = Builder::new_multi_thread()
        // ... do any configuring we want
        .build()
        .unwrap();

    let handle = rt.handle();
    // Store the handle in a Rust-side global so we can access it from other FFI functions.

    runtime.block_on(async {
        // Kick off the root-level async component that manages the sync process.
    })
}

There might be other ways to do this, e.g. on first call of a Rust FFI method that needs the reactor, have Rust make its own thread and start the reactor. That might make it easier on the SDK because it wouldn't need to be as careful with how it interacts with async operations. This is the kind of thing that needs experimentation.

[pacu]

I think we should be able to initialize what's needed on the initialize method of the SDKs. Also there should be a deinit or pause as well for OS interruptions and termination.

[str4d]

Yeah, I think if it's FFI-driven, we'll want a series of FFI methods:

• Initialize runtime, get back handle that can control it.
• In a thread, run the runtime. This blocks the thread (and fires off subthreads as appropriate).
• Use the runtime. This adds tasks that the runtime will process.
• Shutdown the runtime. This uses the handle to inform the runtime of its end, causing the runner thread to cancel or drop any pending tasks and unblock.

[Slesarew]

Hi. Just in case you find this useful, I've had surprisingly successful results in similar situation with letting native android/ios threading system handle multithread management while calling rust code through lazy static mutex: https://github.com/paritytech/parity-signer/blob/master/rust/navigator/src/lib.rs#L21-L60

This is very stable and was shipped in production already; I'm a bit concerned that native environment might start misbehaving if threads are spinned outside of it (this happened with JS/reactnative frontend before, but that's a bit different story), however I'll start investigating that again soon and let you know. But if that fails, it is always possible to route spinning through native handler with some simple logic.

[str4d]

Yeah, that's the other direction I was wondering above (implement an async runtime in a way that it can be driven by Kotlin or Swift directly, which would require techniques like you use). It seems like that might be a significant lift, but if we did do that then it would mean the runtime is directly embedded in the outer environment's existing coroutines or whatever, and could be visible to the Os-level resource management.

[DAOCUONG]

For ios and mac os handle the thread itself we just need to give it a task https://github.com/SSheldon/rust-dispatch